If there are any samples remaining in the output fifo from previous conversion
calls, we have to output those samples first instead of doing direct output
of the current samples.