For stereo we need to use intermediate planar buffers, but mono does not need
to be deinterleaved so the output buffer can be used directly.