The substreams can have different resampling delays, so an additional
level of buffering is needed to synchronize them.

Bug-Id: 876