It is not supposed to be done outside lavc.

This is basically a revert of 818062f2.

It is unclear what issue this was supposed to fix, if it reappears again
it will have to be fixed in a more proper place.

The wtv-demux test change is because the sample starts with a B-frame.