If this happens, it's a sign of parsing issues earlier in the process, or
misuse by the calling module.

Prevents writing invalid bitstreams.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit ef13fafe229a86480305fbb05e83208102ea962f)