If this happens, it's a sign of parsing issues earlier in the process, or
misuse by the calling module.

Prevents writing invalid bitstreams.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 318a1a383dc0312ad5b4afec0ddf0d8d231f5c79)