Without this its possible to make the new decode API decode billions of samples
out of a empty input and never return to the caller before all samples have been
created and discarded.

Fixes: Timeout
Fixes: 2992/clusterfuzz-testcase-6649611793989632
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>