It should not be a value larger than the number of streams we have,
or it will cause invalid reads and/or SIGSEGV.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>