We can't do this in general since we could be reading a file with B-frames while
lacking an index.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>