Not from the underlying frame. Fixes races with frame threading in
field-coded files, where decoding would wait for the wrong field (e.g.
random failures in mixed-nal-coding).

Bug-Id: 954