This makes the changed code-path faster.

Change not tested except with the fuzzer testcase as I found no other testcase.

Improves: Timeout (136sec -> 74sec)
Improves: 16040/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SCPR_fuzzer-5705876062601216
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg