This fixes out of array reads and/or infinite loops.

30 is the maximum number of bits that can be read into
coeff_abs below.

CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Martin Storsjö <martin@martin.st>