• Andreas Cadhalpun's avatar
    golomb: always check for invalid UE golomb codes in get_ue_golomb · 22e960ad
    Andreas Cadhalpun authored
    Also correct the check to reject log < 7, because UPDATE_CACHE only
    guarantees 25 meaningful bits.
    
    This fixes undefined behavior:
    runtime error: shift exponent is negative
    
    Testing with START/STOP timers in get_ue_golomb, one for the first
    branch (A) and one for the second (B), shows that there is practically no
    slowdown, e.g. for the cavs decoder:
    
    With the check in the B branch:
        629 decicycles in get_ue_golomb B, 4194260 runs,     44 skips
        433 decicycles in get_ue_golomb A,268434102 runs,   1354 skips
    
    Without the check:
        624 decicycles in get_ue_golomb B, 4194273 runs,     31 skips
        433 decicycles in get_ue_golomb A,268434203 runs,   1253 skips
    
    Since the B branch is executed far less often than the A branch, this
    change is negligible, even more so for the h264 decoder, where the ratio
    B/A is a lot smaller.
    
    Fixes: mozilla bug 1230239
    Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
    
    Found-by: Tyson Smith
    Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
    Reviewed-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
    Signed-off-by: 's avatarAndreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
    22e960ad
golomb.h 14 KB