• Katerina Barone-Adesi's avatar
    idct8x8: Fix undefined negative shifts · 1389b4c1
    Katerina Barone-Adesi authored
    The original code left-shifts negative values, which is undefined
    in the C99 specification (the one used during normal Libav compilation).
    This change multiplies by (1 << shift), which is functionally equivalent,
    but has defined behavior.
    
    With this change, fate-idct8x8 compiled with --fsanitize=undefined works.
    
    Bug-Id: 686
    1389b4c1
jrevdct.c 36 KB