• Michael Niedermayer's avatar
    avcodec/flacdsp: Avoid undefined operations in non debug builds · 831274fb
    Michael Niedermayer authored
    This fixes ubsan warnings in non debug builds by using unsigned operations
    
    in debug builds the correct signed operations are retained so that overflows
    (which should not occur in valid files and may indicate problems in the DSP code
    or decoder) can be detected.
    
    Alternatively they can be changed to unsigned unconditionally, then its
    not possible though to detect overflows easily if someone wants to test
    the DSP code for overflows.
    
    The 2nd alternative would be to leave the code as it is and accept that
    there are undefined operations in the DSP code and that ubsan output is
    full of them in some cases.
    
    Similar changes would be needed in some other DSP routines
    Suggested-by: 's avatarMatt Wolenetz <wolenetz@google.com>
    Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
    831274fb
flacdsp.c 4.06 KB