• Ronald S. Bultje's avatar
    vp9: fix overflow in 8x8 topleft 32x32 idct ssse3 version. · fd8b90f5
    Ronald S. Bultje authored
    Also disable the mmx/iwht optimization when the bitexact flag is set.
    With synthetically coded coefficients (i.e. these that lead to a
    residual well outside the [-255,255] range), our optimizations will
    overflow. It doesn't make sense to fix the overflows, since they can
    only occur on synthetic input, not on real fwht-generated input. Thus,
    add a bitexact flag that disables this optimization.
    fd8b90f5
vp9dsp_init.c 19.1 KB