• Martin Storsjö's avatar
    checkasm: aacpsdsp: Tolerate extra intermediate precision in stereo_interpolate · aad0e26f
    Martin Storsjö authored
    The stereo_interpolate functions add h_step to the values h
    BUF_SIZE times. Within the stereo_interpolate C functions, the
    values h (h0-h3, h00-h13) are declared as local float variables,
    but the compiler is free to keep them in a register with extra
    precision.
    
    If the accumulation is rounded to 32 bit float precision after
    each step, the less significant bits of h_step end up ignored
    and the sum can deviate, affecting the end result more than
    the currently set EPS.
    
    By clearing the log2(BUF_SIZE) lower bits of h_step, we make sure
    that the accumulation shouldn't differ significantly, regardless
    of any extra precision in the accmulating register/variable.
    
    This fixes the aacpsdsp checkasm test when built with clang for
    mingw/x86_32.
    Signed-off-by: 's avatarMartin Storsjö <martin@martin.st>
    aad0e26f
Name
Last commit
Last update
compat Loading commit data...
doc Loading commit data...
ffbuild Loading commit data...
fftools Loading commit data...
libavcodec Loading commit data...
libavdevice Loading commit data...
libavfilter Loading commit data...
libavformat Loading commit data...
libavresample Loading commit data...
libavutil Loading commit data...
libpostproc Loading commit data...
libswresample Loading commit data...
libswscale Loading commit data...
presets Loading commit data...
tests Loading commit data...
tools Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.travis.yml Loading commit data...
CONTRIBUTING.md Loading commit data...
COPYING.GPLv2 Loading commit data...
COPYING.GPLv3 Loading commit data...
COPYING.LGPLv2.1 Loading commit data...
COPYING.LGPLv3 Loading commit data...
CREDITS Loading commit data...
Changelog Loading commit data...
INSTALL.md Loading commit data...
LICENSE.md Loading commit data...
MAINTAINERS Loading commit data...
Makefile Loading commit data...
README.md Loading commit data...
RELEASE Loading commit data...
configure Loading commit data...