• Martin Storsjö's avatar
    checkasm: aacpsdsp: Tolerate extra intermediate precision in stereo_interpolate · aad0e26f
    Martin Storsjö authored
    The stereo_interpolate functions add h_step to the values h
    BUF_SIZE times. Within the stereo_interpolate C functions, the
    values h (h0-h3, h00-h13) are declared as local float variables,
    but the compiler is free to keep them in a register with extra
    precision.
    
    If the accumulation is rounded to 32 bit float precision after
    each step, the less significant bits of h_step end up ignored
    and the sum can deviate, affecting the end result more than
    the currently set EPS.
    
    By clearing the log2(BUF_SIZE) lower bits of h_step, we make sure
    that the accumulation shouldn't differ significantly, regardless
    of any extra precision in the accmulating register/variable.
    
    This fixes the aacpsdsp checkasm test when built with clang for
    mingw/x86_32.
    Signed-off-by: 's avatarMartin Storsjö <martin@martin.st>
    aad0e26f
Name
Last commit
Last update
..
aarch64 Loading commit data...
arm Loading commit data...
x86 Loading commit data...
.gitignore Loading commit data...
Makefile Loading commit data...
aacpsdsp.c Loading commit data...
af_afir.c Loading commit data...
alacdsp.c Loading commit data...
audiodsp.c Loading commit data...
blockdsp.c Loading commit data...
bswapdsp.c Loading commit data...
checkasm.c Loading commit data...
checkasm.h Loading commit data...
exrdsp.c Loading commit data...
fixed_dsp.c Loading commit data...
flacdsp.c Loading commit data...
float_dsp.c Loading commit data...
fmtconvert.c Loading commit data...
g722dsp.c Loading commit data...
h264dsp.c Loading commit data...
h264pred.c Loading commit data...
h264qpel.c Loading commit data...
hevc_add_res.c Loading commit data...
hevc_idct.c Loading commit data...
hevc_sao.c Loading commit data...
huffyuvdsp.c Loading commit data...
jpeg2000dsp.c Loading commit data...
llviddsp.c Loading commit data...
llviddspenc.c Loading commit data...
opusdsp.c Loading commit data...
pixblockdsp.c Loading commit data...
sbrdsp.c Loading commit data...
sw_rgb.c Loading commit data...
synth_filter.c Loading commit data...
utvideodsp.c Loading commit data...
v210dec.c Loading commit data...
v210enc.c Loading commit data...
vf_blend.c Loading commit data...
vf_colorspace.c Loading commit data...
vf_eq.c Loading commit data...
vf_gblur.c Loading commit data...
vf_hflip.c Loading commit data...
vf_nlmeans.c Loading commit data...
vf_threshold.c Loading commit data...
videodsp.c Loading commit data...
vp8dsp.c Loading commit data...
vp9dsp.c Loading commit data...