Commit 4240e6a9 authored by Michael Niedermayer's avatar Michael Niedermayer

avfilter/vf_hqdn3d: Initialize the whole LUT

With bps > 8 more than 255..255 are used
The initialized table content is left unchanged,
But it could also be adjusted for the slight difference of
the maximum
Signed-off-by: 's avatarMichael Niedermayer <michaelni@gmx.at>
parent 7ae805db
......@@ -180,9 +180,9 @@ static int16_t *precalc_coefs(double dist25, int depth)
gamma = log(0.25) / log(1.0 - FFMIN(dist25,252.0)/255.0 - 0.00001);
for (i = -255<<LUT_BITS; i <= 255<<LUT_BITS; i++) {
for (i = -256<<LUT_BITS; i < 256<<LUT_BITS; i++) {
double f = ((i<<(9-LUT_BITS)) + (1<<(8-LUT_BITS)) - 1) / 512.0; // midpoint of the bin
simil = 1.0 - FFABS(f) / 255.0;
simil = FFMAX(0, 1.0 - FFABS(f) / 255.0);
C = pow(simil, gamma) * 256.0 * f;
ct[(256<<LUT_BITS)+i] = lrint(C);
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment