• Ronald S. Bultje's avatar
    swscale: fix overflows in vertical scaling at top/bottom edges. · d49352c7
    Ronald S. Bultje authored
    This fixes integer multiplication overflows in RGB48 output
    (vertical) scaling as detected by IOC. What happens is that for
    certain types of filters (lanczos, spline, bicubic), the
    intermediate sum of coefficients in the middle of a filter can
    be larger than the fixed-point equivalent of 1.0, even if the
    final sum is 1.0. This is fine and we support that.
    
    However, at frame edges, initFilter() will merge the coefficients
    for the off-screen pixels into the top or bottom pixel, such as
    to emulate edge extension. This means that suddenly, a single
    coefficient can be larger than the fixed-point equivalent of
    1.0, which the vertical scaling routines do not support.
    
    Therefore, remove the merging of coefficients for edges for
    the vertical scaling filter, and instead add edge detection
    to the scaler itself so that it copies the pointers (not data)
    for the edges (i.e. it uses line[0] for line[-1] as well), so
    that a single coefficient is never larger than the fixed-point
    equivalent of 1.0.
    d49352c7
Name
Last commit
Last update
doc Loading commit data...
libavcodec Loading commit data...
libavdevice Loading commit data...
libavfilter Loading commit data...
libavformat Loading commit data...
libavutil Loading commit data...
libpostproc Loading commit data...
libswscale Loading commit data...
presets Loading commit data...
tests Loading commit data...
tools Loading commit data...
.gitignore Loading commit data...
COPYING.GPLv2 Loading commit data...
COPYING.GPLv3 Loading commit data...
COPYING.LGPLv2.1 Loading commit data...
COPYING.LGPLv3 Loading commit data...
CREDITS Loading commit data...
Changelog Loading commit data...
Doxyfile Loading commit data...
INSTALL Loading commit data...
LICENSE Loading commit data...
Makefile Loading commit data...
README Loading commit data...
RELEASE Loading commit data...
avconv.c Loading commit data...
avplay.c Loading commit data...
avprobe.c Loading commit data...
avserver.c Loading commit data...
cmdutils.c Loading commit data...
cmdutils.h Loading commit data...
cmdutils_common_opts.h Loading commit data...
common.mak Loading commit data...
configure Loading commit data...
ffmpeg.c Loading commit data...
library.mak Loading commit data...
version.sh Loading commit data...