• Ganesh Ajjanagadde's avatar
    avutil/mathematics: make av_gcd more robust · b7fb7c45
    Ganesh Ajjanagadde authored
    This ensures that no undefined behavior is invoked, while retaining
    identical return values in all cases and at no loss of performance
    (identical asm on clang and gcc).
    Essentially, this patch exchanges undefined behavior with implementation
    defined behavior, a strict improvement.
    
    Rationale:
    1. The ideal solution is to have the return type a uint64_t. This
    unfortunately requires an API change.
    2. The only pathological behavior happens if both arguments are
    INT64_MIN, to the best of my knowledge. In such a case, the
    implementation defined behavior is invoked in the sense that UINT64_MAX
    is interpreted as INT64_MIN, which any reasonable implementation will
    do. In any case, any usage where both arguments are INT64_MIN is a
    fuzzer anyway.
    3. Alternatives of checking, etc require branching and lose performance
    for no concrete gain - no client cares about av_gcd's actual value when
    both args are INT64_MIN. Even if it did, on sane platforms (e.g all the
    ones FFmpeg cares about), it produces a correct gcd, namely INT64_MIN.
    Reviewed-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
    Signed-off-by: 's avatarGanesh Ajjanagadde <gajjanagadde@gmail.com>
    b7fb7c45
Name
Last commit
Last update
compat Loading commit data...
doc Loading commit data...
libavcodec Loading commit data...
libavdevice Loading commit data...
libavfilter Loading commit data...
libavformat Loading commit data...
libavresample Loading commit data...
libavutil Loading commit data...
libpostproc Loading commit data...
libswresample Loading commit data...
libswscale Loading commit data...
presets Loading commit data...
tests Loading commit data...
tools Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.travis.yml Loading commit data...
COPYING.GPLv2 Loading commit data...
COPYING.GPLv3 Loading commit data...
COPYING.LGPLv2.1 Loading commit data...
COPYING.LGPLv3 Loading commit data...
CREDITS Loading commit data...
Changelog Loading commit data...
INSTALL.md Loading commit data...
LICENSE.md Loading commit data...
MAINTAINERS Loading commit data...
Makefile Loading commit data...
README.md Loading commit data...
RELEASE Loading commit data...
arch.mak Loading commit data...
cmdutils.c Loading commit data...
cmdutils.h Loading commit data...
cmdutils_common_opts.h Loading commit data...
cmdutils_opencl.c Loading commit data...
common.mak Loading commit data...
configure Loading commit data...
ffmpeg.c Loading commit data...
ffmpeg.h Loading commit data...
ffmpeg_dxva2.c Loading commit data...
ffmpeg_filter.c Loading commit data...
ffmpeg_opt.c Loading commit data...
ffmpeg_qsv.c Loading commit data...
ffmpeg_vdpau.c Loading commit data...
ffmpeg_videotoolbox.c Loading commit data...
ffplay.c Loading commit data...
ffprobe.c Loading commit data...
ffserver.c Loading commit data...
ffserver_config.c Loading commit data...
ffserver_config.h Loading commit data...
library.mak Loading commit data...
version.sh Loading commit data...