Fixes: 734/clusterfuzz-testcase-4821293192970240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 733/clusterfuzz-testcase-4682158096515072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: undefined shift
Fixes: 631/clusterfuzz-testcase-6725491035734016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Overhauled version, original patch by Miroslav Slugeň <thunder.m@email.cz>.

Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

Make it clear that there is no timing-dependent behavior. In particular,
there is no state in which both input and output are denied, and where
you have to wait for a while yourself to make progress (apparently some
hardware decoders like to do this).

Avoid wording that makes references to time. It shouldn't be mistaken
for some kind of asynchronous API (like POSIX read() can return EAGAIN
if there is no new input yet). It's a state machine, so try to use
appropriate terms.
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Merges Libav commit 8a60bba0.

Apparently the demuxer outputs the wrong padding for HE-AAC (based on
the raw sample rate, or so). aacdec contains a hack to adjust the muxer
padding accordingly before it's used to trim the decoder output. This
modified the packet side data, which in combination with the old
decoding API would change the packet the user passed to the decoder.
This is clearly not allowed, and it breaks running some gapless fate
tests with "-fflags +keepside" applied (without keepside, the packet
metadata is typically newly allocated, essentially making a copy and not
modifying the user's input packet).

This should probably be fixed in the demuxer (and consequently also the
muxer), but for now only fix the immediate problem.

Regression since 946ed78f (2012).

except filter_length == 1

odd filter_length gives worse frequency response,
even when compared with shorter filter_length

also makes build_filter simpler
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

Fixes: 729/clusterfuzz-testcase-5154831595470848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/pictordec: Fix runtime error: left shift of 64 by 25 places cannot be represented in type 'int'

Fixes: 724/clusterfuzz-testcase-6738249571631104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

unintentionally changed to 0.01 at
'61926b6c'
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

Fixes code with qemu ARM
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The Chen-Shapiro(CS) test was used to test normality for
Lagged Fibonacci PRNG.

Normality Hypothesis Test:

The null hypothesis formally tests if the population
the sample represents is normally-distributed. For
CS, when the normality hypothesis is True, the
distribution of QH will have a mean close to 1.

Information on CS can be found here:

http://www.stata-journal.com/sjpdf.html?articlenum=st0264
http://www.originlab.com/doc/Origin-Help/NormalityTest-AlgorithmSigned-off-by: Thomas Turner <thomastdt@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

this gives better frequency response

update swresample fate and other fates
that depend on resampling
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

This reverts commit faa9d298.

This change became superfluous when support for C11 atomics was introduced.
Reverting it will make the removal of this implementation in an upcoming
merge conflict free.
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

Fixes a gcc warning:
libswscale/slice.c:178:56: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]

Fixes warnings like the following:
libswscale/input.c:951:13: warning: ‘planar_rgb14be_to_a’ defined but not used

Fixes: timeout in 730/clusterfuzz-testcase-5265113739165696 (part 2 of 2)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: timeout in 730/clusterfuzz-testcase-5265113739165696 (part 1 of 2)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Update the fate test as needed.

Update the fate test as needed.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

use ff_thread_once
Suggested-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

use ff_thread_once
Suggested-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

use ff_thread_once
Suggested-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

use ff_thread_once
Suggested-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>

The constants used in the decoder used floating point precision,
and this caused different values to be generated on different
architectures.

So, eradicate floating point numbers and use fixed point (32.32)
arithmetics everywhere, replacing constants with precomputed integer
values.

Signed-off-by: Vittorio Giovara <vittorio.giovara at gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>

The way videotoolbox hooks in as a hwaccel is pretty hacky. The VT decode
API is not invoked until end_frame(), so alloc_frame() returns a dummy
frame with a 1-byte buffer. When end_frame() is eventually called, the
dummy buffer is replaced with the actual decoded data from
VTDecompressionSessionDecodeFrame().

When the VT decoder fails, the frame returned to the h264 decoder from
alloc_frame() remains invalid and should not be used. Before
97472199, it was accidentally being
returned all the way up to the API user. After that commit, the dummy
frame was unref'd so the user received an error.

However, since that commit, VT hwaccel failures started causing random
segfaults in the h264 decoder. This happened more often on iOS where the
VT implementation is more likely to throw errors on bitstream anomolies.
A recent report of this issue can be see in
http://ffmpeg.org/pipermail/libav-user/2016-November/009831.html

The issue here is that the dummy frame is still referenced internally by the
h264 decoder, as part of the reflist and cur_pic_ptr. Deallocating the
frame causes assertions like this one to trip later on during decoding:

  Assertion h->cur_pic_ptr->f->buf[0] failed at src/libavcodec/h264_slice.c:1340

With this commit, we leave the dummy 1-byte frame intact, but avoid returning it
to the user.

This reverts commit 97472199.
Signed-off-by: wm4 <nfxjfg@googlemail.com>

Signed-off-by: James Zern <jzern@google.com>

The code previously completely discarded frames that had any error in a slice
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Without this the FPU state becomes trashed and causes mysterious
fate failures with cpuflags=0
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes memleak
Fixes: 548/clusterfuzz-testcase-5511470875934720

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Ever since the codecpar changes, this has been always printed when
opening a flv file. This is because the codecpar changes made all
streams to be added lazily as read_packet is called.

Now the previous commit takes care of this instead.