Finishes fixing ticket #6171

This is related to, but doesn't solve ticker 6152.

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Fixes: 639/clusterfuzz-testcase-5143866241974272

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/gsmdec_template: Fix runtime error: signed integer overflow: -22527 * 99113 cannot be represented in type 'int'

Fixes: 636/clusterfuzz-testcase-6520876646268928
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/bmp: Fix runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself

There is code checking height and width later, leaving an invalid value invalid
is thus fine.

Fixes: 635/clusterfuzz-testcase-6225161437052928
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Nicolas Roy-Renaud <nicolas.roy-renaud.1@ens.etsmtl.ca>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

When setting the channel layout directly using AVBufferSrcParameters
the channel layout was correctly set however the init function still
expected the old string format to set the number of channels (when it
hadn't already been specified).
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

check if proto is null before av_strcasecmp
CID:  1401346
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>

Fixes: signed integer overflow: 5 + 2147483646 cannot be represented in type 'int'
Fixes: 634/clusterfuzz-testcase-5285420445204480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 633/clusterfuzz-testcase-4553133554401280

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 630/clusterfuzz-testcase-6608718928019456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Fixes #5442.
Signed-off-by: Paul B Mahol <onemda@gmail.com>

Silence a valgrind warning.

Fixes ticket #6160.

avcodec/dca_xll: Fix runtime error: signed integer overflow: -1073741824 * 32768 cannot be represented in type 'int'

Fixes: 629/clusterfuzz-testcase-6697457381539840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: left shift of 32 by 26 places cannot be represented in type 'int'

Fixes: 628/clusterfuzz-testcase-6187747641393152
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 627/clusterfuzz-testcase-5020897033322496
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The quantization table is stored in the natural order, but when we
access it, we use an index that's in zigzag order, causing us to read
the wrong value. This causes artifacts, especially in areas with
horizontal or vertical edges. The artifacts look a lot like the
DCT ringing artifacts you'd expect to see from a low-bitrate file,
but when comparing to NewTek's own decoder, it's obvious they're not
supposed to be there.

Fix by simply storing the scaled quantization table in zigzag order.
Performance is unchanged.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

lavf/mov.c: Correct keyframe search in edit list to return the very first keyframe/frame with matching timestamp. Fixes ticket#5904
Signed-off-by: Sasi Inguva <isasi@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 625/clusterfuzz-testcase-4574924406521856
Fixes: 626/clusterfuzz-testcase-4738718621499392

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes undefined behavior
Fixes: 622/clusterfuzz-testcase-5745722022428672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

ticket #6157
Reported-by: Steven Liu <lq@chinaffmpeg.org>
Signed-off-by: Pavel Koshevoy <pkoshevoy@gmail.com>

Fixes: 618/clusterfuzz-testcase-6594990333493248

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 617/clusterfuzz-testcase-6413875723370496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/h264_ps: Fix runtime error: signed integer overflow: -1094995528 * 2 cannot be represented in type 'int'

Fixes: 615/clusterfuzz-testcase-5488002644049920
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/h264_cabac: runtime error: signed integer overflow: 2147483647 + 14 cannot be represented in type 'int'

Fixes: 614/clusterfuzz-testcase-4931860079575040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 612/clusterfuzz-testcase-4707817137111040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 611/clusterfuzz-testcase-5613455820193792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 610/clusterfuzz-testcase-4831030085156864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

If the PVQ search picked a place to increment/decrement on the y[]
vector which had no pulse then it would cause a desync since it would
change the sum in the wrong direction. Fix this by not considering
places without pulses as viable.

This makes the PVQ search slightly worse at K < 5 which isn't all that
common. Still, this is a workaround to prevent making broken files until
I can think of a better way of fixing it.

Also add an assertion, which can be removed or moved to assert1/2 once
the PVQ search is stable.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Fixes: runtime error: shift exponent 132 is too large for 32-bit type 'int'
Fixes: 609/clusterfuzz-testcase-4825202619842560

See 11.2.2 IHDR Image header

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 608/clusterfuzz-testcase-603978286392934

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes Coverity CID 1396259.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>

Fixes Coverity CID 1396416.
Signed-off-by: Marton Balint <cus@passwd.hu>