libtta 2.3 has a limit of 6 channels, so 16 is substantially above the "official" already

Fixes: OOM
Fixes: 15249/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TTA_fuzzer-5643988125614080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (76 sec -> 24 sec)
Fixes: 15043/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SANM_fuzzer-5699856238116864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array read
Fixes: 15286/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5658245101780992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 20242 * 121095 cannot be represented in type 'int'
Fixes: 15310/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5737051745419264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Infinite loop
Fixes: 15337/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5757428949319680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 2147483646 + 2 cannot be represented in type 'int'
Fixes: infinite loop
Fixes: 15396/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5116605501014016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Instead we ask for a sample, its unclear what to do in this case.

Fixes: index 30 out of bounds for type 'int8_t [30][64]'
Fixes: 15339/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5749441484554240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 1106434976 + 1041773512 cannot be represented in type 'int'
Fixes: 15421/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV2_fuzzer-5669209314426880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: -1027919784 + -1120041624 cannot be represented in type 'int'
Fixes: 15406/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3_fuzzer-5700646528876544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: OOM
Fixes: 15410/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SVQ3_fuzzer-5659464805384192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

This checks the ham value much stricter and avoids hitting cases which cannot be reached
with data from the libavformat demuxer.

Fixes: out of array access
Fixes: 15320/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5080476840099840
Fixes: 15423/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5630765833912320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 6494225984479297536 - -6043795377581187040 cannot be represented in type 'long'
Fixes: 15285/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5632780307791872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: negation of -9223372036854775808 cannot be represented in type 'int64_t' (aka 'long'); cast to an unsigned type to negate this value to itself
Fixes: 15289/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5709034499342336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array access
Fixes: 15360/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLIC_fuzzer-5653837190266880
Fixes: 15412/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLIC_fuzzer-5740537648250880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: index -1 out of bounds for type 'const uint8_t [185][2]'
Fixes: 15250/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3IMAGE_fuzzer-5648992869810176

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

lpc_quant of 0 produces undefined behavior, thus disallow this.
If valid samples use this then such a sample would be quite
usefull to confirm the correct&lossles handling of this.

Fixes: libavcodec/alac.c:218:25: runtime error: shift exponent -1 is negative
Fixes: 15273/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5656388535058432
Fixes: 15276/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5761238417539072
Fixes: 15315/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5767260766994432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Various anomalies
Fixes: 14493/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXV_fuzzer-5071018000908288
Fixes: 14630/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXV_fuzzer-5714888963391488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

This commit changes vp9_metadata to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

If a new sequence display extension had to be added, this was up until
now done at two places: One where a sequence display extension was
initialized with default values and one where the actual sequence
display extension was inserted into the fragment. This division of
labour is unnecessary and pointless; it has been changed.

Furthermore, if a sequence display extension has to be added, the
earlier code set some fields to their default value twice. This has been
changed, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes mpeg2_metadata to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes h265_metadata to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

When looping over an access unit's units in positive direction and
deleting some of them, one needs to make sure that a unit that is at
the position of a unit that just got deleted gets checked, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes h264_redundant_pps to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

If the fragment is empty after parsing (i.e. it contains no OBUs), then
the check for the type of the fragment's first OBU is nonsensical; so
error out in this case just as h264_metadata and hevc_metadata do.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The recent changes to h264_metadata (enabled by the recent changes to
ff_cbs_write_packet) made it possible to add side_data to the output
packet at any place, not only after the output packet has been written
and the properties of the input packet copied. This means that one can
now localize the code to add display orientation side-data to the packet
to the place dealing with said display-orientation.

Furthermore, the documentation of av_display_rotation_set states that
the matrix will be fully overwritten by it, so there is no need to
allocate it with av_mallocz.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes h264_metadata to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

When testing whether a particular unit should be kept or discarded, it
is best to start at the very last unit of a fragment and count down,
because that way a unit that will eventually be deleted won't be
memmoved during earlier deletions; and frag/au->nb_units need only be
evaluated once in this case and the counter is automatically correct
when a unit got deleted.

It also works for double loops, i.e. when looping over all SEI messages
in all SEI units of an access unit.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes filter_units to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props (or, in case of
passthrough, to av_packet_move_ref).

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

According to the BSF API, when a BSF is finished with an input packet,
it should return AVERROR(EAGAIN) to signal that another packet should be
sent to the BSF via av_bsf_send_packet that the actual BSF can receive
via ff_bsf_get_packet[_ref]. filter_units on the other hand simply called
ff_bsf_get_packet again if the first packet received didn't result in
any output. This call of course returned AVERROR(EAGAIN) which was
returned, but it is nevertheless better to not include a fake loop.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

According to the API, the packet structure a bsf receives must not be
touched on failure, yet filter_units nevertheless did it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes av1_metadata to (a) use ff_bsf_get_packet_ref
instead of ff_bsf_get_packet (thereby avoiding one malloc and free per
filtered packet) and (b) to use only one packet structure at all,
thereby avoiding a call to av_packet_copy_props.

(b) has been made possible by the recent changes to ff_cbs_write_packet.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Up until now, ff_cbs_write_packet always initialized the packet
structure it received without documenting this behaviour; furthermore,
the packet's buffer would (on success) be overwritten with the new
buffer without unreferencing the old. This meant that the input packet
had to be either clean (otherwise there would be memleaks) in which case
the initialization is redundant or uninitialized. ff_cbs_write_packet
was never used with uninitialized packets, so the initialization was
redundant. Worse yet, it forced callers to use more than one packet and
made it difficult to add side-data to a packet designated for output,
because said side-data could only be attached after the call to
ff_cbs_write_packet.

This has been changed. It is now allowed to use a non-blank packet.
The currently existing buffer will be unreferenced and replaced by
the new one, as will be the accompanying fields (i.e. data and size).
The rest isn't touched at all.

This change will enable us to use only one packet in the bitstream
filters that rely on CBS.

This commit also updates the documentation of ff_cbs_write_extradata
and ff_cbs_write_packet (to better describe existing behaviour and in
the latter case to also describe the new behaviour).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

set AVFrame decode_error_flags in case h->slice_ctx->er.error_occurred is set
after the call to ff_h264_execute_decode_slices. This allows the user to detect
concealed decoding errors in the call to avcodec_receive_frame
Signed-off-by: Amir Pauker <amir@livelyvideo.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Amir Pauker <amir@livelyvideo.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Equivalent to the previous patch for libx264.

Fix the quantisation offset - use the whole range, and don't change the
offset size based on bit depth.

Iterate the list in reverse order.  The first region in the list is the one
that applies in the case of overlapping regions.

Clarify and add examples for the behaviour of the quantisation offset,
and define how multiple ranges should be handled.