Fixes: signed integer overflow: 40550400 * 128 cannot be represented in type 'int'
Fixes: 20331/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV40_fuzzer-5676685725007872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Marton Balint <cus@passwd.hu>

Currently Musepack allocates an array that needs to be freed later in
the demuxer's read_close-function; it is the sole reason for said
function's existence. But it is unnecessary, because one can store this
array in the stream's priv_data pointer, so that it will be freed
generically.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Chapter titles are added to the chapter's metadata since 6cb6e159,
yet since 012867f0 (the predecessor of) avpriv_new_chapter() already
adds the title to the chapter's metadata. So setting it again in
matroskadec.c is redundant and expensive.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Moreover, putting the Cues in front of the Clusters by reserving space
in advance is also tested.

The new capability of using ffprobe during a remux/transcode test are
used here for information about the chapters.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This is primarily intended to test that muxers correctly write chapters
or metadata; but given that it does this by having our demuxers read the
generated files, it also tests demuxers. And of course it may prove
useful for encoders, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Up until now, they were appended to the FATE_EXTERN-$(CONFIG_FFMPEG)
variable and were therefore activated when ffmpeg was enabled regardless
of whether ffprobe was enabled.

Also the same happened with FATE_SAMPLES_FASTSTART, although the
corresponding test (mov-faststart-4gb-overflow) only requires external
samples.

Furthermore, remove the unused FATE_FULL variable (FATE_EXTERN_FFPROBE has
taken its place).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Peter Ross <pross@xvid.org>
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>

It is a small buffer of a known, fixed size and so it should simply be
put into the muxer's context.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

An AVStream's codecpar is supposed to be filled by the caller before
avformat_write_header(); if the CodecParameters change, the caller
should signal this via packet side data, but not touch the AVStream's
codecpar.

The FLAC muxer checks for packet side data containing updated extradata,
yet if nothing has arrived by the time the trailer is written, the
already written extradata is overwritten by the very same extradata
again, unless the output is unseekable, in which case a warning that the
FLAC header can't be rewritten is emitted.

This commit changes this by only trying to rewrite the extradata if a
new streaminfo arrived via packet side data. Only then is a warning
emitted in case the output is unseekable.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

When no packet could be output, the interleavement functions
nevertheless initialized the packet destined for output (with the
exception of the data and size fields, making the initialization
pointless), although it will not be used at all. So remove the
initializations.
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

write_packet() currently saves the original timestamps of the packet it
got and restores them in case writing fails. This is unnecessary as we
are no longer working directly with the user-supplied AVPacket here; and
it is also pointless because the timestamps may already have been
altered before write_packet().

So remove this and add a general comment to the function that timestamps
may be modified; also remove a long outdated comment about side data.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The documentation of av_write_frame() explicitly states that the function
doesn't take ownership of the packets sent to it; while av_write_frame()
does not directly unreference the packets after having written them, it
nevertheless modifies the packet in various ways:
1. The timestamps might be modified either by prepare_input_packet() or
compute_muxer_pkt_fields().
2. If a bitstream filter gets applied, it takes ownership of the
reference and the side-data in the packet sent to it.
In case of do_packet_auto_bsf(), the end result is that the returned packet
contains the output of the last bsf in the chain. If an error happens,
a blank packet will be returned; a packet may also simply not lead to
any output (vp9_superframe).
This also implies that side data needs to be really copied and can't be
shared with the input packet.
The method choosen here minimizes copying of data: When the input isn't
refcounted and no bitstream filter is applied, the packet's data will
not be copied.

Notice that packets that contain uncoded frames are exempt from this
because these packets are not owned by and returned to the user. This
also moves unreferencing the packets containing uncoded frames to
av_write_frame() in the noninterleaved codepath; in the interleaved
codepath, these packets are already freed in av_interleaved_write_frame(),
so that unreferencing the packets in write_uncoded_frame_internal() is
no longer needed. It has been removed.
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Now that ff_interleave_add_packet() always returns blank packets, the
input packet to ff_interleave_packet_per_dts() will always be blank on
return as well (if supplied) and the same goes for interleave_packet()
in mux.c. Document these facts and remove the redundant resetting that
happened in av_interleaved_write_frame().

The last reference to the (long removed) destruct field that AVPackets
once had has been removed as well when updating the documentation of
ff_interleave_packet_per_dts().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

When an error happened in ff_interleave_add_packet() when adding
a packet to the packet queue, said packet would not be unreferenced
in ff_interleave_add_packet(), but would be zeroed in
av_interleaved_write_frame(), which results in a memleak.

This has been fixed: ff_interleave_add_packet() now always unreferences
the input packet on error; as a result, it always returns blank packets
which has been documented. Relying on this a call to av_packet_unref()
in ff_audio_rechunk_interleave() can be removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

If writing uncoded frames in noninterleaved mode fails at the preparatory
steps (i.e. before it reaches write_packet()), the packet would not be
unreferenced and the frame would leak. This is fixed by unreferencing
the packet in write_uncoded_frame_internal() instead.

This also makes it possible to remove the unreferencing in
write_packet() itself: In noninterleaved mode frames are now freed in
write_uncoded_frame_internal(), while they are freed in interleaved
mode when their containing packet gets unreferenced (like normal
packets).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Currently uncoded frames (i.e. packets whose data actually points to an
AVFrame) are not refcounted. As a consequence, calling av_packet_unref()
on them will not free them, but may simply make sure that they leak by
losing the pointer to the frame.

This commit changes this by actually making uncoded frames refcounted.
In order not to rely on sizeof(AVFrame) (which is not part of the public
API and so must not be used here in libavformat) the packet's data is
changed to a (padded) buffer containing just a pointer to an AVFrame.
Said buffer is owned by an AVBuffer with a custom free function that
frees the frame as well as the buffer. Thereby the pointer/the AVBuffer
owns the AVFrame.

Said ownership can actually be transferred by copying and resetting
the pointer, as might happen when actually writing the uncoded frames
in AVOutputFormat.write_uncoded_frame() (although currently no muxer
makes use of this possibility).

This makes packets containing uncoded frames compatible with
av_packet_unref(). This already has three advantages in interleaved mode:
1. If an error happens at the preparatory steps (before the packet is
put into the interleavement queue), the frame is properly freed.
2. If the trailer is never written, the frames still in the
interleavement queue will now be properly freed by
ff_packet_list_free().
3. The custom code for moving the packet to the packet list in
ff_interleave_add_packet() can be removed.

It will also simplify fixing further memleaks in future commits.
Suggested-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Based on the one in ffmpeg.c and it is not using an extra flush_idx variable.
Signed-off-by: Marton Balint <cus@passwd.hu>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

The data of an attachment file is put into an AVCodecParameter's
extradata. The corresponding size field has type int, yet there was no
check for the size to fit into an int. As a consequence, it was possible
to create extradata with negative size (by using a big enough max_alloc).

Other errors were also possible: If SIZE_MAX < INT64_MAX (e.g. on 32bit
systems) then the file size might be truncated before the allocation;
and avio_read() takes an int, too, so one would not have read as much
as one desired.

Furthermore, the extradata is now padded as is required.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Marton Balint <cus@passwd.hu>

Also change the default to that.
Signed-off-by: Marton Balint <cus@passwd.hu>

Also change the default to that.
Signed-off-by: Marton Balint <cus@passwd.hu>

Based on code from the BMP parser.

Addresses ticket #8574
Reviewed-by: James Zern <jzern@google.com>
Signed-off-by: James Almer <jamrial@gmail.com>

Fixes: left shift of negative value -14336
Fixes: 20298/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AC3_FIXED_fuzzer-5675484201615360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout
Fixes: out of array access
Fixes: 20274/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLAC_fuzzer-5649631988154368
Fixes: 19275/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLAC_fuzzer-5757535722405888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 2145417478 + 76702564 cannot be represented in type 'int'
Fixes: 20313/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RA_144_fuzzer-5734487724130304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Previously, there was no way to flush an encoder such that after
draining, the encoder could be used again. We generally suggested
that clients teardown and replace the encoder instance in these
situations. However, for at least some hardware encoders, the cost of
this tear down/replace cycle is very high, which can get in the way of
some use-cases - for example: segmented encoding with nvenc.

To help address that use case, we added support for calling
avcodec_flush_buffers() to nvenc and things worked in practice,
although it was not clearly documented as to whether this should work
or not. There was only one previous example of an encoder implementing
the flush callback (audiotoolboxenc) and it's unclear if that was
intentional or not. However, it was clear that calling
avocdec_flush_buffers() on any other encoder would leave the encoder in
an undefined state, and that's not great.

As part of cleaning this up, this change introduces a formal capability
flag for encoders that support flushing and ensures a flush call is a
no-op for any other encoder. This allows client code to check if it is
meaningful to call flush on an encoder before actually doing it.

I have not attempted to separate the steps taken inside
avcodec_flush_buffers() because it's not doing anything that's wrong
for an encoder. But I did add a sanity check to reject attempts to
flush a frame threaded encoder because I couldn't wrap my head around
whether that code path was actually safe or not. As this combination
doesn't exist today, we'll deal with it if it ever comes up.

Signed-off-by: James Almer <jamrial@gmail.com>

This copies the behaviour of the libopenjpeg decoder.
Fixes ticket #5919.

Fixes ticket #8612.