As defined in sections F.14.2.8 and F.14.3.8
Reviewed-by: Mark Thompson <sw@jkqxz.net>
Signed-off-by: James Almer <jamrial@gmail.com>

cbs_h2645_read_more_rbsp_data does not handle malformed input very well:
1. If there were <= 8 bits left in the bitreader, these bits were read
via show_bits. But show_bits requires the number of bits to be read to
be > 0 (internally it shifts by 32 - number of bits to be read which is
undefined behaviour if said number is zero; there is also an assert for
this, but it is only an av_assert2). Furthermore, in this case a shift
by -1 was performed which is of course undefined behaviour, too.
2. If there were > 0 and <= 8 bits left and all of them were zero
(this can only happen for defective input), it was reported that there
was further RBSP data.

This can lead to an infinite loop in H.265's cbs_h265_read_extension_data
corresponding to the [vsp]ps_extension_data_flag syntax elements. If the
relevant flag indicates the (potential) occurence of these syntax elements,
while all bits after this flag are zero, cbs_h2645_read_more_rbsp_data
always returns 1 on x86. Given that a checked bitstream reader is used,
we are also not "saved" by an overflow in the bitstream reader's index.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: memleaks on error paths during init
Fixes: 15548/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLASHSV2_fuzzer-6324019382452224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: memleaks on error paths during init
Fixes: 15533/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLASHSV_fuzzer-5647977168764928

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 9151595917793558550 + 297519050751678697 cannot be represented in type 'long'
Fixes: 15496/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5722866475073536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 15328/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5637545171353600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array read
Fixes: 15409/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5758846959616000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 14 + 2147483647 cannot be represented in type 'int'
Fixes: 14794/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5677380695228416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/atrac9dec: Check conditions before apply_band_extension() to avoid out of array read in initialization of unused variables

Fixes: global-buffer-overflow
Fixes: 15247/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5671602181636096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Such a failure should not be fatal and its worth testing this path too
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Replace STnxm_UB and LDnxm_SH with new macros ST_{H/W/D}{1/2/4/8}.
The old macros are difficult to use because they don't follow the same parameter passing rules.
Changing details as following:
1. remove LD4x4_SH.
2. replace ST2x4_UB with ST_H4.
3. replace ST4x2_UB with ST_W2.
4. replace ST4x4_UB with ST_W4.
5. replace ST4x8_UB with ST_W8.
6. replace ST6x4_UB with ST_W2 and ST_H2.
7. replace ST8x1_UB with ST_D1.
8. replace ST8x2_UB with ST_D2.
9. replace ST8x4_UB with ST_D4.
10. replace ST8x8_UB with ST_D8.
11. replace ST12x4_UB with ST_D4 and ST_W4.

Examples of new macro: ST_H4(in, idx0, idx1, idx2, idx3, pdst, stride)
ST_H4 store four half-word elements in vector 'in' to pdst with stride.
About the macro name:
1) 'ST' means store operation.
2) 'H/W/D' means type of vector element is 'half-word/word/double-word'.
3) Number '1/2/4/8' means how many elements will be stored.
About the macro parameter:
1) 'in0, in1...' 128-bits vector.
2) 'idx0, idx1...' elements index.
3) 'pdst' destination pointer to store to
4) 'stride' stride of each store operation.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout
Fixes: 15567/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5758451487080448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The typedef used to define EbmlSyntax already includes a const qualifier
so that it is unnecessary to include another const qualifier in future
definitions and declarations. Given that MSVC warns about this, this
commit removes these redundant const qualifiers.
Suggested-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Unknown-length elements end when an element not allowed in them, but
allowed at a higher level is encountered. In order to check for this,
c1abd95a added a pointer to every syntax level's parent to each
EbmlSyntax. Given that the parent must of course also reference the
child in order to be able to enter said child level, one needs to use
forward declarations.
These forward declarations constitute tentative definitions and tentative
definitions with internal linkage (like our syntaxes) must not be an
incomplete type. Yet they were an incomplete type and while GCC and
Clang did not even warn about this (on default warning levels), it
broke compilation with MSVC. Therefore this commit adds the sizes.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: out of array access
Fixes: 15522/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DNXHD_fuzzer-5747756078989312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

This allows testing parsers with a wider range of input packet sizes.
Which is important and usefull for regression testing, some of our
parsers in fact to not work if the packet size is changed from 1024
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array access
Fixes: 15522/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DNXHD_fuzzer-5747756078989312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: memleak
Fixes: 15446/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5662875831500800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: memleak
Fixes: 15446/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5662875831500800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Assertion failure
Fixes: 15422/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO5_fuzzer-5676625481433088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

If a file uses unknown-length level 1 elements besides clusters and such
elements are after the first cluster, then these elements will usually
be parsed twice: Once during parsing of the file header and once when
reading the file reaches the position where these elements are located.
The second time the element is parsed leads to a "Duplicate element"
error message. Known-length elements are not affected by this as they
are skipped except during parsing the header.

This commit fixes this by explicitly adding a check for whether the
position of the element to be parsed is the same as the position of the
already known level 1 element.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit converts the MatroskaLevel1Element struct to use file-based
offsets, as opposed to the current practice of using offsets relative to
the beginning of the segment in it. This also includes a change from
uint64_t to int64_t.

This is in preparation to another patch that improves the check for
duplicate level 1 elements.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Also use the smallest scope possible for a loop variable.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Up until now, one last kind of unknown-length element hasn't been
properly handled: Unknown-length elements that are supposed to be
skipped, i.e. the level 1 elements that might reside after the
clusters.

This commit changes this. To do this, ebml_parse got a mode that
essentially tries to skip everything except when parsing is needed
(namely for unknown-length elements for which parsing is necessary
as they can't be skipped). This mode is selected by using a NULL
as destination where the parsed data should be written to.
It is used to parse the level 1 elements in matroska_parse_cluster.

The syntax list used for parsing must of course include links to
the syntax of all the master elements that might need to be parsed.
In other words: Instead of matroska_clusters (which contained every
level 1 element except clusters as EBML_NONE elements designated to
be skipped) matroska_segment is needed and used; matroska_clusters has
been removed.

Furthermore, matroska_segment has been reordered so that clusters are at
the front as this is now the most common case for this list.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

matroska_probe did not support the case of an unknown-length EBML header
at all; given that libavformat's Matroska muxer used to produce such
files in the streaming case, support for them has been added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The current Matroska specifications mandate that only two elements may
use an unknown-length length: Segments and clusters. But this was not
always so: For the greater part of Matroska's existence, all master
elements were allowed to make use of the unknown-length feature.

And there were muxers creating such files: For several years
libavformat's Matroska muxer used unknown-length for all master
elements when the output wasn't seekable. This only stopped in March
2010 with 2529bb30. And even afterwards it was possible (albeit
unlikely) for libavformat to create unknown-length master elements
that are in violation of today's specifications, namely if the master
element was so big that the seek backwards to update the size could
no longer be performed inside the AVIOContext's write buffer. This
has only been fixed in October 2016 (with the patches that introduced
support for writing CRC-32 elements).

Libavformat's Matroska demuxer meanwhile has never really supported
unknown-length elements besides segments and clusters. Support for the
latter was hardcoded. This commit changes this: Now all master elements
for which a syntax to parse them is available are supported. This
includes the files produced by old versions of libavformat's muxer.

More precisely, master elements that have unknown length and are about
to be parsed (not skipped) are supported; only a warning is emitted for
them. For normal files, this means that level 1 elements after the
clusters that are encountered after the clusters have been parsed (i.e.
not because they are referenced by the seekhead at the beginning of the
file) are still unsupported (they would be skipped at this point if
their length were known).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

1. Up until now, the error message for EBML numbers whose length exceeds
the limits imposed upon them because of the element's type did not
distinguish between known-length and unknown-length elements. As a
consequence, the numerical value of the define constant
EBML_UNKNOWN_LENGTH was emitted as part of the error message which is
of course not appropriate. This commit changes this by adding error
messages designed for unknown-length elements.

2. We impose some (arbitrary) sanity checks on the lengths of certain
element types; these checks were conducted before the checks depending
on whether the element exceeds its containing master element. Now the
order has been reversed, because a failure at the (formerly) latter
check implies that the file is truly erroneous and not only fails our
arbitrary length limit. Moreover, this increases the informativeness of
the error messages.

3. Furthermore, the error message in general has been changed by replacing
the type of the element (something internal to this demuxer and
therefore suitable as debug output at best, not as an error message
intended for ordinary users) with the element ID. The element's position
has been added, too.

4. Finally, the length limit for EBML_NONE elements has been changed so
that all unknown-length elements of EBML_NONE-type trigger an error.
This is done because unknown-length elements can't be skipped and need
to be parsed, but there is no syntax to parse available for EBML_NONE
elements. This is done in preparation for a further patch which allows
more unknown-length elements than just clusters and segments.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The Matroska (and WebM) file format achieves forward-compability by
insisting that demuxers ignore and skip elements they don't know about.
Unfortunately, this complicates the detection of errors as errors
resulting from loosing sync can't be reliably distinguished from
unknown elements that are part of a future version of the standard.

Up until now, the strategy to deal with this situation was to skip all
unknown elements that are not obviously erroneous; if an error happened,
it was tried to seek to the last known good position to resync from (and
resync to level 1 elements). This is working fine if the input is
seekable, but if it is not, then the skipped data can usually not be
rechecked lateron. This is particularly acute if unknown-length clusters
are in use, as the check for whether a child element exceeds the
containing master element is ineffective in this situation.

To remedy this, a new heuristic has been introduced: If an unknown
element is encountered in non-seekable mode, an error is presumed to
have happened based upon a combination of the length of the row of the
already encountered unknown elements and of how far away skipping this
element would take us.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Cosmetics include reordering EbmlType so that EBML_SINT is adjacent to
the other numbers (and matches the order in the switch in ebml_parse)
and also reordering the switch for assignment of default values so that
it matches the order in EbmlType.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Up until now, avio_tell was used multiple times in ebml_parse and its
subroutines, although the result of these calls can usually be simply
derived from the result of earlier calls to avio_tell. This has been
changed. Unnecessary calls to avio_tell in ebml_parse are avoided now.

Furthermore, there has been a slight change in the output of some error
messages relating to elements exceeding their containing master element:
The reported position of the element now points to the first byte of the
element ID and no longer to the first byte of the element's payload.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit closes the last hole in the system of checks for a
known-length file ending too early: Now an error message is emitted
in case the file ends directly after an EBML element.

Furthermore, this commit adds a check and a corresponding warning
whether there is data beyond the Matroska segment (only reasonable for
known-length segments). If everything looks alright, then parsing is
stopped as soon as EOF is reached (in contrast, the earlier code would
always call matroska_resync at the end).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

By including SimpleBlocks and BlockGroups twice in the same EbmlSyntax
array (with different semantics), one can reduce the duplication of the
other values.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The new code does not rely on whether the cluster's position is set or
not to infer whether a cluster needs to be closed or not (instead, this
is done in ebml_parse), so there is no need to reset the cluster's
position at all any more. It will be automatically set to the correct
value when a cluster is entered.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Before this commit, the parsing of clusters mixed EBML levels by
allowing elements from different levels in a EbmlSyntax (namely
matroska_cluster_parsing). This has been changed. And the level
is now explicitly used to determine how to parse.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This commit changes how levels are handled: If the level used for
ebml_parse ends directly after an element that has been consumed, then
ebml_parse ends the level itself (and any known-length levels that end
there as well) and informs the caller via the return value; if the
current level is of unknown-length, then the level is ended as soon as
an element that is not valid on the current level, but on a higher
level is encountered (or if EOF has been encountered).

This is designed for situations where one wants to parse master elements
incrementally, i.e. not in one go via ebml_parse_nest.

The (incremental) parsing of clusters still mixes levels by using a
syntax list that contains elements from different levels and the level
is still ended manually via a call to ebml_level_end if the last cluster
was an unknown-length cluster (known-length clusters are already ended
when their last element is read), but only if the next element is a
cluster, too. A  different level 1 element following an unknown-length
cluster will currently simply be presumed to be part of the earlier
cluster. Fixing this will be done in a future patch. The modifications
to matroska_parse_cluster contained in this patch are only intended not
to cause regressions.

Nevertheless, the fact that known-length levels are automatically ended
in ebml_parse when their last element has been read already fixes a bogus
error message introduced in 9326117b that was emitted when a known-length
cluster is followed by another level 1 element other than a cluster in
which case the cluster's level was not ended (which only happened when
a new cluster has been encountered) so that the length check (introduced
in 9326117b) failed for the level 1 element as it is of course not
contained in the previous cluster. Most Matroska files were affected by
this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

By linking to the syntax of the parent (i.e. the containing master
element) one can check whether an element is actually part of a higher
level in the EBML hierarchy. Knowing this is important for
unknown-length levels, because they end when an element that doesn't
belong to this, but to a higher hierarchy level is encountered.

Sometimes there are different syntaxes dealing with the same elements.
In this case it is important to use a parent that contains all the
elements at the parent level; whether this is the syntax actually used
to enter the child's level is irrelevant. This affects the list of level
1 elements (which has been used as parent for matroska_cluster, too) and
it affects recursive elements (currently only the SimpleTag), where the
non-recursive parent has to be choosen.

This is in preparation for a patch that redoes level handling.

Finally, the segment id has been added to ebml_syntax. This will enable
handling of unknown-length EBML headers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>