Fixes ticket #6362.

Fixes several warnings of the type
libavformat/nut.c:207:42: warning: ISO C forbids conversion of function pointer to object pointer type [-Wpedantic]

The code has little usefulness and uses the __PRETTY_FUNCTION__ GNU extension.

It is supposed to be a flag. The only currently defined value is
AVIO_SEEKABLE_NORMAL, but other ones may be added in the future.
However all the current lavf code treats this field as a bool (mainly
for historical reasons).
Change all those cases to properly check for AVIO_SEEKABLE_NORMAL.

Needed for av_log() inside that function.
Signed-off-by: Paul B Mahol <onemda@gmail.com>

Currently, AVStream contains an embedded AVCodecContext instance, which
is used by demuxers to export stream parameters to the caller and by
muxers to receive stream parameters from the caller. It is also used
internally as the codec context that is passed to parsers.

In addition, it is also widely used by the callers as the decoding (when
demuxer) or encoding (when muxing) context, though this has been
officially discouraged since Libav 11.

There are multiple important problems with this approach:
    - the fields in AVCodecContext are in general one of
        * stream parameters
        * codec options
        * codec state
      However, it's not clear which ones are which. It is consequently
      unclear which fields are a demuxer allowed to set or a muxer allowed to
      read. This leads to erratic behaviour depending on whether decoding or
      encoding is being performed or not (and whether it uses the AVStream
      embedded codec context).
    - various synchronization issues arising from the fact that the same
      context is used by several different APIs (muxers/demuxers,
      parsers, bitstream filters and encoders/decoders) simultaneously, with
      there being no clear rules for who can modify what and the different
      processes being typically delayed with respect to each other.
    - avformat_find_stream_info() making it necessary to support opening
      and closing a single codec context multiple times, thus
      complicating the semantics of freeing various allocated objects in the
      codec context.

Those problems are resolved by replacing the AVStream embedded codec
context with a newly added AVCodecParameters instance, which stores only
the stream parameters exported by the demuxers or read by the muxers.

Fixes runtime error: null pointer passed as argument 2, which is
declared to never be null
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.

Also fix potential overflow of avio_tell(bc) + value_len.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

And make one more informative.

libc's qsort comparator has a const qualifier on both arguments. This
adds a missing const qualifier to exactly match the comparator API.

Existing usages of av_tree_find, av_tree_insert are appropriately
modified: type signature changes of the comparators, and removal of
unnecessary void * casts of function pointers.
Reviewed-by: Henrik Gramner <henrik@gramner.com>
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes assertion failure

Found-by: durandal_1707
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Otherwise sm_size can be larger than size, which results in a negative
packet size.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

If it fails, the buffers can be (partially) uninitialized.

This fixes 'Conditional jump or move depends on uninitialised value(s)'
valgrind warnings.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Prevents read of uninitialized variable

Based on patch by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

These loops can take a lot of time if count is very large.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This can unnecessarily waste a lot of time.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

nut->last_syncpoint_pos doesn't necessarily change between resync
attempts, so find_any_startcode can return the same startcode again.

Thus remember where the last resync happened and don't try to resync
before that.

This can't be done locally in nut_read_packet, because this wouldn't
prevent infinite resync loops, where after the resync a packet is
returned and while reading a following packet the resync happens again.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Added in 36170266. Modified version that
doesn't use this label merged in 55231323,
thus obsoleting this label.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes CID1041175
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Based on Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com> work.

CC: libav-stable@libav.org

This fixes a segmentation fault when accessing the metadata.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

CC: libav-stable@libav.org

The existing check has two problems:
 1) i + count can overflow, so that the check '< 256' returns true.
 2) In the (i == 'N') case occurs a j-- so that the loop runs once more.

This can trigger the assertion 'nut->header_len[0] == 0' or cause
segmentation faults or infinite hangs.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This fixes a segmentation fault when accessing the metadata.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

A negative frame rate triggers an av_assert2 in av_rescale_rnd.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Bug-Id: CID 205122 / CID 205123

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

This is not an API change; the fields were explicitly declared private
before.
Signed-off-by: Anton Khirnov <anton@khirnov.net>

It is not supposed to be set by demuxers.

CC: libav-stable@libav.org
Bug-Id: CID 733713

Chapters do not have an event_flags field.

Bug-Id: CID 1231990