Signed-off-by: Paul B Mahol <onemda@gmail.com>

Tested functions are internally kept in a binary search tree for efficient
lookups. The downside of the current implementation is that the tree quickly
becomes unbalanced which causes an unneccessary amount of comparisons between
nodes. Improve this by changing the tree into a self-balancing left-leaning
red-black tree with a worst case lookup/insertion time complexity of O(log n).

Significantly reduces the recursion depth and makes the tests run around 10%
faster overall. The relative performance improvement compared to the existing
non-balanced tree will also most likely increase as more tests are added.

Fixes: invalid_read.nut
Found-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Forwards the error code, avoids potential integer overflow
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

* commit 'b2417ee6':
  dxv: Improve error message
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit '1bcd4a47':
  dxv: Support RAW intermediate compression
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'bbf71d46':
  dxv: Print texture information after header parsing
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'fb288969':
  dxv: Support the original first version
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit '588a5619':
  dxv: Parse ancillary encoder information
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'a9b8c638':
  mmal: Fix AVBufferRef usage
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit '49623f53':
  mmal: Remove setting extradata on input format
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'f290e48d':
  mmal: drop the h264 BSF
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

Fix out of bounds access caused by wrongful usage
of swb_offset constants when computing scalefactor
positions.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>

Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>

Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>

It is supposed to only old BT.709 colorspaces.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

CID 1260 (as evidenced by incorrect decoding of a sample from ticket
4876) seems to use incorrect weight tables. It appears those tables
were not zigzag-scanned.

Apply zigzag on weight tables for new CIDs 1258, 1259, and 1260, and
fix an incorrect chroma table for CID 1256.

Fixes last issue from ticket #4876.
Found-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This bit is 1 in some samples, and seems to coincide with interlaced
mbs and CID1260. 2008 specs do not know about it, and maintain qscale
is 11 bits. This looks oversized, but may help larger bitdepths.

Currently, it leads to an obviously incorrect qscale value, meaning
its syntax is shifted by 1. However, reading 11 bits also leads to
obviously incorrect decoding: qscale seems to be 10 bits.

However, as most profiles still have 11bits qscale, the feature is
restricted to the CID1260 profile.

The encoder writes 12 bits of syntax, last and first bits always 0,
which is now somewhat inconsistent with the decoder, but ends up with
the same effect (progressive + reserved bit).

Partially fixes ticket #4876.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Context cookies must be updated when a playlist response return Setcookie header.
Signed-off-by: wm4 <nfxjfg@googlemail.com>

Fixes: 03_cave_girls_h264_slc24_slicecrc1_fuzzed.avi
Found-by: Dinah Handel <dinahrhandel@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes hang.nut
Found-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This patch tweaks search_for_pns to be both more
aggressive and more careful when applying PNS. On
the one side, it will again try to use PNS on zero
(or effectively zero) bands. For this, both zeroes
and band_type have to be checked (some ZERO bands
aren't marked in zeroes). On the other side, a more
accurate rate-distortion measure avoids using PNS
where it would cause audible distortion.

Also fixed a small bug in the computation of freq
that caused PNS usage on low-frequency bands during
8-short windows. This allows re-enabling PNS during
8-short.

Fix segfault with too large slice_count
Fixes Ticket4879
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The change was wrong, also add a comment explaining it

Found-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Kyle Swanson <k@ylo.ph>

Sample-Id: boombox64.mov
Reported-by: Aarni Koskela
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

DXV 1.0 provided DXT1 with LZF only.

Sample-Id: Orange-Strings-07_640_DXV.mov
Reported-by: Carl Eugen Hoyos
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Header does not contain the number of channels, but rather the encoder
version, so rename the variable, and read the additional byte.

AVBufferRef.data and AVPacket.data don't need to have the same value.
AVPacket could point anywhere into the buffer. Likewise, the sizes
don't need to be the same.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

This works only for extradata sizes up to 128 bytes. Additionally, I
could never actually see it doing anything. The new code using
MMAL_BUFFER_HEADER_FLAG_CONFIG now takes care of this.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

We can send mp4-style data directly. But for some reason, this requires
sending the extradata as buffer with MMAL_BUFFER_HEADER_FLAG_CONFIG
set. Reuse the infrastructure for sending AVPackets to do this.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Signed-off-by: Paul B Mahol <onemda@gmail.com>