Make sure the buffer size does not exceed the expected
RLE size.

Prevent an out of array bound write.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Bug-Id: CVE-2013-0852
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Fixes part of Ticket2516
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Might fix overread, no testcase known though.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Add ability to handle multiple palettes and objects simultaneously.
Each simultaneous object is given its own AVSubtitleRect.
Note that there can be up to 64 currently valid objects, but only
2 at any one time can be "presented".
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Also switch from "tbl" to "tab" name suffixes.

Fixes: use of uninitialized memeory
Fixes: msan_uninit-mem_7fa421d0e222_1765_Girl_With_The_Dragon_Tattoo_2_23_56.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

pts should be that of the packet containing the presentation segment.
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Useful for detection of subtitles displayed during foreign language
scenes.
Signed-off-by: Diego Biurrun <diego@biurrun.de>

According to the sample for trac ticket #1722, PGS subtitles
are decoded from several packets at the same DTS and varying PTS.
The PTS from the presentation segment seem to be the valid one;
in particular, clear subtitles are too early with the other PTS.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Also break some long lines, remove codec function placeholder comments
and add spaces in sample/pixel format lists.
Signed-off-by: Martin Storsjö <martin@martin.st>

The previous implementation assumed that a new picture would always
supersede the previous picture. Similarly, presentation segments
were assumed to pertain to the most-recently-read picture.

However, each presentation segment may refer to 0 or more pictures
by their ID. Picture IDs may repeat, and a repeated picture ID
indicates that the old picture for that ID is no longer needed
and may be discarded.

The new implementation allocates a buffer with one slot for each
possible picture ID (the picture ID is a 16-bit field) and
properly decodes presentation segments so that all relevant
pictures are output upon encountering a display segment.

Given that most PGS streams are unlikely to use more than a small
fraction of the available picture IDs, it would probably be better
to use a more memory-efficient data structure. I'm lazy though, so
I leave this to a more motivated individual.

I've tested the code with MKV files in VLC (a recent revision from
their git repo) and with HandBrake (a version that I hacked up to
use ffmpeg's PGS subtitle decoder).
Review-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

It's more readable and less prone to breakage.

On Blu-ray colors are stored in the order YCrCb (and not YCbCr) as mentioned in the specifications:
see System Description Blu-ray Disc Read-Only Format, 9.14.4.2.2.1 Palette Definition Segment

When decoding a Blu-ray subtitle, the colors were incorrectly set.
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Call this new function before decode() to replace the custom and
inconsistant initialization in various decoders.
This function is equivalent to avcodec_get_frame_defaults() for AVFrame.
Signed-off-by: Aurelien Jacobs <aurel@gnuage.org>

On Blu-ray colors are stored in the order YCrCb (and not YCbCr) as mentioned in the specifications:
see System Description Blu-ray Disc Read-Only Format, 9.14.4.2.2.1 Palette Definition Segment

When decoding a Blu-ray subtitle, the colors were incorrectly set.

Signed-off-by: Mans Rullgard <mans@mansr.com>

Done to keep ABI compatible. Otherwise this is just silly

It is pretty hopeless that other considerable projects will adopt
libavutil alone in other projects. Projects that need small footprint
are better off with more specialized libraries such as gnulib or rather
just copy the necessary parts that they need. With this in mind, nobody
is helped by having libavutil and libavcore split. In order to ease
maintenance inside and around FFmpeg and to reduce confusion where to
put common code, avcore's functionality is merged (back) to avutil.
Signed-off-by: Reinhard Tartler <siretart@tauware.de>