Fixes a deadlock with frame threads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

CC: libav-stable@libav.org

Fixes a crash with zuffed files.

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Some invocations include a verb in the log message, others do not.  Yet
av_log_missing_feature expects callers to provide a verb.  Change the
function to include a verb instead and update the callers accordingly.
The result is a more natural function API and correct English in the
function invocations.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This fixes one of the potential races spotted by Helgrind.

When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org
Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

It is possible in various error paths as well as gap handling
that this has already been allocated. It is not clear why that
would be a problem with the current code, thus disable the
assert to avoid a common assert failure when asserts are enabled.
Signed-off-by: Martin Storsjö <martin@martin.st>

This fixes crashes introduced by 2e8f3cbc, the codec can be null
when called from parsers.
Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: Joakim Plate <elupus@ecce.se>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes seeking to the first keyframe
Fixes Ticket1029
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The h264_vdpau decoder crashed if output colorspace was not 8-bit 420.
Add a check to error out instead (current hardware does not support
other colorspaces, so successful decoding is not possible).

Check implemented at a different place by michael, thus blame for bugs goes to michael
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The h264_vdpau decoder crashed if output colorspace was not 8-bit 420.
Add a check to error out instead (current hardware does not support
other colorspaces, so successful decoding is not possible).
Signed-off-by: Martin Storsjö <martin@martin.st>

It is possible in various error pathes as well as gap handling
that this has already been allocated. Its not clear why that
would be a problem with the current code, thus disable the
assert to avoid common assert failure when asserts are enabled.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Write out the NAL decoding loops in full so that they are easier
to parse for a preprocessor without it having to be aware of macros
or other such things in C code.

This also makes the code more readable.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

This works with I, P and B frames
CAVLC and CABAC
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Instead of inlining everything into ff_h264_hl_decode_mb(), use
explicit templating to create versions of the called functions
with constant parameters filled in.  This greatly speeds up
compilation of h264.c and reduces the code size without any
measurable impact on performance.

Compilation time for h264.c on an i7 goes from 30s to 5.5s.
Code size is reduced by 430kB.
Signed-off-by: Mans Rullgard <mans@mansr.com>

This function exports the exact sps.num_reorder_frames value
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Below fixes the maximum reference count check for second reference list in
non-B frames.  There is nothing to prohibit full (field sized) reference
list in this case as far as I can tell, and this fixes several syntax-test
files here (this is a regression caused when this check was made more
stringent by
http://git.videolan.org/?p=ffmpeg.git;a=commit;h=dc9ce40069bde3d28f8d0b3e5bd733ae255fecb5)

Probably a silly corner case seldom seen irl, but thought I'd pass along
in case there was interest in correcting the check.

---------------

h264: Fix maximum reference count check for non-b frames; full range is
technically ok
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This is based on:
	commit 30f51509
	Author: Mans Rullgard <mans@mansr.com>
    		h264: allow cropping to AVCodecContext.width/height

The above commit is not taken as is as its buggy (incorrect handling of
the interlaced case) and didnt apply cleanly.

Fixes ticket156
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes part of Ticket1369

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This fixes out of global array reads.
The alternative solutions of checking the index or modifying the VLC tables
to prevent the index going outside are each about 1-2 cpu cyclces slower
per coded 4x4 block.
The alternative of padding the global tables directly is more ugly and
moving them to the context should benefit cache locality.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

This fixes a null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

During failure conditions ff_h264_decode_init() leaks memory
allocated for nal units.  Found via valgrind.

Valgrind traces: http://pastebin.com/GqTqxs8TSigned-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>