When byte_run is 0, pixel_countdown is not touched and the loop will run
forever.

CC:libav-stable@libav.org

CC:libav-stable@libav.org

range == 0 happens with corrupted files

CC:libav-stable@libav.org

CC:libav-stable@libav.org

CC:libav-stable@libav.org

This fixes crashes in chromium on win64 on machines with AVX
(crashes that apparently aren't triggered by fate).
Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

This gets rid of a number of warnings about casts discarding
qualifiers from the pointer target, present since 7ebfb466.
Signed-off-by: Martin Storsjö <martin@martin.st>

Instead, only extend edges on-demand when the motion vector actually
crosses the visible decoded area using ff_emulated_edge_mc(). This
changes decoding time for cathedral from 8.722sec to 8.706sec, i.e.
0.2% faster overall. More generally (VP8 uses this also), low-motion
content gets significant speed improvements, whereas high-motion content
tends to decode in approximately the same time.
Signed-off-by: Martin Storsjö <martin@martin.st>

Instead, keep them in the bitstream buffer until we read them verbatim,
this saves a memcpy() and a subsequent clearing of the target buffer.
decode_cabac+decode_mb for a sample file (CAPM3_Sony_D.jsv) goes from
6121.4 to 6095.5 cycles, i.e. 26 cycles faster.
Signed-off-by: Martin Storsjö <martin@martin.st>

This allows more transparent mixing of get_bits and whole-byte access
without having to touch get_bits internals.
Signed-off-by: Martin Storsjö <martin@martin.st>

These functions are mostly H264-specific (the only other user I can
spot is bink), and this allows us to special-case some functionality
for H264. Also remove the 16-bit-coeff with >8bpp versions (unused)
and merge the duplicate 32-bit-coeff for >8bpp (identical).
Signed-off-by: Martin Storsjö <martin@martin.st>

These functions do the same thing in 16 bit space and don't need
any depth specific clipping.
Signed-off-by: Martin Storsjö <martin@martin.st>

The "CentaurHauls family 6 model 9 stepping 8" family of CPUs
(flags: fpu vme de pse tsc msr cx8 sep mtrr pge mov pat mmx fxsr sse
up rng rng_en ace ace_en) SIGILLs on long nop codes.
Signed-off-by: Martin Storsjö <martin@martin.st>

The non-alpha and alpha-Y planes are cleared in the idct_put/add()
calls. For the alpha U/V planes, we only care about the DC for entropy
context prediction purposes, the rest of the data is unused.
Signed-off-by: Martin Storsjö <martin@martin.st>

This was caused by unconditionally referencing a conditionally compiled
table. Now the code is also compiled conditionally.
Signed-off-by: Diego Biurrun <diego@biurrun.de>

This avoids SIMD-optimized functions having to sign-extend their
line size argument manually to be able to do pointer arithmetic.
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

CC: libav-stable@libav.org

a7d2861d removed necessary braces.

Encoders requiring it have the dependency expressed in the
configure.

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

The library might provide an encoder in the future, so it's better to
check for the presence of the decoder rather than just the library.

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Convert to diffptr_t the line_size parameters still int.

Remove all the warnings in dsputil.c

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae61045)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a52eb9f7fcd8d9167fb4a5ba015b612)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767b922661132a3c1f4b5ba2c7338b660)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.

The nontrivial parts are:
1) extracting a simplified version of the frame management code from
   mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
   its own more complex system already and those were set only to appease
   the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
   for dxva, the draw_horiz_band() call is moved from
   ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
   because it's now different for h264 and MpegEncContext-based
   decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
   added some very simplistic frame management instead and dropped the
   use of ff_h264_frame_start(). Because of this I also had to move some
   initialization code to svq3.

Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Split out dependency on MpegEncContext.

The H264 context is always uninitialized at this point.

!encoding is always true, we do not have a H.264 encoder

partitioned_frame is never set for h264 (as easily seen from git grep).

There may be more decoders that rely on this. Those should be found and
fixed.