Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This makes vp5 and vp6 independent of dsputil.

This way, the special IDCT permutations are no longer needed. Bfin code
is disabled until someone updates it. This is similar to how H264 does
it, and removes the dsputil dependency imposed by the scantable code.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The non-alpha and alpha-Y planes are cleared in the idct_put/add()
calls. For the alpha U/V planes, we only care about the DC for entropy
context prediction purposes, the rest of the data is unused.
Signed-off-by: Martin Storsjö <martin@martin.st>

The non-alpha and alpha-Y planes are cleared in the idct_put/add()
calls. For the alpha U/V planes, we only care about the DC for entropy
context prediction purposes, the rest of the data is unused.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The function is only used in VP3 and VP5, so no need to have it in
DSPContext.

The function is never used.

Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Fixes CVE-2012-2783

CC: libav-stable@libav.org

Returning 0 for failure is misleading.

CC: libav-stable@libav.org

It's got_frame, not data size

It will be useful in the upcoming transition to refcounted AVFrames.

Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

All whitespace plus a curly brace.
Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The YUV channels of VP6 are encoded in a highly linear fashion which does
not have any slice-like concept to thread.  The alpha channel of VP6A is
fairly independent of the YUV and comprises 40% of the work.  This patch
uses the THREAD_SLICE capability to split the YUV and A decodes into
separate threads.

Two bugs are fixed by splitting YUV and alpha state:
- qscale_table from VP6A decode was for alpha channel instead of YUV
- alpha channel filtering settings were overwritten by YUV header parse
Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Rather than cleverly managing frame pointers with swaps to avoid
re-using "golden" frames, just do brute-force management of the
4 AVFrames.  New strategy is probably no more costly and is easier
to adapt to threaded usage.
Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Makes golden_frame more like other frame data, paves way for threading
alpha channel decode.
Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Pave the way for per-thread context initialization.
Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This moves all VP3-specific function pointers from dsputil to a
new vp3dsp context.  There is no reason to ever use the VP3 IDCT
where an MPEG2 IDCT is expected or vice versa.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

I am not sure these new values are correct, not am I sure
the semantics are a good idea since we do not seem to make any
use of them but they caused a lot of confusion, but this
seems to make things closer to matching the documentation.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Signed-off-by: Janne Grunau <janne-libav@jannau.net>

Signed-off-by: Janne Grunau <janne-libav@jannau.net>

Signed-off-by: Janne Grunau <janne-libav@jannau.net>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This is required specifically for setting frame->format to -1,
otherwise it will be set to 0 = PIX_FMT_YUV420P and code reading
the format from the output decoded frame will get misled.

In particular fix regressions occurring with the pending vsrc_buffer
patch.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

Signed-off-by: Mans Rullgard <mans@mansr.com>