Found with a clang-scan report on http://fate.libav.org/csa/

Based on a patch by Piotr Bandurski.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

If there was a failure inflating, or reinitializing
the zstream, the current frame's buffer would be lost.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

Be consistent with error messages and code formatting.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

Adds some validation for changing parameters after setting the matrix and
fixes a bug in the conversion path setup.

This makes big and little endian systems use the same output
format.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Based on patch by Oana Stratulat <oanaandreeastratulat@gmail.com>.

In the GNU assembler, a relational expression, bizarrely, has the
value -1 if true, whereas in Apple's it is +1.  This patch makes
sure the correct expression is used in both cases.
Signed-off-by: Mans Rullgard <mans@mansr.com>

The clang integrated assembler does not support pre-UAL syntax,
while gcc requires pre-UAL syntax for ARM code.  A patch[1] for
clang to support the old syntax as well has been ignored since
January.

This patch chooses the syntax appropriate for each compiler,
allowing both to build the code.  Notably, this change allows
building for iphone with the latest Apple Xcode update.

[1] http://llvm.org/bugs/show_bug.cgi?id=11855Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

Refactoring mmx2/mmxext YASM code with cpuflags will force renames.
So switching to a consistent naming scheme beforehand is sensible.
The name "mmxext" is more official and widespread and also the name
of the CPU flag, as reported e.g. by the Linux kernel.

This reverts commit 36936080. It was
already applied; no idea why it didn't error out while re-applying it.

This fixes Bugzilla #327:
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

For left HFYU prediction, we predict from the buffer buf+1 using 8- or
16-byte reads. This means that aligning the buffer by 16 bytes is in
itself not sufficient, because if the width itself is 16- or 8-byte
aligned, the buffer will not be padded, and thus a read of size 16 at
buf+1 will overflow boundaries at the right edge. Padding the buffer by
1 byte is sufficient to not overflow its boundaries.

Fixes bug 342.

This makes add_hfyu_left_prediction_sse4() handle sources that are not
16-byte aligned in its own function rather than by proxying the call to
add_hfyu_left_prediction_ssse3(). This fixes a crash on Win64, since the
sse4 version clobberes xmm6, but the ssse3 version (which uses MMX regs)
does not restore it, thus leading to XMM clobbering and RSP being off.

Fixes bug 342.

The scaling process for obtaining direct MVs from co-located field MVs
is the same for interlaced field and progressive pictures.
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>

In VC-1 interlaced field pictures, chroma motion vectors can extend beyond
picture boundary even if luma vectors are bounded. The problem shows up
only for hpel interpolated MVs, and may be due to the way motion vectors
are scaled / cropped.

Thanks to Konstantin Shishkov for suggesting the fix. This fixes
long-known segfaults in MC-VC1.ts from videolan streams archive.
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>

Currently there is a wild mix of 3dn2/3dnow2/3dnowext.  Switching to
"3dnowext", which is a more common name of the CPU flag, as reported
e.g. by the Linux kernel, unifies this.

Fixed codebook mode in 5300 rate may write up to SUBFRAME_LEN + 4 and
that is considered normal by the reference decoder. Without that additional
padding it might overwrite first elements of LPC history.

The same buffer with saved data is used later in LPC reconstruction, so
it should have some head space for LPC history.

This fixes Bugzilla #327:
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

Some calculations were changed in b6a3849a to use mmsize, which was not correct
for the AVX version, which uses INIT_YMM and therefore has mmsize == 32.

Fixes Bug 341.
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Signed-off-by: Mans Rullgard <mans@mansr.com>

This allows building dct-test even if aandcttab.o is not pulled in
by any enabled codec.  The DCT with which these tables are used does
not use them directly, so building it without the tables is possible.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Reordering the members in this struct reduces the holes required
to maintain alignment.  With this order, the only remaining, and
unavoidable, hole is 3 bytes following left_nnz.
Signed-off-by: Mans Rullgard <mans@mansr.com>

These functions are not faster than other mmx implementations on
any hardware I have been able to test on, and they are horribly
inaccurate.  There is thus no reason to ever use them.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

At the moment it only does BGR24, but I plan to add the rest after.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>