This fixes integer multiplication overflows in RGB48 output
(vertical) scaling as detected by IOC. What happens is that for
certain types of filters (lanczos, spline, bicubic), the
intermediate sum of coefficients in the middle of a filter can
be larger than the fixed-point equivalent of 1.0, even if the
final sum is 1.0. This is fine and we support that.

However, at frame edges, initFilter() will merge the coefficients
for the off-screen pixels into the top or bottom pixel, such as
to emulate edge extension. This means that suddenly, a single
coefficient can be larger than the fixed-point equivalent of
1.0, which the vertical scaling routines do not support.

Therefore, remove the merging of coefficients for edges for
the vertical scaling filter, and instead add edge detection
to the scaler itself so that it copies the pointers (not data)
for the edges (i.e. it uses line[0] for line[-1] as well), so
that a single coefficient is never larger than the fixed-point
equivalent of 1.0.

Signed-off-by: Anton Khirnov <anton@khirnov.net>

Previously the decoder only worked if the user had set avctx->pix_fmt
manually. For some reason the libavformat tmv demuxer sets this, so
the problem was not visible in avplay etc.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Many of the test programs directly access internal symbols not
exported from the shared libraries.  This allows tests to run
when configured with shared libraries.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.
Signed-off-by: Mans Rullgard <mans@mansr.com>

This macro is empty since the removal of the A32 bitstream reader.
Signed-off-by: Mans Rullgard <mans@mansr.com>

We're shifting individual components (8-bit, unsigned) left by 24,
so making them unsigned should give the same results without the
overflow.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Add a macro to shorten the if condition.

For certain types of filters where the intermediate sum of coefficients
can go above the fixed-point equivalent of 1.0 in the middle of a filter,
the sum of a 31-bit calculation can overflow in both directions and can
thus not be represented in a 32-bit signed or unsigned integer. To work
around this, we subtract 0x40000000 from a signed integer base, so that
we're halfway signed/unsigned, which makes it fit even if it overflows.
After the filter finishes, we add the scaled bias back after a shift.

We use the same trick for 16-bit bpc YUV output routines.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

The buffer splicing relies on the bitstream reader over-reading
the end of the buffer as declared in init_get_bits(), although
more data is actually present.  Manually moving the bitstream
boundary after init_get_bits() allows this to work as expected.
Signed-off-by: Mans Rullgard <mans@mansr.com>

The sample has an incomplete last frame.  Decoding it is pointless.
The garbage produced was changed by the bitstream reader now
protecting against over-reads.
Signed-off-by: Mans Rullgard <mans@mansr.com>

channels would be 0 sometimes and would cause floating point exception
Fixes bugzilla #124
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

When turned on, H264/CAVLC gets ~15% (CVPCMNL1_SVA_C.264) slower for
ultra-high-bitrate files, or ~2.5% (CVFI1_SVA_C.264) for lower-bitrate
files. Other codecs are affected to a lesser extent because they are
less optimized; e.g., VC-1 slows down by less than 1% (all on x86).
The patch generated 3 extra instructions (cmp, cmovae and mov) per
call to get_bits().

The performance penalty on ARM is within the error margin for most
files, up to 4% in extreme cases such as CVPCMNL1_SVA_C.264.

Based on work (for GCI) by Aneesh Dogra <lionaneesh@gmail.com>, and
inspired by patch in Chromium by Chris Evans <cevans@chromium.org>.

The A32 bitstream reader variant is only used on ARMv5 and for
Prores due to the larger bit cache this decoder requires.

In benchmarks on ARMv5 (Marvell Sheeva) with gcc 4.6, the only
statistically significant difference between ALT and A32 is
a 4% advantage for ALT in FLAC decoding.  There is thus no (longer)
any reason to keep the A32 reader from this point of view.

This patch adds an option to the ALT reader increasing the bit
cache to 32 bits as required by the Prores decoder.  Benchmarking
shows no significant change in speed on Intel i7.  Again, the
A32 reader fails to justify its existence.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Anton Khirnov <anton@khirnov.net>

In the case that (frame_flags & 0x03) == 3, hybrid_maxclip
may have had a signed integer overflow.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

It doesn't make much sense to clip pre-shift,
nor is it correct for proper decoding.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

The keyframe after a POC reset may not be the first to be returned to
the user. Therefore, don't reset the expected next POC once we return
a keyframe to the user, but once we know that the next frame in the
return-queue is a keyframe.

These pointers are already of the correct type.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Martin Storsjö <martin@martin.st>

The encoder clearly needs width * height * 4 bytes.
Signed-off-by: Mans Rullgard <mans@mansr.com>

This file does not use anything from put_bits.h but needs
intreadwrite.h.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Fixes checksum on big endian systems.

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Janne Grunau <janne-libav@jannau.net>