Fix out of bounds read.

Bug-Id: 962
Found by: F4B3CD@STARLAB and Agostino Sarubbo
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes an oversight in 1316df7a.

CC: libav-stable@libav.org

Signed-off-by: Diego Biurrun <diego@biurrun.de>

This was introduced in bc2a3296.

The whole block that the statement was added to is only
relevant when used as a demuxer, but the other statements
there have had other if statements guarding them. Make
sure to only run this whole block if being used as a
demuxer.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

If the input has been decoded from a stream which uses edge cropping
then the whole surface need not be valid.  This defines an input
region for the scaler so we only use the active area of the frame.

There should be an extra offset of 6 on bit_rate_scale and of 4 on
cpb_size_scale which were not accounted for here.

Also remove the redundant casts which were hiding the error here.

This avoids SIMD-optimized functions having to sign-extend their
line size argument manually to be able to do pointer arithmetic.

Also adjust parameter names to be "stride" everywhere.

The latter can do everything the former can do, but also handle conditions
the former cannot like multiple header #includes and checking for headers
and functions in a single test program, which is necessary for certain
library tests.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Reading a value larger than int using atoi() may give the wrong result.

With some old libva versions <va/va.h> does not automatically include
the per-codec subsidiary headers, so we need to include the right one
explicitly ourselves.

Enables VP8 decoding - the decoder places the the bitstream version
in the profile field, which we want to ignore.

Also adds some extra fields to the main context structure that may
be needed by a hwaccel decoder.

ptrdiff_t is the correct type for array strides and similar.

ptrdiff_t is the correct type for array strides and similar.

ptrdiff_t is the correct type for array strides and similar.

ptrdiff_t is the correct type for array pitches and similar.

ptrdiff_t is the correct type for array strides and similar.

ptrdiff_t is the correct type for array pitches and similar.

ptrdiff_t is the correct type for array pitches and similar.

It no longer serves a useful purpose.

See https://software.intel.com/en-us/articles/copying-accelerated-video-decode-frame-buffers

Deprecates struct vaapi_context and the installed header vaapi.h,
to be removed at the next version bump.

Most of the functionality here has moved into lavc.