fix ticket: 7455
Reviewed-by: Pedro Arthur <bygrandao@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>

Fixes: Timeout
Fixes: 10053/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PROSUMER_fuzzer-5636993883570176

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

It seems what this function does is a vertical prediction filter, thus
the new name should improve understanding.

rename the related table_b too
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Simplifies code and reduces needed memory
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Also provide shorter option, and reorder options in documentation.

Also bump the API version requirement to 10.9.5, because on olders versions
there were some reports of crashes using the undocumented, yet available
BMDDeckLinkDeviceHandle.
Signed-off-by: Marton Balint <cus@passwd.hu>

If vaEndPicture() failed in ff_vaapi_decode_issue(), free
the pic->slice_buffers.

Fixes the memory leak issue in ticket #7385
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>

Sets the level based on the stream properties if it is not explicitly
set by the user.  Also add a tier option to set general_tier_flag, since
that affects the level choice.

Adds support for determining for level limits, including mapping PTL
blocks to profiles to check profile-dependent level limits.

Set profile compatibility/constraint flags properly (including the
constraint flags used for RExt profiles, as all streams we can currently
generate are RExt-compatible), and use that to add support for the "Main
Intra" and "Main 10 Intra" RExt subprofiles (for which we can re-use the
existing Main and Main10 VAAPI profiles).

A recent version of the standard changed the max and default to 15, from
16 in older versions.  This updates the default to 15 to match, but the
max stays as 16 so that we don't reject older streams.

Including a unit test.

Give the entries in the VAAPI format map table an explicit type and add
functions to do the necessary lookups.  Add another field to this table
indicating whether the chroma planes are swapped (as in YV12), and use
that rather than explicit comparisons where swapping is needed.

Also adds greyscale, 4:2:2, 4:4:4 and RGB support.

Clarify that the list is the naughty list, and therefore being on it is
not desirable.  The i965 driver does not need to be on the list after
version 2.0 (when the standard parameter buffer rendering behaviour was
changed).

constraint_set1_flag should be set for constrained baseline and main
profiles, because the stream conforms to main profile.

constraint_set3_flag should be set for high profile when the stream
is intra-only.

constraint_set4_flag should always be set for main and high profiles
because interlaced encoding is not supported.

constraint_set5_flag should be set for main and high profiles when
B-frames are not used.

Also fix the setting of max_num_ref_frames - use the gop_size value
to check for intra-only rather than the constraint flag (which is not
necessarily set).

Add a larger warning more clearly explaining the consequences of missing
packed header support in the driver.  Also only write the extradata if the
user actually requests it via the GLOBAL_HEADER flag.

Choose what types of reference frames will be used based on what types
are available, and make the intra-only mode explicit (GOP size one,
which must be used for MJPEG).

This was added in libva 2.1.0 (VAAPI 1.1.0).  Use AVCodecContext.qmax,
matching the existing behaviour for qmin, and clean up the defaults so
that we only pass min/max when explicitly set.

Query which modes are supported and select between VBR and CBR based
on that - this removes all of the codec-specific rate control mode
selection code.

The codec sequence headers may contain fields which can overwrite the
fine parameters given in the specific settings (e.g. a crude bitrate
value vs. the max-rate / target-percentage / etc. values in
VAEncMiscParameterRateControl).  Always reapply all global parameters
after a sequence header to avoid this causing problems.