Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.

The nontrivial parts are:
1) extracting a simplified version of the frame management code from
   mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
   its own more complex system already and those were set only to appease
   the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
   for dxva, the draw_horiz_band() call is moved from
   ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
   because it's now different for h264 and MpegEncContext-based
   decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
   added some very simplistic frame management instead and dropped the
   use of ff_h264_frame_start(). Because of this I also had to move some
   initialization code to svq3.

Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Split out dependency on MpegEncContext.

The H264 context is always uninitialized at this point.

!encoding is always true, we do not have a H.264 encoder

partitioned_frame is never set for h264 (as easily seen from git grep).

There may be more decoders that rely on this. Those should be found and
fixed.

Some changes in the border pixels, visually indistinguishable.

Do it all in config_props().

This error was somehow missed for months.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

In some cases when an input contributes fully to the corresponding
output, other inputs may also contribute to the same output. This is the
case, for example, for the default 5.1 to stereo downmix matrix without
normalization.

Do not skip an output if the corresponding input contributes to other output
channels.

If the matrix reduction ends up with no mixing matrix needed, we need to still
reset the mix function accordingly and log the info to the user.

This will print the new matrix if it is set after initialization.