Fixes integer overflow
Fixes: efe937780e95574250dabe07151bdc23/unknown_unknown_351_849_cov_3187578556_shellymanne.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Without this fate-filter-join failes with
FF_API_GET_CHANNEL_LAYOUT_COMPAT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Signed-off-by: Donny Yang <work@kota.moe>

* commit 'c457bdeb':
  checkasm: Fix floating point arguments on 64-bit Windows
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'b1abd2aa':
  vf_scale: Add an option to pass the scaler params
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'c23999be':
  avconv_opt: Add an option that lists all supported hwaccels
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit '629d4c5b':
  avconv_opt: Add missing comma
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

swscale/utils: If cascaded contexts are used forward sws_setColorspaceDetails() to the first context
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This uses a RGB intermediate, a more optimal solution would be to perform the rematrixing
directly in subsampled YUV, this is quite a bit more complicated though

Fixes Ticket4805
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This should only affect yuv->yuv, and not change behavior
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This fixes fate with FF_API_LAVF_BITEXACT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Based-on-patch-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>

Reviewed-by: Ivan Uskov <ivan.uskov@nablet.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

These are not used outside nor are in installed headers
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

It does not need to be non static
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This was copied from the decoder, but is unneeded for the encoder.
tns_max_bands is unused and set to zero which zeroed out start, end
and size and thus no filter was actually applied.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Since the coefficients are stepped up to order + 1 it was possible
that it went over TNS_MAX_ORDER. Also just return in case the only
coefficient is less than the threshold.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

The encoder-side filter isn't that important. The PSNR
shouldn't change so the FATE test should still be fine.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

The order should never go above TNS_MAX_ORDER (and thus cause
the context to be reinitialized) but this is just in case.

Also fix a comparison, since the coefficients are zero-indexed.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

It also made no sense to actually make the filter span the entire
window including the first band of the next window.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Nicolas George <george@nsup.org>
Signed-off-by: Clément Bœsch <u@pkh.me>

Tests fails on some ARM builds but it's close enough so it's okay.
NEON, half-precision floats, rounding errors, who knows.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

This commit introduces a test for AAC-Main prediction
which was just reworked in this series of commits.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Hopefully without errors like last time, but I'm prepared.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Pulses are already on the way so expect to see the list
gone in the close future.

TNS is already of sufficiently high quality to be enabled
by default (but isn't yet, so you too can help by testing!).
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

This commit abandons the way the specifications state to
quantize the coefficients, makes use of the new LPC float
functions and is much better.

The original way of converting non-normalized float samples
to int32_t which out LPC system expects was wrong and it was
wrong to assume the coefficients that are generated are also
valid. It was essentially a full garbage-in, garbage-out
system and it definitely shows when looking at spectrals
and listening. The high frequencies were very overattenuated.
The new LPC function performs the analysis directly.

The specifications state to quantize the coefficients into
four bit index values using an asin() function which of course
had to have ugly ternary operators because the function turns
negative if the coefficients are negative which when encoding
causes invalid bitstream to get generated.

This deviates from this by using the direct TNS tables, which
are fairly small since you only have 4 bits at most for index
values. The LPC values are directly quantized against the tables
and are then used to perform filtering after the requantization,
which simply fetches the array values.

The end result is that TNS works much better now and doesn't
attenuate anything but the actual signal, e.g. TNS removes
quantization errors and does it's job correctly now.

It might be enabled by default soon since it doesn't hurt and
helps reduce nastyness at low bitrates.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

This commit removes the array which was made redundant with
the last commit. The current prediction system gets the
quantization error directly (and without the single-frame delay)
in the search_for_pred function.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>