Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Purely a cosmetic change, most of the zeroing of encoder resources
should happen at the top of the main loop.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

This commit reworks the TNS implementation to a hybrid between what
the specifications say, what the decoder does and what's the best
thing to do.

The filter application function was copied from the decoder and
modified such that it applies the inverse AR filter to the
coefficients. The LPC coefficients themselves are fed into the
same quantization expression that the specifications say should
be used however further processing is not done, instead they're
converted to the form that the decoder expects them to be in
and are sent off to the compute_lpc_coeffs function exactly the
way the decoder does. This function does all conversions and will
return the exact coefficients that the decoder will generate, which
are then applied to the coefficients.
Having the exact same coefficients on both the encoder and decoder
is a must since otherwise the entire sfb's over which the filter
is applied will be attenuated.

Despite this major rework, TNS might not work fine on some audio
types at very low bitrates (e.g. sub 90kbps) as it can attenuate
some coefficients too much. Users are advised to experiment with
TNS at higher bitrates if they wish to use this tool or simply
wait for the implementation to be improved.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Turns out autocorrelating more than 750 coefficients at once
will cause a segfault, despite there being enough space to
hold an entire frame of samples into the buffer.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

This commit adds a function to get the reflection coefficients on
floating point samples. It's functionally identical to
ff_lpc_calc_ref_coefs() except it works on float samples and will
return the global prediction gain. The Welch window implementation
which is more optimized works only on int32_t samples so a slower
generic expression was used.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Not needed anymore, it was only used by the AAC TNS
encoder and was replaced with a more suitable function
in the following commit.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Needed for following commits. Contains the starting sfb for
every samplerate and window type.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Needed for the following commits.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Fixes out of array access
Fixes: 87196d8bbc633629fc9dd851fce73e70/asan_heap-oob_26f6853_862_cov_585961513_sonic3dblast_intro-partial.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes assertion failure
Fixes: 56dcafde14a8397161bb61a16c511179/signal_sigabrt_7ffff6ac8cc9_686_cov_1897408623_microsoft_new_way_to_shove_mpeg2_in_asf.dvr_ms

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes memleak
Fixes: efe937780e95574250dabe07151bdc23/unknown_unknown_351_849_cov_3187578556_shellymanne.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes integer overflow
Fixes: efe937780e95574250dabe07151bdc23/unknown_unknown_351_849_cov_3187578556_shellymanne.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Without this fate-filter-join failes with
FF_API_GET_CHANNEL_LAYOUT_COMPAT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Signed-off-by: Donny Yang <work@kota.moe>

* commit 'c457bdeb':
  checkasm: Fix floating point arguments on 64-bit Windows
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'b1abd2aa':
  vf_scale: Add an option to pass the scaler params
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit 'c23999be':
  avconv_opt: Add an option that lists all supported hwaccels
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

* commit '629d4c5b':
  avconv_opt: Add missing comma
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>

swscale/utils: If cascaded contexts are used forward sws_setColorspaceDetails() to the first context
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This uses a RGB intermediate, a more optimal solution would be to perform the rematrixing
directly in subsampled YUV, this is quite a bit more complicated though

Fixes Ticket4805
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This should only affect yuv->yuv, and not change behavior
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This fixes fate with FF_API_LAVF_BITEXACT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Based-on-patch-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>

Reviewed-by: Ivan Uskov <ivan.uskov@nablet.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

These are not used outside nor are in installed headers
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

It does not need to be non static
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This was copied from the decoder, but is unneeded for the encoder.
tns_max_bands is unused and set to zero which zeroed out start, end
and size and thus no filter was actually applied.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

Since the coefficients are stepped up to order + 1 it was possible
that it went over TNS_MAX_ORDER. Also just return in case the only
coefficient is less than the threshold.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

The encoder-side filter isn't that important. The PSNR
shouldn't change so the FATE test should still be fine.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>