Commit 2469ed32 authored by George Boyle's avatar George Boyle Committed by Michael Niedermayer

avcodec/flacenc: Fix Invalid Rice order

Fixes ticket #4628.

The problem arose, in the sample file at least, in the last block where the
minimum and maximum Rice partition orders were both 0. In that case, and any
other where pmax == pmin, the original UINT32_MAX placeholder value for
bits[opt_porder] was getting overwritten before the comparison to check if the
current partition order is a new optimal, so the correct partition order and
RiceContext params were not being set.
Signed-off-by: 's avatarMichael Niedermayer <michaelni@gmx.at>
parent 202188a2
......@@ -705,7 +705,7 @@ static uint64_t calc_rice_params(RiceContext *rc,
bits[pmin] = UINT32_MAX;
for (i = pmax; ; ) {
bits[i] = calc_optimal_rice_params(&tmp_rc, i, sums, n, pred_order, kmax, exact);
if (bits[i] < bits[opt_porder]) {
if (bits[i] < bits[opt_porder] || pmax == pmin) {
opt_porder = i;
*rc = tmp_rc;
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment