Commit d47f1607 authored by Peter Ross's avatar Peter Ross Committed by Aurelien Jacobs

vp6: ensure that huffman decoding table is sorted with descending symbol order

Previous code worked by pure luck with glibc qsort implementation.
patch by Peter Ross   pross _at_ xvid _dot_ org

Originally committed as revision 13887 to svn://svn.ffmpeg.org/ffmpeg/trunk
parent d43cf688
...@@ -202,10 +202,11 @@ static void vp6_parse_vector_models(vp56_context_t *s) ...@@ -202,10 +202,11 @@ static void vp6_parse_vector_models(vp56_context_t *s)
model->vector_fdv[comp][node] = vp56_rac_gets_nn(c, 7); model->vector_fdv[comp][node] = vp56_rac_gets_nn(c, 7);
} }
/* nodes must ascend by count, but with descending symbol order */
static int vp6_huff_cmp(const void *va, const void *vb) static int vp6_huff_cmp(const void *va, const void *vb)
{ {
const Node *a = va, *b = vb; const Node *a = va, *b = vb;
return a->count >= b->count; return (a->count - b->count)*16 + (b->sym - a->sym);
} }
static void vp6_build_huff_tree(vp56_context_t *s, uint8_t coeff_model[], static void vp6_build_huff_tree(vp56_context_t *s, uint8_t coeff_model[],
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment