Commit 52954075 authored by Michael Niedermayer's avatar Michael Niedermayer

avcodec/simple_idct_template: adjust coeffs for 12bit idct

This prevents overflows with 12bit jpegs
Fixes Ticket2952
Signed-off-by: 's avatarMichael Niedermayer <michaelni@gmx.at>
parent d59213b5
......@@ -64,6 +64,7 @@
#elif BIT_DEPTH == 10 || BIT_DEPTH == 12
#if BIT_DEPTH == 10
#define W1 90901
#define W2 85627
#define W3 77062
......@@ -72,13 +73,20 @@
#define W6 35468
#define W7 18081
#if BIT_DEPTH == 10
#define ROW_SHIFT 15
#define COL_SHIFT 20
#define DC_SHIFT 1
#else
#define ROW_SHIFT 17
#define COL_SHIFT 18
#define W1 45451
#define W2 42813
#define W3 38531
#define W4 32767
#define W5 25746
#define W6 17734
#define W7 9041
#define ROW_SHIFT 16
#define COL_SHIFT 17
#define DC_SHIFT -1
#endif
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment