swscale: fix another yuv range conversion overflow in 16bit scaling.

(cherry picked from commit 81cc7d0b)

swscale: fix another yuv range conversion overflow in 16bit scaling.
(cherry picked from commit 81cc7d0b)
afb9d4e8 · Ronald S. Bultje · Michael Niedermayer · ebb0d603 · afb9d4e8
Commit afb9d4e8 authored Jun 30, 2011 by Ronald S. Bultje Committed by Michael Niedermayer Jul 01, 2011
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

swscale.c libswscale/swscale.c +1 -1

No files found.
--- a/libswscale/swscale.c
+++ b/libswscale/swscale.c
@@ -2001,7 +2001,7 @@ static void lumRangeToJpeg16_c(int16_t *_dst, int width)
    int i;
    int32_t *dst = (int32_t *) _dst;
    for (i = 0; i < width; i++)
-        dst[i] = (FFMIN(dst[i],30189<<4)*19077 - (39057361<<4))>>14;
+        dst[i] = (FFMIN(dst[i],30189<<4)*4769 - (39057361<<2))>>12;
 }
 static void lumRangeFromJpeg16_c(int16_t *_dst, int width)
 {