Commit bcff32d1 authored by Kostya Shishkov's avatar Kostya Shishkov

Mask all unused bits for packed pixel format instead of green and alpha mask only.

That fixes the case when converting 15-bit RGB/BGR to YUV and high bit is set
for input value(s).

Originally committed as revision 28916 to svn://svn.mplayerhq.hu/mplayer/trunk/libswscale
parent 2500d8be
......@@ -1663,7 +1663,7 @@ static inline void RENAME(name ## _half)(uint8_t *dstU, uint8_t *dstV, uint8_t *
{\
int pix0= ((type*)src)[2*i+0];\
int pix1= ((type*)src)[2*i+1];\
int g= (pix0&(maskg|maska))+(pix1&(maskg|maska));\
int g= (pix0&~(maskr|maskb))+(pix1&~(maskr|maskb));\
int b= ((pix0+pix1-g)&(maskb|(2*maskb)))>>shb;\
int r= ((pix0+pix1-g)&(maskr|(2*maskr)))>>shr;\
g&= maskg|(2*maskg);\
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment