Commit fa211943 authored by Andreas Rheinhardt's avatar Andreas Rheinhardt Committed by Michael Niedermayer

avfilter/vf_hqx: Fix undefined left shifts of negative numbers

Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
Signed-off-by: 's avatarAndreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: 's avatarPaul B Mahol <onemda@gmail.com>
Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
parent 5cf593ad
......@@ -523,7 +523,7 @@ static av_cold int init(AVFilterContext *ctx)
int startg = FFMAX3(-bg, -rg, 0);
int endg = FFMIN3(255-bg, 255-rg, 255);
uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
c = bg + (rg<<16) + 0x010101 * startg;
c = bg + rg * (1 << 16) + 0x010101 * startg;
for (g = startg; g <= endg; g++) {
hqx->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
c+= 0x010101;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment