vc1dec: fix invalid memory access for small video dimensions
For small video dimensions, these calculations of the upper bound
for pixel access may have a negative result. Using an unsigned
comparison to bound a potentially negative value only works if
the greater operand is non-negative. Fixed by doing edge emulation
when the upper bound is probably negative, everywhere that this
pattern appears.
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
Showing
Please
register
or
sign in
to comment