Commit 531ddbac authored by Michael Niedermayer's avatar Michael Niedermayer

avcodec/mv30: Fix integer overflows in idct2_1d()

Fixes: signed integer overflow: 6500736 * 473 cannot be represented in type 'int'
Fixes: 23259/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MV30_fuzzer-5179394271477760

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3b8d5bcc3189c6c46279889f1176c0caba4466e4)
Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
parent d25345bb
......@@ -200,10 +200,10 @@ static inline void idct2_1d(int *blk, int step)
{
const int t0 = blk[0 * step];
const int t1 = blk[1 * step];
const int t2 = t1 * 473 >> 8;
const int t2 = (int)(t1 * 473U) >> 8;
const int t3 = t2 - t1;
const int t4 = (t1 * 362 >> 8) - t3;
const int t5 = ((t1 * 277 >> 8) - t2) + t4;
const int t4 = ((int)(t1 * 362U) >> 8) - t3;
const int t5 = (((int)(t1 * 277U) >> 8) - t2) + t4;
blk[0 * step] = t1 + t0;
blk[1 * step] = t0 + t3;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment