Commit 151067bb authored by Michael Niedermayer's avatar Michael Niedermayer

dpx: Fix rounding in 10bit total_size calculation

This could have caused out of array reads
Signed-off-by: 's avatarMichael Niedermayer <michaelni@gmx.at>
parent d24de459
......@@ -159,7 +159,7 @@ static int decode_frame(AVCodecContext *avctx,
return -1;
}
avctx->pix_fmt = AV_PIX_FMT_GBRP10;
total_size = (4 * avctx->width * avctx->height * elements) / 3;
total_size = (avctx->width * avctx->height * elements + 2) / 3 * 4;
break;
case 12:
if (!packing) {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment