Commit 18b4404d authored by Stefano Sabatini's avatar Stefano Sabatini

lavc/imgconvert: fix check on av_image_check_size() return code in avpicture_get_size()

The documentation states that av_image_check_size() will return a
negative value in case of error, while the check is done on ret != 0.

Also return a proper error code rather than -1 in case the check fails.
parent 0b2ecf82
......@@ -346,8 +346,8 @@ int avpicture_layout(const AVPicture* src, enum PixelFormat pix_fmt, int width,
int avpicture_get_size(enum PixelFormat pix_fmt, int width, int height)
{
AVPicture dummy_pict;
if(av_image_check_size(width, height, 0, NULL))
return -1;
if (av_image_check_size(width, height, 0, NULL) < 0)
return AVERROR(EINVAL);
if (av_pix_fmt_descriptors[pix_fmt].flags & PIX_FMT_PSEUDOPAL)
// do not include palette for these pseudo-paletted formats
return width * height;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment