Fixes: 15116/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5715173567889408
Fixes: memleak

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Lauri Kasanen
Reviewed-by: Tomas Härdin

It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.

Also make sure that image->size is positive, so that it can't match a
negative error code.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Avoid invalid memory read/crash when frame offset >= 0xfffffff8.
Base64-encoded example: AAABADAwMDAwMAAAMAAwMDAw/P///w==
(The previous commit verifies that p->buf_size >= 22.)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fix cases where unknown data (data beyond p->buf_size) could produce a
higher ico probe score than if the unknown data was known and valid.
For example:
    Header:  OK, 2 frames
    Frame 0: Unknown (offset points beyond end of probe buffer)
    Frame 1: Invalid
Previously this example had a score of 25, even though the score would
be 1 if the unknown frame was known to be valid or 0 if it was known
to be invalid.  For this example the score is now 1.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Avoids misdetection of MPEG (B-)frames.

Reviewed-by: Michael Bradshaw

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The ico probe function is pretty weak just checking a few bytes for being 0, 1 or not 0

Fixes probetest failure
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Fixes ticket 759.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes ticket 905.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>