Fixes: Timeout (85sec -> 0.1sec)
Fixes: 17634/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5666410809786368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (42sec -> 379 ms)
Fixes: 16323/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5679178099195904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (100sec -> 5sec)
Fixes: 15509/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5724297261219840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegReviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 15248/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5087440458481664
Fixes: signed integer overflow: 3 + 2147483647 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

This was forgotten in df05fd31.

Remove now unused step variable.

Express bitfields more simply.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket1221
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Coccinelle profile used:

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_get_buffer(ctx, f, flags)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str;
  @@

  -if ((r = ff_reget_buffer(ctx, f)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_reget_buffer(ctx, f)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0)
  +    return r;

...along with some manual patches for the remaining ones.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

CC:libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket1222
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

It's got_frame, not data size

It will be useful in the upcoming transition to refcounted AVFrames.

The return value was the number of bytes left,
it is supposed to be the number of bytes used.

Commit 2bf09826 introduced an overflow check in loco_decode_plane,
but the error code is never taken into account, leading to
completely idiotic return values.

Found-by: Clément Bœsch <ubitux@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes a part of ticket #1222.

Also break some long lines, remove codec function placeholder comments
and add spaces in sample/pixel format lists.
Signed-off-by: Martin Storsjö <martin@martin.st>

Fixes null pointer dereference / http://www.google-melange.com/gci/task/view/google/gci2011/7120335Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

It's more readable and less prone to breakage.

This is required specifically for setting frame->format to -1,
otherwise it will be set to 0 = PIX_FMT_YUV420P and code reading
the format from the output decoded frame will get misled.

In particular fix regressions occurring with the pending vsrc_buffer
patch.

Signed-off-by: Mans Rullgard <mans@mansr.com>

None of these symbols should be accessed directly, so declare them as
hidden.
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d36beb3f)