Fixes ticket #4799.
Analyzed-by: Aleksandr Ustinov

It's considerably faster than libaom in most systems.

Reviewed-by: BBB
Signed-off-by: James Almer <jamrial@gmail.com>

Signed-off-by: Rick Kern <kernrj@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Rick Kern <kernrj@gmail.com>

On iOS 11, encoding a frame may return error with log
"Error encoding frame 0", which means vtenc_output_callback
is called with status=0 and sample_buffer=NULL. Then the
encoding session will be crashed on next callback wether or not
closing the codec context.

Let us look through the link below introducing VTCompressionOutputCallback,

https://developer.apple.com/documentation/videotoolbox/vtcompressionoutputcallback?language=objc

"status=0" (noErr) means compression was successful.
"sampleBuffer=NULL" means the frame was dropped when compression
was successful (status=0) or compression was not successful (status!=0).

So we should not set AVERROR_EXTERNAL on "status=0" and "sample_buffer=NULL"
as it is not a error.

The fix is that we only set AVERROR_EXTERNAL with status value non zero.
When sample_buffer is NULL and status value is zero, we simply return
with no other operation.

This crash often occurs on iOS 11 for example encoding 720p@25fps.
Signed-off-by: sharpbai <sharpbai@gmail.com>
Signed-off-by: Rick Kern <kernrj@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 65535 * 65312 cannot be represented in type 'int'
Fixes: 16704/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-6294115603447808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: 57342 * 120830 cannot be represented in type 'int'
Fixes: 16426/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5758744817827840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg

snow allows creating large output from tiny input, like other
wavelet codecs

Fixes: Timeout (82sec -> 1.5sec)
Fixes: 9520/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-6286612576075776

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This avoids 3 warnings
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Multiple memleaks
Fixes: ffmpeg-memory-leak
Found-by: Francis Provencher <francis@protekresearchlab.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (151sec -> 173ms)
Fixes: 16053/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-5647069169057792
Fixes: 16053/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-5108957126852608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpegSigned-off-by: Michael Niedermayer <michael@niedermayer.cc>

Changing details as following:
1. The previous order of parameters are irregular and difficult to
   understand. Adjust the order of the parameters according to the
   rule: (RTYPE, input registers, input mask/input index/..., output registers).
   Most of the existing msa macros follow the rule.
2. Remove the redundant macro SLDI_Bn_0 and use SLDI_Bn instead.
Reviewed-by: Shiyou Yin <yinshiyou-hf@loongson.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This was partially fixed by 233cd890, but it made changes to AVFrame fields
from within end_frame, which doesn't work consistently when multithreading
is enabled. This is what the post_process function is for.
Signed-off-by: Aman Gupta <aman@tmm1.net>

Some of these symbols are only defined in newer kernel releases.
Signed-off-by: Aman Gupta <aman@tmm1.net>

Signed-off-by: James Almer <jamrial@gmail.com>

Regression since 78f52b4fSigned-off-by: James Almer <jamrial@gmail.com>

Fixes ticket 8143.
Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>