Fixes overflows for large image sizes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Use uintptr_t instead of plain int.  Without this change, the
comparisons will come out wrong for pointers in certain ranges.
Fixes random failures on ppc64.  Also fixes some compiler warnings.
Signed-off-by: Mans Rullgard <mans@mansr.com>

This allows using more specific implementations for chroma/luma, e.g.
we can make assumptions on filterSize being constant, thus avoiding
that test at runtime.

It just does that part in scalar form, I doubt using a vector store
over 2 array would speed it up particularly.

The function should be written to not use a scratch buffer.

For 9/10bit, it means we don't have to upscale to 16bit before
actual scaling or pixel format conversion, and thus a performance
gain.

This means that precision is retained when scaling between sample
formats with >8 bits per component (48bit RGB, 16bit grayscale,
9/10/16bit YUV).

Signed-off-by: Mans Rullgard <mans@mansr.com>

Remove unused variables "flags" and "dstFormat" in yuv2packed1,
merge source rows per plane for yuv2packed[12], and make every
source argument int16_t (some where invalidly set to uint16_t).
This prevents stack pollution and is part of the Great Evil Plan
to simplify swscale.

This will likely lead to a considerable performance boost,
since it removes a branch from the inner loop. Part of the
Great Evil Plan to simplify swscale.

Make yuv2yuvX16_c a function pointer for yuv2yuvX(), so that the
function pointer becomes bitdepth-independent.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Also remove code that overwrites the C versions of functions in
sws_init_swScale_altivec(), so that it uses the C functions of files
if no altivec-optimized version exists.

Fix handling of input if not in native endianness, and add support for
9/10-bit output. This allows us to force endianness of YUV420P 9/10bit
in the H264/10bit fate tests, which should fix them on big-endian
systems.

PPC and x86 code is split off from swscale_template.c. Lots of code is
still duplicated and should be removed later.

Again uniformize the init system to be more similar to the dsputil one.

Unset h*scale_fast in the x86 init in order to make the output
consistent with the previous status. Thanks to Josh for spotting it.

Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Janne Grunau <janne-ffmpeg@jannau.net>