Although gcc guarantees 16 byte stack alignment, threads under WinXP
don't appear to be guaranteed to start stack aligned.  So fix the
alignment.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

The issue had been introduced in
c4356536Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Altivec does unaligned reads from this buffer in
hscale_altivec_real(), and can thus read up to 16 bytes beyond
the end of the buffer. Therefore, add an extra 16 bytes of
padding at the end of the conversion buffer.

This fixes fate-lavfi-pixfmts_scale on AltiVec-enabled builds
under valgrind.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

The vertical scaler handles 2 rows at a time and thus requires
alignment by 2, or else it'll read invalid memory and result in
corrupt output.

This fixes assembling using "nasm".

Also implement MMX/MMX2 versions and SSE4 versions.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Janne Grunau <janne-libav@jannau.net>

The intermediate buffer is always aligned.

Use uintptr_t instead of plain int.  Without this change, the
comparisons will come out wrong for pointers in certain ranges.
Fixes random failures on ppc64.  Also fixes some compiler warnings.
Signed-off-by: Mans Rullgard <mans@mansr.com>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

SSE-optimized hScale() scales up to 4 pixels at once, so we need to
allocate up to 3 padding pixels to prevent overreads. This fixes
valgrind errors in various swscale-tests on fate.

Speed: from 3.9x to 9.6x speed improvement over C, and some small
(up to 15%) speed improvements over existing MMX code (particularly
for bigger filters).

This allows using more specific implementations for chroma/luma, e.g.
we can make assumptions on filterSize being constant, thus avoiding
that test at runtime.

It just does that part in scalar form, I doubt using a vector store
over 2 array would speed it up particularly.

The function should be written to not use a scratch buffer.

The logged information is possibly false, and it tends to be outdated
after each change since the logging code needs to be manually updated.

Simplify and prevent confusing wrong debug messages.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Also remove the unnecessary isSupportedIn/Out macros.

Make the code more compact/readable, and simplify the access to
lsws-specific pixel format information.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

This reverts commit ac0fb593. It
causes valgrind errors which I'll want to investigate before
resubmitting this.

When converting RGB format to RGB format with the same bits per sample,
unscaled path performs conversion on the whole buffer at once. For
non-multiple-of-16 BGR24 to RGB24 conversion it means that padding at the
end of line will be converted too. Since it may be of arbitrary length
(e.g. 8 bytes), operating on the whole buffer produces obviously wrong
results.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Signed-off-by: Mans Rullgard <mans@mansr.com>

Fixes a crash when forcing libc to strictly adhere to malloc sizes.

Prevents overflows on very bright scenes when adding dither,
which may lead to black dots.

ptrdiff_t can be 4 bytes, which leads to the next element being 4-byte
aligned and thus at a different offset than intended. Forcing 8-byte
alignment forces equal offset of dither16/32 on x86-32 and x86-64.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Based on a somewhat similar idea in FFmpeg's swscale.