• Mans Rullgard's avatar
    x86: swscale: fix fragile memory accesses · 90540c2d
    Mans Rullgard authored
    To access data at multiple fixed offsets from a base address, this
    code uses a single "m" operand and code of the form "32%0", relying on
    the memory operand instantiation having no displacement, giving a final
    result of the form "32(%rax)".  If the compiler uses a register and
    displacement, e.g. "64(%rax)", the end result becomes "3264(%rax)",
    which obviously does not work.
    
    Replacing the "m" operands with "r" operands allows safe addition of a
    displacement.  In theory, multiple memory operands could use a shared
    base register with different index registers, "(%rax,%rbx)", potentially
    making more efficient use of registers.  In the cases at hand, no such
    sharing is possible since the addresses involved are entirely unrelated.
    
    After this change, the code somewhat rudely accesses memory without
    using a corresponding memory operand, which in some cases can lead to
    unwanted "optimisations" of surrounding code.  However, the original
    code also accesses memory not covered by a memory operand, so this is
    not adding any defect not already present.  It is also hightly unlikely
    that any such optimisations could be performed here since the memory
    locations in questions are not accessed elsewhere in the same functions.
    
    This fixes crashes with suncc.
    Signed-off-by: 's avatarMans Rullgard <mans@mansr.com>
    90540c2d
rgb2rgb_template.c 102 KB