• Christophe Gisquet's avatar
    x86: simple_idct: 12bits versions · 7ece8b50
    Christophe Gisquet authored
    On 12 frames of a 444p 12 bits DNxHR sequence, _put function:
    C:         78902 decicycles in idct,  262071 runs,     73 skips
    avx:       32478 decicycles in idct,  262045 runs,     99 skips
    
    Difference between the 2:
    stddev:    0.39 PSNR:104.47 MAXDIFF:    2
    
    This is unavoidable and due to the scale factors used in the x86
    version, which cannot match the C ones.
    
    In addition, the trick of adding an initial bias to the input of a
    pass can overflow, as the input coefficients are already 15bits,
    which is the maximum this function can handle.
    
    Overall, however, the omse on 12 bits samples goes from 0.16916 to
    0.16883. Reducing rowshift by 1 improves to 0.0908, but causes
    overflows.
    Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
    7ece8b50
simple_idct.h 1.54 KB