-
Claudio Freire authored
1. Fix sf_idx and band_type addressing to address only the first subwindow in the group (others could hold garbage values) 2. Don't step on ms_mask when is_mask is set. I/S selection already sets the ms_mask properly and shouldn't be overridden. 3. Use mid/sid cb/sf when computing coding error, as should be since those are the cb/sfs that will eventually be set. 4. Fix distortion computation on multi-subwindow groups (was subtracting the bits terms multiple times) 5. Clear ms_mask when one side uses PNS and the other doesn't. When using PNS, ms_mask signals correlated noise, which can be detected just like regular M/S detection, so we don't skip noise bands, but when only one side uses PNS setting the flag can confuse some encoders, so avoid that.
6711aa21