There is in general no reason for the currently active SPS to be the one
referenced by the PPS being parsed.