While it is a per-frame variable, it is only really used in the
low-level decoding code, so it is more efficient to store it in the
slice context.