Normally, all channel ids are between 0 and 10, while they in
uncommon cases can have values up to 64k.

This avoids allocating two arrays for up to 64k entries (at a total
of over 6 MB in size) each when most of them aren't used at all.
Signed-off-by: Martin Storsjö <martin@martin.st>