If we fill with black then the generated palette will have one color more
than what the user requested. This also resulted in unwanted black specks in
the output of paletteuse, especially when generating small palettes.