It might be possible to make your quantizer antialiasing-resistant, but I'm not sure how well it will work. The basic idea would be like so:
Like I say, I'm not sure how well it will work, but it seems like it should get rid of the problem where black lines get blue and yellow pixels due to the original antialiasing.
- First Pass: find all the pixels whose color is "close enough" to one of the output colors, and flag them as "good", while assigning them to the appropriate output color. This should handle all the filled in areas as long as your output colors are acceptable.
- Second Pass: for all pixels that didn't get set above, look at all their adjoining pixels that were given a value step one. Then, set their color to the color from that set that most closely matches.
- Repeat until done? Note: if you use a pass number for your "good" flag, you can avoid having pixels that are set during pass 2 being considered pre-good in pass 2.
Like I say, I'm not sure how well it will work, but it seems like it should get rid of the problem where black lines get blue and yellow pixels due to the original antialiasing.