It's oft been remarked that calculating the "distance" between two colors in RGB space doesn't match human visual perceptions very well. There are many other color spaces that work better, but HSV (hue, saturation, value) is an easy one to use and could dramatically improve the results without much work. The relevant code to change is, I believe, calculateDistance at the end of ColorData.java.