Problem

Color Mixing

I love to paint, it's incredibly therapeutic and gives me a way to extend myself onto a canvas. Like any craft, becoming a skilled artist takes time and consistent practice. One thing that's helped me the most is attempting to master the art of observation. Our brains aren’t always reliable when it comes to accurately interpreting objects, shadows, or colors. We might think we’re seeing the real thing, but often that's just our imagination filling in the gaps.

And after observation comes replication, arguably the trickiest part. One of my biggest challenges as an artist has been mastering color theory well enough to mix and match hues with precision. That’s where technology can step in to support us, helping to close the gap in our paint mixing limitations.

Let's Test it Out!

Reference Photo

Our starting point:

As you can see I already have a thin layer put in place for the sky (right) and my dad's sweater yet I wasn't too happy with the color I developed. These are the two places where I'm going to test out my code.

Attempt 1: Replicating the Sky

My code outputted the following information, letting me know I should mix together 43.87% white, 35% yellow, and 21% blue. While I didn't have these exact paint tubes, I used what I had - primary yellow, primary blue, and titanium white - which could have impacted the results slightly. To take this into account, cerulean blue is much more warmer than primary blue and lies slightly on the blue-green side, I mixed my primary blue with green.

Here's the pastel green on the canvas (below). I'm happy with how it turned out, it definitely doesn't look green on the canvas and has that old photo feel.

Attempt 2: My Dad's Sweater

My code outputted the following information, letting me know I should mix together 40.86% cadmium yellow, 33.82% titanium white, and 25% raw umber.

Once again I didn't have raw umber ( a grayish cool brown) but I did have burnt umber (reddish brown which is pictured as the darkest brown color on the left side of the palette). To make raw umber I mixed together black and white (grey) and a tiny bit of blue.

On the left is the color my code produces and on the right is the color I thought was appropriate, clearly two vastly different colors. In the bottom image you can see me testing out these two colors with the color my code produced on the left and mine on the right.

Thoughts?

I was happy with the results that my code produced, while it wasn't an exact match it was definitely a starting point which was much easier to mold than starting from scratch. The only major area of concern I had are the percentages, I found I had to mix different proportions to get my desired color. However, especially when working with acrylic paint which dries down really fast, this provides an option to reproduce the same color multiple times.

My Coding Process

Starting this project was like opening a can of worms, I very quickly figured out that translating the human perception of color into the digital color sphere would be a feat of it's own, and mixing color based on the color theory even worse. This project really tested me by forcing me to think outside the box on how to get the closest color match.

I started off with a python script outputting a specific color palette when an image was inputted. I then provided instructions for mixing paints for this color palette. The second feature was utilizing a color picker and then outputting mixing instructions for that one specific color.

I started off utilizing RGB, the digital color space most commonly used for a lot of functions. I found that it failed to properly mix colors based on pigments, giving weird random hues instead.

My second course of action was utilizing HSL, a color space that was supposed to be closer to our human perception. The results were better but still lacking considerably when it came to color mixing.

I started thinking that maybe AI would be able to identify colors in the digital space and use it's understanding of the color theory to replicate said color with tangible paints/pigments. I utilized RAG by giving the AI paint colors with it's corresponding RGB values (which I later tested with HSV).This took a lot of trial and error of getting the right prompt for prompt engineering. However, even AI fell flat in delivering the results that I wanted. I found that the results were quite random, even after playing around with the temperature variable.

This is when I took a break and randomly found HSLUV which follows the CIELUV color space. HSLUV's main feat is its perceptual color uniformity, which is very similar to the human perception of color. For example, if you look at the HSV color wheel the colors look very saturated or vibrant, yet, we never perceive color in that manner in the real world. CIELUV basically mutes everything to where gradual mathematical changes in color represent gradual visual changes in color. However, once again, I struggled a lot with color mixing.

It was apparent that my main problem with every color space was not necessarily color identification but rather color mixing. Paints are very dynamic, meaning a pure paint color can look completely different depending on how much paint is applied, lighting, and brand which is impossible to replicate with one static color and thus tonal differences are very apparent when mixing paints. I started to create my own database of paint colors with Roboflow where I can train a LLM on these paint datasets, each paint color needs a considerate amount of pictures therefore this is something I'm doing on the side. For the mean time, I used a color picker for color swatches of paints to create my own RGB value to produce the closest digital color translation. My code then picks out a base color that mostly closely matches the target color, from there I tried to reverse engineer it where it measures out the distance to every other color based on the base color and picks the best choice. While this is the best result I've gotten so far, there is definitely room for improvement.