Data is Plural and the Data Visualization Society teamed up to create a visualization challenge around a couple databases of Bob Ross’s paintings. One, from Walt Hickey at FiveThirtyEight, has content tags for each episode of the show. Another, from Jared Wilber‘s catalog scraping, has details about the paints used in each episode and thumbnails of each painting.
Noticing that the paintings have fairly descriptive titles, such as “Autumn Days” and “Quiet Woods,” I thought it would be interesting to look at which paint colors correspond to the common title words. Looking at color occurrences s in general would have been just as meaningful from a title-word-to-color correspondence perspective, but I thought limiting it to actual paint colors would connect the result more in the actual painting process.
Mapping the millions of final colors into the dozen base paint colors took some effort, which I will explain. But I’ll start with my final result.
Each panel shows the base colors for each painting, sized by the amount of inferred usage, including the white header and the black frame (like the frame of a painting, get it?) I ordered the panels along two axes, bottom-to-top by amount of white and left-to-right by amount of green (see axis legend in bottom left). Given the difficulty of unmixing colors, it was nice to seem basic expectations met:
- Snow and Winter at the top of the whiteness scale
- Black at the bottom of the whiteness scale
- Valley, Forest, and Meadow on the green extreme
- A sliver of red for Barn
- A lot of red for Sunset
- Plenty of blue for Blue!
My approach depended on there being plenty of paintings (~400) so that each title word has enough occurrence and reasonable quality images for each painting (thumbnails were about 400×300, with only a minority noticeably cropped).
Unmixing
With only a dozen or so paint colors per episode but a continuum of colors in each final painting, I had to find some way of unmixing the colors. Colors are hard to manipulate mathematically in a visual-perception-aligned manner, and mixing them like paints mix is especially hard. I watched a few Bob Ross videos to research his mixing (and now YouTube thinks I’m a big fan and want to see even more!), and saw that much of it was in lightening the dark raw paint colors.
Instead of directly unmixing, my strategy was to mix the base colors to get a new expanded palette, concentrating mostly on lightening, and then matching each image pixel to the nearest palette color. Then I could recall the formulation of each palette color to infer the amount of each base color in each pixel. For example, here’s the expanded palette for “Autumn Days.”
It’s probably overkill to have so many colors, but I wanted to make sure I had plenty of light tints to avoid a light color finding white as its nearest palette color. The first 10 colors in the first column are the base colors in this case. Then there are a few hue mixes and then darkening and lightening mixes. I used L*a*b* color space for mixing and avoided mixing hues that weren’t in the same quadrant. For example, blue and yellow are on opposite sides of the hue space and mixing them together mathematically would produce gray, which is in the middle.
I scaled the lightness coordinate to de-emphasize it, and then used Euclidean distance in L*a*b* space to find the nearest palette color to each image pixel. Here is the below and after, original thumbnail and the recreation from closest palette colors, for “Autumn Days.”
Not bad. The continuous gradients look poor, but hopefully they still capture the base color contributions. Light purples and blues are especially hard since that’s an area where the mathematical lightening doesn’t match perception well. I made some effort to adjust for that when creating my expanded palette.
Happy Accident
For the title words, I did some manual recoding to merge word variations and synonyms, but I accidentally overlooked Waterfall and Falls. Happily (and luckily) they ended up next to each other in the white/green panel sorting, making it look like there is some real meaning in my unmixing.
Bonus chart
While the color usage was mostly consistent across all 31 seasons of the show, having made the paint breakdowns I could look for any long-term trends. The most obvious trends were the rise of Prussian Blue and the switch from Burnt Umber to Dark Sienna.
