Scented candle ratings trends

Late in 2020 Kate Petrova shared on Twitter an interesting analysis of scented candles reviews that links a decline in review ratings with the rise of covid-19 and its associated loss-of-smell symptom. Looking at the smoking-gun plot raises a few questions.

• Why are there dots at non-integer rating values?
• Why is a there a blob of only high ratings around Jan 2020?
• Why doesn’t the smooth fit go through the blob?

Fortunately, the ratings data and code was shared on GitHub, so I might figure out some answers. All of those questions are mostly addressed by slowing down and actually reading the graph, particularly the y-axis title.

Why are there dots at non-integer rating values?

Each dot is an average rating for a given day. Days with only one rating have integer dot positions but other averages are usually not integers. Here’s a bar chart of the rating counts for those three scented candles.

That’s a lot of bars (one per day), but it’s easy to see there is a surge of reviews around January of 2020. There are also smaller bumps at the end of other years. Zooming in to the January 2020 period, we get:

We can see that many of the days outside that region have only one or no ratings, while all the days in the surge period have plenty of ratings.

Why is a there a blob of only high ratings around Jan 2020?

Since there are lot of values being averaged and most ratings are high, any low ratings will get only slightly pull down the high average rating. Here are all the raw (unaveraged) ratings values. Since they’re all at integer values, I’ve added jitter and transparency to get a better sense of the counts per day.

Not only are there a lot of ratings in that period, but the vast majority appear to be 5s, which explains why the average ratings are near 5.

Why doesn’t the smooth fit go through the blob?

The simple answer is that the smoother parameters are set to capture long term trends. The more subtle answer is that the smoother is being computed on the already averages values instead of the raw ratings. As a result, all those high ratings don’t carry as much weight since only one average per day is seen by the smoother. My rule:

Don’t smooth a smoother.

Smoothers and other model fitters are designed to deal with variation, so there’s usually no need to pre-smooth the data for a fitter. Here’s my attempt at recreating the original loess fit (teal) over my jittered points overlaid with the same fit on the raw data (red).

Without the pre-summarization the smooth fit comes closer to the cluster, and the drop in ratings looks more pronounced. However, the stiff smoother parameters are still preventing the surge of high ratings from getting a strong say. If we relax the stiffness (using α = 0.4 and λ = 1 instead of the default α = 0.75 and λ = 2), I think we come close to seeing the pattern in the data.

Notice the pre-summarized (teal) version still doesn’t pick up the surge group. The new fit suggests three phases of ratings.

1. A steady pre-2020 ratings average of about 4.
2. A bump around January 2020 in both volume and rating. We can’t tell why from this data. It could be a new promotion, a new scent formula or even ratings spammers.
3. A new steady average of around 3.6.

The final assessment is not too different from the original: a slight decline in average ratings coinciding with the spread of covid-19; however, I’m happier with the updated model.

More to the story

The original analysis nice includes a control group of unscented candles to check that their ratings didn’t also decline. And the data, but not the original graphs, includes ratings for two more scented candles beyond the top three scented candles. If we look at the ratings trends for all eight of those candles separately, we see more angles.

Only the first scented candle “s1” experiences the January 2020 bump. Scented candle s4 fits in more with the unscented candles and doesn’t decline. Unscented candle u3 does experience a 2020 decline but it also have a wide range of past ratings. Investigating those angles would likely require learning more about the individual candles, so I’ll leave it at that.