Using AI to Find Where Clowns End and Juggalos Begin

By Nhoral

Photo by Nate Smith

Clowns and Juggalos generally have pretty distinctive looks. So distinctive in fact, that without much work (or rather leaning entirely on the work of others) we can build a super accurate classifier. But that’s not what we’ll do here.

Disclaimer: This article skips over a lot of the core techniques involved in preparing your data and creating a Learner to train a model. If you are interested in how that works, check out my article on Using Transfer Learning to Detect Yoga Poses.

No, we will explore a deeper question.

When does Clown become Juggalo and vice versa? When is it too hard for even a modern AI classifier to confidently tell them apart?

Modifying the same steps we did previously, we can prepare our data from two Google Image datasets. We are using a DataBunch again, so we can easily inspect a batch of training data.

One of these things is not like the other

We have a fairly small data set, roughly 300 pictures per category. Even so, the model is super accurate in under a minute of training.

93% accurate out of the box is pretty impressive for make-up analysis

We can trust that our model is pretty good at understanding the features that separate Clowns and Juggalos. Now we can explore the gray area between the two.

To see where the model made the greatest mistake, we can evaluate images that produce the highest loss. The images with the highest loss are where the model chose incorrectly or was not as confident in the prediction.

The overlap of Scary Clown and Juggalo is definitely high

The images above are pretty representative of our top losses, groups of clowns and scary clowns.

Some baseless guesses:

  • Many of our Juggalo photos are from large events, and crowds tend to make the model less confident about a prediction of clown.
  • Scary clowns tend to have more dark paint around the eyes, which is also a feature of Juggalo make-up.

It’s worth noting that the model classified all but the first image correctly, it was just less confident about these choices.

Checking out the other side, we can see where our model gets confused with Juggalo face paint and outfits. Some of these images could go either way, so I am not surprised that our model had a hard time.

This is really showing us which Juggalos are thinking outside the box

My takeaway is that red color is generally pretty indicative of high Clownosity in an image.

Maybe more interesting, the model has an easy time distinguishing the two categories of face paint. This means that Juggalo make-up really has a pretty consistent style. One entirely separate from your average door-to-door Clowns.

Thanks to the Clowns and Juggalos that put on make-up and allowed my model into their culture and into their hearts.

If any of this is interesting to you, check out