When Algorithms Go Wrong

ArCompany is a social business intelligence company; we collect, analyze and strategize using data from the internet. I like to call myself a ‘data freak’ because I believe that usually the data will get you to the truth. In a debate, I constantly challenge friends to “show me the data;” I’m an honesty freak too.

Our team is acutely aware of the current debate about the future of privacy; Joe Cardillo once pointed out that privacy is the new currency on the web. I’m a big fan of what Mark Cuban is trying to do with his app, Cyber Dust; not everyone on my team is, or at least, some of us are torn. That is because Cuban’s app allows users to erase their tracks on the internet, meaning that marketers like us can’t use the data that they previously left behind.

I’m a fan of Cuban’s attempt because I don’t believe we should all forfeit our privacy as an inevitable price of being online. I believe that we deserve a choice that is broader than ‘be online and give up privacy, or stay off.’ Staying off is no longer realistic. As much as I believe in the choice Cuban is trying to provide, I won’t be using Cyber Dust, because I enjoy what algorithms provide me.

I am a big fan of algorithms used properly; I love shopping on Amazon, and have since its inception. I’ve been a Prime member since the first year it was available. I constantly compare Amazon’s use of data and algorithms to Facebook’s atrocious attempts at it. Shopping on Amazon is enjoyable because they constantly steer me towards things I’m interested in. The shopping experience appears seamless and organic, unlike the sometimes jarring ads that Facebook puts on my wall.

Sometimes the Facebook algorithm gives me laugh out loud moments, like earlier this week when THIS appeared in my feed:

If you know ANYTHING about me, you know that I am far from a vegetarian. For the record, I have many Facebook only friends – people I’ve never met in real life – and even THEY know this about me: I love food and wine of all varieties. This gave me a huge chuckle because there is only one vegetarian I communicate with regularly, Ryan Pannell, but we primarily instant message. So, besides chuckling, I also felt a bit uneasy that FB must be scanning my IMs in order to think feeding me ads about dating a vegetarian was a smart thing to do; it’s clear that there is no ‘thinking’ going on in this instance.

My other chuckle came by simply looking at the url; the internet may have transformed us into a giant global village, but it also allows us to segment ourselves down into quadrants so small that we actually have a space not just for vegetarians, but for SINGLE vegetarians.

The good and the bad of algorithms

In general, our team at ArCompany advocates for using algorithms, as long as a) they use data properly, and b) it is understood that they cannot work without a human brain operating in conjunction with them, protecting the business using them from errors that can only be seen with the human eye.

There are two stories that illustrate what can happen when algorithms are used without oversight. The first one I wrote about last year: the case of Solid GoldBomb Tees.

When the t-shirt company’s algorithm spit out an option for “Stay Calm and Rape A Lot,” an image of a shirt, with that saying, went up on Amazon.com for all the world to see and purchase. Amazon acted quickly and ceased selling the shirt, banning Solid Gold from the site entirely. With their only channel of sales closed off, and a crushing PR blow, the company quickly crumbled.

Solid Gold’s algorithm was built with no safety measures or precautions allowing a human eye to pick up a mistake, and it caused the company to instantaneously fold. Many PR or Social Media professionals watching saw the Gold Bomb incident as a clear example as to why there must always be an experienced, wizened hand overseeing technology, automation, and yes, even algorithms.

Since the Gold Bomb fiasco there have been other examples of algorithms gone awry, and my team has spent countless hours both using algorithms, and discussing the fact that as wonderful as they are, a human hand and mind is a necessary part of the equation when utilizing their outcomes. Susan Silver, our Social Media Business Intelligence Analyst (yes, a MOUTHFUL of a title, but with good reason), often describes the work she does in social audits as part science, part art, because it takes a human eye and mind to make sense, and make appropriate use of the data we gather.

Algorithm Heartbreak

This next example of algorithms run wild tells the story as clearly as any can. If you haven’t read Inadvertent Algorithmic Cruelty, prepare yourself to shed tears; it is a heartbreaking read. A father, having lost his beautiful 6 year old daughter to brain cancer in the past year, did not want to be reminded by seeing her image surrounded by dancers and confetti, and the title “Eric, here’s what your year looked like!” –  as if he didn’t know.

Prior to reading his story I felt Facebook had been lazy with their patronizing “It’s been a great year, thanks for being part of it!” headline for every single year in review post. In many ways 2014 was not great for me, and there was no way I wanted to be reminded of most of it. I thought, how lazy of them; could they not make the background images and quotes customizable? How little forethought had they given this yearly round up?

But the story of a father’s loss punched me right in the gut; my struggles paled in comparison. And it reinforced what we already know; the advancements that algorithms have brought are amazing, but the wise do not use them without setting limits. The need for human oversight is now greater than ever; if you intend on building a customer-centric business you must ensure that your customers are protected from your technology… that they are never meant to FEEL like their personal information is being callously misused.

photo credit: ruiwen via photopin cc

Amy Tobin

VP of Content & Strategy at ArCompany. She has an extensive background in Sales, and focuses on generational marketing and content. With Hessie Jones she founded ArCompany’s Millnnnial, GenX and Boomer Think Tanks and writes and speaks on those topics from an insights/strategy perspective.