Facebook's spam filter blocked the most popular articles about its 50m user breach

When news broke yesterday that Facebook had suffered a breach affecting at least 50,000,000 stories, Facebook users (understandably) began to widely share links to articles about the breach.

The articles were so widely and quickly shared that they triggered Facebook's spam filters, which blocked the most popular stories about the breach, including an AP story and a Guardian story.

There's no reason to think that Facebook intentionally suppressed embarrassing news about its own business. Rather, this is a cautionary tale about the consequences of content filtering on big platforms.

Facebook's spam filter is concerned primarily with stopping spam, not with allowing through storm-of-the-century breaking news headlines that everyone wants to share. On a daily basis, Facebook gets millions of spams and (statistically) zero stories so salient that every Facebook user shares them at once. Any kind of sanity-check on a spam filter that allowed through things that appeared to be breaking news would represent a crack in Facebook's spam defenses that would let through much more spam than legitimate everywhere-at-once stories, because those stories almost never occur, while spam happens every second of every minute of every hour of every day.

And yet, storm-of-the-century stories are incredibly important (by definition) and losing our ability to discuss them -- or having that ability compromised by having to wait hours for Facebook to discover, diagnose and repair the problem -- is a very high price to pay.

It's a problem with the same underlying mechanics as the incident in which a man was sent an image of his mother's grave decorated with dancing cartoon characters and party balloons on the anniversary of her funeral. Read the rest