AI is like a magic trick: amazing until it goes wrong, then revealed as a cheap and brittle effect

I used to be on the program committee for the O'Reilly Emerging Technology conferences; one year we decided to make the theme "magic" -- all the ways that new technologies were doing things that baffled us and blew us away.

One thing I remember from that conference is that the technology was like magic: incredible when it worked, mundane once it was explained, and very, very limited in terms of what circumstances it would work under.

Writing in Forbes, Kalev Leetaru compares today's machine learning systems to magic tricks, and boy is the comparison apt: "Under perfect circumstances and fed ideal input data that closely matches its original training data, the resulting solutions are nothing short of magic, allowing their users to suspend disbelief and imagine for a moment that an intelligent silicon being is behind their results. Yet the slightest change of even a single pixel can throw it all into chaos, resulting in absolute gibberish or even life-threatening outcomes."

And just like magicians, the companies and agencies that use machine learning systems won't let you look behind the scenes or examine the props: Facebook won't reveal its false positive rates or allow external auditors for its machine learning system, which is why Instagram's anti-bullying AI is going to be a fucking catastrophe.

Another important parallel: magic tricks depend on the deliberate cultivation of a misapprehension of what's going on. A magician convinces you that they're doing the same trick three times in a row, while really it's three different tricks, so the hypothesis you develop the first time is invalidated when you see the trick "again" a second time. Read the rest