A catalog of ingenious cheats developed by machine-learning systems

When you train a machine learning system, you give it a bunch of data -- a simulation, a dataset, etc -- and it uses statistical methods to find a way to solve some task: land a virtual airplane, recognize a face, match a block of text with a known author, etc.

Like the mischievous genies of legend, machine learning systems will sometimes solve your problems without actually solving them, exploiting loopholes in the parameters you set to find shortcuts to the outcome you desired: for example, if you try to train a machine learning system to distinguish poisonous and non-poisonous mushrooms by alternating pictures of each, it might learn that all odd-numbered data-points represent poisonous mushrooms, and ignore everything else about the training data.

Victoria Krakovna's Specification gaming examples in AI is a project to identify these cheats. It's an incredibly fun-to-read document, a deep and weird list of all the ways that computers find loopholes in our thinking. Some of them are so crazy-clever that it's almost impossible not to impute perverse motives to the systems involved.

* A robotic arm trained to slide a block to a target position on a table achieves the goal by moving the table itself.

* Game-playing agent accrues points by falsely inserting its name as the author of high-value items

* Creatures exploited physics simulation bugs by twitching, which accumulated simulator errors and allowed them to travel at unrealistic speeds

* In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children). Read the rest