Beyond GIGO: how "predictive policing" launders racism, corruption and bias to make them seem empirical

"Predictive policing" is the idea that you can feed crime stats to a machine-learning system and it will produce a model that can predict crime. It is garbage.

It's garbage for a lot of reasons. For one thing, you only find crime where you look for it: so if you send out the cops to frisk all the Black people in a city, they will produce statistics that suggest that all concealed weapons and drugs are carried by Black people, in Black neighborhoods.

But once you feed that biased data to an algorithm, the predictions it reaches acquire a veneer of empirical respectability, as the mathematical oracle tells you where the crime will be based on calculations that can never be fully understood and so cannot be interrogated, let alone objected to.

Some of the dirtiest police forces in America have bought predictive policing tools, often in secret.

The iron law of computing states: "Garbage in, garbage out," but predictive policing is worse than mere GIGO: it produces computer-generated marching orders requiring cops to continue the bad practices that produced the bad data used to create the bad model.

In a forthcoming paper for The New York University Law Review, Rashida Richardson (AI Now Institute, previously), Jason Schultz (NYU Law, previously) and Kate Crawford (Microsoft Research, previously) describe how at least thirteen cities whose police departments experimented with predictive policing after being placed under federal investigations or consent decrees for "corrupt, racially biased, or otherwise illegal police practices."

That means that in at least 13 cities, the data that cops were feeding to the predictive policing systems had been generated by practices known to be biased. Read the rest