How to recognize AI snake oil

Princeton computer scientist Arvind Narayanan (previously) has posted slides and notes from a recent MIT talk on "How to recognize AI snake oil" in which he divides AI applications into three (nonexhaustive) categories and rates how difficult they are, and thus whether you should believe vendors who claim that their machine learning models can perform as advertised.

Narayanan's categories are:

* Perception, such as facial recognition and song identification, where there is a definitive correct answer, which is making "genuine, rapid progress."

* Automating judgment, such as spam detection, copyright violation, essay grading, where humans routinely make judgments that can be used to train a model, which is "far from perfect, but improving," albeit with limits, because "reasonable people can disagree about the correct decision."

* Predicting social outcomes, such as predictive policing, predicting terrorist risk, predicting which kids are at risk, which is "fundamentally dubious" because regression analysis and other statistical tools do not work better than "manual scoring using just a few features" -- and this doesn't work very well (and that's before you get into areas like training data bias, etc).

Moreover, the use of AI to predict social outcomes doesn't just produce bad predictions, it also drives demand for more surveillance to feed the machine-learning models, and uses up energy that could be deployed on better-performing techniques for mitigating these harms.

This is a great, compact presentation, but I feel the need to weigh in critically on Narayanan's claim that ML can be used for judging "copyright violation": this is a common misconception among computer scientists who lack a nuanced understanding of copyright law and its limitations and exceptions. Read the rest