Babysitter vetting and voice-analysis: Have we reached peak AI snakeoil?

The ever-useful Gartner Hype Cycle identified an inflection point in the life of any new technology: the "Peak of Inflated Expectations," attained just before the sharp dropoff into the "Trough of Disillusionment"; I've lived through the hype-cycles of several kinds of technology and one iron-clad correlate of the "Peak of Inflated Expectations" is the "Peak of Huckster Snakeoil Salesmen": the moment at which con-artists just add a tech buzzword to some crooked scam and head out into the market to net a fortune before everyone gets wise to the idea that the shiny new hypefodder isn't a magic bullet.

Machine Learning has enjoyed an extraordinarily long and destructive peak, with hucksters invoking AI to sell racist predictive policing systems, racist sentencing and parole systems, and other Weapons of Math Destruction.

But those were Long Cons run by sophisticated hucksters with huge gangs of confederates; lately, we've been seeing a lot of short cons run by petty grifters who prey on fears to target individuals and small businesses, rather than cities, nations and Fortune 100 multinationals.

Here's an example: Predictim uses a secret "black-box algorithm" to mine your babysitters' social media accounts and generate a "risk rating" that you're entrusting your kid to someone who is a drug abuser, a bully, a harasser, or someone who has a "bad attitude" or is "disrespectful."

This system does not weed out risky people. It is a modern-day ducking stool, used to brand people as witches. What's more, it's a near-certainty that its ranking system is racially biased and also discriminates on the basis of class (because poor and racialized people are overpoliced and more likely to be arrested or otherwise disciplined for offenses that wealthier, whiter people get away with, so if you train a machine-learning system to find the correlates of anti-social behavior, it will just tell you to steer clear of brown people and poor people). Read the rest