Lime scooters have a software bug that causes them to hurl their riders to the ground

Lime scooters have been recalled in Switzerland and cleared off the streets of New Zealand following a string of injuries, including multiple broken bones, caused by a software bug that brings the scooters to an abrupt halt, throwing their riders off (the scooters are still available in the USA despite an account of a similar incident in Texas).

The company says it has found the bug: "[I]n very rare casesusually riding downhill at top speed while hitting a pothole or other obstacleexcessive brake force on the front wheel can occur, resulting in a scooter stopping unexpectedly."

There's an important underlying issue here that illustrates one of the ways in which devices whose rental terms are enforced by software do not fail safe: Lime scooters are designed so that they can be remotely immobilized, over the internet, if your credit runs out or if the scooter is doing something else the company disfavors.

This design constraint means that the users of the scooter can't (in some circumstances) override the brakes. Malicious code, or code with errors in it, poses a constant risk for the scooter rider, because if it triggers this braking function, then by design the system will treat attempts by the rider override the immobilization command as an attack.

In an ideal world, we'd design the control systems for devices that can harm their users to fail safe, with overrides for owners that let them judge when safety features are inappropriately triggered. But when the "safety" that these features ensures is the safety of a rental company, not the user of the device, then the "fail safe" mode is one that elevates the protection of the owner's capital investment over the user's physical wellbeing. Read the rest