Last weekend, a gnarly power outage in San Francisco took out a number of traffic lights, which, in turn, sent a number of self-driving Waymo robotaxis into a sort of fugue state. Instead of driving, some of the Waymos responded to these now-analog intersections by turning on their hazard lights, blocking traffic, and, well, not doing much of anything. There were multiple instances of Waymo cars clogging up roads, turning futuristic technology into glorified bollards. The city quickly asked the company to turn off the service.
The immediate issue has been resolved—the power is back on and the Waymo service had resumed in San Francisco as of Sunday. But questions linger about whether Waymo, or the city, had a plan for a relatively predictable type of municipal emergency—a blackout that crowds communications networks—or how they’re adjusting now.
One of the big solutions to AI failures is the much-discussed human in the loop. The idea: At some point in an automated process — whether it be a job-application screening system or powerful self-driving car algorithms — humans have the opportunity to intervene and fix the hard stuff that artificial intelligence can’t handle. AI doesn’t understand every complex situation, the logic goes. So there are safeguards built into a system to ensure that, at some point, an actual live person can set an automated system back on the right path.
The problem, as recently events demonstrated, is that sometimes this human-in-the-loop doesn’t always answer the phone. Or can’t.
Over the weekend, a remote assistance team was supposed to help the cars navigate when they encountered a confusing traffic situation, a Waymo spokesperson explains. But networks were overwhelmed – because of the power outage – making it difficult for the Waymo Driver software in some of the cars to connect with that team and receive confirmations.
Waymo spokesperson Ethan Teicher tells Fast Company that the company prioritizes safety and tests and refines its emergency preparedness and response protocols on a regular basis. He also defends the company’s response to other emergencies, including Hurricane Helene in Atlanta and previous tsunami warnings in San Francisco.
“We are committed to continuous improvement, and we will use learnings from the weekend to strengthen our resilience under even the most challenging conditions,” Teicher says. “Ahead of entering any city, we work to understand the types of issues that impact the region.”
