On Wednesday, June 18, Mr. Li, driving the Xiaomi Su7 flamboyant new max of his sister, was the target of around twenty vocal and visual alerts enjoining him to “focus on driving”. The cause, according to him, was obvious: his eyes, of a naturally discreet form, were constantly interpreted by AI as heavy or closed eyelids. “When I forced my eyes to open in large, the alarm stopped. But when I let my eyes come back to their normal state, she started again ”he said, stressing the absurdity of the situation.
The video of this misadventure, in which we hear his sister laughing from the passenger seat as he tries to keep his eyes widely, went around Chinese social networks. It highlights a potential gap in training fatigue detection algorithms, particularly sensitive to eye movements and facial expressions via a camera on the steering wheel. If the system is designed to prevent accidents, in particular by slowing down or even stopping the car if the alerts are ignored, it seems that it has not been sufficiently “educated” to the diversity of morphologies. The driver’s sister also claims to have never encountered such a problem with the same vehicle.
Subscribe to WorldOfSoftware
Xiaomi admits that the system could be “too sensitive”
Xiaomi reacted by confirming the presence of this function in all the MAX, saying that it aims to strengthen security. The company admitted a possible “over-sensitivity” of the system, offering the possibility of deactivating it manually (although not recommended for security reasons).
Mr. Li’s incident is not the first to put the Xiaomi SU7 in the spotlight for security and AI issues. In March, a fatal accident involving three students aboard an SU7, while the assisted mode of assisted Noa (Navigate on Autopilot) was activated, had already raised strong concerns. The car had detected an obstacle and slowed down, but still struck a lateral element of the road at 97 km/h, leading to the death of the occupants and a fall in the course of Xiaomi’s action. This drama had already initiated a debate on the blind confidence granted to intelligent driving systems and on the need for increased vigilance of drivers, even with the assistance of AI.
These combined events are promoting to question the adequacy of tests carried out by Chinese car manufacturers, and more broadly about the existence of biases in artificial intelligence systems which would not be universal sufficiently. The on-board AI, a major asset of the modern car, must still prove its ability to adapt to the richness of humanity, beyond sometimes limited training standards.
🟣 To not miss any news on the Geek newspaper, subscribe to Google News and on our WhatsApp. And if you love us, .
