The social media giant has announced a significant update to its conversational artificial intelligence. The platform will now proactively inform parents or legal guardians when a supervised adolescent’s exchanges with the chatbot reveal a risk of self-harm or dark thoughts. This initiative takes place in a context where the mental health of young users is a growing concernespecially after similar cases involving other AIs.
How does this alert system work in practice?
The device is based on a Specialized AIdesigned to identify even subtle references to self-harm in discussions with Meta AI. When a conversation is reported, it does not result in an automatic alert. To avoid false alarms and unnecessary stress, each case is systematically subjected to a human review before a notification is sent to the legal guardians.
Meta specifies that this feature only applies to accounts for which the parental supervision has been activated. The company takes a cautious approach: in the event of doubt about the adolescent’s intention regarding a risk of suicidethe alert will still be sent. The measure is already in place in the United States, the United Kingdom, Australia and Canada, with a global deployment expected by the end of the year.
What other security measures does Meta have?
Beyond this notification, Meta is strengthening its emergency protocol. The company develops the capacity to contact the emergency services directly if a conversation, whether involving an adult or a teenager, suggests a risque is imminent of taking action. This practice is already in effect for publications on Facebook and Instagram, having led to more than 19,000 reports first aid last year.
The company also consulted with more than 75 clinicians specializing in pediatric mental health. adolescents to refine the responses of its AI. The goal is to ensure that the chatbot responds appropriately, validating the user’s feelings without abruptly end the conversation. Additionally, the “Limited Content” setting on Instagram will also apply to AI, further restricting sensitive topics.
Is this initiative unanimously supported?
If the intention is welcomed by some, such as the American organization ConnectSafely which sees it as a fair balance between privacy and protection, critical voices are being raised. The Molly Rose Foundation, created in memory of a young girl who was the victim of harmful content online, expresses its serious concerns. She fears that these alerts cause panic and force unwanted revelations from young people.
For this foundation, Meta’s priority should be to “ repair the security issues at the heart of its products » rather than setting up alert systems in retrospect. The debate highlights the complex tension between the need to inform parents of potential danger and the right to privacy young people, who often use chatbots as a confidential space to express their discomfort.
