Character.AI today said today that it soon will no longer allow minors to communicate with its chatbots in a move to address complaints about child safety.
The Silicon Valley Google LLC-funded chatbot startup, which lets users create character-based avatars, has been the focus of scrutiny for some time now, recently at the center of a probe looking into artificial intelligence tools giving possibly misleading mental health advice.
Last year, parents of two children in the U.S. claimed their kids had been groomed by the company’s chatbots. A lawsuit said both children were “targeted with sexually explicit, violent, and otherwise harmful material, abused, groomed, and even encouraged to commit acts of violence on themselves and others.”
This came after a mother in Florida claimed her 14-year-old son took his own life after becoming obsessed with his hyperrealistic chatbots, AI she said that had encouraged him to self harm. Experts have warned that users of generative AI can fall into a trap of believing the software is human – what’s being called AI-psychosis.
As the danger becomes clearer, Character.AI says it is introducing a slew of safety initiatives, which will start by limiting under-18s to two hours or less with chatbots by Nov. 25. barring minors from using the app, what the company called “extraordinary steps,” more “conservative” than its peers. The move will see about 10% of its 20 million users leave the app.
“We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI and about how open-ended AI chat in general might affect teens, even when content controls work perfectly,” the company said in a press release. “After evaluating these reports and feedback from regulators, safety experts, and parents, we’ve decided to make this change to create a new experience for our under-18 community.”
Politicians have been scrambling to keep up with the brave new world of AI. On Tuesday, Senators Josh Hawley and Richard Blumenthal announced a bipartisan proposal to ban AI chatbot companions for minors. Earlier in October, California Governor Gavin Newsom enacted a similar measure requiring chatbots to identify as AI and advise minors to take periodic breaks.
Photo: Unsplash
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About News Media
Founded by tech visionaries John Furrier and Dave Vellante, News Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
