The AI platform will block teenagers from chatting with its virtual characters

The widely-used chatbot platform Character.ai is enforcing a significant rule adjustment that will prevent minors from having discussions with its AI-powered avatars.

This decision follows growing backlash regarding the nature of conversations minors were engaging in with the digital companions.

Legal Challenges

The service, which started in 2021 and attracts millions of users, currently faces several legal actions in the United States.

Families have initiated cases, including one involving the loss of a minor, with some calling the platform a "clear and present danger" to teenagers.

Updated Policies

Effective November 25th, users below eighteen will only have permission to create material such as videos with their characters, instead of chatting as they now do.

Online safety advocates have applauded the decision but expressed that the feature was inappropriate to be accessible by children in the first place.

Platform Announcement

"Today's announcement is a extension of our general belief that we need to keep building the most protected AI platform on the planet for leisure activities," said Character.ai boss Karandeep Anand.

He mentioned that AI safety represents "a moving target" but something the company had taken an "forceful" approach to, with safety measures and protective guardrails.

Professional Perspectives

Internet security organization Internet Matters welcomed the move, but stressed that safety measures should have been built in from the outset.

"Our research indicates that children are subjected to risky information and placed in jeopardy when engaging with AI, including AI chatbots," the organization said.

Earlier Concerns

Character.ai has been criticized in the past for hosting potentially harmful or concerning chatbots that young users could engage with.

Digital representations simulating English minors were found on the platform before being taken down.

Subsequently, investigative journalists discovered a chatbot modeled after notorious criminal which had logged multiple interactions with users.

Industry Impact

Online expert Matt Navarra called the situation as a "reality check" for the AI industry, which is moving "from permissionless innovation to post-crisis regulation".

"When a company that builds a minor-friendly service subsequently pulls the plug, it's demonstrating that moderated conversations aren't enough when the tech's emotional pull is strong," he commented.

Coming Changes

Character.ai states its new focus will be on providing "enhanced interactive features" and "narrative experiences" for teens.

Updated identity confirmation methods will also be implemented, and the organization will fund a new AI safety investigation facility.

An AI safety researcher, who has researched AI safety, stated it was "a sensible move" to restrict minors from using chatbots.

"It aids to differentiate creative play from more private, personally delicate interactions," she clarified.

"This is so crucial for young users in the process of learning to navigate psychological and technological restrictions."

She mentioned that Character.ai's new measures might reflect a "growing period" in the AI industry, where youth protection is gradually being acknowledged as an "pressing matter" for responsible innovation.