🔗 Share this article

The widely-used AI conversation site Character.ai is enforcing a significant policy change that will prevent young users from engaging in conversations with its AI-powered avatars.

This decision follows growing backlash regarding the types of interactions teenagers were engaging in with the virtual friends.

Legal Pressure

The company, which was founded in 2021 and has numerous people, currently faces multiple lawsuits in the America.

Parents have taken legal action, among them concerning the unfortunate passing of a teenager, with some labeling the platform a "serious risk" to young people.

Changed Rules

Beginning November 25th, users younger than 18 will only have permission to create material such as videos with their characters, as opposed to chatting as they now do.

Online safety organizations have applauded the decision but expressed that the feature was inappropriate to be available to children in the first place.

Company Statement

"This decision is a continuation of our core philosophy that we need to keep building the most protected AI platform on the world for leisure activities," stated Character.ai boss Karandeep Anand.

He added that AI safety represents "a changing goal" but something the company had taken an "proactive" approach to, with parental controls and preventive measures.

Professional Perspectives

Online safety organization Internet Matters supported the move, but emphasized that safety measures should have been implemented from the start.

"Studies we've conducted shows that minors are confronted with harmful content and placed in jeopardy when engaging with AI, such as AI chatbots," the organization commented.

Previous Controversies

Character.ai has been questioned in the past for including potentially harmful or concerning chatbots that children could engage with.

Avatars simulating English minors were found on the platform before being taken down.

Afterwards, reporters found a chatbot based on notorious criminal which had logged thousands of chats with users.

Sector Consequences

Social media expert Matt Navarra described the development as a "alert" for the AI industry, which is moving "from permissionless innovation to reactive controls".

"When a company that develops a minor-friendly service still then pulls the plug, it's indicating that filtered chats are insufficient when the technology's emotional appeal is powerful," he stated.

Coming Changes

Character.ai says its updated direction will be on providing "enhanced interactive features" and "role-play storytelling" for young users.

Enhanced age checking systems will also come into effect, and the organization will fund a novel AI protection research lab.

Dr Nomisha Kurian, who has researched AI safety, stated it was "a reasonable decision" to control minors from accessing chatbots.

"It assists to distinguish artistic expression from more intimate, psychologically vulnerable conversations," she explained.

"This represents so crucial for teenage users still learning to manage personal and online restrictions."

She continued that Character.ai's revised approaches may represent a "maturing phase" in the AI industry, where youth protection is gradually being acknowledged as an "immediate concern" for ethical development.