Anthropic bars cruel treatment of Claude AI

Anthropic has updated its usage policy to bar users from sustained and needless abusive or cruel behavior toward its Claude AI system.

The San Francisco-based AI lab said Claude's ability to end these interactions will remain the main enforcement mechanism. The updated policy does not explicitly cite “model welfare,” the idea that AI systems might deserve protections usually reserved for living things.

Anthropic had already said last year that Claude could end conversations in rare, extreme cases of persistently harmful or abusive user interactions. In February, CEO Dario Amodei told The New York Times he was unsure whether AI models could be conscious, saying: “We don't know if the models are conscious... But we're open to the idea that it could be.”

Jackson Stakeman, a general manager at Atlanta-based AI services provider Sparq, told AFP that the consciousness debate may be the wrong focus. “These systems reflect what we put in, at scale. That's reason enough for the policy change,” he said.

Others reject the idea of machine consciousness. Microsoft AI chief Mustafa Suleyman wrote last month that AIs “are not conscious” and warned against granting rights and moral protections to technological entities. Pope Leo XIV, speaking Thursday at St. Peter's Basilica in Vatican City, suggested machines lack a soul and merely compile data quickly.