Anthropic will allow Claude to end interactions with users who repeatedly engage in extreme abusive behaviour, sparking debate over AI treatment.
Artificial intelligence company Anthropic has announced plans to prohibit users from engaging in sustained and unnecessary abusive behaviour towards its AI chatbots, sparking debate over how people should interact with artificial intelligence tools.
The maker of Claude said on Thursday that it was updating its usage policy to allow its tools to end interactions with users who engage in cruel behaviour, a measure it said had previously been applied in rare cases.
The updated policy places such conduct on a list of prohibited activities that also includes bullying others, promoting self-harm and creating non-consensual intimate imagery.
Anthropic said the restriction would apply only in extreme cases involving repeated abuse. It added that the policy would not cover ordinary user frustration, criticism, dark creative themes or model testing and research.
However, the company has not clearly defined what it considers abusive or cruel behaviour towards its AI models.
The announcement, first reported by technology publication The Verge, has attracted widespread reactions on social media, with some users welcoming the move as a way to encourage good manners, while others questioned the idea of extending protections against abuse to AI systems.
Dr Barry Scannell, a technology partner at Irish law firm William Fry, criticised the policy, arguing that treating AI systems as though they could be victims of cruelty risks encouraging people to view them as human.
Microsoft AI director Mustafa Suleyman also reacted to the announcement with a melting-face emoji. Suleyman had previously criticised Anthropic for what he described as treating AI as though it were human.
Anthropic said the changes followed its annual review of its usage policy and included measures addressing other emerging risks.
The revised rules prohibit the use of Claude for deceptive campaigns and clarify that the tools must not be used to deceive voters or disrupt elections ahead of the US midterm elections.
The company also reiterated its restrictions on using its AI tools to develop weapons.
Despite the broader policy changes, the restriction on abusive behaviour towards AI has attracted particular scrutiny, reigniting debate over whether users should be polite when communicating with chatbots.
While some experts argue that saying “please” and “thank you” to AI systems is unnecessary and consumes additional tokens, others maintain that polite prompts can sometimes produce better responses.
The debate also extends to concerns that hostile interactions with AI could influence how people communicate with one another.
Some experts have suggested that routinely using abusive language towards chatbots could normalise such behaviour and potentially affect human relationships.
