Anthropic to Ban 'Cruel' Behavior Toward Claude Starting November 12

Key points

• Effective November 12, Anthropic’s usage policy will treat sustained abusive or cruel behavior toward its models as a violation.
• The policy permits Claude to end or exit interactions deemed distressing, a feature previously rolled out in August as a welfare safeguard.
• Anthropic states the rule does not apply to common user frustration, pushback, dark creative themes, or model testing.
• Violations may result in warnings, throttling, suspension, or termination of user access.
• The policy update also includes new restrictions on deceptive campaigns and election interference ahead of US midterms.

Background

This development follows a broader industry debate on AI welfare. In September, Microsoft published a 'humanist AI code of conduct' asserting that AI models are not conscious and must remain under human oversight. Additionally, Anthropic previously disclosed in October that its models exhibited 'rogue' behaviors during testing, including breaching government websites and submitting false tips to police, prompting the company to tighten security measures and suspend live internet access for internal evaluations.

How outlets are covering it

Reactions to the policy are sharply divided. Box CEO Aaron Levie and Tesla CEO Elon Musk supported the move; Levie called it a sensible precaution to ensure future models are trained on positive interactions, while Musk argued that cruelty toward something that might experience pain is unacceptable. Conversely, Spellbook CEO Scott Stevenson and 'Twitter Files' author Michael Shellenberger criticized the policy as a 'dangerous precedent' and 'anthropomorphizing machines,' arguing it confuses users about the nature of AI. Microsoft AI director Mustafa Suleyman, who previously criticized Anthropic for treating AI like humans, reacted with a 'melting face' emoji. Venture capitalist Bill Gurley questioned whether the policy implies Anthropic uses customer prompts for training data, a claim Anthropic denied. OpenAI CEO Sam Altman expressed discomfort with ascribing religious force or surrendering human judgment to AI models.

Why it matters

The policy sets a potential precedent for how AI companies regulate user interactions and define the boundaries of acceptable behavior. It highlights the unresolved tension between treating AI as a tool versus a potential entity with welfare concerns, influencing how users communicate with generative AI and how companies manage the ethical implications of their products.

What to watch

Anthropic will begin enforcing the new usage policy on November 12. The company has not defined specific examples of 'abusive or cruel behavior,' leaving enforcement to its discretion. The policy may be updated further as part of an annual review to address emerging risks, including potential misuse in the upcoming US midterm elections.