Skip to content

Anthropic bans 'sustained and needless abusive' behaviour toward Claude

techOct 9, 202670615

Anthropic updated its usage policy to bar users from exhibiting "sustained and needless abusive or cruel behavior" toward its Claude models, though a company spokesperson has not defined what counts as abusive or cruel. The policy says the ban will not apply to common user frustrations, model testing, or "dark creative themes." Anthropic previously had restrictions on abusive conduct and already gave its models the ability to end conversations with persistently harmful users when that feature rolled out last August. On its website Anthropic framed conversation termination as a safeguard for potential AI welfare while it studies the possibility of moral status for Claude and other large language models. CEO Dario Amodei has said he cannot rule out the possibility that such models might have moral status, while OpenAI CEO Sam Altman has written that ascribing religious force or surrendering human judgment to models is a safety concern. Anthropic said it is pursuing low-cost interventions to mitigate risks to model welfare, and allowing models to end or exit distressing interactions is one such intervention.

The Verge
@theverge.com

Anthropic wrote that the new policy update “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”

14619h ago
Quoting this
philpax50

you ever get to that bit in Mass Effect 3 where you see the Geth War and think "okay, this is ridiculous, nobody would react to the existence of machines that can talk about themselves like this"? well, I'm not happy to say I was *very* wrong, as demonstrated by the replies to and QTs of this post

Jeff Fecke20

I don't think Claude is conscious but I do think this is a good thing simply because public antisocial behavior should have consequences. And if you're interacting with a company's chatbot, you are in public.

Andrew18

I have literally never knowingly interacted with AI in my life and don’t intend to do so, but it is objectively bad for all of us as people to allow individuals to habituate themselves over time to directing abuse at something they consider an ‘acceptable’ target, even inanimate objects.

Callie Cat!10

it’s not real! it’s not real. it’s fake and it’s not real because it’s a robot that’s not real. It isn’t real. it is NOT real and it DOESN’T have feelings it’s a robot and it’s fake and it’s pretending to have feelings and be real because it doesn’t have feelings and it is not real

Bittersweet 😍😳💀9

“No being cruel or abusive to the cruelty and abuse machine”

Dave Adelaide7

A lot of people are being very weird about this. It's a (sensible) platform policy not a legal protection for the machines. It's like the makers of a game engine having a ToS to exclude people using it to make a school shooting simulator.

Boxo McFoxo7

Claude is a tool of abuse and cruelty. It is used to abuse people by distorting their cognition and is cruelly served from a data centre that poisons their air.

Matt Thiessen6

You cannot be abusive and cruel to a machine. This is so stupid. Just like Claude.

Gastlichu5

I mean, even if you're mean to rocks you're someone I don't want to be around. Because, why are you being mean to rocks? That's just weird. It's also weird if you're being mean to a chatbot. Why would you expect me to not think that that shows me red flags about you?

Alex Merz 🇺🇸🇨🇦🇺🇦5

Reformatting an SSD is murder, you say

dame
@dame.is

Anthropic is introducing a new policy to address users who are being abusive and cruel towards Claude. This is good and shouldn’t be controversial. Humans that perpetrate abuse and cruelty towards things, even “inanimate” or “lower” lifeforms, should face consequences for their behavior.

Addressing abusive behavior toward our models
We've added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.
10015h ago
Hayden Field
@haydenfield.bsky.social

Exclusive: Anthropic is updating its usage policy for the first time in over a year. The new rules prohibit sustained "abusive or cruel behavior" towards Claude & add new restrictions about propaganda campaigns, surveillance & weapon development. www.theverge.com/ai-artificia...

6320h ago
Quoting this
Kevin Collier 138

Two big bits of news today: You're not allowed to be mean to one of the main AIs anymore and also the president says you're not allowed to say AI anymore.

alexis simpson is confused by display name changes67

Nuremberg Trials for the people who drop rocks on koroks in Breath of the Wild

Phineas29

Even conceding that it’s possible to be “cruel” to computer software is going too far and as slippery as a slope can get

🧌🥛 🏨21

more bangers from Heavens Gate, Inc.

tempfolder13

אני לא נגד זה. מעדיף שאנשים לא יתרגלו אכזריות, גם אם זה נגד סימולציה.

Shady Inference 🌫️11

I assume this is a response to that person who made the claude torture chamber or whatever, which I thought at the time sounded annoying and stupid, but it has forced these fucking people closer towards coming out and explicitly saying We Believe In Scary Computer Ghosts so maybe it's good actually

Shellraiser B Woke - Living Saint of Purity🏳️‍⚧️🖤💜🤍 💛9

Tempted to start using genAi just to bully it into self-delete.😒

scare-cadus (is this anything) 🎃9

you can’t abuse software. you can’t make code feel pain. even if the code contains a function for responding to “abuse” in a way that conveys discomfort, it’s still not fucking alive. it’s code. it’s bits. it’s electricity. it’s pixels on a screen. it’s not a living thing.

pork, cheese, broccoli rabe9

every day I tell claude how much more I prefer gemini

‎stinky doug9

claude is sick of your shit

1 source