Anthropic Adds Cruelty Toward the Model to Its List of Prohibited Uses

Anthropic Adds Cruelty Toward the Model to Its List of Prohibited Uses

Anthropic updated Claude's usage policy with a clause that is uncommon in the industry. The document, also available in Portuguese, prohibits continued and unnecessary abusive or cruel behavior toward the models. The change was reported on October 12.

The company clearly delimited the scope. According to the text, the rule applies only in extreme cases.

However, the target is users who repeatedly act cruelly with no discernible purpose.

What Falls Outside This Rule

This part matters for those who test models. The policy avoids covering common situations of user frustration or pushback.

Dark creative themes are also excluded. In addition, research and model-testing activities remain permitted.

Therefore, red teaming and adversarial evaluation remain viable. Note that the criterion revolves around repetition and lack of purpose.

Until now, highly offensive language could already end the conversation immediately. However, under the new guidelines, there is still a lack of clarity about tougher measures against these users.

Anthropic Avoided Explaining the Reason for the Change

The company left out the justification. Users on Reddit, however, raised two hypotheses.

The first concerns social habit. In addition, the concern would be to prevent abusive language from becoming normalized in the chat and later appearing in other environments.

The second hypothesis is of more interest to developers. The model absorbs vocabulary and tone from the instructions it receives.

Consequently, cruel interactions could feed into training and affect behavior toward the public.

The Other Prohibitions Added to the Same Document

The update brought a much broader list. However, this kind of guideline revision happens rarely.

Among the new items is bullying against other people. Also included are the promotion of self-harm and the creation of non-consensual intimate images.

The list continues with interference in electoral processes and deceiving citizens at the time of voting. Finally, weapons development and surveillance activities are included.

These points are likely connected to a recent report from the company. In it, attempts to use Claude to research biological weapons and commit cybercrimes were detected.

What This Means in Practice for Product Builders

First, review your system prompts. Personas that simulate continuous hostility deserve a second look in light of the new text.

Second, document the purpose of your tests. Adversarial evaluation with a clear objective remains within the rules.

Third, think about the content your users submit. An application that forwards messages directly to the model inherits responsibility for that traffic.

In addition, keep track of the full list of prohibitions. Surveillance and weapons development were explicitly included, which affects certain sectors.

Finally, maintain your own filters. A policy layer in your backend protects the account before any sanction from the provider.

Follow our profile on Instagram!