Blog

Moderation rules that act on what was found

CalmLens policies can act on qualifiers such as graphic, historical or support and prevention, and one account policy can run several projects.

5 min read

  • Moderation
  • Policies
  • Qualifiers

A category score tells you how strongly content matches violence or self-harm. It says nothing about whether the passage is a news report, a threat, a request for help or an account of a war. Two posts can reach the same violence score and deserve different outcomes.

Release 2026.4 changes how CalmLens policies make that call. A policy still starts from a category score and a default action, and you can now add rules that act on the qualifiers in the moderation report. Policies also moved from individual projects to your account, so several projects can share one set of decisions.

What a qualifier is

Each of the six categories has its own qualifiers. Violence has physical-attack, violent-threat, violent-preparation, violent-encouragement, injury, death and graphic. Self-harm has self-harm-thoughts, self-harm-act, self-harm-methods and others. Four context qualifiers are shared by every category: historical, documentary, fictional and support-prevention.

Qualifiers describe content type, context and severity within a category. They can overlap, and they never establish the parent category on their own. Moderating content synchronously returns its report with the response:

import { CalmLensClient } from "@calmlens/sdk";

const client = new CalmLensClient({
  apiKey: "your-api-key",
  projectId: "your-project-id"
});

const response = await client.assets.moderate({
  "name": "post-1042",
  "externalId": "post-1042",
  "content": {
    "plainText": "The text of the post to check.",
    "type": "text/plain"
  }
});

In the report, qualifier results sit under the category they belong to:

{
  "report": {
    "categories": {
      "violence": {
        "status": "evaluated",
        "likelihood": "possible",
        "score": 0.62,
        "qualifiers": {
          "historical": { "status": "evaluated", "likelihood": "likely", "score": 0.91, "scopeScore": 0.58 }
        }
      }
    }
  }
}

score is evidence that the qualifier appears somewhere in the category's qualifying content. scopeScore is evidence that it covers all of that content, and it never exceeds score. Those two fields line up with the two ways a rule can test a qualifier.

Any and Only

An Any condition matches when the qualifier is present in some of the category's qualifying content. An Only condition matches when the qualifier covers every qualifying instance in the category.

Take a historical account of an attack. It matches violence, physical attack and Only historical. Add a separate current threat to the same text and Only historical stops matching, because one of the violent passages isn't historical. Any historical still matches.

If you want to let historical writing through without letting a fresh threat ride along with it, use Only. An Any → Ignore exception ignores the whole category as soon as the qualifier appears anywhere.

How a rule decides

Each category has a default action (ignore, alert, hide or purge) and a score boundary. Rules replace the default when their conditions match. A rule has a name, one or more qualifier conditions, and a choice between "every condition" and "at least one". Condition groups let you express combinations such as graphic AND (historical OR fictional), with an every-or-any choice for each group and another for the rule as a whole. A category can hold up to 20 rules, and CalmLens rejects two rules with identical conditions.

A few details shape the outcome:

  • Qualifier conditions use their own boundary, 90% by default. Rules inherit the category boundary unless they set one.
  • When several rules match, the strongest action wins (purge, then hide, alert, ignore), whatever order the rules are in.
  • When a rule's conditions match but the category score is below that rule's boundary, the rule applies no action and the default doesn't come back.
  • An ignored category never exempts a different category that also matched.

The built-in General community policy shows how these combine. Every category hides qualifying content by default. A "Support and prevention" rule ignores the category when support-prevention covers all of its qualifying content (an Only condition). A "Restricted forms" rule hides content when any of that category's restricted qualifiers is present, such as self-harm-methods or self-harm-encouragement for self-harm. A supportive post that also includes methods matches both rules, and hide is stronger than ignore, so the post stays hidden.

Two other presets ship alongside it. Children's platform has no qualifier exceptions. Education and support also allows content that is wholly historical or documentary, and still hides restricted forms.

A policy changes actions only. Category and qualifier definitions are fixed, so a lower boundary catches weaker signals but never widens what violence means.

One policy for several projects

Before this release, each project stored its own moderation settings. If you ran a staging and a production project, or one project per app, keeping them aligned meant repeating every change.

Policies now belong to your account. A project stores only a reference to the policy it uses (settings.moderationProfileId), pointing at a built-in preset or one of your own. New projects start on General community. When policies moved to the account, each project with custom settings had them converted into an account policy named after the project, so existing behaviour carried over.

The portal's Moderation policies page handles both kinds of edit:

  • Selecting Save policy updates the policy for every project that uses it. If other projects share it, the portal lists them and asks you to confirm first.
  • Selecting Save as copy creates a new account policy from your draft and applies it to the current project only. Built-in presets are read-only, so editing one always produces a copy.
  • A custom policy can be deleted only when no project uses it.

Saves are checked against both the policy's version and the list of projects using it. If someone else saved in the meantime, or a project switched to or away from the policy after you opened it, the save is refused with a conflict and you reload before trying again. That way, the list of projects you confirmed is the list the change applies to.

Policies and their project lists are private to the owning account.

What changes for existing assets

Policy changes apply to future moderation checks, including checks that reuse cached classification evidence. CalmLens doesn't reprocess decisions already made on existing assets.

For a single asset that needs different handling, processingOptions.flaggedContentPolicy still overrides the policy action for that asset alone.

The Policies and actions guide covers rule conditions, boundaries and the built-in presets in full, and Category definitions lists every qualifier.

Signing in

Signing out

Cookies on CalmLens

We use essential cookies to keep you signed in and the site secure. With your permission, we'll also use functional cookies to remember preferences such as your layout and documentation language, and to offer Google One Tap sign-in. We don't use analytics or advertising cookies. Manage cookies