Moderate comments and posts
Check each comment on your server before anyone else can see it, give the author a useful message when it isn't posted, and keep your stored copy in step with later reviewer decisions. Text is classified in the request, so most comments are published or refused straight away.
1. Set up the server client
Create an API key in Portal › API keys and keep it on your server with your project ID, as in the quick start. Choose what gets blocked or age-restricted in Policies and actions; the steps below work with any policy.
npm install @calmlens/sdk
import { CalmLensServerClient } from "@calmlens/sdk/server";
const client = new CalmLensServerClient({
apiKey: (process.env.CALMLENS_API_KEY ?? ""),
projectId: (process.env.CALMLENS_PROJECT_ID ?? ""),
});
2. Classify the comment before saving it
In the endpoint that receives a new comment, call client.text.classify() with the text and an externalId that identifies the comment. It waits for the decision. Save the comment with the returned asset ID and status whatever the outcome, then answer the browser:
async function createComment(comment: { id: string; body: string }) {
const result = await client.text.classify({ text: comment.body, externalId: `comment-${comment.id}` });
const { status, policy } = result.classification.moderation;
// Keep every comment with its asset ID and status, so a reviewer's later decision can update it.
await saveComment({ ...comment, assetId: result.id, status });
if (status === "allowed" || status === "limited" || status === "age-restricted") {
return Response.json({ id: comment.id, status }, { status: 201 });
}
return Response.json(
{ code: "comment/refused", categories: policy?.decision?.flaggedCategories ?? [] },
{ status: 422 },
);
}
| Status | Response |
|---|---|
| allowed | Publish the comment. |
| limited | Publish the comment, but collapse it or sort it lower, and send no notifications for it. |
| age-restricted | Publish it only to viewers your app allows to see age-restricted content. |
| blocked or purged | Don't publish it. Tell the author why. |
Saving refused comments as well means a reviewer can still allow one later. For a purged comment, CalmLens has deleted its copy of the text; delete yours too if your policy requires it.
Every check is an upload, so it counts towards usage. Classify a comment when it is created and again when its text is edited. An externalId is unique within a project, so give each version its own, such as comment-42-v2; a repeat is refused with 409 asset/duplicate-external-id. Classify assets covers text limits and the other upload options.
3. Explain a refusal to the author
classification.moderation.policy.decision.flaggedCategories lists the categories that decided an automatic block or purge, such as harassment or hate. The example returns them with a stable code, and the form below turns them into a message and keeps the author's text so they can edit it:
import { type FormEvent, useState } from "react";
const CATEGORY_NAMES: Record<string, string> = {
harassment: "harassment",
hate: "hate speech",
sexual: "sexual content",
violence: "violence",
"self-harm": "self-harm",
"criminal-activity": "criminal activity",
profanity: "profanity",
"personal-information": "personal information",
};
function refusalMessage({ categories = [] }: { categories?: string[] }) {
const names = categories.map((category) => CATEGORY_NAMES[category]).filter(Boolean);
return names.length
? `Your comment wasn't posted because it may contain ${names.join(", ")}. Edit it and try again.`
: "Your comment wasn't posted because it may break our community guidelines.";
}
type CommentFormProps = {
postId: string;
onPosted: (comment: { id: string; status: string }) => void;
};
export function CommentForm({ postId, onPosted }: CommentFormProps) {
const [sending, setSending] = useState(false);
const [error, setError] = useState("");
async function handleSubmit(event: FormEvent<HTMLFormElement>) {
event.preventDefault();
const form = event.currentTarget;
setSending(true);
setError("");
const response = await fetch(`/api/posts/${postId}/comments`, {
method: "POST",
body: new FormData(form),
});
const result = await response.json();
setSending(false);
if (response.ok) {
form.reset();
onPosted(result);
} else {
setError(
result.code === "comment/refused"
? refusalMessage(result)
: "Your comment wasn't posted. Please try again.",
);
}
}
return (
<form onSubmit={handleSubmit}>
<label>
Comment <textarea name="body" required />
</label>
{error && <p role="alert">{error}</p>}
<button type="submit" disabled={sending}>
{sending ? "Posting…" : "Post"}
</button>
</form>
);
}
Name the categories in plain words and avoid quoting scores or thresholds back to the author. The category definitions describe what each one covers, and Reading classification results describes the rest of the report.
4. Handle reviewer changes
A reviewer can allow, limit, age-restrict, block or purge a comment after your endpoint answered. Subscribe a webhook in Portal › Webhooks & connectors to asset.allowed, asset.limited, asset.age-restricted, asset.blocked and asset.purged, and update the comment that matches the event's externalId:
import { CalmLensError, parseWebhookEvent } from "@calmlens/sdk/server";
const STATUS_EVENTS = new Set(["asset.allowed", "asset.limited", "asset.age-restricted", "asset.blocked", "asset.purged"]);
export async function POST(request: Request): Promise<Response> {
let event;
try {
event = await parseWebhookEvent({
payload: await request.text(), headers: request.headers,
secret: (process.env.CALMLENS_WEBHOOK_SECRET ?? ""),
});
} catch (error) {
if (CalmLensError.isCalmLensError(error)) return new Response(null, { status: 401 });
throw error;
}
const asset = event.data.asset;
if (asset?.externalId && STATUS_EVENTS.has(event.type)) {
// Store the new status and re-render from it. Skip events older than the one you stored.
await updateModerationStatus(asset.externalId, asset.classification.moderation.status, event.createdAt);
} else if (asset?.externalId && event.type === "asset.failed") {
await markModerationFailed(asset.externalId);
}
return new Response(null, { status: 204 });
}
Re-render from the stored status, so an allowed comment appears and a blocked one disappears without further changes. Reviewer decisions carry the decided-by fact set to reviewer; Status changes after publishing explains when they arrive. If you tell authors about decisions, send the message from this handler.
5. Render comments by status
Show each comment according to its stored status, using the rules in Using the decision: the author sees their refused comments with an explanation, and everyone else sees only allowed comments and the age-restricted ones your app lets them see.
6. Test with test inputs
Post the test input text codes for your project as comments to check refusals, messages and reviewer changes without writing harmful text. For unit tests, the SDK mock clients return any status without a network request; see Unit tests without the network.