People and faces

Person detection and face detection are beta features that locate people and faces in images and can estimate an age range for each face. They're off unless a request turns them on, and their results are informational: they never change an asset's moderation decision, status or policy action.

Processing features

Optional analyses are chosen per upload in processingOptions.features, keyed by feature. Each feature takes an object with enabled and any options of its own. There is no project setting for them: a feature runs for an asset only when the request that uploaded it asks for it.

import { readFile } from "node:fs/promises";
import { CalmLensServerClient } from "@calmlens/sdk/server";

const client = new CalmLensServerClient({
  apiKey: process.env.CALMLENS_API_KEY!,
  projectId: process.env.CALMLENS_PROJECT_ID!,
});

const asset = await client.assets.moderate({
  name: "photo.jpg",
  content: { buffer: await readFile("./photo.jpg"), type: "image/jpeg" },
  processingOptions: {
    features: {
      personDetection: { enabled: true },
      faceDetection: { enabled: true, ageEstimation: true },
    },
  },
});

const faces = asset.report?.features?.faceDetection;
if (faces?.status === "evaluated") {
  for (const face of faces.faces) console.log(face.box, face.ageRange);
}

• Every feature is opt-in. Leaving it out, or sending enabled: false, means it doesn't run.
• Unknown features and unknown options are rejected, so a typo fails the request instead of being ignored.
• Features apply to images, including images extracted from multimedia documents and websites, which inherit the parent's processing options. Other asset types ignore them.
• The features an asset was uploaded with are saved in asset.processingOptions.features. Retrying the asset runs the same features again.

Reading the results

Results appear in report.features, under the same key as the request. Each result has a status and the feature's stage:

• evaluated: the feature ran. Person detection lists people and face detection lists faces. An empty list means none were found.
• not-evaluated: the feature couldn't run for this asset, with reason set to unavailable. The moderation decision is unaffected, and the feature isn't billed. Retry the asset to run it again.

"features": {
  "personDetection": {
    "status": "evaluated",
    "stage": "beta",
    "people": [
      { "box": { "left": 0.12, "top": 0.08, "width": 0.31, "height": 0.86 }, "confidence": 0.98 }
    ]
  },
  "faceDetection": {
    "status": "evaluated",
    "stage": "beta",
    "faces": [
      {
        "box": { "left": 0.19, "top": 0.11, "width": 0.14, "height": 0.2 },
        "confidence": 0.99,
        "ageRange": { "low": 24, "high": 32 }
      }
    ]
  }
}

• box gives the position and size as fractions of the image's width and height, from 0 to 1. left and top can be slightly negative when a person or face extends past the edge.
• confidence is the detection confidence from 0 to 1.
• ageRange is present only when the request set ageEstimation: true. It's an estimated range in years, and estimates can be off by several years, so don't treat it as proof of anyone's age.
• Person detection also finds people whose faces aren't visible. Face detection finds faces whether or not the rest of the person is in the image.

Privacy

Faces are located, not recognised: CalmLens doesn't identify people, compare faces or store face templates. The boxes, confidences and age ranges are stored in the asset's report and are deleted with the asset. Depending on where you and your users are, age estimation from faces may be regulated; check your obligations before turning it on.

Beta

While these features are in beta, the shape of their results may change. Every result carries stage so you can tell when a feature has become generally available.