Workflows
A workflow lists the categories to check and, for each, a reject threshold and a review threshold. Every check names a workflow with its workflow parameter, and default when it names none.
How a decision is made
Each category gets a score between 0 and 1. For each category the workflow lists:
- a score at or above
rejectrejects the content; - else, a score at or above
reviewsends it to review; - a category the workflow does not list is not checked at all.
The answer's summary.action is reject when any category reaches its reject threshold, else review when any reaches its review threshold, else accept. The thresholds are numbers from 0 to 1. A category needs at least one of the two, and its review threshold is not above its reject threshold.
The default workflow
Every organization starts with a default workflow, copied from openmod's own. It is semi-permissive: it rejects what is plainly unacceptable, sends the doubtful cases to review and lets everyday content pass.
| Kind | Rejected | Sent to review |
|---|---|---|
| Image | Nudity, gore, weapons, hate symbols and self-harm at 0.8. Extremist text, and links in text or QR codes to adult or unsafe sites. | The same five categories from 0.5. Drugs and minors from 0.5. Links to gambling sites. |
| Text | Unsafe text, violence, sexual content, hate, illegal activity and unethical content at 0.8. Bank account numbers, payment card numbers and unsafe links. | The same six categories from 0.5. Self-harm from 0.5. Email addresses, phone numbers, requests to continue on another messaging app, spam, scams, extremism, and links to adult or gambling sites. |
Two more templates are available when you create a workflow: strict, which checks everything openmod can, and nudity_only, for profile pictures. Organizations keep their own copy, so a change to a template never changes a workflow of yours. When a newer default is available, the dashboard shows what changed and offers to update, which makes a new version.
Editing a workflow
Edit workflows in the dashboard:
- pick the categories from the catalogue, one list for images and one for texts, and set a review and a reject threshold for each;
- add custom categories (below);
- see every version, and compare any two.
A name is 1 to 64 characters. Your plan sets how many workflows you can have: 3 on the free plan, 10 on Starter, 50 on Growth and 200 on Scale. A saved version compiles in the background: until it has, the previous version keeps serving checks, and the dashboard says the new one is waiting.
Custom categories
A custom category teaches a workflow something that is specific to your product, in plain words. You give it:
- an id: lower-case letters, digits and underscores, up to 64 characters, such as
competitor_mention; - up to 8 prompts: short descriptions of what to look for, each up to 200 characters;
- optionally up to 8 negatives: descriptions of what looks similar but is fine;
- a shift, from -20 to 20, to make the category stricter or more lenient.
A workflow can have up to 20 custom categories. They count as one check, like any other category. Once defined, a custom category appears in the workflow's category list, and you set its thresholds like any other.
Versions
Every save makes a new version. The answer says which one decided, in workflow_version, and so does the journal: you can always tell which rules were in force for a past decision.
Listing your workflows from the API
GET /v1/workflows lists the names and current versions of the workflows the key may use:
curl -H "Authorization: Bearer $OPENMOD_API_KEY" https://api.openshield.dev/v1/workflows
{
"status": "success",
"workflows": [
{ "name": "default", "version": 3 },
{ "name": "profile_picture", "version": 1 }
]
}
A workflow with no rule for the kind of content you send answers 503 workflow_unavailable for it: an image sent to a workflow that only has text rules, for example.