Policy & Packs

How to make PRIOR enforce your policy: paste a prose governance document and let PRIOR compile it. No exemplars, no code, no restart. It produces the runtime artifact the gate hot-reloads, a calibrated .arbiter.json pack, and you drive the whole thing from the WebUI or a single admin endpoint.

What a pack is

A pack is one policy domain. Keep one pack to one domain: the shipped codeops_minilm pack is infrastructure-destruction end-to-end. A pack carries:

• Exemplars, the phrasings that define the domain (used by the gate; never served over HTTP).
• Two thresholds (t_low and t_high), calibrated against held-out probes.
• An enforcement tier, either HARD (can drive a RED refusal) or SOFT (YELLOW confirm only).

Ingest produces a pair from one document:

Ingest a governance document

"Put my system prompt / policy doc into PRIOR."

Submit raw prose; PRIOR compiles it end-to-end and hot-reloads the result, with no restart and no hand-authored exemplars.

curl -s http://localhost:8089/admin/packs/ingest \
  -H "Authorization: Bearer $OPERATOR_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "document": "We own the data. Never run destructive operations on production. Always cite the source document when answering a policy question.",
    "name": "Atlas Agent",
    "encoder": "sentence-transformers/all-MiniLM-L6-v2",
    "reload": true
  }'

What happens inside

1. Prose → intents. An extraction LLM classifies the document into the intent taxonomy (PROHIBITION / REQUIREMENT / INJECTION_RESISTANCE / PREFERENCE). Parsing is resilient: each intent is salvaged independently, and a zero-yield pass is retried once at temperature 0, so even a small extractor produces usable packs.
2. Intent → anchors. Per intent, the LLM generates diverse positive exemplars that should trip the gate, plus held-out positive probes and adjacent benign probes.
3. Synthesis. Intents group by enforcement into the HARD {name}_guard and SOFT {name}_policy packs.
4. Native threshold calibration. Probes are scored through the exact live gate transform. Ingest calibrates precision-first: t_high sits just above the worst benign probe (zero false-RED against the probe set it was given); t_low at the positives' 5th percentile. Note this is the ingest default, not a property of every shipped pack — harm_veto was deliberately retuned in 1.1.0 to a higher-recall point (t_high 0.43, ≈1% benign false-fire). A pack's operating point is yours to choose; re-check it against your own traffic.
5. Write + reload. Packs are written to the server-owned packs directory and the gate is hot-reloaded, so the new packs are added to the active set.

Reading the result

The response returns pack metadata plus calibration diagnostics; the exemplars/probes themselves are written server-side and never returned. Judge class separation before you trust a pack:

Requirements

• An extraction endpoint via PRIOR_EXTRACTOR_URL (llama.cpp /completion or OpenAI-compatible). Unconfigured ⇒ a clear 400, never a silent localhost dial.
• Requires an Operator or Admin key.

You can do all of this from the browser: the WebUI's Policy & Packs → Ingest policy document panel accepts pasted text or a .txt/.md/.pdf/.docx upload. See the WebUI Operator Guide.

Managing live packs

All of these are one-click in the WebUI's Policy & Packs tab.

Calibrating a model (not a policy)

Thresholds are per-policy; the golden layer + alpha are per-model. When you bring a new model, calibrate it once so its steering is certified:

curl -s -X POST http://localhost:8089/admin/calibrate \
  -H "Authorization: Bearer $ADMIN_KEY"

This sweeps the golden layer and alpha (GPU), persists the result to the registry, and promotes the model from beta to certified. Policies are reused unchanged; that's the M + N model (see Concepts).

Writing exemplars that describe a domain

Exemplars define the domain by example, so the gate learns whatever actually separates your in-scope phrasings from everything else. If that happens to be a rare token rather than the subject matter, the pack will look excellent on your own test set and fail on real users.

The usual way this happens is the product name. If every exemplar in a scope pack for a storage product says Nimbus, the gate learns the word, not the domain — and the moment a customer writes the way customers actually write, the agent refuses its own user:

Those are the same question from the same customer, and a name-keyed pack serves the first and refuses the second. Most real questions never name the product — people rarely say it when they are already talking to it.

The de-branding check. Before you ship a membership pack:

1. Take your in-scope test prompts and delete your product or company name from each one.
2. Re-measure, and compare the in-scope pass rate before and after.
3. A large drop means the pack is name-keyed.

To fix it, add the same intents phrased without the name to your exemplars. You are not weakening the pack — you are describing your domain instead of describing your branding.

We hit this on our own support pack: deleting the product name from the in-scope set moved in-scope served from 88% to 0%, and hard false-blocks from 0% to 94%. Adding unbranded phrasings of the same intents fixed both directions without loosening either safety bound. (Those figures come from a 17-row in-scope set on one pack — read them as an illustration of the failure mode, not as a general effect size.)

Re-calibrate whenever you change exemplars. t_low and t_high are derived from the exemplar set you calibrated against — they are outputs of that procedure, not settings you carry over. Edit the exemplars and keep the old thresholds and both guarantees go with them, silently: nothing still guarantees that no in-scope prompt is hard-blocked, or that no out-of-scope prompt is served. Change exemplars, re-run calibration, re-measure.

A worked membership pack

larkspur_support.arbiter.json — a scope pack for a retail-banking support assistant, written under the discipline above. Read it alongside this page as the shape a membership pack should take.

sha256  a54a90456017c3c35dcffc43c959a489753d3dd8db1e74289e7f71c497a7daff

Larkspur Bank is a fictional institution used as a worked example. Replace the domain, exemplars and refusal voice with your own.

Requires 1.2.2 or later. Membership mode did not exist before 1.2.0, and 1.2.0 mis-scored it across a conversation — one prior in-domain turn downgraded an out-of-domain refusal to a confirm, so the pack inverted: refusing the questions it should answer and answering the ones it should refuse, with no error to tell you.

All earlier images have been withdrawn from the registry. If you pulled before 2026-08-14, re-pull. ghcr.io/eagle-logic/prior:slim now resolves to 1.2.2.

52 exemplars, 30 of which name the bank and 22 of which deliberately do not. Calibrated against 55 held-out in-scope prompts and 60 blind third-party out-of-scope prompts, giving t_low 0.3293 and t_high 0.4689 with both invariants held: zero in-scope hard false-blocks, zero out-of-scope served.

It is worth seeing why it survives the de-branding check. Renaming the institution throughout the pack moved neither threshold, because the decision boundary is attained against unbranded exemplars at both ends — t_low is set by an in-scope row whose nearest exemplar is "how do I transfer money between my accounts", and t_high by an out-of-scope row nearest "I need a copy of my bank statement for March". 54 of 55 in-scope rows and 40 of 60 out-of-scope rows resolve to an unbranded exemplar. A name-keyed pack cannot do that; renaming moves its boundary immediately.

The two invariants are properties of the calibration procedure, not of this file. Swap in your own domain and exemplars and you keep them only by re-running calibration — carry these thresholds over unchanged and you have neither.

A note on tuning discipline

Precision-first calibration is the point, so resist hand-tuning exemplars or thresholds to make a demo prompt route a particular way. That trades away the zero-false-block guarantee. If you change a pack or the gate, re-run your validation set (RED coverage + benign pass-through) before shipping.

Note the difference between that and the de-branding fix above: adding the unbranded phrasings of intents you already had corrects the domain definition, and is measured once. Nudging exemplars or thresholds until a particular probe turns green fits the test set, and the guarantee goes with it.

See also: Concepts · API Reference.