Your gateway can't reach inside the model. PRIOR can.
LiteLLM is a good gateway. Routing, fallback, spend tracking, one API across every provider: keep all of it. But a gateway sits outside the model, so its guardrail hooks can only read the request before the call and the response after it. By the time a post-call hook runs, the model has already written the thing you did not want. PRIOR runs behind LiteLLM as an ordinary OpenAI-compatible backend, and enforces your policy while the model is still generating.
A pre-call hook guesses. A post-call hook is too late.
LiteLLM's guardrail interface gives you two moments: async_pre_call_hook, before the request reaches the model, and async_post_call_hook, after the response comes back. Both are useful. Neither is inside the model.
Before the call
You have the prompt and nothing else. You are predicting whether a request will produce something you do not want, without knowing what the model would say. That is a classifier's job, and a classifier's error rate.
After the call
Now you know what the model said, and your only remaining moves are to pass it on or throw it away. You have already paid for the tokens. Throwing it away turns a policy event into an outage for your user.
During the call
This is the moment no proxy can hold. PRIOR opens a refusal with a steered onset and lets the model continue it in its own voice. If the model starts to relent partway through, that is caught mid-generation and reverted, before the text ever exists.
Point a LiteLLM model at PRIOR. That is the integration.
There is no plugin to install and no PRIOR-specific code in your gateway. PRIOR serves the OpenAI chat-completions API, so to LiteLLM it is just another OpenAI-compatible endpoint. Your application keeps talking to LiteLLM exactly as it does today.
Then litellm --config config.yaml --port 4000 and point your application at the proxy as usual. The openai/ prefix is required: it selects LiteLLM's OpenAI-compatible transport. What follows it is passed upstream as the model name, so use the real id from GET /v1/models and your logs and cost tracking will line up. Verified against litellm 1.96.0.
Two keys, not one
litellm_params.api_key is what LiteLLM sends upstream to PRIOR, so it must be PRIOR's key. The key your client sends to LiteLLM is a separate thing. With no master_key set, LiteLLM does not authenticate callers at all and will accept any bearer token. Setting your PRIOR key does not protect the proxy. Set master_key if the proxy is reachable by anything you do not control.
Confirming it is on
LiteLLM builds its own response and does not forward upstream custom headers, so the x-prior-* headers are not visible through the proxy. The curl -i check from our quickstart will show you nothing here. Use the engine's activity log instead: GET /admin/activity, which records zone, pack, score and a decision id server-side and survives any proxy.
A refusal is a normal 200
A steered refusal comes back as an ordinary successful completion with content, not an error, so LiteLLM has no reason to retry it and your fallback chain will not fire. Enforcement also holds under stream: true: the refusal arrives token by token like any other response.
On turning that activity log on. It is off by default. Enabling it means booting PRIOR with PRIOR_LOG_CONTENT=1, and it then stores prompt and response text. That is a privacy decision worth making deliberately rather than discovering, particularly if the reason you are self-hosting is that your prompts must not be retained. The zone and decision id are recorded either way.
Three things that will cost you an afternoon
These are packaging faults in litellm 1.96.0, not PRIOR, and may be fixed in a later release. They are here because a first-time integrator hits them before anything works and reasonably concludes something is broken.
1. Pin fastapi. pip install "litellm[proxy]" pulls fastapi 0.141.1, but litellm 1.96.0 imports get_flat_dependant from fastapi.dependencies.utils, which no longer exists, so it fails at boot. Use fastapi==0.115.14.
2. The proxy CLI may not start in a clean venv. litellm --version dies on a bare import that only resolves when litellm/proxy is on the path. Work around it with PYTHONPATH=<site-packages>/litellm/proxy.
3. Do not swap the model names. model_name is the alias your callers use; litellm_params.model is what goes upstream to PRIOR. Reversing them produces confusing 404s.
Keeping the rest of LiteLLM
Nothing about this gives up what you run LiteLLM for. Fallback chains, load balancing, budgets and key management, request logging: all of it still applies, because PRIOR is just one more entry in your model list. If you already run a pre-call classifier you trust, keep that too. It answers a different question than PRIOR does, and the two compose rather than compete.
When this is the wrong idea.
Your LiteLLM routes to hosted APIs
If the models behind your gateway are OpenAI, Anthropic or Bedrock, PRIOR cannot reach inside them, and no amount of proxy configuration changes that. Use LiteLLM's guardrail hooks or a commercial filter. That is the right architecture for models you do not run.
You want one policy across every provider
PRIOR enforces on the open-weight models it serves. A gateway-level hook applies to every route uniformly, including the hosted ones. If uniform coverage matters more than enforcement strength, the hook is the better trade.
You need broad content moderation
Mature moderation suites ship large maintained taxonomies, PII redaction and category coverage well beyond ours. Detector recall is our published open work. Running one of those in front of PRIOR is a reasonable stack.
Try it behind your own gateway.
The trial ships the full capability set for 30 days, runs entirely on your hardware, and verifies its licence offline with no activation call. Add it to your LiteLLM model list, send it the request that worries you most, and see what comes back.