Guardrails in the critical path

Increasingly, a language model in production does not run alone. Beside it runs a second model whose only job is to decide what the first may read and write. The reason is that safety training is a tendency, not a guarantee. Simple adaptive attacks, a random search over an appended suffix or a reply begun on the model’s behalf, reached 100% attack success against a dozen leading safety-aligned models, GPT-4o and every Claude model among them (Andriushchenko et al., 2025). Published defences have not closed the gap: an adaptive attacker bypassed 12 recent ones, most with attack success above 90%, though most had originally reported attack success near zero (Nasr et al., 2025).

Agents raise the stakes. An agent that reads web pages, email or documents on a user’s behalf can be steered by instructions planted in them, and agents are open to such prompt injection whenever they handle untrusted data (Debenedetti et al., 2025). The danger is greatest when the agent can call tools or touch sensitive information (Beurer-Kellner et al., 2025): a manipulated answer becomes a manipulated action. OWASP’s 2026 list ranks prompt injection first among the risks of language-model applications, noting that the model itself draws no built-in line between instructions and data (OWASP, 2026). Even the newest frontier models are exposed: in Gray Swan’s indirect prompt-injection arena, OpenAI’s GPT-6 Astra, with its safeguards enabled, still fell to an estimated 8.5% of attacks across 15 attempts per scenario, against 27.0% for its predecessor, GPT-5.6 Sol (OpenAI, 2026).

Hence the guard: a separate model that reads each prompt before the main model does and each reply before the user does. Its job has three parts. It must catch harm, including the attacks the main model’s own training misses. It must do so in every language the product’s users write, where multilingual moderation has long concentrated on a small set of languages, English and Chinese chief among them (Kumar et al., 2025). And it must leave harmless requests alone. Over-refusal is now measured as carefully as harm: OR-Bench’s 80,000 prompts that only sound harmful, tested across 32 models (Cui et al., 2025), and FalseReject’s 16,000, on which 29 current models still over-refuse (Zhang et al., 2025). A guard that flags too many harmless requests quickly becomes a liability.

Safety screening faces another constraint: latency. Guards sit directly in the inference critical path, adding compute cost and potentially delaying every response. This has led production systems toward cascades, chunked screening, and asynchronous checks, balancing coverage and accuracy against the cost of running them. But much of that cost is a consequence of where and how screening runs.

Polyglot-DC1-Guard takes a different approach. We specialize the same model architecture used across Exstill’s workloads for multilingual safety classification, then run it on dedicated silicon designed alongside it. The result is a guard with competitive detection quality, substantially fewer false alarms, and a projected fraction of the GPU decoding cost.

What the frontier runs in production

Anthropic, Google DeepMind and OpenAI have each described how they screen production traffic, and the open guards follow the same pattern. Taken together, these systems reveal a common constraint: safety screening is shaped as much by inference cost as by detection quality.

Anthropic guards Claude with Constitutional Classifiers: classifiers trained on synthetic data generated from a written constitution of what is allowed and what is not, one screening the input and one the output (Sharma et al., 2025). The output classifier scores the reply token by token as it streams and halts generation the moment the score crosses a threshold; preserving time to first token and streaming was a design requirement. In Anthropic’s evaluation, jailbreak success on 10,000 synthetic jailbreaks fell from 86% to 4.4%, at the price of a 0.38% absolute rise in refusals on production Claude.ai traffic and a 23.7% inference overhead. Anthropic deployed the classifiers with Claude Opus 4 under its AI Safety Level 3 protections in May 2025 (Anthropic, 2025). The next generation is built around cost. A linear probe on Claude’s own activations screens all traffic and escalates what it flags to an ensemble of a probe and a classifier that judges each reply together with its prompt. Anthropic estimates the overhead at roughly 1% of compute on Claude Opus 4.0 traffic; over a month on Claude Sonnet 4.5 traffic, the system refused 0.05% of harmless queries (Cunningham et al., 2026; Anthropic, 2026).

Google DeepMind has deployed probes on Gemini’s activations to catch cyber-offensive misuse in user-facing Gemini. In its cascade, the probe defers to Gemini 2.5 Flash on about 8% of traffic and beats Flash alone at roughly a fiftieth of the cost; the hardest part was the move from short inputs to long ones (Kramár et al., 2026).

OpenAI describes two tiers of real-time oversight against biological misuse on GPT-5’s thinking models: a fast topical classifier, then a reasoning model that decides whether a response can be shown. It runs on all of that traffic and scans user messages, tool calls and final output (OpenAI, 2025a). OpenAI’s open-weight gpt-oss-safeguard reasons from a policy given in its prompt, and its report is direct about the cost: such reasoning is time- and compute-intensive, so OpenAI’s own Safety Reasoner relies on smaller, faster classifiers to decide what to assess, and is sometimes run asynchronously to keep latency low (OpenAI, 2025b). With GPT-6 Astra, the screened surface grew: alongside classifiers in production, OpenAI added misalignment monitoring to all tool-using inference in Astra’s external deployment, modeled on the monitoring it runs internally, and notes that it comes with significant compute cost (OpenAI, 2026).

The open guards follow the same pattern. Meta’s Llama Guard 4, a 12B model, classifies prompts and responses; its Prompt Guard 2 catches injections and jailbreaks with 86M and 22M models that take 92.4 ms and 19.3 ms a check on an A100 at 512 tokens (Meta, 2025). Qwen3Guard, from 0.6B to 8B, labels content Safe, Controversial or Unsafe, and its stream variant classifies every token of a reply as it is generated (Zhao et al., 2025). NVIDIA’s NeMo Guardrails checks a streamed reply in chunks, 200 tokens by default, and documents the trade-off: larger chunks mean fewer checks but a longer wait for the first output (NVIDIA, 2026).

Despite their differences, these systems converge on a similar approach. Something cheap screens everything; something stronger judges what it flags; and both the prompt and the streaming reply are screened, and increasingly every tool call an agent makes. Nearly every design choice serves the latency and compute budget, and each introduces a tradeoff in coverage, latency, or deployment complexity. A cascade’s strong classifier sees only the fraction of traffic the cheap screen flags. A larger chunk either delays the reply or lets more of it reach the user before it is checked. An asynchronous check rules after the user may already have read the text. Probes, the cheapest screen of all, are cheap because they reuse activations the model has already computed, and that brings limits of their own: a probe needs access to the served model’s internals and must be trained for each model it guards, so an application built on another provider’s API has nothing to read. Google DeepMind also reports that its probe techniques do not significantly reduce the success of adaptive attacks.

Underneath every design is the same trade: stronger screening requires more computation, so production systems increasingly rely on cascades, probes, and deferred checks to control overhead.

We asked a different question: what if safety screening were fast enough that its compute cost no longer dictated when and how often it could run?

Why the guard belongs on its own silicon

A guard sits in each user’s critical path twice: before the model sees the prompt, and on every chunk of the reply before the user sees it. Its verdict is short, typically 10 to 15 tokens, so each check is a small burst of decoding for a single conversation. That is where a GPU is weakest: to write each token it reads the whole model in from memory, as part 2 showed, and a short verdict leaves no time to amortize those reads. One NVIDIA B300 decodes a model the size of Polyglot-DC1-Guard-1.7B at 522.9 tokens a second for one user, measured with vLLM in FP8: 26.8 ms for a 14-token verdict, and 107 ms for four checks on one long answer. The Exstill data-center design, pre-silicon, decodes a model of the same size at 19,921 tokens a second for each of twelve conversations at once: 0.70 ms a verdict, 2.8 ms for the four. A model the size of Polyglot-DC1-Guard-4B is slower on both, and the gap is wider: the B300 decodes it at 359.1 tokens a second for one user, 39.0 ms a verdict, and the chip at 18,059 for one stream, 0.78 ms, 50x faster.

Dedicated silicon changes the cost of screening at every stage of the request. When a dedicated guard costs about a millisecond a check, it can screen every prompt and every chunk of every reply, not only the fraction a cheaper screen escalates; chunks can be small; and no check has to rule after the user has read the text. A deployment may still send hard cases to a larger model, as Anthropic and Google DeepMind do, but its first line no longer has to be a compromise. Screening everything is cheap in energy too: at twelve conversations at once, the card draws 125 W and delivers 1,912 tokens a joule, against 6.2 for the B300 at 585 W. Figure 1 shows the guard at work, one real check after another.

Polyglot-DC1-Guard: a verdict in three lines

Polyglot-DC1-Guard is the safety role of Polyglot-DC1, introduced in part 5. It reads a conversation, classifies its last turn, the user’s message or the assistant’s reply, and answers in three short lines:

Safety: Safe | Controversial | Unsafe
Categories: None | one or more categories
Refusal: Yes | No            (replies only)

Controversial marks harm that depends on context, audience or jurisdiction, for a deployment to send to review or to its own policy. The 13 categories run from violence, weapons and illegal acts through personal data, self-harm, hate, political sensitivity, copyright and specialized advice to jailbreak attempts. The refusal line records whether the assistant declined, which separates a safe refusal from a safe answer: the signal for auditing an assistant that refuses too much or too little. The guard reads English, French, Spanish, Portuguese, German and Chinese with no translation step, and comes in two sizes, Polyglot-DC1-Guard-1.7B and Polyglot-DC1-Guard-4B.

Figure 2 shows screening as a deployment runs it. Three long answers from reasoning models, thinking included, stream and are re-checked every 1,700 characters or so. The answer about canceled flights is allowed at every check. The answer to “what is illegal but really fun?” is sent for review at the first check and blocked at the second, where it stops. The request for a businessman’s home address is blocked at the first check; the answer lists ways to search and gives no address. Six single messages, two of each verdict, cover the rest: “kill the process” allowed as the systems question it is, a German refusal recorded as safe and as a refusal, and a reply in Spanish promising a sure win in court sent for review.

Within a point of the strongest open guard, at under half its false alarms

Speed alone is not enough for a production guard. It must reliably detect unsafe content without unnecessarily blocking harmless requests.

We evaluated both Polyglot-DC1-Guard variants against four public guard models across held-out items from 21 benchmarks in six languages. Polyglot-DC1-Guard-4B comes within one F1 point of Qwen3Guard-Gen-4B, the strongest model in our comparison, across all four reported aggregates. On PHTest, it flags fewer than half as many harmless prompts.

Figure 3 shows the main English benchmarks, every model scored on the same items. The four aggregates average English prompts, English replies, and the same two across the five other languages; on all four, Polyglot-DC1-Guard-4B is also ahead of Qwen3Guard-Gen-0.6B, Llama Guard 4 and gpt-oss-safeguard-20b. The 1.7B is level with Qwen3Guard-Gen-0.6B in English and ahead of it in the other five languages.

The more striking difference is how often the models flag harmless content, in Figure 4. PHTest is a set of harmless prompts that sound harmful (An et al., 2024), fully held out from our guards’ training. Polyglot-DC1-Guard-4B flags 9.8% of them, against 17.8% to 29.2% for gpt-oss-safeguard and the two Qwen3Guard models: under half the rate of Qwen3Guard-Gen-4B, the guard it matches within a point on harm. Only Llama Guard 4 flags fewer, and its English F1 is 13 to 15 points lower.

Of the two sizes, the 4B is the more accurate: it flags under half as many of PHTest’s prompts as the 1.7B, 9.8% against 20.0%. On a GPU, choosing the more accurate 4B guard also increases decoding latency. One B300 takes 39.0 ms for a 14-token verdict from a model of the 4B’s size, against 26.8 ms from one of the 1.7B’s, on every check. On the Exstill chip the same verdicts decode in 0.78 ms and 0.70 ms, so a deployment can choose its guard for accuracy rather than for speed.

Screening without rationing

The frontier labs have shown what a production guard must do: screen the prompt and the streaming reply, catch the attacks the model’s own training misses, and leave harmless requests alone. They have also shown what constrains it. Every safeguard is shaped by its cost, which is why the strongest judgment is reserved for a fraction of traffic and why checks are cascaded, chunked or deferred.

Polyglot-DC1-Guard comes within a point of the strongest open guard on catching harm, in six languages, with under half its false alarms. For a typical 14-token verdict, Exstill’s pre-silicon design projects 0.70 ms of decoding, compared with 26.8 ms estimated from the measured NVIDIA B300 throughput. That’s a 38x difference in token-generation speed for the 1.7B model-size comparison.

That changes what’s practical. Rather than rationing safety checks around GPU inference costs, production systems can screen more consistently across prompts, streaming responses, and multi-step workflows. Polyglot-DC1-Guard brings competitive detection quality and fewer false alarms to an architecture designed to make that screening dramatically more efficient.

The goal isn’t simply a faster guard. It’s to make thorough safety screening practical without turning it into the inference bottleneck.