Polyglot-DC1: one architecture, many roles

Every frontier model works with a crew of small models around it. Part 1 followed one request through an AI service and met them at every step: one screens the prompt before the frontier model sees it, one drafts ahead while it writes, one picks an agent’s next tool call, and one reasons through a filing while a trader waits for the answer. The frontier model gets the headlines, but these small models decide how long a person waits, and today each of them is a different architecture, built, tuned and served on its own.

This post introduces Polyglot-DC1. We didn’t design specialized silicon to accelerate four unrelated models. Instead, we co-designed one model architecture and the silicon it runs on, then specialized that architecture through distillation for four distinct AI workloads.

Three principles guided our approach: smaller models should be specialized for the tasks they perform, model architectures and hardware should be optimized together, and a shared architectural foundation should support multiple workloads without requiring a new chip for each. The name says the same thing in two words:

Polyglot

One model architecture, fluent in many roles. The architecture is the same for every job; only the weights change.

DC1

Built for, and designed with, our data-center design, the chip that part 4 arrived at. The 1 marks the first generation.

Polyglot-DC1 comes in two sizes, 1.7B and 4B parameters. We explore how Polyglot-DC1 was designed, how its models are specialized, and what their results tell us about the advantages of ML–silicon co-design.

Small models, trained for the job

A frontier model is a generalist. It has to be ready for any question, so it carries hundreds of billions of parameters, even trillions. The jobs around it are the opposite: narrow, repetitive and well defined. A guard sorts one message into a fixed set of labels. A draft model predicts the next few tokens of one specific model. An agent’s step is usually a single tool call in a fixed format. Tasks like these are where a small model trained for the task can do as well as a model a hundred times its size, at a fraction of the cost to serve.

NVIDIA researchers made that case for agents in 2025, in “Small Language Models are the Future of Agentic AI.” Their argument is that agents use only a narrow slice of what a language model can do, and need strict, consistent output formats, which a model under about 10B parameters can be fine-tuned to deliver. They estimate that serving a 7B model costs 10 to 30 times less than serving one of 70 to 175B, in latency, energy and compute, and that 40% to 70% of the large-model calls in three open agent frameworks could go to a small model instead.

Newer results agree. In October 2026, Zhou and colleagues post-trained Qwen3-8B to select tools from a library of more than ten thousand; on ToolBench it beat Llama-4-Maverick, a 402B model. Guards show the same pattern. In the benchmarks we ran for this post, Qwen3Guard-Gen-0.6B, a dedicated guard with only 0.6B parameters, scores 88.1 F1 on English prompt classification, ahead of both Llama Guard 4 at 12B (75.8) and gpt-oss-safeguard-20b (86.3).

Our own models make the cleanest comparison, because we can set them against a general model of exactly the same size. Against Qwen3-1.7B, our 1.7B finance model scores 8.3 points higher on average across ten finance benchmarks, and our draft for Qwen3-Next keeps 26% more of the tokens it proposes (4.18 accepted per verification against 3.32, over nine benchmarks). Same size, same task: the model trained for the task wins.

The model and the silicon, designed together

A smaller parameter count doesn’t automatically translate into lower latency. Real performance depends on how efficiently the model architecture maps to the underlying hardware, and the field has started to design the two together. The method is an architecture search in which every candidate model is scored not just on accuracy but on how fast it runs on the target hardware. Puzzle used it to derive a model from Llama-70B that fits on a single H100 and runs 2.17 times faster while keeping 98.4% of the parent’s accuracy. Jet-Nemotron’s search reaches the accuracy of Qwen3, Gemma 3 and Llama 3.2 at 2B parameters while generating up to 53.6 times faster. On phones, Meta’s MobileLLM-Flash searches against latency measured on the phone itself and decodes up to 1.6 times faster at comparable quality, and MobileMoE sizes a mixture of experts to the phone’s memory and compute and decodes 2.2 to 3.4 times faster than a dense model that takes the same memory. A 2026 co-design scaling law takes its latency estimates from a roofline model of the target chip and finds an architecture with 19% lower perplexity than Qwen2.5-0.5B at the same speed.

These approaches demonstrate the value of hardware-aware model design, but they share a constraint: the underlying hardware is fixed.

At Exstill, we remove that constraint. Rather than optimizing the model for an existing accelerator, we explore the model architecture and silicon together, treating both as variables in the same design space. This allows decisions about the silicon and the model to inform one another from the outset.

As part 4 showed, our search measures every candidate’s tokens, their time and energy, on placed and routed circuits, varying the silicon architecture and the model architecture together. The chip that comes out is shaped around one architecture: its memory, its engines and its data paths are sized for that architecture and no other. That has a direct consequence. Optimizing silicon independently for every model architecture would fragment the hardware platform. A shared architecture allows us to specialize for different tasks without redesigning the silicon each time.

So Polyglot-DC1 is one architecture for every job, fixed in the silicon, and each job is a separate set of weights, made by distillation. The next question is how one architecture can learn to perform such different tasks.

Four jobs, four distillations

The four jobs are four different ML tasks. Drafting for speculative decoding is next-token prediction aligned to one target model, judged by how many of the draft’s proposals the target accepts. Guardrails are classification: a label for each prompt and each response (safe, controversial or unsafe), the harm category, and whether the assistant refused, across six languages. Agentic work is multi-turn tool use: reading a policy and a conversation, choosing the right tool call with the right arguments, and reasoning between calls. Real-time reasoning is chain-of-thought question answering over financial text and tables, where the model writes out its reasoning before it answers.

Each is its own set of weights for the same architecture. The challenge is not simply compressing a larger model into a smaller one. It’s preserving the capabilities that matter for a specific workload while discarding general-purpose capacity the task doesn’t require. Knowledge distillation provides the foundation for this specialization, allowing smaller models to learn from the behavior of more capable teachers: a larger model that already does the job well, the teacher, supervises a small model, the student.

In its original form (Hinton et al., 2015) the student is trained to match the teacher’s whole output distribution over the next token, the probability it gives every alternative, not just the one correct label, so every training token carries far more signal than a hard label does. For language models the teacher can also generate complete sequences for the student to imitate (sequence-level distillation).

The strongest recent methods go a step further and train on the student’s own generations, scored against the teacher, so that the student is corrected on the mistakes it actually makes rather than on text it would never have written (on-policy distillation). The choice of divergence matters as well: with forward KL the student spreads over the teacher’s whole distribution, with reverse KL it concentrates on the teacher’s most likely answers. Because each Polyglot-DC1 model serves one narrow task, the student can spend all of its capacity on the teacher’s judgment for that task, none on general ability it will never use.

There is a large literature on this, and its finding is consistent: a small model distilled for one task can reach, and sometimes pass, models several times its size. Apple’s distillation scaling laws show that when a capable teacher already exists, distilling a student beats training it from scratch, up to a compute budget that grows with the student’s size. DistiLLM-2 takes Qwen2.5-Math-1.5B from 27.1 to 42.9 on MATH, above its 7B teacher’s 41.3, and a 1.5B coding model from 30.7 to 42.2 on HumanEval. Agent Distillation distills a 32B agent’s whole behavior, retrieval and code tools included, into students of 0.5B to 3B that match students twice their size trained on reasoning traces alone. For drafting, DistillSpec showed that distilling the draft model from the target it serves, so that its guesses line up with the target’s choices, raises the speedup of speculative decoding by 10 to 45%. And on-policy self-distillation lifts Qwen3-1.7B from 37.1 to 43.4 on AIME and HMMT with far fewer training tokens than reinforcement learning needs.

Polyglot-DC1 applies this job by job: each of the four is distilled from systems that already do it well, into the one architecture the chip is built for. Distillation itself is well known. What is hard is making a single fixed architecture, chosen for the silicon rather than for any one task, match purpose-built models on jobs as different as these. Making that work across such different tasks requires more than conventional distillation. We’ve developed task-specific training techniques to specialize Polyglot-DC1 while preserving its common architecture. What the jobs share is the person or the agent waiting while the model writes, one token at a time, which is what the chip is built for: on the data-center design, each of twelve people gets 19,921 tokens a second, about 50 µs a token, pre-silicon. The times in the sections below are decode times, the time to write the tokens; reading the prompt in, the network and the rest of an application add to them, on our chip and on the GPU alike.

Speculative decoding: frontier answers, sooner

A frontier model writes one token per forward pass. For one person, each pass costs almost the same whether it handles one token or a dozen, because the time goes to reading the model’s weights from memory, not to the arithmetic. Speculative decoding exploits that. A small draft model guesses the next several tokens, and the frontier model, the target, checks them all in a single pass. It keeps the guesses that match what it would have written, up to the first one that doesn’t, and adds one token of its own. Under greedy decoding the result is exactly the text the target would have written alone. Only the time changes.

The draft has to run before every check, so the time it takes is subtracted from the gain. A 2026 study of speculative decoding for 1B to 2B models found that the drafter’s overhead, not the quality of its guesses, was the main limit on the speedup. On our chip, a 14-token draft takes 0.7 ms, well inside the 2.3 ms gpt-oss-120b needs to check it, and token IDs and lightweight control information cross the PCIe link between the two.

Many speculative decoding approaches depend on a particular target model’s internal activations, tightly coupling the drafter to the target. Polyglot-DC1-Draft takes a different approach. It is a standalone model that needs only the target’s tokens, not its internals, and works in the target’s own tokenizer. The same card that drafts for gpt-oss-120b can draft for DeepSeek-V4-Pro, Kimi K3, Qwen3-Next or Muse Spark Open by loading other weights. When a new frontier model appears, it needs new draft weights, not a new drafter design or a new chip. Simulated on a DGX B300 on SPEED-Bench coding, gpt-oss-120b (reasoning effort high) decodes 4.4 times faster with our draft, and Qwen3-Next 6.3 times faster.

Guardrails: screening every prompt and every response

A production assistant reads text it cannot trust and writes text that people act on. It needs a safety classifier on both sides: one that checks each prompt before the frontier model sees it, and one that checks each response before the user does. A guard that slows every request tempts teams to screen less, checking the prompt but not the response, or checking the response only after it has finished streaming, when the user has already read it.

Our guard writes its verdict in 10 to 15 tokens: safe, controversial or unsafe, the harm category, and, for a response, whether the assistant refused. It works in six languages: English, French, Spanish, Portuguese, German and Chinese. For a model of the 1.7B size, decoding that verdict takes under a millisecond on our chip, against 19 to 29 ms on an NVIDIA B300 serving one person: 38 times faster, and fast enough to screen a response chunk by chunk as it streams. On accuracy, the 4B is within 1.0 F1 point of Qwen3Guard-Gen-4B, a purpose-built guard of the same size, on every average, and it flags fewer harmless prompts.

Agentic: delays that add up

An agent works in a loop. It reasons, calls a tool, reads the result and decides what to do next, dozens of times in a single task. Each step has to wait for the one before it, so the delay of every step adds up over the whole task. Increasing concurrency can improve aggregate throughput, but it doesn’t necessarily reduce the latency of an individual agent’s sequential execution.

To measure what that costs, we timed the agent’s own decoding on eight customer-service episodes from τ³-bench, replayed turn by turn at the speeds of a model of the 1.7B size. An agent turn averages 1,145 tokens, reasoning and tool calls included: 2.5 s on one B300 against 57 ms on our chip. All eight episodes together, 123,677 tokens, take 266 s on the B300 and 6.2 s on the chip, 43 times faster. Reading the growing conversation into the model and the tools’ own delays come on top of both.

Real-time reasoning: an answer while it still matters

Reasoning models can generate thousands of intermediate tokens before producing a final answer. That sequential generation can dominate response latency, particularly for time-sensitive workloads. The longer the chain of thought, the more of the wait is decoding, one token after another: the work a chip built for a single stream speeds up most. In finance the wait matters, because an answer can lose its value within seconds: a quote, a risk check, a question about a filing that has just landed.

Decoding a 2,000-token chain of thought takes about 3.8 s on one B300 and about 0.1 s on our chip; reading the question and its documents in comes on top of both. Our finance model reasons through filings, valuation, risk and derivatives. The 4B averages 72.2 across ten finance benchmarks, ahead of all seven open models we compared, up to 8B in size, the best of which averages 68.2.

Choosing a size

The 1.7B is for jobs that run on every request. The chip figures above are for this size, with twelve conversations held on the chip at once, and as a guard it scores within 1.2 F1 points of the 4B. The 4B is for when accuracy comes first: in finance it averages 72.2 against the 1.7B’s 63.6, and as a guard it flags fewer harmless prompts. For one person it is nearly as fast. On a design our search found for the 4B, within the same 125 W, one person with a 16K-token conversation gets 18,059 tokens a second, against the 1.7B’s 19,921. The difference is capacity: that design holds one conversation at a time, where the 1.7B’s holds twelve.

What it unlocks

Polyglot-DC1 lets one chip do many jobs without being a general-purpose chip. Every job is a set of weights on the same silicon. The same card can draft for any of five frontier models, screen prompts and responses, run an agent or reason through a filing, and switching jobs means loading other weights. For an operator that is one part to qualify, one card to deploy and one supply chain, whatever mix of jobs the traffic needs.

That changes what a new job costs. A model hard-wired into a chip is the right answer when one model will run unchanged for years, but the work around the frontier model changes every few months. With one architecture, a new frontier model needs a new draft and a new domain needs a new distillation. Each is a training run on silicon that already exists, not a new chip.

This advantage compounds as we expand to new workloads. The silicon remains optimized for a shared execution architecture, while each new distillation extends the range of tasks it can support. The deeper challenge, and our differentiation, is making that same architecture perform effectively across fundamentally different applications.

The architecture is the part that connects all of this. The weights give Polyglot-DC1 its role; the co-designed silicon gives that role its execution path. The large target model can remain on the GPU, while the specialized work around it moves to dedicated inference hardware: the frontier model’s co-processor.

One architectural foundation. Different learned skills. Silicon built for both.