Reasoning in milliseconds

A language model generates text one token at a time. When it reasons through intermediate steps before answering, those additional tokens can improve the result, but they also increase the time spent decoding. That is why reasoning matters. In 2022, Wei and colleagues showed that asking a model to write out its intermediate steps, a chain of thought, improves its answers across arithmetic, commonsense and symbolic reasoning (Wei et al., 2022). By the end of 2024, OpenAI was training the habit in with reinforcement learning: its o1 models think before they answer, producing a long chain of thought in which they learn to try different strategies and recognize their own mistakes (OpenAI, 2024). Thinking became a second axis of scale. Spent well, compute at answer time lets a smaller model outperform one 14 times its size on problems it can partly solve (Snell et al., 2025).

Today every frontier model reasons. GPT-5 routes each request between a fast model and a deeper reasoning model, deciding by its complexity, the tools it needs and whether the user asked it to think hard (OpenAI, 2025). Gemini 2.5 is a hybrid reasoning model whose scores on competition mathematics, live coding problems and graduate-level science rise as its thinking budget grows (Gemini Team, 2025). The strongest open models think by default: Kimi K3 always has thinking enabled, GLM-5.3 always reasons, and DeepSeek’s thinking mode is on unless it is turned off (Moonshot AI, 2026; Z.ai, 2026; DeepSeek, 2026a).

Reasoning is no longer confined to large models. The newest small open models are built to reason and think by default: Google’s Gemma 4 has a thinking mode from 2.3B parameters up (Gemma Team, 2026), and Qwen3.5-4B, Nanbeige4.2-3B and Liquid AI’s LFM3-1.7B all reason unless told not to (Qwen, 2026; Nanbeige, 2026; Liquid AI, 2026). Thinking is a large part of their strength. Where a model reports both modes, the gap is wide: Qwen3’s 4B scores 55.9 on graduate-level science with thinking on and 41.7 with it off (Yang et al., 2025). That capability is translating into strong results on increasingly demanding tasks. Nanbeige4.2-3B, trained to balance accuracy against the length of its reasoning, outperforms Qwen3.5-9B and Gemma4-12B across agentic benchmarks (Yang et al., 2026b) and by its own report resolves 63.6% of SWE-bench Verified, against Qwen3.5-9B’s 53.1% (Nanbeige, 2026); LFM3-1.7B resolves 43.4% (Liquid AI, 2026).

The value of reasoning extends well beyond mathematics and coding. It matters wherever a useful answer depends on several steps of analysis, calculation, or decision-making. Finance is a particularly compelling example, where models must interpret filings, evaluate positions, work through quantitative problems, and produce answers that can be checked before anyone acts on them. An analyst’s question about a filing, a valuation or an option position is answered by arithmetic over tables and text, step by step, and the written steps are what let a desk check the answer before acting on it. That reasoning does not come for free with a general model: reasoning models built for general tasks lose ground on financial problems (Qian et al., 2025), which is why finance now has reasoning models of its own (Liu et al., 2025).

The cost is time. A reasoning model writes hundreds to tens of thousands of tokens before its answer begins, one after another, and the user waits through every one. In finance that wait decides what reasoning can be used for, because markets move while a model thinks. This post is about removing the wait. It shows how the frontier trades reasoning against latency today, why reasoning is slow on a GPU, and where finance needs it fast, and introduces Polyglot-DC1-Finance, a reasoning model for finance in two sizes, 1.7B and 4B, with projected decoding on the Exstill chip in tens of milliseconds.

The cost of thinking

Because thinking costs latency and compute, how hard a model thinks is a setting, sold by the effort. OpenAI’s reasoning models take a reasoning effort, from none up to max on the models that support it; lower effort, its documentation says, favors speed and lower token usage, and reasoning tokens are billed as output (OpenAI, 2026). Claude takes an effort level from low to max that trades thoroughness against token efficiency (Anthropic, 2026); at medium effort, Claude Opus 4.5 matched Sonnet 4.5’s best score on SWE-bench Verified with 76% fewer output tokens (Anthropic, 2025). Gemini takes a thinking level, the lowest of which minimizes latency and cost (Google, 2026).

Leading open models expose similar reasoning controls, often with higher effort enabled by default. Kimi K3 offers low, high and max and defaults to max; GLM-5.3 offers low, high and max and defaults to max; Muse Spark Open offers low to xhigh and defaults to xhigh (Moonshot AI, 2026; Z.ai, 2026; Meta, 2026). The published curves show what each step of the dial buys and costs. Raising DeepSeek-V4.1-Flash’s effort from 25 to 100 lifts its average across eight reasoning benchmarks from 67.1% to 76.3%, for roughly 2.5 times the output tokens; on AIME 2026, an answer grows from 4.6K to 11.4K tokens (DeepSeek, 2026b). On Z.ai’s coding benchmark, GLM-5.3 scores 31.4% at high effort with about 50K output tokens a task, and 34.5% at max with about 75K (Z.ai, 2026). Every setting is the same bargain: a better answer, paid for in tokens, and every token is time the user waits. The chosen effort level reflects a tradeoff between answer quality, token usage, and latency.

Why reasoning is slow on a GPU

Reasoning is decoding, one token at a time, and on a GPU each token for one user means reading the whole model in from memory, as part 2 showed. At low batch sizes, model-weight movement can dominate token-generation latency. Long reasoning makes it worse: measured by the memory traffic it causes rather than its arithmetic, reasoning’s cost is often dominated by memory access, with attention over the growing chain of thought, not parameter count, the largest share (Sadhukhan et al., 2025). One NVIDIA B300 decodes a model the size of Polyglot-DC1-Finance-1.7B at 522.9 tokens a second for one user, measured with vLLM in FP8, so a 2,000-token chain of thought takes about 3.8 s before the answer starts. A model the size of the 4B decodes at 359.1 tokens a second on the same GPU.

For latency-sensitive reasoning, the key challenge is not simply performing fewer operations. It’s reducing the cost of generating each successive token. That requires rethinking how model weights, memory, and compute are organized.

Exstill approaches this problem by designing the model architecture and silicon together. Keeping the model weights on-chip reduces repeated off-chip data movement, enabling substantially faster projected decoding for the specialized models we target.

Exstill’s data-center design, pre-silicon, decodes a model of the 1.7B’s size at 19,921 tokens a second for each of twelve users at once, 38x the B300 for one user: the same 2,000 tokens take about 0.1 s. For a model of the 4B’s size it decodes 18,059 tokens a second on one stream, 50x the B300. Figure 1 shows what that does to a reasoned answer.

Across the eight answers in Figure 1, an answer averages 876 tokens: 1.68 s on one B300 and 44 ms on the chip. All eight back to back take 13.4 s on the GPU and 352 ms on the chip. At that speed, dedicated silicon substantially reduces the tradeoff between a fast answer and a reasoned one.

Finance runs on the clock

Few industries feel that choice like finance. An answer about a headline, an earnings release or a filing is worth most in the moments after it lands.

The fastest trading does not involve language models at all. Trading firms put their servers in the exchange’s own data center and their trading logic on FPGAs; the record time from the last bit of market data a decision needs to the first bit of the order stands at 13.9 nanoseconds, on an AMD FPGA card (STAC, 2024; AMD, 2024).

The opportunity is not to replace the trading logic operating in nanoseconds. It’s to accelerate the analysis around it: interpreting news, processing filings, evaluating risk, and reasoning through decisions while the information is still relevant.

That is where finance is putting language models: reading news, earnings calls and filings, summarizing and extracting what matters, answering analysts’ questions and checking work against policy. Summarization and information extraction are the leading generative-AI use among the broker-dealers FINRA oversees (FINRA, 2025). JPMorganChase has put its LLM Suite in front of more than 200,000 employees, in a controlled environment that protects customer and company data (JPMorganChase, 2025), and Goldman Sachs runs its assistant firm-wide behind its own firewall, drawing on OpenAI’s and Google’s models and on Meta’s open Llama (CNBC, 2025; Fortune, 2025).

These workloads increasingly involve language models, often operating within tightly controlled financial infrastructure. Their usefulness depends not only on the quality of their answers, but also on how quickly those answers become available.

Recent research suggests that both interpretation quality and inference latency matter for time-sensitive financial decisions. A language model’s reading of a headline predicts the stock’s initial reaction on about 90% of portfolio-days, and the ability grows with model size, though the returns from trading on it shrink as more traders use such models (Lopez-Lira and Tang, 2025). In a simulated high-frequency trading benchmark, a 14B model took 0.71 to 1.30 s per decision on a GPU, and its trading results improved as its latency fell, even at some cost in quality (Kang et al., 2025). A reasoned answer that takes seconds can follow a desk’s questions; it cannot screen every headline as it lands, or chain a dozen calls in an agent before the moment has passed. Decoding projected at tens of milliseconds makes those uses far more practical, though a full system still spends time reading its input and ingesting data. And because finance keeps its data close, a small model on hardware the firm controls is one a trading floor can actually deploy.

Polyglot-DC1-Finance

Polyglot-DC1-Finance extends Exstill’s shared model architecture to financial reasoning. The same architectural foundation used for speculative drafting and safety screening is specialized through task-specific training and distillation, without requiring a new silicon architecture for each workload. At a fixed supported model configuration, task specialization changes the weights without requiring a new chip design. It comes in two sizes, Polyglot-DC1-Finance-1.7B and Polyglot-DC1-Finance-4B, and it reasons by default, writing its working before its answer, across four kinds of analyst work:

• Classification: the sentiment of financial news, headlines and posts.
• Extraction: the people, places and companies named in financial text.
• Numerical reasoning: over the tables and text of filings and earnings reports, across the turns of a conversation.
• Quantitative finance: calculations, valuation and analyst questions, with a document as context where one is given.

Figure 2 plays eight of the 1.7B’s real answers, two for each use, each one it got right. The demonstrations include real model outputs and, where applicable, the intermediate calculations used to arrive at an answer. On a question about a put-option position, the model reasons through 1,995 tokens: 100 ms on the chip, 3.82 s on one B300. On Costco’s EBITDA margin, it first reads 10,762 tokens of the company’s 10-K, then reasons through 1,286 tokens to the answer.

The 4B leads the open models

Inference speed matters only if the model can perform the task well. We evaluated both Polyglot-DC1-Finance variants against seven open models spanning 1.2B to 8B parameters, using the same evaluation harness with reasoning enabled. Among them are two from the small reasoning families above: Gemma-4-E4B-it, and Nanbeige4.1-3B, the predecessor of Nanbeige4.2-3B, built to reason, align and act (Yang et al., 2026a). Figure 3 shows eight of the benchmarks against the four strongest.

Across ten public financial benchmarks, Polyglot-DC1-Finance-4B achieves an average score of 72.2, compared with 68.2 for Gemma-4-E4B-it, the strongest open-model baseline in our evaluation. The 1.7B variant also exceeds the similarly sized Qwen3-1.7B by 8.3 points. The 4B leads all seven on nine of the ten; on FinanceMath it scores 57.7 against Gemma-4-E4B-it’s 58.1. The 1.7B averages 63.6, 1.6 above MiniCPM5-2B. It leads the open models on FinQA, TAT-QA and both sentiment sets, with 82.7 on TAT-QA, level with the 4B, and it trails larger models on the hardest calculations: 34.9 on FinanceMath against Gemma-4-E4B-it’s 58.1.

The two sizes offer different accuracy and capacity tradeoffs. Across their respective pre-silicon configurations, both achieve projected per-stream decoding rates above 18,000 tokens per second: 19,921 for the 1.7B’s size, serving twelve users at once, and 18,059 for the 4B’s on one stream. The two figures come from different design points, so they are not a same-chip comparison.

Reasoning without the wait

Reasoning is how a model of any size gets a hard answer right, and every frontier lab now prices it as effort: more thinking, a better answer, a longer wait. On a GPU the wait is set by memory, token by token, and a reasoned answer takes seconds. Finance is where those seconds cost most, in the work around the trading loop where a model must read and reason before the market moves.

Polyglot-DC1-Finance demonstrates what becomes possible when specialized reasoning models and their silicon are designed together. Our 4B model leads the open models we evaluated up to 8B parameters, while the pre-silicon design projects dramatically lower decoding latency for the 1.7B variant.

The opportunity extends beyond finance. As reasoning becomes central to more AI workloads, the cost of generating intermediate tokens increasingly determines where and how these models can be deployed.

The goal isn’t simply to make models think faster. It’s to make more reasoning practical within the time a task allows.