A small model that knows when to hand off cuts inference cost 96%
The cheapest way to get frontier-level math reasoning just got a working open-source demo: teach a 4B model to admit when it's stuck.
PyroMind Dynamics says its PyroDash method lets a Qwen3.5-4B model solve hard reasoning benchmarks at near-frontier accuracy while paying roughly 4% of what a big model alone would cost. The trick is a control token. The small model streams an answer normally, and the moment it decides it can't finish, it emits <|llm_offload|>. A Collaborate Engine stops decoding, strips the token, and hands the original question plus the partial reasoning trace to a frozen GLM-5.2, which continues from that point. Control never goes back — one request costs at most one big-model call. On five math benchmarks (GSM8K, Minerva, Olympiad, AIME24, AIME25), the paper reports that the accuracy-leaning variant beats GLM-5.2 running alone, while the cost-leaning variant cuts inference cost by about 96%, a figure the company's Chinese coverage pegs at $1.78 versus $49 for the same workload.
What makes this different from every router shipped in the last two years is where the decision lives. Request-level routing picks a model before generation starts, so a problem that turns out to be hard mid-solve has nowhere to go. Cascades can escalate after scoring, but they restart generation from scratch instead of continuing where the small model stopped. PyroDash decides token by token, and the small model decides, not an external classifier. Training is three stages: teach the control-token embedding, cold-start the handoff behavior with supervised fine-tuning, then GRPO reinforcement learning against a reward of accuracy minus a cost penalty scaled by λ. The cost term is the honest part — every token the small model writes before handoff is billed twice, once at its own rate and again as input context for the big model. That's why a single λ knob produces models with visibly different personalities: spend more big-model tokens, or spend less and accept the accuracy hit.
The scaffold is real and public: an arXiv paper (2607.20327), the EasyHard-24k dataset of 24,061 filtered easy/hard samples, and three checkpoints on Hugging Face. PyroMind's CEO Kevin Ding frames it as a bet on the "agent swarm" route to ASI rather than one gigantic centralized model — and points out that sub-100B local models account for the bulk of Hugging Face downloads. The open question is whether self-assessed difficulty survives contact with messy production traffic, where the hard part is often knowing you're wrong in the first place.
What to watch: whether anyone reproduces the 96% number on non-math workloads, where partial-solution handoffs are far less clean than a solved equation.
If you're running a 4B model behind a frontier fallback today, is the router your code or the model's own confidence? Tell us in the comments.
Sources: PyroDash (GitHub)
