AllSpark's Iris beats every open-weight search agent — and says its scaffolding did the work
A paper out of AllSpark Research claims the strongest open-weight deep-research agents in their size classes, and the authors' most interesting admission is not a benchmark number: it is that half the story belongs to the harness wrapped around the model.
AllSpark released Iris-mini and Iris-pro, two open-weight search agents it says beat every comparable open model on multi-hop browsing benchmarks. The recipe is the unusual part. Instead of scraping question-answer pairs, the team builds training tasks backward from the hyperlink structure of a web corpus: pick a seed page, walk its outgoing links into an entity graph, then generate a multi-hop question whose answer requires chaining several of those links. Every entity except the final answer gets rewritten as a descriptive reference, so no clue can be resolved by string matching — the agent has to reason, not grep. Only questions a reference model fails closed-book but solves once the right sources are in hand survive into the dataset. Solution paths come from a stronger teacher model, then pass two rounds of filtering: a whole-path check for correctness and repetition loops, and a step-by-step judge whose criteria were derived from the data itself. Training alternates supervised fine-tuning with reinforcement learning against live web search — "SFT-RL climbing" — feeding the hardest solved tasks and most efficient rollouts back into the next round, with the judge and summarizer running inside the cluster on the team's own large Qwen model so nothing depends on external APIs.
The scores: with context management enabled, Iris-pro reaches 88.6 on BrowseComp and 56.4 on Humanity's Last Exam; Iris-mini hits 82.2 and 52.3, leading its class on three of four benchmarks and beating XYZ-Aquila-mini by 3.4 points on BrowseComp. Both are single ReAct agents — no sub-agents, no test-time verification.
The caveat the authors raise themselves deserves more attention than the leaderboard entry. Context management moves BrowseComp by up to 21.2 points on the smaller model — not because its token budget is tighter, but because it burns context faster and hits the limit more often. That gap is larger than most reported differences between competing systems, which means any score published without an ablation is measuring the scaffolding as much as the weights. AllSpark reports both conditions, which is more than the field usually does. The paper also flags a BrowseComp-ZH item where the agent answered "Bolton" against a "Lannister" ground truth and was marked wrong — the character does marry Ramsay Bolton.
Related: we covered a comparable open-weight claim last month — A 33B model from Singapore just matched DeepSeek V4 Pro — on free tokens.
What to watch: the weights and the full data-construction recipe are promised, not yet shipped. Reproduction on someone else's harness is the real test.
Is a benchmark score meaningful when the scaffolding can move it 21 points? Tell us in the comments.
Sources: The Decoder · arXiv: Iris — Climbing to the Search Frontier · AI Weekly · Iris (GitHub)
