Apodex launches TRACES, a benchmark that evaluates AI on real scientific discovery
Most AI benchmarks test whether a model can recall the right answer. TRACES, introduced today by Apodex, asks a harder question: can an AI system actually investigate a problem it hasn't seen before, form hypotheses, use tools, and arrive at a verifiable solution?
The benchmark comes with a technical report on arxiv and a press release backed by multiple outlets, giving it more credibility than the usual "we made a leaderboard" announcement. What makes TRACES different is its scope and its evaluation philosophy. Apodex surveyed 561 industries across 16 sectors, assembled 423 high-value real-world problems, and selected 20 for the initial release. Each problem comes with a full environment — data, tools, constraints, feedback loops, and verification of intermediate artifacts — not just a question and an answer key.
The evaluation framework, called HDS6, scores six dimensions independently: Tools (can the system use available instruments effectively?), Repair (can it recover from errors?), Alternatives (does it explore multiple paths?), Coherence (do its steps form a logical chain?), Evidence (does it ground claims in data?), and Scope (does it stay within the problem's boundaries?). This is a meaningful departure from single-score benchmarks, because a model could ace the final answer while failing badly on the investigative process — and TRACES would catch that.
The early results are concrete. In AAV capsid design — a real problem in gene therapy where engineering the right viral shell is critical — Apodex's system surpassed the published state of the art by 7% across viability, tropism, structure prediction, and generative design. In drug repurposing and reformulation, wrapping GPT-5.5 and GPT-5.6-sol in a task-specific biomedical environment improved their mean normalized prediction scores by 2.5 and 7.6 points respectively, compared to running the same models closed-book. The improvement comes not from a bigger model but from a better investigation harness — exactly the point TRACES is designed to measure.
The broader significance is about where AI evaluation is headed. Current benchmarks like MMLU or HumanEval test knowledge and code generation in controlled settings. TRACES argues that the real frontier is "discoverative AI" — systems that can navigate messy, open-ended problems the way a research scientist would. That framing matters because it shifts the conversation from "which model scores highest" to "which system can actually do useful work in the real world." If the benchmark gains traction, it could change how labs prioritize their safety and capability evaluations, especially for high-stakes domains like medicine and materials science.
What to watch: whether other labs adopt TRACES for their own evaluations, and whether the 20-problem initial release expands quickly enough to cover the full 423-problem set Apodex has assembled.
Should AI benchmarks evaluate the investigation process, not just the final answer? Tell us in the comments.
