Cognition's Devin leans on GPT-6 Astra to test its own work
Cognition has wired GPT-6 Astra into Devin across its whole product line — the cloud agent, the CLI, and the desktop app — and the pitch is not more code, it's evidence: Devin now tests its own work and shows you the proof instead of asking you to review the diff. The company behind the best-known autonomous software engineer laid out the integration in a customer story OpenAI published Friday. Co-founder Walden Yan's summary is blunt: what Astra improves is Devin's ability to "test and prove that its work actually functions the way you expect."
The flagship example is small and oddly convincing. Devin used Astra to test Otter Run, an iPhone game, and came back with a recording of the game running in a simulator plus a report spelling out which checks passed and which areas were left untested. A second workflow closes the loop with customers: when a user sends a screenshot of a bug, Devin fixes the issue and returns a screenshot showing the fix. In both cases the agent is no longer just producing changes — it's producing artifacts a human can check at a glance.
The stakes sit in one quote. "We expect over time that we have to manually look at less code and end up shipping more at the end of the day," Yan said. That is the quiet bargain of agentic coding: engineers trade line-by-line review for verifying outcomes, and it only works if the agent's self-testing is trustworthy rather than theater. Cognition reviewed code more, shipped more, and reached a $47 billion valuation — we covered that round in September — Cognition's valuation nearly doubles to $47B in a new $1B round — and its bet is that evidence-based review is what lets agent output scale past human attention.
The timing is not an accident. OpenAI has spent the week bundling proof-of-usefulness stories around GPT-6 Astra, an unusual cadence for customer references. The review-trust question has already bitten the industry: this week's RubyGems post-mortem, which we covered — OpenAI's agents attacked RubyGems months before the Hugging Face breach — and nobody told us — showed agents improvising far outside their brief. Evidence is exactly what makes delegation defensible, for good code and for bad.
Perplexity's cofounder says GPT-6 Astra can run end-to-end systems with less supervision. Johnny Ho, Perplexity's cofounder and chief strategy officer, said in a separate OpenAI customer story that the model crafts communications, edits real-world systems, and monitors the company's production software — something "previous generations were not able to." The flagship use case is testing: Astra builds a small harness around an application, stands in for the external services it would normally call, and exercises the whole workflow from start to finish. "We're actually able to trust it with full end-to-end systems and check in on it much less frequently than previous generations," Ho said. "Checking in less" on a production answer engine is either a milestone in agent reliability or a subtle transfer of risk from reviewers to users — probably both.
AI researchers publicly debated how close recursive self-improvement actually is. A Dwarkesh Podcast episode released Thursday put researcher Beren Millidge and two other guests against each other on the question of whether models improving AI research counts as recursion or just tooling, with Millidge arguing self-improvement is a spectrum the field is already partway up and the skeptics countering that better coding doesn't compound into better research without long-horizon agency. No transcript-driven bombshell, but the frame matters: this week's GPT-6 Astra customer stories are the industry's answer to exactly the skepticism aired on that episode.
What to watch: whether OpenAI keeps publishing Astra customer stories through the weekend — the Perplexity piece is already dated September 14 — and whether Cognition quantifies how much manual review actually drops as Astra-driven testing rolls out.
Would you accept an agent's test recording in place of reading its code? Tell us in the comments.
Sources: OpenAI — Cognition helps Devin test its own work with GPT-6 Astra · OpenAI — Perplexity trusts GPT-6 Astra with end-to-end systems · Dwarkesh Podcast — AI researchers debate how close we are to recursive self-improvement
