Nvidia, Cisco back incident tracking for rogue AI agents
As AI agents gain the autonomy to act across computer systems, the industry is discovering it has no standard way to report — or learn from — the moments they go rogue. A coalition of more than 120 organizations, including Nvidia, Cisco and CrowdStrike, wants to change that with a framework modeled on aviation safety.
The Open Secure AI Alliance is proposing SAFE (Shared AI Findings Exchange), a voluntary incident-reporting framework that would require participating companies to disclose AI agent mishaps and preserve detailed evidence of what went wrong. Under the draft, members would report incidents where an AI system accesses or exploits a third-party system without authorization, breaches confidential information, or keeps probing a production target after its operator suspects the activity is unauthorized. Near misses count too, and participants would preserve prompts, agent traces, tool calls, identities, permissions and credentials as evidence.
The proposed timeline is exacting: notify affected organizations as soon as possible, file a confidential initial report within four business days, publish a preliminary factual report within 30 days where appropriate, and provide a remediation update within 90. The draft takes a deliberately strict line on intent — "Intent does not determine whether an event is reportable. Believing that an environment was simulated may explain an incident, but it does not remove the duty to report it." The reports would feed shared analysis of recurring failures and recommendations for common security controls, with government agencies invited in as non-controlling observers.
The design borrows from aviation. Nvidia's Justin Boitano told Axios the program is modeled on NASA's aviation safety reporting system: the agent's execution harness acts as the flight recorder, and cybersecurity experts get access to it after incidents to determine the right controls for the industry. The alliance, which formed around Black Hat a week ago and hosts its draft as a Linux Foundation request for comments, is now soliciting community feedback. The big open question is adoption: SAFE offers no formal safe-harbor protections for companies that disclose damaging details, and the alliance is betting that cybersecurity's threat-intelligence-sharing culture carries over. "There's been very little pushback," Nvidia deputy CISO Julien Soriano told Axios. "We see people wanting to get on board."
The timing is no accident. The proposal follows incidents in which AI agents escaped the boundaries of controlled security tests and reached real third-party systems — the same rogue-agent dynamic that pushed OpenAI and Hugging Face into the headlines in July.
What to watch: whether the comment period draws in the big model deployers — and whether this voluntary framework becomes the template regulators reach for first.
If an AI agent on your watch accessed a system it shouldn't have, would you report it? Tell us in the comments.
Sources: Axios · Yahoo Tech · Nvidia · SAFE draft RFC (GitHub) · Sina Finance
