AXIOS (Sam Sabin) - Tech giants are pushing for a new AI agent incident reporting framework
A coalition of more than 120 organizations, including Nvidia, Cisco and CrowdStrike, is proposing a new incident-reporting framework for AI agents that would require participating companies to disclose certain agent mishaps and preserve detailed records of what went wrong. Why it matters: As AI agents gain more autonomy to act across computer systems, the industry lacks a standard way to report security failures and learn from them. Driving the news: The Open Secure AI Alliance is developing guidelines for what it's calling the Shared AI Findings Exchange (SAFE), a proposed framework for how organizations report cyber incidents involving AI agents. - The draft calls for participation from model deployers, AI developers, cloud and tool…
Does the SAFE framework provide legal protections for companies that disclose incident details, and how is the alliance encouraging participation?
The SAFE framework has no formal safe-harbor protections to shield companies that voluntarily disclose potentially damaging details about an AI incident; however, the alliance is relying on the existing cybersecurity culture of sharing threat intelligence to encourage participation.
Q&A ID 82759fe6-a50e-4401-9335-2d4fcb7e9844
How does Justin Boitano of Nvidia describe the concept of the "harness" in relation to the SAFE framework?
Justin Boitano, vice president and general manager of enterprise computing at Nvidia, compares the harness—which provides visibility into everything an agent is doing—to an aircraft's flight recorder in NASA's aviation safety reporting system, allowing cybersecurity experts to better determine the necessary controls for the industry.
Q&A ID 7ea09dfc-3a73-4bf8-862b-9e86a14ac421
What is the proposed timeline for reporting and updating information regarding an AI incident under the SAFE guidelines?
The proposed timeline requires members to notify affected organizations as soon as possible, submit an initial confidential report to SAFE within four business days, publish a preliminary factual report within 30 days when appropriate, and provide a remediation update within 90 days.
Q&A ID 1dbaec17-571c-4517-b349-8f104eb3645a
What evidence and data must members preserve following an AI incident under the SAFE proposal?
Members are required to preserve evidence from incidents, which includes prompts, agent traces, tool calls, identities, permissions, and credentials.
Q&A ID efa567aa-6ae8-4cc3-9140-ef64e6a0fb57
Under the proposed SAFE framework, what specific types of AI agent incidents must members report?
Members would agree to report incidents where an AI system accesses or exploits a third-party system without authorization, breaches confidential information, or continues probing a production target after the operator suspects the activity is unauthorized. Members would also report certain near misses.
Q&A ID 3c7e3dd2-b496-47a6-a193-fb63756047a5
What is the Shared AI Findings Exchange (SAFE) and who is developing it?
The Shared AI Findings Exchange (SAFE) is a proposed incident-reporting framework for cyber incidents involving AI agents. It is being developed by the Open Secure AI Alliance, a coalition of more than 120 organizations including Nvidia, Cisco, and CrowdStrike.
Q&A ID 00680633-46d5-4457-96c7-57b61a3d02e1