How Agent Anomaly Detection scores on benchmarks we didn't write
How Agent Anomaly Detection scores on benchmarks we didn't write
Google tested its Private Preview Agent Anomaly Detection feature on five public, non-Google agent-security benchmarks (AgentDojo, InjecAgent, Agent Security Bench, AgentDyn and AgentHarm), running 1,774 sessions through an ADK agent using gemini-2.5-flash. It flagged 87% of harmful instructions the agent actually carried out, with false alarms on 2% to 16% of benign sessions, but caught only 4 of 18 sessions where poisoned long-term memory drove the agent to act. The post details per-benchmark and per-app results, known gaps such as resisted instructions and small actions, and recommended defenses including Model Armor, Semantic Governance Policy and human approval for high-impact actions.
Source: Google Developers Blog · developers.googleblog.com