iCreatAI News
Skip to content
Google Developers Blog·· 3 days agoAI score58

How Agent Anomaly Detection scores on benchmarks we didn't write

How Agent Anomaly Detection scores on benchmarks we didn't write

AI summary

Google tested its Private Preview Agent Anomaly Detection feature on five public, non-Google agent-security benchmarks (AgentDojo, InjecAgent, Agent Security Bench, AgentDyn and AgentHarm), running 1,774 sessions through an ADK agent using gemini-2.5-flash. It flagged 87% of harmful instructions the agent actually carried out, with false alarms on 2% to 16% of benign sessions, but caught only 4 of 18 sessions where poisoned long-term memory drove the agent to act. The post details per-benchmark and per-app results, known gaps such as resisted instructions and small actions, and recommended defenses including Model Armor, Semantic Governance Policy and human approval for high-impact actions.

Source: Google Developers Blog · developers.googleblog.com