The agent evaluation playbook: Built-in, DeepEval, and AutoEval on Amazon Bedrock AgentCore Evaluations

This AWS post explains how to evaluate production agents on Amazon Bedrock AgentCore by starting from business outcomes, breaking them into observable behaviors, and mapping each to a trace- or session-level evaluator. It covers the built-in AWS evaluators, managed third-party metrics from DeepEval and AutoEval, custom and multi-dimension LLM-as-a-judge evaluators, and suggested evaluator stacks for customer support, research, and domain-content agents. It also describes three evaluation modes—on-demand, online, and batch—that share the same evaluator IDs, plus the ListEvaluators API for retrieving the full catalog.