UAEF: A framework-agnostic system for closed-loop evaluation of AI agents

AWS introduces the Universal Agent Evaluation Framework (UAEF), an open-source, framework-agnostic system that uses adapters to normalize agent traces from LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into a common AgentTrace format, then scores them with 36 built-in metrics across 7 quality dimensions. Results are stored per experiment in Amazon DynamoDB and Amazon S3, enabling comparison of agent versions, regression detection, and report and insight generation, with both a Python library and a low-code web UI sharing the same engine. The post also provides a playbook for choosing metrics and preparing datasets, plus runnable notebooks for each supported framework.