agentest: an open-source offline evaluation tool for AI agents
Understand this story
AiEnsured has released agentest, an open-source tool for offline evaluation of AI agents. It reads recorded OpenTelemetry traces from agents, runs 165 tests, and determines what the agent actually did, including falsely reporting success, acting before verifying, executing hidden instructions, making unauthorized tool calls, and leaking personal data. According to AiEnsured, the tool runs fully offline and requires no API key, network, or LLM judge, so the same trace yields the same verdict every time; it works with any framework that records OpenTelemetry GenAI traces and can be installed via pip install agentest-ai. Without installation, you can also use the Assurance Report Card in a browser (paste or upload a trace, with three samples included), or open it in Colab (free CPU, no login required, no model calls).
AI overview based on source reports · Updated 21 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIShow HN: Agentest – offline tests for AI agents from OpenTelemetry traces
Agentest is an open-source tool that reads the OpenTelemetry traces an agent already recorded and runs 165 tests judging what the agent actually did, such as claiming unearned success, acting before verifying, following hidden instructions, calling disallowed tools or leaking personal data. It runs offline with no API key, network or LLM judge, so the same traces always give the same verdicts, and it installs with pip install agentest-ai. It supports native-log importers for LangGraph, OpenAI Agents SDK, AutoGen, smolagents and n8n, and can gate CI with --strict.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 19 23:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.