AgentCore evaluators can connect to decision models
Understand this story
AWS described how to connect decision models to Amazon Bedrock AgentCore Evaluations through code-based evaluators: the evaluator converts structured states and fixed options into calibrated probabilities that can be thresholded, then converts the evaluator input into a decision model request and returns the result. The article demonstrates with two backends: TypeSafe-hosted Jev and the open-source Strands Decider 2B (2 billion parameters). Prerequisites include: an AWS account in a region where AgentCore Evaluations is available; Jev requires a TypeSafe API key; Strands Decider requires a host to run its service; the decision model evaluator is opt-in.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogUse decision models with Amazon Bedrock AgentCore Evaluations
This post explains how to connect decision models—Jev from TypeSafe and the open-source Strands Decider 2B—to Amazon Bedrock AgentCore Evaluations through code-based evaluators running in AWS Lambda, converting evaluator input into decision model requests and probabilities back into evaluation results. It describes noul, choice, and score question types, positions decision models as a calibrated screening layer alongside rather than a replacement for built-in LLM-as-a-judge evaluators, and outlines a five-step process for validating a decision-model evaluator against labeled traffic. A full code sample in the AgentCore samples GitHub repository deploys an HR Assistant agent evaluated by deterministic checks, built-in evaluators, and decision-model evaluators.
Activity for this story
- Current comparable activity
- 4
- Peak comparable activity
- 10Nov 5 04:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.