Use decision models with Amazon Bedrock AgentCore Evaluations

This post explains how to connect decision models—Jev from TypeSafe and the open-source Strands Decider 2B—to Amazon Bedrock AgentCore Evaluations through code-based evaluators running in AWS Lambda, converting evaluator input into decision model requests and probabilities back into evaluation results. It describes noul, choice, and score question types, positions decision models as a calibrated screening layer alongside rather than a replacement for built-in LLM-as-a-judge evaluators, and outlines a five-step process for validating a decision-model evaluator against labeled traffic. A full code sample in the AgentCore samples GitHub repository deploys an HR Assistant agent evaluated by deterministic checks, built-in evaluators, and decision-model evaluators.