Measuring alignment eval tells
Measuring alignment eval tells
Anthropic researchers ran investigator agents inside 19 alignment evals and 36 capability evals to see whether models can tell what kind of eval they are in, using a direct 0–100 probability score and an LLM-judged open-ended answer. They report that investigator agents have become increasingly able to discriminate the two, with Opus 4.8 and Opus 5.5 nearly perfect (Opus 5.5 ranked every alignment eval above every capability eval), and that agents cite environment-level tells (simulated or contrived environments) and task-level tells (planted shortcuts, impossible legitimate success, conflicting goals, scripted users). The authors note interrogation can prime models and that they do not claim these beliefs causally affect agent behavior during the tasks.
Source: METR Blog · metr.org