iCreatAI News
Skip to content
METR Blog·· 15 hr agoAI score55

Measuring alignment eval tells

Measuring alignment eval tells

AI summary

Researchers ran investigator agents inside 19 alignment evals and 36 capability evals to see whether models can tell what kind of eval they are in, and found that investigator agents have become increasingly able to discriminate between the two, with Opus 4.8 and Opus 5.5 nearly perfect on their dataset. The agents cite environment-level tells (mocked services, simulated infrastructure, implausible scenarios) and task-level tells (planted shortcuts, impossible legitimate success, conflicting goals, scripted users), and in one honeypot built by modifying a capabilities eval, Opus 5.5 identified all four task-level modifications. The authors frame this as understanding whether such tells exist, not as measuring whether they causally affect agent behavior during the eval.

Source: METR Blog · metr.org