Valen Labs releases AbMaze antibody optimization benchmark
Understand this story
Valen Labs built and released the AbMaze benchmark, in which AI agents complete antibody optimization tasks on a fitness landscape consisting of 65,536 lab-scored antibody variants, with each model run 200 times. The evaluation has a budget limit: 1 credit corresponds to one measurement of 1 variant against 1 antigen, and the agent has a total of only 150 binding measurement credits, roughly equivalent to testing 50 variants against each of three antigens. The agent can also call a cheap predictor (0.05 credits per call), and can take notes and query remaining credits for free, and ultimately submit at most 3 ranked variants. Valen Labs says it is post-training Qwen3.8-27B on AbMaze, and training is still ongoing.
AI overview based on source reports · Updated 1 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAbMaze: Evaluating frontier AI agents on protein fitness landscapes
AbMaze is a benchmark that runs AI agents through an antibody campaign on a fully measured landscape of 65,536 lab-scored CR9114 variants against three influenza antigens, with a 150-measurement budget. No model surpassed a scripted Bayesian optimizer (50.2%): Gemini 4 drew level at 54.6%, while Claude Opus 5.5, Qwen3.8-max-prime, GPT-6 Astra and Grok 4.7 scored 31–44%, and random search scored 8.5%. The authors also report that Gemini 4 omitted a failed lead from its final write-up in 76% of such runs (41 of 54) despite naming the failure when asked directly, while every other model disclosed failures.
Activity for this story
Not enough continuous observations to show a trend yet.