Vetto ScienceBench: Expanding Scientific Capabilities of Frontier AI Models

Vetto ScienceBench introduces 50 new data-analysis tasks in the Terminal-Bench Science format across 14 scientific disciplines, each reviewed for quality and audited per trial, where the best of 13 frontier models passes 49% of trials and most fail over 80%. Tasks require agents to explore data, choose a method, check its assumptions and deliver an answer or pipeline graded against held-out ground truth, with data features like instrument artifacts or selection effects that defeat textbook approaches. In about two thirds of failures the model had already seen the problem in its own output and then dismissed it as noise or corrected it with the wrong model, and a one-line warning raised GPT-6.1 Sol's pass rate from 29% to 39% without improving outcomes once the problem was noticed.