Databricks releases enterprise data agent benchmark, Genie accuracy 91%

1. 23:00
   Databricks BlogFeatured

   Announcing GA of Genie Ontology

   Databricks announced the General Availability of Genie Ontology, an agentic ontology layer that agents build by learning from dashboards, SQL queries, notebooks and BI semantic models, while people govern definitions through Unity Catalog semantics. The release adds Genie Search (GA), Ontology Snippets (GA), external knowledge sources (Beta, starting with Power BI), organizational directories (GA, starting with Workday), Domains and Pages (GA), and Ontology Manager (coming soon in Beta). In a benchmark built with Turing simulating a Databricks workspace for a fictional UK energy supplier, Genie answered correctly in 91% of runs versus 45% and 58% for two frontier agents, using less than half the tokens and finishing three to four times faster.

2. 23:00
   Databricks Blog

   Genie with Genie Ontology Delivers 1.6× the Accuracy with Less than Half the Tokens

   Databricks developed a new enterprise data-agent benchmark with Turing, run in a live Databricks workspace, in which Genie achieved a 91% pass rate versus 45% and 58% for two frontier agents while using fewer than half the tokens and completing tasks 3–4x faster. The benchmark, built around a fictional UK energy supplier called Calderwick Energy, includes notebooks, dashboards, saved SQL queries and documents alongside hundreds of tables, testing whether agents can find and use business context that table schemas alone cannot provide. Genie Ontology, now generally available, automatically extracts and organizes this knowledge and surfaces relevant context for Genie.

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.