iCreatAI News
Skip to content

Technology areas

Agent Latest updates

The technical direction of enabling models to autonomously plan, call tools, and complete multi-step tasks—all developments from Claude Code and Manus to various Agent frameworks and evaluation benchmarks.

161 条精选Past 30 days: 147 updatesTotal: 1,929 updates

Updated

精选归档 · 第 9 页

Oct 5Mon 161–161 updates
  1. GitHub Blog · AI & ML61

    ReviewBench: An open benchmark for AI code review

    GitHub introduces ReviewBench, an open benchmark for AI code review built from analysis of 103.9 million GitHub pull requests, containing 219 public pull requests across 19 languages with a multi-source golden set labeled for severity and category. It reports grounded and augmented precision, recall, and F1 metrics, uses Claude Sonnet 5 as the LLM grader under a published rubric, and had its ground-truth findings independently re-labeled by senior engineers with 96.6% agreement. GitHub says ReviewBench's offline predictions for a Copilot code review multi-model ensemble matched a later A/B test, which showed an 8.0% rise in addressed rate, 13.6% in recall, 61% in comment volume, and an 8.0% drop in cost per review.

    Why it matters: It offers a concrete, reproducible methodology and public dataset for comparing AI code reviewers, plus evidence on how offline scores tracked a production experiment.