- Who it helps
- Developers comparing automated code reviewers.
- Availability
- GitHub announced the benchmark October 5 and describes its dataset as public. This review inspected the announcement; the dataset and evaluation code were not run.
The actual test set
The evaluation contains 219 pull requests from 187 repositories across nineteen languages. The much larger 103.9-million-pull-request analysis informed sampling; it is not the benchmark’s test count.
Read the reference rules
Candidate issues came from humans, models and static analysis, with duplicate findings merged. Grounded and augmented measures ask different questions about agreement and additional discoveries.
What you can do
- Compare the benchmark’s languages, change sizes and issue categories with your repository before treating its aggregate score as a purchasing decision. Open reference ↗
Where the evidence stops
A reference set can miss valid findings. GitHub’s reported validation and production correlation were not independently checked here.
Sources read. Original sources read October 23, 2026 UTC. No external project, model inference or reported experiment was executed.
Sources & further reading
Web · paper
ReviewBench: an open benchmark for AI code review ↗
Read the October 5 announcement and corpus/ground-truth description. The benchmark contains 219 pull requests, not the 103.9 million used to characterize distributions.
GitHub / Michelle Zhou and Alejandro Carderera de Diego · page checked 2026-10-23
Coverage note: No independent end-to-end reproduction or complete video demonstration was verified in this bounded sample.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
ReviewBench provides a public evaluation set for AI code-review agents. GitHub describes representative pull requests, a multi-source reference set of findings and metrics that distinguish known issues from additional valid findings.