- Who it helps
- Teams evaluating AI-generated software tests
- Availability
- October 27 research abstract available; artifact setup and licensing were not verified in this review.
A stricter test target
The study describes 5,086 instances from eight open-source projects across five languages and four behavior-driven testing frameworks. Agents receive Gherkin requirements and interfaces while implementation is withheld.
Passing is only one measure
The strongest evaluated setup reaches 82.8% passing tests but 33.8% kill-complete tests. Kill-complete means detecting every deliberately faulty version detected by the reference test. It does not mean finding every possible bug.
What you can do
- Inspect the definition of kill-complete and the withheld implementation setup before comparing agents. Open reference ↗
Where the evidence stops
Abstract checked November 11, 2026; benchmark and generated tests were not executed. Results depend on the provided requirements, fault set and evaluated agent configurations.
Sources read. Public source text checked November 11, 2026 UTC. No third-party project or complete workflow executed.
Sources & further reading
Paper · paper
Req2Test: requirements-driven testing ↗
October 27 abstract compares passing tests with tests that detect deliberately faulty code.
Binhang Qi and coauthors · page checked 2026-11-11 · Published 2026-10-27
Coverage note: Only the abstract was checked; artifact installation and benchmark reproduction remain unverified.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
Req2Test checks whether generated tests detect faulty behavior, beyond whether they pass against a working implementation.