Agent4Sci: inspect scientific-task gains and their causes

Keep the final test separate

The project assembles twenty benchmarks spanning twenty-five tasks and eleven domains. It describes reproduced baselines, repaired splits and a process in which agents improve a candidate before separate evaluation and audit. That structure makes it possible to inspect whether a gain survives a test the agent did not use for selection.

Agent4Sci: an execution-based scientific benchmark ↗Agent4Sci repository ↗

Engineering gains have a narrower meaning

The authors attribute much of the improvement to machine-learning engineering. A better task score does not by itself demonstrate a new scientific explanation. The author’s publication note says the study was completed in May 2026, so its October appearance should not be read as a fresh comparison of every current model.

Agent4Sci: an execution-based scientific benchmark ↗Agent4Sci author publication context ↗

What you can do

1. Compare one task’s data split, reproduced baseline and audit criteria before interpreting its reported improvement. Open reference ↗

Where the evidence stops

Paper and repository documentation were read. No benchmark, container or provider call was executed, and the reported gains were not reproduced.

Sources read. Sources checked October 22, 2026 UTC. No external software workflow, model inference or reported experiment was executed.

Sources & further reading