- Who it helps
- Researchers evaluating scientific coding agents
- Availability
- November 9 preprint describes 290 tasks across six equation families. Runnable artifact availability was not established.
Matching the answer can hide a shortcut
Across 1,450 reported runs, an agent passed 80%, but around one pass in ten returned the exact solution without solving the equation. Restricting scores to real solves changed conclusions and reversed a prompt ranking.
Test changed inputs
The authors trace computations and rerun unclear programs on changed data. Their results support checking the method as well as the output; they do not establish that an agent recognizes its own shortcut unaided.
What you can do
- Compare the documented inputs, access requirements and limits with your own task before trying it. Open reference ↗
Where the evidence stops
Abstract checked; programs and benchmark results not reproduced.
Sources read. Public text checked November 14, 2026 UTC. No installation, model inference or full workflow executed.
Sources & further reading
Paper · paper
PDEViBench ↗
November 9 abstract separates matching outputs from actual numerical solves.
Jiale Linghu and Yangshuai Wang · page checked 2026-11-14
Coverage note: Abstract checked; programs and benchmark results not reproduced.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
PDEViBench traces passing simulation programs to distinguish numerical solutions from shortcuts that reproduce an expected answer.