- Who it helps
- Researchers evaluating agents that control scientific instruments.
- Availability
- October 30 research paper; the inspected abstract describes instrMCP and 37 virtual QCoDeS tasks.
What the evaluation covers
instrMCP connects coding agents to instruments through Model Context Protocol. The benchmark reconstructs tasks in electronic transport, scanning-probe imaging and calibration.
Measurement and interpretation diverge
The authors report stronger measurement scores than scientific-analysis scores. Supplying clean reference data closes much of that gap, suggesting that measurement-session context contributes to errors.
What you can do
- Read the task definitions and scoring rubrics before comparing the results with your laboratory. Open reference ↗
Where the evidence stops
Abstract read November 4. Results concern virtual instruments and the tested agents. No physical experiment, software installation or benchmark reproduction performed.
Sources read. Primary sources read November 4, 2026 UTC. No third-party software, model inference or full workflow executed.
Sources & further reading
Paper · paper
instrMCP and q-devBench test scientific instrument agents ↗
Identifies the subject, documented capability or specific verification limit.
instrMCP and q-devBench authors · page checked 2026-11-04
Coverage note: Independent attention and end-to-end execution remain unverified.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
q-devBench separates taking measurements from interpreting them in a virtual laboratory benchmark.