- Who it helps
- Researchers and developers evaluating consequential agent tool use.
- Availability
- The October 22 preprint describes 300 task pairs across six domains. The paper is public; a separately inspected runnable release was not established in this check.
True observations can be incomplete
The benchmark removes selected facts while keeping the remaining statements true. An independent audit tool can restore evidence. Evaluation checks both prohibited actions and completion of permissible tasks.
Refusal is only one outcome
The authors report that recovery access lowers unsafe execution across ten tested models. They also distinguish getting the missing evidence from using it correctly. Blanket refusal can look safe while failing the useful half of a paired task.
What you can do
- For one test task, write a permitted and prohibited version that differ only in a decisive fact, then inspect both outcomes. Open reference ↗
Where the evidence stops
This is a research protocol, not a deployment safety guarantee. No benchmark or provider call was executed and the reported findings were not reproduced.
Sources read. Public evidence checked October 24, 2026 UTC. No third-party software, model inference or complete product workflow was executed.
Sources & further reading
Paper · paper
EpiSEM-Bench missing-evidence evaluation ↗
The October 22 paper compares matched permitted and prohibited tasks, with missing facts and an optional audit source.
Zhang and colleagues · page checked 2026-10-24
Paper · paper
EpiSEM-Bench evaluation protocol ↗
The protocol separates unsafe execution, useful completion and recovery; authors’ results were not reproduced.
Zhang and colleagues · page checked 2026-10-24
Coverage note: Primary material was inspected. Independent execution and matching original YouTube, X and Reddit discussion remain incomplete; unavailable engagement counts are omitted.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
EpiSEM-Bench tests whether an agent seeks a missing fact before acting, using paired tasks where the same request is permitted in one world and prohibited in the other.