- Who it helps
- Developers evaluating tool-using agents.
- Availability
- October 2026 EMNLP demonstration paper. The abstract describes open-source software; installation and repository availability were not verified here.
Replay the same fault
The method perturbs tool responses, replays matching calls from cache and returns to live tools after the agent diverges. MCP connects an agent to external tools.
Recovery depends on the failure
Reported retry improvements on timeouts did not solve stale-data failures. A passing recovery case cannot establish reliability across all tool faults.
What you can do
- Read the fault definitions and scoring rules before adapting the evaluation to your own tools. Open reference ↗
Where the evidence stops
Abstract and metadata checked October 31. No agent run, fault injection or independent replication performed.
Sources read. Sources checked October 31, 2026 UTC. No third-party software, model inference or complete workflow was executed.
Sources & further reading
Web · paper
AgentCheck tests how agents recover from faulty tools ↗
Supports the cited capability or limit in AgentCheck tests how agents recover from faulty tools.
Aritra Mazumder and Nusrat Jahan Lia · page checked 2026-10-31
Coverage note: Independent attention and end-to-end execution remain unverified.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
AgentCheck is a workbench for replaying tool failures and comparing an agent’s behavior before and after a mitigation.