- Who it helps
- Teams evaluating browser agents and their completion checks.
- Availability
- The September 20 paper is public and links an artifact site. This review checked the abstract; artifact setup and access were not verified.
What stays fixed
Each pair preserves the instruction and backend success condition while changing the environment. This makes a failed result easier to relate to a particular intervention.
Why completion claims matter
The authors report that agents sometimes declare success without the required state change. A useful evaluation therefore checks the saved outcome as well as the agent’s final message.
What you can do
- Read the intervention families and outcome criteria before adapting this evaluation idea to a browser task. Open reference ↗
Where the evidence stops
The reported failure patterns were not reproduced. Full methods and independent comparisons remain to be examined.
Sources read. Sources read October 20, 2026. No installation, model inference or complete external workflow was executed.
Sources & further reading
Paper · paper
BreakingWeb: controlled browser environment changes ↗
Original evidence for the subject’s scope, availability and limitations.
BreakingWeb authors · page checked 2026-10-20 · Published 2026-09-20
Coverage note: The linked evidence establishes the documented scope. Independent execution and complete YouTube, X and community coverage remain unavailable.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
BreakingWeb pairs browser tasks with controlled website changes to measure whether an agent still reaches the intended result.