- Who it helps
- Researchers evaluating multimodal assistants and developers who need inspectable video answers.
- Availability
- The October 20 preprint, benchmark materials and MIT-licensed evaluation code are public. Python 3.10+ and a model API adapter are required. Model calls can incur charges; dataset terms need separate review.
Answer quality and supporting evidence
The benchmark pairs 392 videos with 1,027 human-written questions across 15 capabilities. Models return answers and audio/video time intervals. One score measures coverage of supporting evidence; another penalizes a weak evidence match in either channel.
EvaOmni: evaluating answers and audio-visual evidence ↗EvaOmni evaluation code and data instructions ↗
A high answer score can hide a weak explanation
The authors report that their top overall model scored 79.4% for answers and 35.1% on the stricter grounding measure. This is a result on their selected benchmark, not a general ranking of video understanding.
What reproduction requires
The repository requires a real model-call adapter. Its mock option checks the pipeline only. Video preprocessing and interval formatting affect comparability, so a successful script run alone does not reproduce the paper’s evaluation.
What you can do
- Inspect the annotations, score definitions and API-adapter instructions before planning an evaluation. Open reference ↗
Where the evidence stops
Paper abstract and repository documentation checked. No model call, dataset inspection or benchmark run was executed; results remain author-reported.
Sources read. Sources checked October 21, 2026. No product workflow or benchmark was executed.
Sources & further reading
Paper · paper
EvaOmni: evaluating answers and audio-visual evidence ↗
Read the October 20 abstract and submission history. Answer accuracy and evidence localization are distinct metrics; reported results were not reproduced.
EvaOmni authors · page checked 2026-10-21 · Published 2026-10-20
GitHub · implementation
EvaOmni evaluation code and data instructions ↗
Read README metrics, Python requirement and evaluation setup. The user must implement the API adapter; mock mode only checks the pipeline. Code is MIT; dataset terms require their own review.
VisionXLab · page checked 2026-10-21
Coverage note: Independent hands-on testing and cross-platform discussion are incomplete. No missing links, creator identities or engagement counts are inferred.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
EvaOmni tests whether an audio-video model can both answer a question and identify the moments that support its answer.