Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence

Source

arXiv

arxiv.org/abs/2606.24797v1

Read the source

Source

Plain-Language Brief

Researchers have developed a new benchmark called Evidence-Grounded Video Question Answering Benchmark (EG-VQA) to evaluate the performance of video question answering models. The benchmark is unique because it requires models not only to generate correct answers but also to provide supporting evidence from the video, which is often overlooked in existing benchmarks. EG-VQA includes 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations. A new metric called Evidence-Grounded F1 (EG-F1) has been introduced to evaluate the temporal alignment and semantic consistency of predicted evidence against ground-truth evidence. The study shows that even strong proprietary models struggle to accurately ground their predictions, indicating a fundamental discrepancy between answer correctness and faithful evidence localization. To address this issue, the researchers propose EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision. EG-Reasoner achieves state-of-the-art performance among open-source models and performs competitively against proprietary systems, especially on reasoning-intensive tasks such as counterfactual questions.

What the Document Says

  • EG-VQA is a new benchmark for evaluating video question answering models.
  • It requires models to provide supporting evidence from videos along with correct answers.
  • The benchmark consists of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations.
  • A new metric called Evidence-Grounded F1 (EG-F1) is introduced to evaluate predicted evidence.
  • Experimental results show that even strong proprietary models struggle to accurately ground their predictions.
  • To address this issue, EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision, is proposed.
  • EG-Reasoner achieves state-of-the-art performance among open-source models and competes with proprietary systems.

Key Concepts Mentioned

  • Video Large Language Models
  • VideoQA
  • Evidence-Grounded Video Question Answering Benchmark (EG-VQA)
  • Evidence-Grounded F1 (EG-F1)
  • EG-Reasoner

Claims Made by the Document

  • Existing benchmarks for video question answering focus on answer correctness but overlook evidence understanding.
  • EG-VQA requires joint reasoning and precise evidence localization.
  • Even strong proprietary models struggle to accurately ground their predictions.
  • Scaling alone is insufficient for robust video understanding.
  • Structured evidence supervision is essential for the development of more reliable and interpretable VideoQA systems.

Evidence Used

  • The text mentions that existing benchmarks are evaluated through answer correctness but don't examine the grounding of predictions in relevant video evidence.
  • The construction of EG-VQA is described as requiring each QA pair to be explicitly annotated with supporting temporal evidence.
  • The experimental results show that even strong proprietary models struggle to accurately ground their predictions.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Recent advances in Video Large Language Models (Video-LLMs) have yielded promising performance on video question answering (VideoQA). Nevertheless, existing benchmarks are predominantly evaluated through answer correctness, while the grounding of predictions in relevant video evidence remains largely unexamined. This disconnect between answer generation and evidence understanding motivates the construction of the Evidence-Grounded Video Question Answering Benchmark (EG-VQA), an open-ended evaluation protocol in which each QA pair is explicitly annotated with supporting temporal evidence, thereby requiring joint reasoning and precise evidence localization. EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations. To evaluate predicted evidence, Evidence-Grounded F1 (EG-F1) is introduced as a unified metric in which temporal alignment and semantic consistency against ground-truth evidence are jointly measured. Experimental evaluation reveals that even strong proprietary models struggle to accurately ground their predictions, exposing a fundamental discrepancy between answer correctness and faithful evidence localization. To bridge this gap, EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision, is proposed. State-of-the-art performance is achieved among open-source models, with results competitive against proprietary systems, particularly pronounced gains are observed on reasoning-intensive tasks such as counterfactual questions. These findings demonstrate that scaling alone is insufficient for robust video understanding and that structured evidence supervision is essential for the development of more reliable and interpretable VideoQA systems.