Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Confidence-Aware Tool Orchestration for Robust Video Understanding

Source

arXiv

arxiv.org/abs/2606.26904v1

Read the source

Source

Plain-Language Brief

The researchers propose a framework called Robust-TO for video understanding that addresses the 'Blind Trust Problem' in video reasoning language models. The proposed method incorporates per-frame trustworthiness into every stage of reasoning and uses a unified evidence interface to integrate various visual perception tools.

What the Document Says

  • The paper discusses the Blind Trust Problem, where video reasoning language models assume all input frames are equally reliable.
  • Robust-TO is an agentic video understanding framework that integrates per-frame trustworthiness into every stage of reasoning.
  • Robust-TO organizes heterogeneous visual perception tools under a unified evidence interface.
  • Each tool receives a sub-query and a set of trustworthy frames.
  • The tools return evidence in a shared format: prediction, temporal grounding, and a calibrated reliability score.
  • During reasoning, the calibrated scores guide evidence weighting and define a confidence-cost GRPO reward for optimizing correctness, reliability, and efficiency.
  • On two video reasoning benchmarks, Robust-TO outperforms the strongest open-source baseline by 10.6% and Gemini-2.5-Pro.
  • Under realistic corruption types, Robust-TO maintains accuracy better than other methods while facing less drop in performance.

Key Concepts Mentioned

  • Video Understanding
  • Blind Trust Problem
  • Robust-TO
  • Visual Perception Tools
  • Trustworthiness
  • Reliability Score
  • Evidence Interface

Claims Made by the Document

  • Robust-TO outperforms the strongest open-source baseline by 10.6%.
  • Robust-TO maintains higher accuracy under corruption types than other methods.

Evidence Used

  • Results from two video reasoning benchmarks
  • Comparison with other methods in realistic corruption types

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Video reasoning language models implicitly assume that every input frame is equally reliable. This leads to what we term the Blind Trust Problem: under realistic perturbations such as motion blur, glare, or occlusion, frontier video reasoning models can suffer 15-30%p accuracy drops on real-world embodied benchmarks, while remaining unaware that their visual evidence has been degraded. To address this challenge, we propose Robust-TO, an agentic video understanding framework that explicitly integrates per-frame trustworthiness into every stage of reasoning. Robust-TO organizes heterogeneous visual perception tools under a unified evidence interface. Each tool receives a sub-query derived from the original question and a set of trustworthy frames selected by the reliability-relevance score. It returns evidence in a shared format: a concrete prediction (e.g., a bounding box, motion trajectory, recognized text, or action label), temporal grounding, and a calibrated reliability score. During reasoning, these calibrated scores guide evidence weighting in a three-tier synthesis process (high/medium/low) and define a confidence-cost GRPO reward that jointly optimizes correctness, evidence reliability, and efficiency. On two video reasoning benchmarks spanning eight tasks, Robust-TO achieves 56.4% average accuracy on clean inputs, surpassing the strongest open-source baseline by 10.6%p and outperforming Gemini-2.5-Pro (46.2%). Under five realistic corruption types, Robust-TO maintains 54.3% average accuracy, 5.8%p above the strongest open-source baseline, while exhibiting the smallest clean-to-corrupted accuracy drop among all compared methods.