Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Omni-Decision: A Progressive Evidence-State Agent System for Omni-Modal QA

Source

arXiv

arxiv.org/abs/2607.11433v1

Read the source

Source

Plain-Language Brief

The paper introduces Omni-Decision, a system aimed at answering questions where evidence is spread across multiple modalities like videos, audio, images, web pages, and computation results. The system maintains a structured evidence state for each query, which includes confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs. This design enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping. Omni-Decision improves the accuracy on two datasets (OmniGAIA and WorldSense) by 27.3% and 30.2% respectively compared to baselines.

What the Document Says

  • The system is designed for omni-modal evidence-seeking QA.
  • Existing systems leave evidence in scratchpads, tool trajectories, or free-form histories making it difficult to track what has been grounded, what remains missing, and when the evidence is sufficient to answer.
  • Omni-Decision maintains a structured evidence state for each query containing confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs.
  • The system enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping.
  • Omni-Decision improves accuracy on OmniGAIA and WorldSense datasets by 27.3% and 30.2% respectively compared to baselines.
  • No-state ablations and trajectory audits support the role of explicit evidence-state control in multi-step omni-modal evidence seeking.

Key Concepts Mentioned

  • Omni-modal QA
  • Evidence-state system
  • Structured evidence state
  • Targeted evidence acquisition
  • Cross-modal cues
  • Accuracy improvement
  • Baselines
  • No-state ablations
  • Trajectory audits

Claims Made by the Document

  • Omni-Decision improves accuracy on two datasets (OmniGAIA and WorldSense) compared to baselines.

Evidence Used

  • The paper mentions an improvement of 45.6% on OmniGAIA and 58.3% on WorldSense by Omni-Decision.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Omni-modal evidence-seeking QA requires agents to answer questions whose evidence is sparsely distributed across videos, audio, images, web pages, and computation results. Existing agentic multimodal systems often leave evidence in scratchpads, tool trajectories, or free-form histories, making it difficult to track what has been grounded, what remains missing, and when the evidence is sufficient to answer. We propose Omni-Decision, a training-free evidence-state system that turns omni-modal QA into a query-scoped evidence-closure process. For each query, Omni-Decision maintains a structured evidence state containing confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs. A shared state view conditions planning, evidence acquisition, validation, repair, and finalization. Heterogeneous observations from media, web, computation, and verification modules are normalized, judged, and committed through deterministic state updates. This design enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping. Omni-Decision achieves 45.6% accuracy on OmniGAIA and 58.3% on WorldSense, improving over the baselines by +27.3 and +30.2 percentage points, respectively. No-state ablations and trajectory audits further support the role of explicit evidence-state control in multi-step omni-modal evidence seeking.