Omni-Decision: A Progressive Evidence-State Agent System for Omni-Modal QA

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Ming Ma, Yi Zhu, Yiran Zhong, Feida Zhu, Weigao Sun, Junhan Shi, Lingrui Mei, Tianming Yang, Steven Hoi
• Venue: arXiv: cs.AI
• Published: 2026-07-13
• Collected: 2026-07-14
• URL: http://arxiv.org/abs/2607.11433v1
• PDF: https://arxiv.org/pdf/2607.11433v1
• DOI: UNKNOWN

Plain-Language Brief

The paper introduces Omni-Decision, a training-free system designed to answer questions across multiple modalities such as videos, audio, images, web pages, and computation results. It focuses on maintaining a structured evidence state for each query, including confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs. This shared state view enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping. The system achieved improved results in two datasets compared to baselines.

What the Document Says

• The paper presents Omni-Decision, a system for omni-modal question answering.
• Omni-Decision maintains a structured evidence state for each query.
• The evidence state includes confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs.
• The system enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping.
• The system achieved 45.6% accuracy on OmniGAIA and 58.3% on WorldSense.
• Omni-Decision improved over the baselines by +27.3 and +30.2 percentage points, respectively.

Key Concepts Mentioned

• omni-modal question answering
• evidence state
• confirmed evidence
• unresolved conflicts
• fact and computation dependencies
• open evidence needs
• targeted evidence acquisition
• sparse cross-modal cues
• inspectable control
• repair and stopping

Claims Made by the Document

• Omni-Decision improves results in omni-modal question answering.
• The system's design enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping.

Evidence Used

• The system achieved improved results in two datasets compared to baselines.

Methods / Approach

• Maintaining a structured evidence state for each query.
• Normalizing, judging, and committing heterogeneous observations.

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The text does not provide information about potential limitations of the system other than those mentioned by the authors.

Open Questions

• What specific methods are used for normalizing, judging, and committing heterogeneous observations?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper focuses on the design and performance of Omni-Decision, but it does not provide detailed information about the specific components or algorithms used.

Raw Abstract / Excerpt

Omni-modal evidence-seeking QA requires agents to answer questions whose evidence is sparsely distributed across videos, audio, images, web pages, and computation results. Existing agentic multimodal systems often leave evidence in scratchpads, tool trajectories, or free-form histories, making it difficult to track what has been grounded, what remains missing, and when the evidence is sufficient to answer. We propose Omni-Decision, a training-free evidence-state system that turns omni-modal QA into a query-scoped evidence-closure process. For each query, Omni-Decision maintains a structured evidence state containing confirmed evidence, unresolved conflicts, fact and computation dependencies, and open evidence needs. A shared state view conditions planning, evidence acquisition, validation, repair, and finalization. Heterogeneous observations from media, web, computation, and verification modules are normalized, judged, and committed through deterministic state updates. This design enables targeted evidence acquisition, preserves sparse cross-modal cues, and provides inspectable control over repair and stopping. Omni-Decision achieves 45.6% accuracy on OmniGAIA and 58.3% on WorldSense, improving over the baselines by +27.3 and +30.2 percentage points, respectively. No-state ablations and trajectory audits further support the role of explicit evidence-state control in multi-step omni-modal evidence seeking.