Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Junhao Shi, Zezheng Huai, Siyin Wang, Jia Chen, Yubang Wang, Zhaoye Fei, Hechang Chen, Jingjing Gong, Xipeng Qiu, Yu-Gang Jiang
• Venue: arXiv: cs.RO
• Published: 2026-06-25
• Collected: 2026-06-26
• URL: http://arxiv.org/abs/2606.27251v1
• PDF: https://arxiv.org/pdf/2606.27251v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers introduce OmniAct, a framework designed for creating persistent embodied agents in unstructured environments. The framework is built on a hierarchical asynchronous architecture that separates planning, memory, and verification. It integrates a multimodal semantic planner, an adaptive hierarchical memory, and an asynchronous visual preemption engine. OmniAct improves end-to-end success in long-horizon tasks across multiple robotic platforms and IoT devices, while maintaining near-constant token consumption and delivering mid-scale open-weight models at proprietary-level performance.

What the Document Says

• The paper introduces OmniAct, a framework that aims to create persistent embodied agents in unstructured environments.
• OmniAct is built on a hierarchical asynchronous architecture with separate planning, memory, and verification components.
• It integrates a multimodal semantic planner for skill routing across unified action spaces.
• The framework also includes an adaptive hierarchical memory with event-boundary-driven compression to limit context growth.
• Additionally, it has an asynchronous visual preemption engine to close the semantic loop during physical execution.
• Across 40 real-world long-horizon tasks on two robotic platforms and four IoT devices, OmniAct improved end-to-end success across all complexity levels.
• The framework maintains near-flat token consumption over under 100k+ accumulated interaction tokens.
• OmniAct elevates mid-scale open-weight models to proprietary-level performance.

Key Concepts Mentioned

• persistent embodied agents
• unstructured environments
• hierarchical asynchronous architecture
• multimodal semantic planner
• adaptive hierarchical memory
• asynchronous visual preemption engine
• long-horizon tasks
• robotic platforms
• IoT devices
• end-to-end success
• token consumption
• open-weight models

Claims Made by the Document

• OmniAct improves end-to-end success in long-horizon tasks.
• OmniAct maintains near-flat token consumption.
• OmniAct elevates mid-scale open-weight models to proprietary-level performance.

Evidence Used

• Across 40 real-world long-horizon tasks on two robotic platforms and four IoT devices, OmniAct improved end-to-end success across all complexity levels.
• OmniAct maintains near-constant token consumption over under 100k+ accumulated interaction tokens.
• OmniAct elevates mid-scale open-weight models to proprietary-level performance.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Building persistent embodied agents in unstructured environments demands unified orchestration of heterogeneous tools spanning both cyber (APIs, IoT) and physical (manipulation, navigation) domains, coupled with autonomous recovery from physical failures that inevitably arise over extended operation. Existing systems treat these as separate problems: VLM-based planners lack a unified cyber-physical action space, agent frameworks accumulate unbounded context that degrades temporal coherence, and VLA policies execute open-loop without detecting their own failures. We argue that persistent autonomy requires not a monolithic model but a hierarchical asynchronous architecture with explicit separation of planning, memory, and verification. To this end, we present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces, an adaptive hierarchical memory with event-boundary-driven compression for sub-linear context growth, and an asynchronous visual preemption engine that closes the semantic loop during physical execution. Across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct achieves consistent improvements in end-to-end success across all complexity levels, maintains near-flat token consumption over under 100k+ accumulated interaction tokens, and elevates mid-scale open-weight models to proprietary-level performance.