Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Junhao Shi, Zezheng Huai, Siyin Wang, Jia Chen, Yubang Wang, Zhaoye Fei, Hechang Chen, Jingjing Gong, Xipeng Qiu, Yu-Gang Jiang
• Venue: arXiv: cs.RO
• Published: 2026-06-25
• Collected: 2026-06-26
• URL: http://arxiv.org/abs/2606.27251v1
• PDF: https://arxiv.org/pdf/2606.27251v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers present OmniAct, a framework that aims to advance the development of persistent embodied agents in unstructured environments. The framework integrates a multimodal semantic planner for skill routing, an adaptive hierarchical memory for efficient context management, and an asynchronous visual preemption engine for closed-loop physical execution. The research claims that OmniAct achieves consistent improvements in success rate across various complex tasks, maintains low resource usage, and outperforms mid-scale open-weight models.

What the Document Says

• The research introduces OmniAct, a framework for building persistent embodied agents.
• OmniAct integrates a multimodal semantic planner, an adaptive hierarchical memory, and an asynchronous visual preemption engine.
• The goal is to unify cyber (APIs, IoT) and physical (manipulation, navigation) domains for persistent autonomy in unstructured environments.
• The framework aims to recover from physical failures during extended operation.
• The researchers argue for a hierarchical asynchronous architecture for persistent autonomy.
• 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices were used in the study.

Key Concepts Mentioned

• persistent embodied agents
• unstructured environments
• cyber-physical action space
• autonomous recovery
• hierarchical asynchronous architecture
• OmniAct
• multimodal semantic planner
• adaptive hierarchical memory
• asynchronous visual preemption engine

Claims Made by the Document

• OmniAct achieves consistent improvements in end-to-end success across all complexity levels.
• OmniAct maintains near-flat token consumption over under 100k+ accumulated interaction tokens.
• OmniAct elevates mid-scale open-weight models to proprietary-level performance.

Evidence Used

• Across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct was tested.

Methods / Approach

• The researchers developed the OmniAct framework for advancing persistent embodied agents in unstructured environments.

Limitations Stated by the Authors

Not stated in source

Limitations Noted by the Agent

Agent interpretation — not from the source.

The study lacks information on potential limitations of the presented method.

Open Questions

• What are the potential challenges and limitations of the OmniAct framework?
• How does the OmniAct framework compare to other similar methods in terms of efficiency and effectiveness?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The research focuses on the presentation of a new framework for advancing persistent embodied agents, but it does not provide detailed comparisons with existing methods or discussions on potential challenges.

Raw Abstract / Excerpt

Building persistent embodied agents in unstructured environments demands unified orchestration of heterogeneous tools spanning both cyber (APIs, IoT) and physical (manipulation, navigation) domains, coupled with autonomous recovery from physical failures that inevitably arise over extended operation. Existing systems treat these as separate problems: VLM-based planners lack a unified cyber-physical action space, agent frameworks accumulate unbounded context that degrades temporal coherence, and VLA policies execute open-loop without detecting their own failures. We argue that persistent autonomy requires not a monolithic model but a hierarchical asynchronous architecture with explicit separation of planning, memory, and verification. To this end, we present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces, an adaptive hierarchical memory with event-boundary-driven compression for sub-linear context growth, and an asynchronous visual preemption engine that closes the semantic loop during physical execution. Across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct achieves consistent improvements in end-to-end success across all complexity levels, maintains near-flat token consumption over under 100k+ accumulated interaction tokens, and elevates mid-scale open-weight models to proprietary-level performance.