Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai, Jing Xiong, Ganlin Yang, Jinliang Zheng, Jiafei Cao, Xueyuan Wei, Jiangmiao Pang, Yuan Shen, Tai Wang
• Venue: arXiv: cs.RO
• Published: 2026-07-06
• Collected: 2026-07-07
• URL: http://arxiv.org/abs/2607.05377v1
• PDF: https://arxiv.org/pdf/2607.05377v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers introduce Cortex, a framework for long-horizon manipulation tasks that addresses the limitations of current Vision-Language-Action (VLA) models by creating executable and tractable subtask plans from high-level planning semantics to low-level execution kinematics. The framework uses 32 canonical skill primitives, tractability principles, event-balanced sampling strategy, and harness engineering for training data generation and fine-tuning.

What the Document Says

• The text presents Cortex, a bidirectionally aligned embodied agent framework for long-horizon manipulation tasks.
• Cortex addresses the Markovian nature of recent VLA models, which struggle with long-horizon tasks by creating executable and tractable subtask plans.
• The framework standardizes manipulation subtasks into 32 canonical skill primitives and uses tractability principles in the data generation pipeline.
• Over 4k hours of open-source video data and 30 hours of simulation data are automatically annotated using this method.
• An event-balanced sampling strategy is devised to construct training data for fine-tuning the framework.
• Harness engineering from task contexts to skill constraints during inference is also implemented.
• Evaluations demonstrate Cortex's efficacy, outperforming monolithic baselines by 3.1% on Libero-long and 4.1% on RoboTwin.
• Cortex enables zero-shot completion of unseen real-world long-horizon tasks, such as multi-stage chemistry experiments.

Key Concepts Mentioned

• Vision-Language-Action (VLA) models
• long-horizon manipulation tasks
• bidirectionally aligned embodied agent framework
• subtask plans
• 32 canonical skill primitives
• tractability principles
• open-source video data
• simulation data
• event-balanced sampling strategy
• harness engineering
• Libero-long
• RoboTwin

Claims Made by the Document

• Cortex addresses the limitations of current VLA models for long-horizon tasks.
• Cortex outperforms monolithic baselines on Libero-long and RoboTwin.

Evidence Used

• The text describes Cortex's ability to create executable and tractable subtask plans, which addresses the limitations of current VLA models.
• Evaluations showing Cortex's performance compared to monolithic baselines on Libero-long and RoboTwin.

Methods / Approach

• standardization of manipulation subtasks into 32 canonical skill primitives
• injecting tractability principles into the data generation pipeline
• event-balanced sampling strategy
• harness engineering

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent may not be aware of any limitations mentioned by the authors.

Open Questions

• What are the specific tractability principles injected into the data generation pipeline?
• What is the detailed harness engineering methodology?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The text mentions tractability principles and harness engineering but does not provide detailed information about them.

Raw Abstract / Excerpt

While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations. Hierarchical dual-system methods address this but suffer from a gap between high-level planning semantics and low-level execution kinematics. We introduce Cortex, a bidirectionally aligned embodied agent framework with a customized planning interface that conveys executable and tractable subtask plans from high-level VLM to low-level VLA. Specifically, we standardize manipulation subtasks into 32 canonical skill primitives and inject tractability principles, such as representative object attributes and improved trajectory reachability, into the data generation pipeline. This enables automatic annotation of over 4k hours of open-source video data and generation of 30 hours of simulation data. We further devise an event-balanced sampling strategy to construct training data for fine-tuning the framework to better handle planning ambiguity during subtask transitions, enhanced by carefully designed harness engineering from task contexts to skill constraints during inference. Both open-loop VLM and closed-loop system evaluations demonstrate Cortex's efficacy, e.g., it outperforms monolithic baselines by 3.1% on Libero-long and 4.1% on RoboTwin. Notably, Cortex's generalist VLM enables zero-shot completion of unseen real-world long-horizon tasks, such as multi-stage chemistry experiments, by simply combining with a fine-tuned VLA-a capability infeasible through VLA fine-tuning alone.