Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Martin Schuck, Maks Sorokin, Simone Manni, Duy Ta, Angela P. Schoellig, Marco Hutter, Simon Le Cleac'H, Jan Brüdigam
• Venue: arXiv: cs.RO
• Published: 2026-08-12
• Collected: 2026-08-13
• URL: http://arxiv.org/abs/2608.12063v1
• PDF: https://arxiv.org/pdf/2608.12063v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The paper proposes a method to integrate robot locomotion and manipulation using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC). The approach generates offline datasets from SMPC simulations, which are used to train an RL agent with sparse task rewards. This allows for faster learning of new skills without the need for manual tuning. The method integrates a high-level RL agent and a low-level dynamic stability controller to produce more optimal behaviors that align with true task objectives. The framework was tested successfully on different robot morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.

What the Document Says

• The study aims to integrate locomotion and manipulation for robots using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC).
• They use SMPC simulations to generate offline datasets, which are used to train an RL agent with sparse task rewards.
• The approach allows for faster learning of new skills without manual tuning.
• The high-level RL agent is integrated with a low-level dynamic stability controller.
• The method produces more optimal behaviors that align with true task objectives.
• The learned policies surpass the original optimal control teacher.
• The study validates the robustness of the sim-to-real framework by testing it on different morphologies, such as an arm-equipped Spot quadruped and a G1 humanoid.

Key Concepts Mentioned

• Robotics
• Reinforcement Learning
• Sample-based Model Predictive Control
• Locomotion
• Manipulation

Claims Made by the Document

• The proposed method allows for faster learning of new skills without manual tuning.
• The learned policies surpass the original optimal control teacher.

Evidence Used

• The paper demonstrates the successful deployment of complex loco-manipulation skills across different morphologies.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.