Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Martin Schuck, Maks Sorokin, Simone Manni, Duy Ta, Angela P. Schoellig, Marco Hutter, Simon Le Cleac'H, Jan Brüdigam
• Venue: arXiv: cs.RO
• Published: 2026-08-12
• Collected: 2026-08-13
• URL: http://arxiv.org/abs/2608.12063v1
• PDF: https://arxiv.org/pdf/2608.12063v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The authors propose a method to integrate locomotion and manipulation for robots using a combination of Sample-based Model Predictive Control (SMPC) and Reinforcement Learning (RL). By leveraging SMPC, they create an automated expert that generates large offline datasets, enabling the RL agent to learn new skills more quickly with minimal manual tuning. The high-level agent and a low-level dynamic stability controller work together to produce optimal behaviors that align with task objectives. This method has been successfully tested on different robotic morphologies.

What the Document Says

• The paper presents a method for integrating locomotion and manipulation using SMPC and RL.
• SMPC is used to generate large offline datasets in simulation.
• Off-policy RL agent uses sparse task rewards due to the generated dataset solving the exploration problem.
• A high-level agent and a low-level dynamic stability controller are integrated for optimal behaviors.
• The method has been tested on an arm-equipped Spot quadruped and a G1 humanoid.

Key Concepts Mentioned

• locomotion
• manipulation
• Reinforcement Learning (RL)
• Sample-based Model Predictive Control (SMPC)
• offline datasets
• optimal behaviors
• simulation
• robot autonomy

Claims Made by the Document

• The proposed method bypasses the limitation of slow reward shaping in standard RL.
• The off-policy RL agent can learn new skills more quickly with sparse task rewards.
• Integrating the high-level agent and low-level dynamic stability controller leads to optimal behaviors aligning with true task objectives.

Evidence Used

• The authors state that their method uses SMPC for rapid tuning and large offline datasets, reducing the time required for learning new skills.
• The paper describes how the sparse task rewards enable the RL agent to learn efficiently.
• The integration of the high-level agent and low-level controller is detailed in the text.

Methods / Approach

• Sample-based Model Predictive Control (SMPC)
• Reinforcement Learning (RL)
• offline datasets
• high-level agent
• low-level dynamic stability controller

Limitations Stated by the Authors

• The authors did not provide information on the specific challenges or limitations encountered during their research.

Limitations Noted by the Agent

Agent interpretation — not from the source.

• Unknown: The quality and generality of the results beyond the tested morphologies.
• Unknown: The applicability to real-world scenarios with varying environmental conditions.

Open Questions

• How does the proposed method perform compared to other RL methods in terms of efficiency and effectiveness?
• What are the potential challenges or limitations when applying this method to more complex tasks or robotic morphologies?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The paper focuses on demonstrating the method's efficiency, but it does not provide a direct comparison with other RL methods.
• The success of the method has been shown for various morphologies, but its applicability to different tasks or complex robotic designs is unclear.

Raw Abstract / Excerpt

Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.