Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

Source

arXiv

arxiv.org/abs/2606.23617v1

Read the source

Source

Plain-Language Brief

The paper proposes a new approach for training Vision-Language-Action (VLA) models through active, continual learning. This method aims to address the inefficiencies of traditional passive imitation learning by collecting data only when needed and focusing on states where the policy is performing poorly. However, the authors find that fine-tuning solely on actively-collected recovery data can lead to forgetting previously learned behaviors. The paper evaluates several techniques for continual learning and highlights trade-offs between adapting to new data and retaining previous knowledge.

What the Document Says

  • The approach focuses on Vision-Language-Action (VLA) models.
  • Traditional fine-tuning is done through passive imitation learning.
  • Passive imitation learning has downsides: it requires robot failure, offers little guidance about which states need supervision, and wastes effort on well-performed parts of the task.
  • The proposed method is an active, continual learning paradigm for VLAs.
  • Active data collection leads to more efficient fine-tuning than passive demonstrations.
  • Fine-tuning only on actively-collected recovery data can lead to catastrophic forgetting.
  • Techniques for continual learning are evaluated, including replay-based data mixing and elastic weight consolidation.
  • Tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors are identified.
  • The work contributes an empirical study of active continual learning for autoregressive VLAs.

Key Concepts Mentioned

  • Vision-Language-Action (VLA) models
  • passive imitation learning
  • active learning
  • continual learning
  • replay-based data mixing
  • elastic weight consolidation
  • catastrophic forgetting

Claims Made by the Document

  • Active, uncertainty-guided data collection leads to more efficient fine-tuning.
  • Fine-tuning only on actively-collected recovery data can lead to catastrophic forgetting.
  • Tradeoffs exist between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors.

Evidence Used

  • Improved fine-tuning results from active, uncertainty-guided data collection compared to passive demonstrations.
  • Fine-tuning only on actively-collected recovery data leads to forgetting previously learned behaviors.
  • Identification of tradeoffs between plasticity and retention in the study of continual learning.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies.