Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Continual Learning in Transition

Source

arXiv

arxiv.org/abs/2608.06216v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua
  • Venue: arXiv: cs.LG
  • Published: 2026-08-06
  • Collected: 2026-08-07
  • URL: http://arxiv.org/abs/2608.06216v1
  • PDF: https://arxiv.org/pdf/2608.06216v1
  • DOI: UNKNOWN

Plain-Language Brief

The paper examines the evolution of Continual Learning (CL), a method that allows models to update and retain knowledge while learning new tasks. The authors suggest a transition from traditional parameter-centric learning to system-level adaptation, with three main dimensions: When, How, and Where learning occurs. The 'How' dimension covers off-policy, on-policy, and beyond-gradient optimization mechanics. The 'When' dimension captures evolution across pre-training, post-training, and inference-time stages. The 'Where' dimension delineates updates occurring within internal parameters versus external structural constraints.

What the Document Says

  • The paper discusses the transition from traditional Continual Learning (CL), which focuses on parameter-centric mechanisms, to system-level adaptation.
  • The evolution of CL is examined through three dimensions: When, How, and Where learning occurs.
  • The 'How' dimension includes off-policy, on-policy, and beyond-gradient optimization mechanics.
  • The 'When' dimension captures evolution across pre-training, post-training, and inference-time stages.
  • The 'Where' dimension delineates updates occurring within internal parameters versus external structural constraints.

Key Concepts Mentioned

  • Continual Learning
  • parameter-centric mechanisms
  • off-policy learning
  • on-policy learning
  • beyond-gradient optimization
  • pre-training
  • post-training
  • inference-time
  • internal parameters
  • external structural constraints

Claims Made by the Document

  • There is a transition from parameter-centric learning to system-level adaptation in Continual Learning.
  • The paper proposes a tri-axial framework to characterize this transition.

Evidence Used

  • The text provides an overview of the paper's content and purpose.

Methods / Approach

  • The paper uses a tri-axial framework to examine the evolution of Continual Learning.

Limitations Stated by the Authors

Unknown

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent might not fully understand the specific methods or results of each represented method in the survey.

Open Questions

  • What are the specific methods and results discussed in each represented method in the survey?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper is a review, so it does not present original research or results. Instead, it discusses various Continual Learning methods and their evolution.

Raw Abstract / Excerpt

Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.