Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy

Source

arXiv

arxiv.org/abs/2607.18984v1

Read the source

Source

Plain-Language Brief

The paper proposes a taxonomy to better understand curriculum learning (CL) in Natural Language Processing (NLP). The taxonomy aims to provide a systematic analysis of CL strategies by separating difficulty evaluation from training scheduling, and formalizing CL schedulers. The research highlights that prior works in NLP conflate distinct notions of difficulty and scheduling, hindering comparison and the accumulation of a coherent evidence base.

What the Document Says

  • The paper discusses curriculum learning (CL) research in NLP, which has been ongoing for over a decade but lacks a unified account on the choice of difficulty function or scheduler for a specific problem.
  • A fine-grained taxonomy is proposed to analyze CL strategies. This taxonomy separates difficulty evaluation from training scheduling.
  • For difficulty evaluation, the paper distinguishes between attribution source and task dependence, viewing difficulty as a perspectival concept with different assumptions about what makes an instance hard to learn.
  • The paper provides the first formalization of CL schedulers in terms of expected training contribution. This formalization enables comparison across implementations by introducing retention regimes and monotonicity properties.
  • An analysis of CL works in NLP using this taxonomy reveals a systematic incomparability problem. Prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label, which hinders comparison and the accumulation of a coherent evidence base.
  • Beyond diagnosing the issue, the taxonomy is expected to support the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.

Key Concepts Mentioned

  • curriculum learning
  • NLP
  • difficulty function
  • scheduler
  • taxonomy

Claims Made by the Document

  • There is a lack of a unified account on the choice of difficulty function or scheduler for a specific problem in NLP CL research.
  • A proposed taxonomy aims to provide a systematic analysis of CL strategies by separating difficulty evaluation from training scheduling.
  • Difficulty in this context is viewed as a perspectival concept with different assumptions about what makes an instance hard to learn.
  • The paper provides the first formalization of CL schedulers in terms of expected training contribution.
  • An analysis of prior CL works in NLP reveals a systematic incomparability problem due to conflation of distinct notions of difficulty and scheduling.

Evidence Used

  • More than a decade of CL research in NLP
  • The proposed taxonomy
  • Distinction between attribution source and task dependence
  • First formalization of CL schedulers
  • Analysis of prior CL works in NLP

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Despite more than a decade of curriculum learning (CL) research in NLP, the field lacks a principled account of which difficulty function or scheduler to use for a given problem. To understand what has hindered progress towards this account, we propose a fine-grained taxonomy separating difficulty evaluation from training scheduling to enable systematic analysis of CL strategies. For difficulty evaluation, we distinguish attribution source and task dependence, revealing difficulty as a perspectival concept encoding different assumptions about what makes an instance hard to learn. For scheduling, we provide the first formalisation of CL schedulers in terms of expected training contribution, enabling comparison across implementations by introducing retention regimes and monotonicity properties. Applied in a dedicated analysis of CL works in NLP, our taxonomy reveals a systematic incomparability problem: prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label -- hindering comparison and the accumulation of a coherent evidence base. Beyond diagnosis, the taxonomy supports the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.