Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
5 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on on-policy self-distillation
5 notes
H$^2$SD: Hybrid Hindsight Self-Distillation
The paper introduces H$^2$SD, a framework for reinforcement learning with verifiable rewards (RLVR) that addresses the limitations of previous methods. H$^2$SD uses a hybrid approach to self-distillation, where it…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a novel framework for training large language models (LLMs) that addresses the issues of overfitting, suppression of exploration, cross-domain generalization problems, and privileged…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework designed to address issues in on-policy self-distillation (OPSD) for training large language models. The main problems being addressed are overfitting, suppression of…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
DemoPSD is a new framework for training large language models that addresses overfitting, privileged information leakage, and exploration suppression issues in on-policy self-distillation (OPSD). It does this by…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework for training large language models that addresses the issues of overfitting, suppressed exploration, and privileged information leakage in on-policy self-distillation (OPSD)…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
