FIELD NOTES TOPIC
leakage attenuation
2 notes, newest first.
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a novel framework for training large language models (LLMs) that addresses the issues of overfitting, suppression of exploration, cross-domain generalization problems, and privileged…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework designed to address issues in on-policy self-distillation (OPSD) for training large language models. The main problems being addressed are overfitting, suppression of…
