FIELD NOTES TOPIC
privileged information leakage
3 notes, newest first.
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework designed to address issues in on-policy self-distillation (OPSD) for training large language models. The main problems being addressed are overfitting, suppression of…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
DemoPSD is a new framework for training large language models that addresses overfitting, privileged information leakage, and exploration suppression issues in on-policy self-distillation (OPSD). It does this by…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework for training large language models that addresses the issues of overfitting, suppressed exploration, and privileged information leakage in on-policy self-distillation (OPSD)…
