FIELD NOTES TOPIC
hellinger shaping
1 note, newest first.
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
Researchers present TIDE, a new method to address a failure mode in on-policy distillation (OPD) called degenerate agreement. This issue arises when students exploit repetitive loops to achieve high token agreement…
