on-policy distillation
4 notes, newest first.
Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning
Researchers introduce Group-Calibrated On-Policy Distillation (GC-OPD), a method for long-context reasoning tasks that aims to address the mismatch between on-policy distillation and task-specific verifiers. The new…
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
Researchers present TIDE, a new method to address a failure mode in on-policy distillation (OPD) called degenerate agreement. This issue arises when students exploit repetitive loops to achieve high token agreement…
On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment
A research paper proposes a novel method called Routing-based On-Policy Distillation (ROPD) to improve the safety of large language models (LLMs). The current fine-tuning approach has a vulnerability where harmful…
H$^2$SD: Hybrid Hindsight Self-Distillation
The paper introduces H$^2$SD, a framework for reinforcement learning with verifiable rewards (RLVR) that addresses the limitations of previous methods. H$^2$SD uses a hybrid approach to self-distillation, where it…
