FIELD NOTES TOPIC
annotation-free self-distillation
2 notes, newest first.
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
Researchers propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without human annotations in specialized domains where they are costly to obtain. This method…
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
Researchers have developed a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without the need for real-world interaction feedback or human-labeled supervision…
