Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch
FIELD NOTES TOPIC

neuron on-policy self-distillation

3 notes, newest first.

  1. Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

    Source note ·

    Researchers propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without human annotations in specialized domains where they are costly to obtain. This method…

  2. Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

    Source note ·

    Researchers have developed a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without the need for real-world interaction feedback or human-labeled supervision…

  3. Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

    Source note ·

    In this paper, the authors propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for annotation-free self-distillation of large language models. This method uses internal neuron activations to…