FIELD NOTES TOPIC
gpt-style
1 note, newest first.
Can Scale Save Us From Plasticity Loss in Large Language Models?
Researchers study plasticity loss in GPT-style transformer models trained on a multilingual continual learning problem. They find evidence of plasticity loss across different sizes of models and under stationary…
