FIELD NOTES TOPIC
hardware optimization
1 note, newest first.
Long-Context Fine-Tuning with Limited VRAM
Researchers introduce Hierarchical Global Attention (HGA) to reduce memory usage during long training sequences in large language models. They use segment-wise backpropagation, tiered KV storage, and detach older data…
