FIELD NOTES TOPIC
model compression
2 notes, newest first.
CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation
The research introduces Correlation Calibration-based Redundancy Suppression (CoCaRS), a method for knowledge distillation in heterogeneous settings where teacher and student models have different architectures. The…
SLORR: Simple and Efficient In-Training Low-Rank Regularization
The paper introduces SLORR, a simple and efficient in-training low-rank regularization framework for neural networks. It aims to improve compressibility without significant accuracy loss, while avoiding the limitations…
