FIELD NOTES TOPIC
throughput optimization
1 note, newest first.
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
DSpark is a new speculative decoding framework for Large Language Models (LLMs) that aims to improve inference speed and maintain draft quality. It does this by using a semi-autoregressive architecture, which combines…
