DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Source

Source name: arXiv
Source kind: paper
Authors / organization: Xin Cheng, Xingkai Yu, Chenze Shao, Jiashi Li, Yunfan Xiong, Yi Qian, Jiaqi Zhu, Shirong Ma, Xiaokang Zhang, Jiasheng Ye, Qinyu Chen, Chengqi Deng, Jiping Yu, Damai Dai, Zhengyan Zhang, Yixuan Wei, Yixuan Tan, Wenkai Yang, Runxin Xu, Yu Wu, Zhean Xu, Xuanyu Wang, Muyang Chen, Rui Tian, Xiao Bi, Zhewen Hao, Shaoyuan Chen, Huanqi Cao, Wentao Zhang, Anyi Xu, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang
Venue: arXiv: cs.AI
Published: 2026-07-06
Collected: 2026-07-07
URL: http://arxiv.org/abs/2607.05147v1
PDF: https://arxiv.org/pdf/2607.05147v1
DOI: UNKNOWN

Plain-Language Brief

DSpark is a new speculative decoding framework for Large Language Models (LLMs) that aims to improve inference speed and maintain draft quality. It does this by using a semi-autoregressive architecture, which combines a parallel backbone with a lightweight sequential module to model intra-block dependencies and mitigate suffix decay. DSpark also employs confidence-scheduled verification, which dynamically adjusts the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks, DSpark improves accepted length over state-of-the-art autoregressive and parallel drafters. When deployed in a live user traffic system (DeepSeek-V4), it reduces verification waste, speeds up per-user generation by 60-85%, and enables performance tiers previously unattainable under strict interactivity constraints.

What the Document Says

DSpark is a speculative decoding framework for LLMs
It uses a semi-autoregressive architecture with a parallel backbone and lightweight sequential module
The goal is to maintain draft quality while improving throughput
DSpark employs confidence-scheduled verification for adaptive, load-aware verification
On offline benchmarks, DSpark improves accepted length over state-of-the-art autoregressive and parallel drafters
When deployed in DeepSeek-V4 system, it reduces verification waste, speeds up per-user generation, and enables new performance tiers

Key Concepts Mentioned

Speculative decoding
Large Language Models (LLMs)
DSpark framework
Semi-autoregressive architecture
Confidence-scheduled verification
DeepSeek-V4 serving system

Claims Made by the Document

DSpark improves accepted length over state-of-the-art autoregressive and parallel drafters on offline benchmarks
DSpark reduces verification waste when deployed in DeepSeek-V4 system under live user traffic
DSpark accelerates per-user generation speeds by 60 to 85% at matched throughput levels compared to the established production baseline (MTP-1)

Evidence Used

Improved accepted length is mentioned in the text
Reduction of verification waste is stated in the text
Acceleration of per-user generation speeds is stated in the text

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.