Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
4 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on speculative decoding
4 notes
Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Researchers propose a new method called EcoSpec for optimizing the performance of Large Language Models (LLMs) using Sparse Mixture-of-Experts (MoE). This method aims to improve decoding speed by considering the cost…
A Practical Investigation of Training-free Relaxed Speculative Decoding
The paper investigates training-free methods to speed up the sampling process of autoregressive Language Models (LLMs) by relaxing some strict guarantees in standard speculative decoding. The authors provide insights…
A Practical Investigation of Training-free Relaxed Speculative Decoding
The study investigates training-free methods of relaxed speculative decoding, which can speed up sampling from an autoregressive Language Model (LLM) by using a faster auxiliary model to draft tokens that are then…
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding
NVIDIA announced DFlash Speculative Decoding, an inference optimization technique for Blackwell GPUs targeting multi-agent AI workloads. The claimed performance improvement is up to 15x on NVIDIA Blackwell hardware.
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
