Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

Source

arXiv

arxiv.org/abs/2606.23406v1

Read the source

Source

Plain-Language Brief

Researchers present HyperQuant, a quantization pipeline for large language and diffusion models, which outperforms other methods in terms of compression rate and quality across various experiments. The method combines four known techniques to make weight and activation distributions more Gaussian, use optimal lattices for quantization, compress lattice indices with Rice coding, and correct biases in the KV cache.

What the Document Says

  • HyperQuant is a unified post-training quantization pipeline for large language and diffusion models.
  • It outperforms HIGGS on weights quantization from 3 to 5 bits per scalar (bps) in self-contained experiments (Table 1).
  • It beats TurboQuant and OCTOPUS on KV quantization down to 1.7 bps.
  • In the LLM setting, HyperQuant compresses a 19B-parameter LTX-2 DiT video model with no observable per-frame artifacts.
  • On an H100 at 4 bps, it compresses linear weights ~3.9x and the KV cache ~3.79x at near-lossless quality.
  • HyperQuant combines four known ideas: a Randomized Hadamard Transform, quantization to an optimal lattice, Rice coding, and bias-correction methods for the KV cache.
  • It is integrated with 8-bit and 4-bit Tensor-Core MMA paths, and int8 beats fp8 on the post-RHT lattice output.

Key Concepts Mentioned

  • HyperQuant
  • quantization pipeline
  • large language models
  • diffusion transformers
  • HIGGS
  • TurboQuant
  • OCTOPUS
  • Rice coding
  • Randomized Hadamard Transform
  • optimal lattice
  • KV cache
  • bias-correction
  • 8-bit
  • 4-bit
  • Tensor-Core MMA paths
  • fp8
  • int8

Claims Made by the Document

  • HyperQuant outperforms HIGGS, TurboQuant, and OCTOPUS in various experiments.
  • HyperQuant can compress large language models with no observable artifacts.
  • Int8 beats fp8 on the post-RHT lattice output.

Evidence Used

  • Results from self-contained experiments (Table 1) provided.
  • Compression results for a 19B-parameter LTX-2 DiT video model mentioned.

Methods / Approach

  • Quantization to an optimal lattice
  • Randomized Hadamard Transform
  • Rice coding
  • bias-correction methods for the KV cache

Limitations Stated by the Authors

Not stated in source.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The effectiveness of HyperQuant may depend on the specific model and settings used.

Open Questions

  • How does HyperQuant perform in other types of models or tasks?
  • What are the potential downsides or trade-offs of using HyperQuant?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper does not provide specific details about the self-contained experiments (Table 1) or the benchmark used.

Raw Abstract / Excerpt

We present HyperQuant (Hadamard, optimallY Packing, Entropy Rice-coding), a unified post-training quantization pipeline for the weights and the KV cache of large language and diffusion transformers. Across a suite of self-contained experiments (Table 1), HyperQuant outperforms the recent HIGGS scheme at every operating point from 3 to 5 bits per scalar (bps) on weights, and beats both TurboQuant and OCTOPUS on KV quantization down to 1.7 bps. Beyond the LLM setting, HyperQuant quantizes the 19B-parameter LTX-2 DiT video model with no observable per-frame artifacts. End-to-end on an H100 at 4 bps, HyperQuant compresses the linear weights ~3.9x and the KV cache ~3.79x at near-lossless quality. HyperQuant combines four known ideas into a single construction: (i) a per-tile Randomized Hadamard Transform that makes the per-coordinate distribution of weights and activations approximately Gaussian; (ii) quantization to a low-dimensional optimal lattice (E8, D4, A2, or Z); (iii) lossless bit-stripping and near-entropy-optimal variable-length Rice coding of the lattice indices; and (iv) bias-correction methods for the KV cache that keep the reconstruction unbiased under inner products, preserving attention semantics. We further integrate the pipeline with 8-bit and 4-bit Tensor-Core MMA paths (fp8-e4m3, int8, nvfp4, mxfp4), and find that int8 beats fp8 on the post-RHT lattice output. Project page: https://moonmath.ai/hyperquant/

Links to this note 2