Local Stability and Gaussian Smoothing of Quantized Neural Networks

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Sergey Salishev, Anton Makarov, Oleg Granichin
• Venue: arXiv: cs.LG
• Published: 2026-07-22
• Collected: 2026-07-23
• URL: http://arxiv.org/abs/2607.20153v1
• PDF: https://arxiv.org/pdf/2607.20153v1
• DOI: UNKNOWN

Plain-Language Brief

The authors investigate using Gaussian averaging as a smooth alternative for quantized neural networks. They provide bounds on the difference between the original function (f) and the smoothed version (g), linking it to the stability analysis of discontinuous networks. They also compute closed-form Gaussian averages for rectified linear unit (ReLU) and sign activation functions, and demonstrate their method using a high-dimensional binary perceptron.

What the Document Says

• The authors study the use of Gaussian averaging as a replacement for quantized neural networks. They discuss a local dimension-dependent bound on |f-g| related to the stability analysis of discontinuous networks under bounded local oscillation.
• They compute closed-form Gaussian averages for rectified linear unit (ReLU) and sign activation functions.
• An example application is shown using a high-dimensional binary perceptron, where layer-preactivation aggregation under an explicit quantization-noise surrogate leads to the Gaussian envelope used in both smoothing during inference and smooth surrogate gradients during training.

Key Concepts Mentioned

• Gaussian averaging
• quantized neural networks
• local oscillation
• bounded local oscillation
• stability analysis
• discontinuous networks
• rectified linear unit (ReLU)
• sign activation functions
• binary perceptron
• smoothing during inference
• smooth surrogate gradients
• layer-preactivation aggregation
• quantization-noise surrogate

Claims Made by the Document

• Gaussian averaging can serve as a smooth alternative for quantized neural networks.
• Under bounded local oscillation, there is a connection between Gaussian smoothing and the stability analysis of discontinuous networks.

Evidence Used

• The authors discuss Gaussian averaging as a substitute for quantized neural networks.
• They derive a local dimension-dependent bound on |f-g| related to the stability analysis of discontinuous networks under bounded local oscillation.
• They provide closed-form solutions for Gaussian averages of rectified linear unit (ReLU) and sign activation functions.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

We study Gaussian averaging as a smooth surrogate for quantized neural models. Under bounded local oscillation, we derive a local dimension-dependent bound on |f-g|, linking Gaussian smoothing to the stability analysis of discontinuous networks. We compute closed-form Gaussian averages of the rectified linear unit (ReLU) and sign activation functions, and illustrate the mechanism on a high-dimensional binary perceptron, where layer-preactivation aggregation under an explicit quantization-noise surrogate yields the Gaussian envelope used in inference-side smoothing and training-side smooth surrogate gradients.