Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization

Source

arXiv

arxiv.org/abs/2606.26002v1

Read the source

Source

Plain-Language Brief

The paper presents HiReLC, a framework that uses reinforcement learning to compress deep neural networks by pruning and quantizing them. The framework divides the compression search into two levels: low-level agents (LLAs) make decisions per block about bitwidth, keep-ratio, quantization type, and granularity, while high-level agents (HLAs) coordinate global budget allocation. An iterative active learning loop is used to reduce computational cost by optimizing the policy with surrogate-guided reinforcement learning and post-compression fine-tuning.

What the Document Says

  • The authors present HiReLC, a framework for compressing deep neural networks using hierarchical reinforcement learning (RL).
  • The framework decomposes the compression search into two levels: low-level agents (LLAs) and high-level agents (HLAs).
  • LLAs operate independently per block and select per-kernel configurations over a multi-discrete action space.
  • HLAs coordinate global budget allocation via ensemble voting guided by Fisher Information-based sensitivity estimates.
  • An iterative active learning loop is used to reduce computational cost by interleaving surrogate-guided RL optimization with post-compression fine-tuning.
  • A lightweight MLP surrogate is used to amortize expensive evaluations and a logit-MSE proxy is used during cold-start.
  • The surrogate is used for reward shaping rather than as a replacement for final post-compression evaluation.
  • The controller is architecture-agnostic by design.

Key Concepts Mentioned

  • deep neural networks
  • hierarchical reinforcement learning
  • neural network compression
  • pruning
  • quantization
  • reinforcement learning
  • active learning
  • surrogate models

Claims Made by the Document

  • The framework effectively compresses deep neural networks.
  • The framework achieves parameter-storage compression ratios of 5.99 - 6.72$ imes$.
  • The framework gains 3.83 % in one setting and has 0.55 - 5.62 % accuracy drops elsewhere.

Evidence Used

  • Experiments across Vision Transformer and CNN benchmarks support the effectiveness of the framework.

Methods / Approach

  • reinforcement learning
  • active learning
  • surrogate models

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The iterative active learning loop might not be efficient for all types of neural networks or compression tasks.

Open Questions

  • What is the impact of HiReLC on different types of deep neural networks and compression tasks?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper does not specify the specific neural network architectures or compression benchmarks used in the experiments.

Raw Abstract / Excerpt

We present HiReLC, a hierarchical ensemble-reinforcement learning framework for automated joint quantization and structured pruning of deep neural networks. The framework decomposes the compression search across two levels of abstraction: low-level agents (LLAs) operate independently per block, selecting per-kernel configurations over a multi-discrete action space spanning bitwidth, pruning keep-ratio, quantization type, and granularity, while high-level agents (HLAs) coordinate global budget allocation via ensemble voting guided by Fisher Information-based sensitivity estimates. To mitigate the computational cost of policy evaluation, an iterative active learning loop interleaves surrogate-guided RL optimization with post-compression fine-tuning, using a lightweight MLP surrogate to amortize expensive evaluations and a logit-MSE proxy during cold-start. The surrogate is used for reward shaping rather than as a replacement for final post-compression evaluation. The controller is architecture-agnostic by design, with a modular layer abstraction decoupling the RL environment from the underlying network topology. Experiments across Vision Transformer and CNN benchmarks demonstrate effective parameter-storage compression ratios of 5.99 - 6.72$\times$ with a 3.83 % gain in one setting and 0.55 - 5.62 % accuracy drops elsewhere, supporting hierarchical policy decomposition and sensitivity-aware guidance as practical design choices for joint neural network compression.