SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow, Lin Wang
• Venue: arXiv: cs.LG
• Published: 2026-07-20
• Collected: 2026-07-21
• URL: http://arxiv.org/abs/2607.18081v1
• PDF: https://arxiv.org/pdf/2607.18081v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers present SelectInfer, a neuron-level optimization framework for Large Language Models (LLMs) that can run efficiently on resource-constrained edge devices. It reduces memory footprint and computation by selectively loading and computing important neurons identified during offline profiling.

What the Document Says

• The paper discusses SelectInfer, a framework for efficient LLM inference on edge devices.
• SelectInfer achieves this efficiency through selective neuron loading and computation.
• Offline LLM profiler identifies task-specific and general-purpose neurons.
• Selective loading reduces memory footprint by loading only important neurons identified during offline stage.
• Selective computation computes only relevant neurons at runtime.

Key Concepts Mentioned

• Large Language Models (LLMs)
• Natural Language Processing (NLP)
• Resource-constrained edge devices
• Neuron-level optimization
• Offline LLM profiler

Claims Made by the Document

• SelectInfer achieves significant reductions in memory footprint and computation.
• SelectInfer preserves task performance.

Evidence Used

• Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices. Existing approaches to model compression and optimization often rely on coarse-grained pruning or quantization, which can compromise accuracy or require re-training and fine-tuning. In this work, we introduce SelectInfer, a neuron-level optimization framework that enables efficient LLM inference on edge devices through selective neuron loading and computation. By profiling and identifying both task-specific and general-purpose neurons using an offline LLM profiler, SelectInfer implements two key optimizations: selective loading, which reduces memory footprint by selectively loading a subset of neurons that were identified to be most important during the offline stage, and selective computation, which dynamically computes only the most relevant neurons at runtime. Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance, making it a practical step towards enabling LLM deployment on edge devices