Efficiency and inference
50 notes, newest first. Includes the vault tags efficiency, inference, quantization, performance improvement, mixture-of-experts, distillation, knowledge distillation.
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
The preprint proposes Reasoning Adaptation through Cross-Turn Estimation (RACE), which estimates when prior reasoning remains sufficient for subsequent actions. Its LoGiC procedure identifies reasoning turns whose…
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
EvoAlloc is described as a self-evolving resource-allocation agent for LLM-based program evolution. It consolidates prior search and allocation outcomes into reusable experience, revises its allocation strategy, and…
HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment
The preprint uses hidden states before generation and at the end of an answer to predict correctness for local-model deployment. HARISSA fine-tunes both states and uses a single policy to choose among increasingly…
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
The preprint studies whether reasoning efficiency can improve without directly optimizing length or adding an inference-time stopping mechanism. It fine-tunes models to predict confidence at intermediate points in…
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
Jev-Mobile separates low-frequency vision-language-model planning from high-frequency lightweight execution. The VLM supplies local goals, the accessibility tree defines a structured action space, and Jev repeatedly…
Learning the Cost of Reliable Inference
The paper proposes a procurement platform where model providers compete on task-level token prices through a reverse second-price auction. The platform learns provider quality and routes requests to a cost-competitive…
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
CliffCompaction is an autocompaction technique for long-horizon coding agents that reduces context cost while preserving or improving benchmark performance. It only truncates or drops content rather than rephrasing it…
CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference
CompKV selects token blocks jointly with the downstream compensation mechanism for sparse attention rather than choosing tokens only by attention mass. The preprint derives a block-level residual approximation based on…
Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
Growing Harness learns the agent harness itself from a strategy-free scaffold. Function-level traces localize failures, an optimizer repairs bounded code surfaces, and a held-out success-first gate rolls back repairs…
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
The preprint evaluates a decision-only judge as an economical first pass and escalates low-confidence cases to stronger evaluation. It compares the judge with 16 generative and reward-model judges using blinded human…
Harness-Zero: Harness Distillation via Agent-as-Harness
Harness-Zero uses a domain- or instance-optimized harness as training-time guidance, then transfers its behavior into model weights so the specialized harness can be removed at deployment. An agent-as-harness corrects…
Pinocchio: Fast Uncertainty Estimates for Black-Box Language Models
Pinocchio is an external calibrator for estimating response correctness from black-box language-model APIs without logits, weights, internal states, or fine-tuning access. The preprint reports a single-forward-pass…
An Empirical Study of Harness Design for Coding Agents
The paper varies planning, action space, and context management in a fixed coding harness and evaluates 176 matched settings across four models on SWE-Bench Verified and Terminal-Bench 2.1. It reports that context…
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
The record describes a 552B-backbone multimodal mixture-of-experts model with up to one million tokens of context, activating 16B parameters during decode and 8B during prefill. It reports cross-layer KV reuse, FP4 KV…
Score Centering Stabilizes Off-policy Reinforcement Learning
The preprint attributes instability in large-language-model reinforcement learning under training-inference mismatch to persistent drift between engines. It derives an additive score-centering correction that cancels…
SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness
SoL-Pi applies recursive self-improvement at the harness layer across many environments. The abstract describes mechanisms for action execution, context compaction, observation handling, and delegated reading; on a…
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video DeltaNet combines local Softmax attention with bidirectional linear memory for long-range video context. Its Video Delta Attention updates memory once per frame, with separate projections and gates for…
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The authors propose a method to utilize multiple Intel AI PCs with integrated GPUs and NPUs, even when they have insufficient memory for large models like a 70B-parameter LLM. The method involves breaking the model…
HiLS-Attention: Hierarchical Sparse Attention Done Right
Tencent's Hunyuan team released official code for HiLS-Attention, a chunk-based sparse attention method meant to make very long-context language models faster to run while keeping accuracy, learning which chunks of…
Argus: A General-Purpose Agentic Runtime for Long-Horizon Reasoning
The paper introduces Argus, a persistent, self-evolving runtime system designed for long-horizon reasoning. Argus consists of Manager, Planner, Engineer, and Reviewer that execute bounded missions over durable project…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
The researchers present a study to make knowledge distillation training efficient for small language models deployed under tight constraints. They propose two methods: offline knowledge distillation (caching teacher's…
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Researchers introduce CACHE-UK, a framework designed for efficiency and accuracy in deploying large language models (LLMs) in the dynamic field of finance. The system addresses the 'quantization stability crisis' that…
Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs
The research focuses on improving the performance of local autonomous computer-use agents (CUAs) under strict hardware constraints. The study investigates inference-time scaling across contextual, temporal, structural…
GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
The authors present GRADRAG, a framework for improving Retrieval-Augmented Generation (RAG) systems by coordinating updates across multiple agents in the pipeline. The system uses feedback from an Evaluator to update…
Taming the Security-Energy Paradox: A Green AI Approach to Optimized Android Malware Detection
Researchers have developed an energy-efficient method for Android malware detection using deep learning models, aiming to balance performance and battery life. They used quantized neural networks (QNNs) with reduced…
DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoning
The paper introduces Dependency-Aware Intermediate QA Supervision (DAIS), a framework that improves the accuracy of answering complex questions by providing intermediate rationales and conditioning local answers on…
Reachability-Aware Pretraining for Efficient Target-Oriented Path Exploration in Temporal Knowledge Graph Reasoning
The paper proposes a new method called RAPTOR for temporal knowledge graph (TKG) reasoning under the extrapolation setting. This method aims to improve the efficiency of target-oriented path exploration by using…
Scaling Behavior Foundation Model for Humanoid Robots
Researchers have developed a new method for improving the performance of Behavior Foundation Models (BFMs) in controlling humanoid robots. The method involves coordinating three core components: motion tracking…
Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution
Researchers have proposed a method called E3 (Estimate, Execute, Expand) to improve the efficiency of large language model (LLM) agents. E3 aims to estimate the difficulty of a task, execute a minimum viable path, and…
Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Researchers propose a new method called EcoSpec for optimizing the performance of Large Language Models (LLMs) using Sparse Mixture-of-Experts (MoE). This method aims to improve decoding speed by considering the cost…
Solution of the Hempel's statistical ambiguity problem and Causal AI
The paper presents a solution to Hempel's statistical ambiguity problem in inductive-statistical inference by introducing the concept of Maximally Specific Causal Relationships (MSCRs). The MSCRs are derived from a…
Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
Researchers propose a proactive memory agent for long-horizon tasks in which an action agent must make decisions based on state information scattered across an expanding trajectory. The proposed memory agent updates a…
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
The paper 'SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling' presents a new approach to scheduling Language Model (LLM) requests for AI agents, rather than humans. The…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors of this paper introduce a new method to evaluate the effects of post-training quantization on large language models, as this technique is commonly used to deploy models in resource-limited settings. The…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors investigate how the quantization process used to deploy large language models in resource-limited settings can lead to unnoticed behavioral changes, even when task performance seems unaffected. They…
Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass
The researchers present an approach called Jailbreak that allows analytical workloads to bypass the traditional database driver and read storage files directly, using Large Language Models (LLMs) to regenerate table…
DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail
Researchers present DT-Guard, a content safety guardrail model for large language models in open-world applications. The model uses a Reasoning-Active Training, Reasoning-Free Inference paradigm and aims to improve…
TREK: Distill to Explore, Reinforce to Refine
TREK is a procedure that uses distillation for exploration support expansion to improve policy optimization on tasks where the current policy stalls on hard prompts. It identifies low-performing prompts, generates…
TREK: Distill to Explore, Reinforce to Refine
The paper proposes a method called TREK (Teacher-Routed Exploration via Forward KL) to improve Group Relative Policy Optimization (GRPO). TREK uses distillation for exploration support expansion and can work with…
Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs
Researchers propose AgentSymbiotic, an iterative framework that enhances both large and small Language Model (LLM) performance in web browsing tasks by combining data synthesis with task-performance. Large LLMs…
Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation
Researchers propose a method to create motion-language agents that can understand and generate human movements in dynamic environments without forgetting previously learned skills. They introduce low-rank adaptation…
Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization
The authors present HiReLC, a framework that uses reinforcement learning to automatically compress deep neural networks by pruning and quantizing them. The framework works in two levels: low-level agents (LLAs)…
Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization
The paper presents HiReLC, a framework that uses reinforcement learning to compress deep neural networks by pruning and quantizing them. The framework divides the compression search into two levels: low-level agents…
Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization
The paper introduces HiReLC, a hierarchical reinforcement learning framework that automates the joint quantization and structured pruning of deep neural networks. It operates at two levels: low-level agents (LLAs)…
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
A study finds that using low-bit quantization to reduce the cost of large language models can lead to longer chains of thought, which offsets the expected per-token speedup in reasoning tasks. The study introduces a…
Semantic Consistency Policy Optimization for Reinforcement Learning of LLM Agents
Researchers propose Semantic Consistency Policy Optimization (SCPO), a method to optimize reinforcement learning for Language Model (LLM) agents. SCPO aims to reduce semantic credit inconsistencies that occur when…
WinDOM: Self-Family Distillation for Small-Model GUI Grounding
Researchers present WinDOM, a large dataset of bounding box training data for small GUI-grounding agents, which can be used for on-device deployment, accessibility tooling, and low-cost iteration. They also introduce…
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding
NVIDIA announced DFlash Speculative Decoding, an inference optimization technique for Blackwell GPUs targeting multi-agent AI workloads. The claimed performance improvement is up to 15x on NVIDIA Blackwell hardware.
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
The paper presents a method called GRINQH (GRaded INput-based Quantization Hierarchy) to accelerate the decoding process of language models during inference, particularly beneficial for edge-computing settings. The…
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Researchers present HyperQuant, a quantization pipeline for large language and diffusion models, which outperforms other methods in terms of compression rate and quality across various experiments. The method combines…
