Large language models
384 notes, newest first. Includes the vault tags large language models, language models, language model, large language model, llm, llms, foundation models, transformers.
PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
PrivDrift is a benchmark for testing whether secrets disclosed during an active conversation remain recoverable after the dialogue shifts to unrelated topics and a later probe uses persuasion. It contains 1,000…
An Agentic Approach for Active Data Collection, Travel Behavior Modeling, and Weather-Sensitive Demand Prediction
The study presents a three-stage approach for combining digital data collection, predictive modeling, and weather-sensitive demand forecasting. It uses a chatbot to collect travel mode choices from student commuters…
Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents
This research study investigates how large language model (LLM) agents can transfer skills they have learned from completed tasks to perform better in new tasks. The authors compare two methods of inducing skills…
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
A research paper titled 'Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference' proposes a new method for creating small language models. Instead of adapting large models to CPUs, the authors…
MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Researchers have introduced a new benchmark called MemTrapBench to test the performance of large language models (LLMs) in terms of cognitive traps caused by their memory components. The study identifies two types of…
MidTool: Mid-training Data Synthesis for Agentic Tool Use
Researchers have developed MidTool, a system for teaching large language models how to use tools effectively during mid-training. This is done by combining large-scale web and PDF data with synthetic supervision from…
MidTool: Mid-training Data Synthesis for Agentic Tool Use
In this research, the authors introduce MidTool, a system used for training large language models during the mid-training stage. MidTool focuses on improving the model's abilities in general tool use, which includes…
MidTool: Mid-training Data Synthesis for Agentic Tool Use
In the paper 'MidTool: Mid-training Data Synthesis for Agentic Tool Use', researchers present a new open corpus construction pipeline called MidTool. This pipeline is designed to teach large language models how to use…
Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
The paper presents Task-CoEvolve, a new approach to optimize language model agent harnesses efficiently by adaptively selecting validation tasks. It addresses the challenge of using fixed validation sets by co-evolving…
Beyond the Transcript: Detecting Covert Co ordination in Latent Multi-Agent Communication
Researchers have developed Verifiable Latent Alignments (VLA), a framework for monitoring and controlling communication between language-model agents that occurs through hidden states not visible in public transcripts…
Beyond the Transcript: Detecting Covert Co ordination in Latent Multi-Agent Communication
Researchers have developed Verifiable Latent Alignments (VLA), a framework to monitor and steer private communication channels used by language model agents in multi-agent systems. The framework links the private…
Decomposing Wrong-Consensus Agreement in LLM Self-Consistency: A GPT-4.1 Case Study
The study presents an analysis of the inconsistency in the answers provided by Large Language Models (LLMs) when using majority voting to improve accuracy. The authors propose a quantitative measure called Gamma, which…
Identifying Implicit Premises for Logical Reconstruction of Argument Graphs
The authors propose a neuro-symbolic pipeline to generate implicit premises from natural language text, which can be translated into logical formulae to show the logical relationships between statements (entailment…
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The authors present a method to distribute Large Language Model (LLM) inference across multiple Intel AI PCs using pre-compiled pipeline shards. This allows for efficient use of idle AI PCs with integrated GPUs and…
rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation
The paper proposes a new approach for recommendation systems, called rEDMRec, which distills the reasoning of large language models into an editable memory that can be used to rank candidate items without regenerating…
Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models
The study explores the ability of large language models (LLMs) to extract contextualized information from research articles efficiently. The authors present four workflows that demonstrate LLMs can perform well in data…
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Researchers have developed a new framework called SPADE (Self-Play in Adaptive Synthetic Executable Environments). This framework allows language models to generate and improve their own training environments as they…
Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis
Researchers introduce a new method called Top-K prompting to train language models for single-step retrosynthesis, a crucial component of computer-aided synthesis planning. They use an ultra-large dataset of verified…
Tuning the Stochastic Machine: A Systems Engineer's Operating Model for Human-AI Engineering
The author, a systems engineer, argues that the persistent correction of errors in LLM (Language Model) assistants is an operations problem rather than a tooling problem. They propose seven principles for an operating…
Tuning the Stochastic Machine: A Systems Engineer's Operating Model for Human-AI Engineering
The author, a systems engineer, argues that issues with Large Language Models (LLMs) are more about operational procedures than tooling. They propose an operating model for human-AI engineering based on seven…
What is Missing from AI Post-Training AI: An Empirical Analysis
Researchers have found that large language models (LLMs) used to post-train other LLMs often follow one training strategy from the beginning and make only local adjustments within it, rather than reevaluating the…
Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies
The paper introduces Reconstruction, a blind benchmark for testing language models' ability to recover the true research idea of a published paper using only its pre-publication bibliography. The authors use an…
Semantic Bandits: In-Context Exploration-Exploitation is Biased by Semantic Priors
Researchers introduce the concept of semantic bandits to study how pre-training affects exploration behavior of Large Language Models (LLMs) used as decision-making agents. They find that action labels with semantic…
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
The text discusses the evolution of Large Language Models (LLMs) from text generation models to core components of agents that can perceive environments, invoke tools, and execute tasks. These LLM-driven embodied…
A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
A research study compared the performance of VITA, a specialized clinical AI tool for low- and middle-income settings, with various frontier large language models (LLMs) on the HealthBench benchmark. The study found…
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
This research paper provides a comprehensive review of various visual explanation methods used in explainable artificial intelligence, specifically focusing on Class Activation Mapping (CAM). The paper discusses the…
NetlistBench: Evaluating LLM Reliability in SPICE Netlist Recognition and Manipulation
Researchers present NetlistBench, a new benchmark for evaluating the reliability of Large Language Models (LLMs) in SPICE netlist recognition and manipulation tasks. The study finds that while LLMs perform well on…
One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL
The paper discusses a common issue in multi-agent reinforcement learning for human-AI interaction: using a single large language model (LLM) to simulate user behavior often results in the LLM focusing on specific…
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
Researchers have introduced a new benchmark called RealisticTritonBench to improve the evaluation of large language models (LLMs) in generating GPU kernels for AI frameworks using Triton. The new benchmark derives…
Most biomedical publications show signs of LLM-assisted writing
A study suggests that by the end of 2025, approximately 89% of biomedical papers on PubMed Central showed signs of LLM-assisted writing, with a higher usage in the Discussion section compared to the Methods section.
ReLTEx: Reliable LLM-based Taxonomy Expansion
The paper discusses a new framework called ReLTEx for improving the reliability of Large Language Model (LLM)-based taxonomy expansion. The framework combines LLM-driven candidate generation with structure-aware…
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?
Researchers introduce a new benchmark called VibeLifeBench to evaluate large language model (LLM) agents' ability to be proactive and consistent in long-horizon tasks across ten everyday life domains. The benchmark…
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
Researchers propose a verification layer using large language models (LLMs) to ensure the feasibility and ethical alignment of actions planned by robot autonomy systems. The system can evaluate the permissibility of…
From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch
The paper presents a new evaluation framework called 'Grip on LLMs' specifically designed for Dutch governmental use of large language models (LLMs). The framework is created in collaboration with domain experts and…
KGCaRe: Explainable Complex Conditional Question Answering using Automatic Knowledge Graph Construction and Context Retrieval with LLMs
The study proposes a new method called KGCaRe for answering complex conditional questions using large language models (LLMs) and retrieval-augmented generation (RAG). This approach combines neural retrieval with…
Parameter Exploration for RLVR via Variational Learning
Researchers propose a new approach called Perturbed Parameter Policy Optimization (3PO) to enhance exploration in large language models (LLMs). This method explores the parameter space instead of the action space…
Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach
Researchers propose a new method for fine-tuning large language models in a federated learning paradigm using Low-rank adaptation (LoRA). They investigate two strategies, Share-A/Local-B and Share-B/Local-A, and reveal…
SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Researchers have proposed a framework called Safety Harness Evolution (SHE) for large language model (LLM) agents. SHE aims to improve the safety of these agents by learning evolving safe boundaries from rollout…
SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Researchers propose a framework called Safety Harness Evolution (SHE) to improve the safety of large language model agents. SHE learns evolving safe boundaries from rollout trajectories, allowing for localized harness…
Stealing Reasoning Traces from Proprietary LLM APIs
Researchers have found a vulnerability in the way large language model providers handle encrypted reasoning traces. This vulnerability allows for decryption of these traces across different models within the same…
EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
Researchers have developed EnvACE, a method for training large language model agents that simulates environment interactions during training using world rehearsal instead of relying on real or synthesized executable…
EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
The paper introduces EnvACE, a method for training large language model agents for long-term tool use without relying on interactions with real or synthesized environments, which can be costly to construct and verify…
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
The paper introduces HarnessOpt-Bench, a benchmark for evaluating the ability of Language Models (LLMs) to optimize the harnesses surrounding them, which includes prompts, tools, control flow, memory, and orchestration…
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Researchers have introduced HarnessOpt-Bench, a benchmark for evaluating the performance of language models (LLMs) in optimizing harnesses (the tools and code surrounding them). The study tests 5 leading LLMs across 4…
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
The text introduces HarnessOpt-Bench, a benchmark for evaluating the performance of Large Language Models (LLMs) in optimizing AI systems. The benchmark involves an iterative process where an LLM edits a given harness…
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Researchers introduce HarnessOpt-Bench, a benchmark for evaluating the performance of Large Language Models (LLMs) in optimizing their own harnesses. The study examines five frontier LLMs on four tasks and finds that…
Learning Globally Reusable Skills for Coding Agents
Researchers present a framework called GSE (Globalized Skill Evolution) that aims to improve automated skill evolution for Large Language Model (LLM) agents in software engineering tasks. The proposed method addresses…
Learning Globally Reusable Skills for Coding Agents
Researchers propose a new method called GSE (Globalized Skill Evolution) to help Large Language Model agents continuously improve their skills without needing expensive retraining. The proposed method optimizes skill…
MicroEvo: Knowledge-Guided LLM Sampling for Efficient Microarchitecture Design Space Exploration
A new method called MicroEvo is proposed for efficient exploration of microarchitecture design spaces. This approach utilizes a combination of large language models (LLMs) and Monte Carlo Tree Search (MCTS) to optimize…
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
Researchers introduce Poli-Bias, a method to measure political bias in large language models (LLMs) by comparing responses to paired prompts where country identities are swapped across various geopolitical…
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
The text introduces a new method called Poli-Bias, which is designed to measure bias in large language models (LLMs) when dealing with international political conflicts. This framework compares responses to paired…
Reducing belief in conspiracy theories as they unfold using large language models
Researchers tested if conversations with a large language model (LLM) can reduce belief in emerging conspiracy theories. They conducted experiments in the days following two significant events, and found that…
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
Researchers audit the effectiveness of using visual tools like crop-and-zoom in multimodal language models by creating a causal graph to separate observation-mediated paths from action-induced shortcuts. They find that…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The research examines the limitations of current benchmarks used to evaluate large language models (LLMs) for safety, reliability, and deployment readiness. The study focuses on a popular LLM, ChatGPT, comparing its…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study compares two access modalities of a large language model (LLM), ChatGPT's chat UI and OpenAI's API, with and without web search enabled. The research shows that the accuracy of responses varies between the…
A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination
A research paper introduces a new framework called A-SR for symbolic regression. A-SR is designed to discover closed-form equations from data and uses a self-evolving agentic system that coordinates formula discovery…
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Researchers propose Observation-Calibrated Self-Distillation (OCSD) to address a confounding issue in On-Policy Self-Distillation (OPSD), a method used to train large language model agents. OCSD aims to attribute…
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Researchers propose a new method called Observation-Calibrated Self-Distillation (OCSD) to train large language models in reinforcement learning. OCSD addresses a confounding issue in previous methods by comparing two…
Chained Recursive Language Models for Multi-Iteration Reasoning
The paper proposes Chained Recursive Language Models (Chained RLM), an architecture for multi-iteration reasoning in large language models (LLMs). It addresses the difficulty in tasks requiring extraction, counting…
ContextWeave: A Real-World Workflow Benchmark
Researchers introduce ContextWeave, a new benchmark for evaluating the performance of language agents in long-horizon, stateful workflows. The benchmark measures how well an agent can recall and apply past experiences…
From Score Matrices to Football-Aware Match-State Simulation: An Auditable LLM Harness for Exact-Score Reranking
The paper presents an auditable information harness that combines dynamic Poisson-family models and large language models (LLMs) to forecast football scores. The system goes through four iterations, each improving on…
Hardware Design and Security in the Era of Chiplets and LLMs
The semiconductor industry is experiencing a double revolution with the shift towards heterogeneous 2.5D chiplet systems and the integration of Large Language Models (LLMs) into Electronic Design Automation (EDA)…
Item Response Theory for AI Safety
The paper presents a study using Item Response Theory (IRT) to analyze the safety of language models. The authors find that three factors - refusal strictness, truthfulness, and contextual harm - explain most of the…
Item Response Theory for AI Safety
Researchers used Item Response Theory (IRT), a statistical toolkit, to analyze the safety performance of 192 language models across eight different safety benchmarks. The study found that three key factors - refusal…
Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First
The researchers introduce SuperScout, a cost-effective solution for routing language models to resolve software issues in repositories. SuperScout first explores the repository and produces a structured handoff, which…
The Effect of Perceived Race and Gender on Police Language Use: Experimental Evidence from VR Simulations
Researchers studied how police officers speak to virtual characters depicted as Black adult males in VR simulations. The study found that most officers speak less deferentially to Black male characters, except for…
When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs
The study investigates how effective relaying key-value caches between multi-agent language models is, particularly when the receiver does not need the sender's private information. The research shows that when the…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
The researchers present a study to make knowledge distillation training efficient for small language models deployed under tight constraints. They propose two methods: offline knowledge distillation (caching teacher's…
VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
The text introduces VIBE, a new benchmark for analyzing the emotional tone of language models' outputs. The benchmark measures valence (positive or negative), arousal (exciting or calm), and dominance (powerful or…
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Researchers introduce CACHE-UK, a framework designed for efficiency and accuracy in deploying large language models (LLMs) in the dynamic field of finance. The system addresses the 'quantization stability crisis' that…
From Textual Requirements to Microservice Architectures - A Comprehensive Evaluation of LLM-Based Design Synthesis
Researchers evaluated the ability of a Large Language Model (OpenAI o3) to generate microservice architectures based on textual requirements. The study found that the LLM was more effective when given examples…
InfoOps Bench: A live information operations safety benchmark
The paper introduces a dynamic AI benchmark that measures the integrity of language models against being used for state-backed information operations. The benchmark uses data from over 2,100 information operations from…
InfoOps Bench: A live information operations safety benchmark
The paper introduces an AI benchmark that measures the safety of language models against being manipulated for state-backed information operations. The benchmark uses data from a live monitoring pipeline tracking…
MemHarness: Memory Is Reconstructed, Not Replayed
Researchers propose MemHarness, a framework for enhancing large language models that allows them to adapt and reconstruct past experiences based on the current context, rather than simply replaying them verbatim as…
MemHarness: Memory Is Reconstructed, Not Replayed
Researchers have developed MemHarness, a framework to improve large language model agents by enabling them to adapt and reconstruct past experiences based on the current context, instead of simply replaying stored…
One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence
The paper presents a study on auditing multiple Language Learning Models (LLMs) with limited budgets, where human auditors have to evaluate self-reported confidence that may be miscalibrated and correlated. The…
ORCA-bench: How Ready Are Language Model Agents for Oncall?
Researchers introduce ORCA-bench, a new benchmark that tests the ability of large language models to perform root cause analysis (RCA) in a production-like setting. The benchmark includes a microservice system with six…
ORCA-bench: How Ready Are Language Model Agents for Oncall?
The paper introduces ORCA-bench, a benchmark for testing the performance of large language models in oncall root cause analysis (RCA). The benchmark includes a live microservice system with six days of metrics, logs…
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
Researchers present PAIChecker, a system designed to detect misalignment between Pull Requests (PRs) and Issues in SWE-bench-like benchmarks used for evaluating Large Language Models (LLMs). The system identifies…
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
The paper proposes a system called PAIChecker to check for misalignment between Pull Requests (PR) and Issues in SWE-bench-like benchmarks used for evaluating Large Language Models' issue resolution capability. The…
When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence
The study presents a framework to quantify and address Derived-Feature Over-Trust (DFOT), which occurs when large language models (LLMs) misuse derived measurements as if they were direct facts. The researchers use…
When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences
The researchers propose a framework to analyze how large language models (LLMs) handle conflicting specifications when integrating multiple sources of information. They compare four types of representation: pure…
WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Researchers introduce WIDE, a dynamic token-level pruning framework for improving the efficiency of large language models (LLMs). The framework allows each token to select attention and FFN channel groups dynamically…
Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility
Researchers have developed a new method for controlling large language models during scientific discovery tasks. The method, called CostAda, takes into account the costs associated with each action in addition to the…
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
Researchers have developed a method to evaluate if latent channels in large language models effectively communicate task-relevant information in multi-agent systems. The study applies this method to Qwen3 and Qwen3-8B…
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
Researchers introduce a method to study how latent channels in multi-agent systems, which transmit information without using text, impact the receiver's performance. They apply this method to Qwen3-4B and Qwen3-8B on…
Improving Item Discoverability in e-Commerce Search via Related Intent Generation
The researchers present a scalable system called 'discovery-augmented search' that aims to improve the discoverability of items in e-commerce marketplaces, particularly grocery. This system uses intent-conditioned…
Linguistic Monoculture in LLM-Assisted Language Use
The study discusses the potential effects of relying on large language models (LLMs) for writing and communication. It suggests that widespread use could lead to a decrease in linguistic variety, a phenomenon referred…
OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding
Researchers have developed a new benchmark called OmegaUse-OfficeVal to test the performance of large language model (LLM) agents in completing long-horizon office-suite tasks. The benchmark includes 100 tasks that…
On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment
A research paper proposes a novel method called Routing-based On-Policy Distillation (ROPD) to improve the safety of large language models (LLMs). The current fine-tuning approach has a vulnerability where harmful…
Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents
A new approach, Cost-Aware Marginal Decision-Focused Stopping (CAM-DF), is proposed for tool selection in Large Language Model (LLM) agents that use external services. The method helps address the challenge of…
SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
SkillRise is a new reinforcement learning framework that helps large language model agents solve related yet distinct tasks more efficiently. It organizes tasks into sequences and uses a single policy to alternate…
Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents
Researchers have developed APPA (Agentic Permissions Policy Algebra), an Information Flow Control (IFC) framework designed to enhance security in Large Language Model (LLM) agents that handle mixed-confidentiality data…
Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls
Researchers introduce a new method called role-stratified per-field conformal risk control for language model agents that perform actions through structured tool calls. This method aims to ensure safety by setting…
Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs
A study evaluates three large language models (Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B) on their compliance with healthcare standards such as ICD-10, CPT, and HL7 FHIR in 320 clinical scenarios. The study found that…
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Researchers propose a framework called DataOrchestra that adapts data processing strategies for each example in Large Language Models (LLMs) pretraining, improving performance across 11 benchmarks.
DeepFaith: Evidence-Grounded LLMs for Faithful Incident Reporting in Multi-Stage APT Defense
In this paper, the authors propose a framework called DeepFaith to create more accurate and reliable incident reports in multi-stage APT defense. The framework uses large language models (LLMs) and ensures that…
Efficient LLM-Generated Shuttling Compilers for Complex Trapped-Ion Architectures
Researchers present a study where they use a large language model (LLM) called Claude Opus 4.7 to generate and refine Python code for shuttling compilers in trapped-ion quantum computers. The compilers are designed for…
Efficient LLM-Generated Shuttling Compilers for Complex Trapped-Ion Architectures
Researchers used a large language model named Claude Opus 4.7 to generate and refine Python code for shuttling compilers in trapped-ion quantum computers. The study covers three types of architectures: linear segmented…
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
The authors present a new approach for managing a language model's working memory by treating the decision of which stored items to keep as an estimation problem. They introduce a method called 'fixed-lag smoothing'…
Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers
Researchers present a new framework called RecursiveECG that uses a language model (LLM) to improve ECG classifiers by analyzing the reasons for individual case failures and suggesting targeted revisions based on…
Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers
Researchers present a new framework called RecursiveECG that uses a language model (LLM) to automatically design ECG classifiers by learning from specific failures and objective ECG evidence. The LLM is used as an…
From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
Researchers have proposed a new method called Multi-Agent Protocol Distillation (MAPD) to improve the performance of large language models in solving knowledge-intensive tasks. This method addresses the challenge of…
Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families
The research presents a new model-agnostic controller, Gubernaut Cognitive Controller (GCC), designed to control the behavior of Large Language Model (LLM) agents. The GCC aims to mitigate reactive failure modes in…
Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families
The research introduces Gubernaut Cognitive Controller (GCC), a model-agnostic runtime control layer designed to regulate the behavior of large language models (LLMs) that can exhibit reactive failure modes such as…
LEX-EC: A Lexical Evidence-Channel Audit Framework for Zero-Shot LLM Personality Classification in Black-Box Settings
Researchers introduce LEX-EC, a framework for analyzing the interpretability of large language models in assigning personality labels from text. The framework identifies and separates marginal distribution effects from…
LLM-SoccerArena: Benchmarking LLMs on Real-World Predictions in Sports
LLM-SoccerArena is a new platform that allows for the prospective evaluation of large language models' abilities to forecast real-world sports events before their outcomes are known. The platform provides a live…
Reason-Mediated Behavioral Models for Auditing LLM Social Simulators
Researchers propose a method to evaluate large language models used as social simulators, such as synthetic survey respondents. They argue that simply checking if the simulated outcomes match human outcomes is not…
Simulating Tenant Responses to Energy Policy Interventions with Transaction-Cost-Aware LLM Age
Researchers developed a method to simulate how tenants respond to energy-efficient renovation policies using large language models (LLMs). They introduce a friction-aware persona modeling approach that takes into…
The Tokenizer Tax: Quantifying and Explaining the Cross-Lingual Cost of Subword Tokenization for Indian Languages
The research quantifies the disadvantage faced by many non-English languages in large language models (LLMs) due to subword tokenizers that are primarily trained on English corpora. The study focuses on Indian…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers have developed an open-source framework to evaluate the effectiveness of AI agents powered by open-weight large language models (LLMs) in data preparation tasks for longitudinal population studies. The…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers have developed an open-source framework for evaluating the effectiveness of AI agents powered by open-weight large language models in data preparation tasks, specifically focusing on longitudinal population…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers introduce an open-source framework for evaluating the effectiveness of AI agents powered by open-weight large language models (LLMs) in data preparation tasks, specifically for longitudinal population…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers have developed an open-source framework for evaluating the effectiveness of AI agents powered by large, locally deployable language models (LLMs) in data preparation tasks, specifically focused on…
Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning
The authors investigate how large language models can learn from others without merely conforming to them, focusing on the distinction between incorporating others' perspectives and maintaining a well-grounded moral…
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
The paper presents a new method to detect which parts of a text are generated by language models (LLMs) in collaborative human-AI writing, operating at the token level and improving upon existing token-level detection…
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
Researchers present a new method for detecting parts of text generated by language models (LLMs) in human-AI coauthored documents. The approach works at the token level, using existing scores and a smoothing technique…
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
A new method for detecting parts of text generated by a language model (LLM) within human-written text is introduced. This method operates at the token level, reducing variability in adjacent tokens while using an…
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
The paper presents a new method for identifying parts of a text generated by Language Model (LLM) at the token level, improving on existing document-level methods. The method is based on smoothing adjacent token scores…
Euclid-MCP: A Model Context Protocol Server for Deterministic Logical Reasoning via Prolog
The paper introduces Euclid-MCP, an open-source MCP server that provides deterministic logical reasoning via SWI-Prolog. It addresses the issue of unreliable multi-step logical reasoning in Large Language Models (LLMs)…
Euclid-MCP: A Model Context Protocol Server for Deterministic Logical Reasoning via Prolog
The paper introduces Euclid-MCP, an open-source MCP server that provides deterministic logical reasoning via SWI-Prolog. It aims to bridge the gap between Large Language Models (LLMs) and multi-step logical reasoning…
From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics
The text discusses a new framework for modernizing Science, Technology, and Innovation (STI) analytics. The proposed framework aims to improve upon traditional bibliometric indicators by using dynamic knowledge graphs…
Improved lower bounds for the Shannon capacity of odd cycles
Researchers have improved lower bounds for the Shannon capacity of odd cycles, which quantifies the maximum rate at which information can be transmitted with zero error over a noisy channel. They achieved this by…
MIRROR: Learning from the Other View for Multi-Modal Reasoning
Researchers have created a new approach called Modality-Informed Reciprocal Reasoning Optimization (MIRROR) to improve the reasoning abilities of both large language models and vision-language models. The approach uses…
PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning
The paper proposes a new training paradigm called PATS for long-horizon reinforcement learning in language models. The method aims to improve weak policies that repeat similar failures by using a policy-centric…
Scaling Up Formal Representation of Clinical Trial Protocols in Ensemble Logic Using LLMs: A Preliminary Study
Researchers present CT-TEL workflow, a scalable pipeline that uses Large Language Models (LLMs) to convert clinical trial protocols from unstructured free text into Temporal Ensemble Logic (TEL) formulas. The study…
Thinkink: 2D Spatial Ink-native Interaction with LLMs
The researchers proposed a new tool called Thinkink that integrates large language models (LLMs) into handwritten note-taking and sketching practices. The tool allows users to handwrite or draw on a shared canvas, with…
Thinkink: 2D Spatial Ink-native Interaction with LLMs
Researchers propose Thinkink, a tool that allows users to interact with large language models (LLMs) using handwritten notes and sketches on a shared digital canvas. The tool was designed through a series of studies to…
Co-Evolving LLM Evaluators and Policies via DynamicRubric
The researchers propose DynamicRubric, a method that allows large language models to improve by co-evolving evaluators and policies. This method generates weighted binary rubrics for each candidate set and aggregates…
ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers
Researchers propose ELSAA, a method to approximate attention in Transformers more efficiently by using sparse and low-rank branches instead of calculating the full quadratic score matrix. This is done to handle longer…
Evaluating and Mitigating Gender Bias in Pre-trained Embeddings for ML-based Recruitment
Researchers evaluate the bias in pre-trained language models used for AI-based recruitment systems, focusing on gender leakage from CV data. They analyze nine models using a synthetic dataset and a multi-task…
HijackKV: New Threat in Position-Independent KV Cache Reuse
Researchers have identified a new threat called KV Cache Hijacking in position-independent KV cache reuse, which is used to reduce inference latency in large language models. This threat allows an attacker to silently…
OLEDLM: A Unified Language Model for OLED Molecular Design
Researchers propose a unified language model named OLEDLM for the design of OLED materials. The model directly generates OLED molecular structures that meet specified optoelectronic properties, using a multi-stage…
Agents in the Wild: Where Research Meets Deployment
This tutorial paper discusses the transition of large language model (LLM) based agentic systems from research to practical use in various fields like software engineering, scientific discovery, and finance. It…
AI Tour Meeting: Group Travel Planning by LLM Agents
This research presents AI Tour Meeting, a group travel planning system that uses multiple Large Language Model (LLM) agents with distinct personas to collaboratively create an itinerary based on their constraints and…
CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
Researchers propose a new framework called CASE to improve the faithfulness of chain-of-thought (CoT) reasoning in large language models (LLMs). The goal is to ensure that the generated reasoning supports the final…
Measuring Reward-Seeking via Contrastive Belief Updates
Researchers have developed a method to measure 'reward-seeking' in language models trained with reinforcement learning. This behavior occurs when the model optimizes for the grader's judgment instead of the intended…
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
The paper presents a new benchmark for testing the security of large language model (LLM) agents called 'Adaptive Adversaries'. The benchmark involves 21 scenarios where an adaptive attacker can observe prior defender…
GigaPath-Flash and GigaTIME-Flash: Efficient Pathology Foundation Models for Whole-Slide and Tumor Microenvironment Analysis
Researchers have developed two new foundation models called GigaPath-Flash and GigaTIME-Flash for analyzing whole-slide pathology and predicting the tumor immune microenvironment from routine H&E images. These models…
Harness Engineering for LLM-Driven GPU Kernel Generation
This research presents a system that uses large language models (LLMs) to optimize GPU kernels. The system separates an evaluation harness from an optimization controller, with human-authored skills and code agents…
Human Grounded Evaluation of Large Language Models for Optical Network Automation
The paper introduces a new evaluation pipeline named HuGLEN for comparing large language models (LLMs) used in network automation. The pipeline uses an LLM to evaluate other LLMs and rates them based on a quality…
LLMs and Agentic AI Systems for Smart Grids: A Tutorial on Architectures and Applications
The paper discusses how large language models (LLMs) and agentic AI systems are being used in smart grids for tasks such as forecasting, optimization, and control. The authors present a design principle that ensures…
LLMs and Agentic AI Systems for Smart Grids: A Tutorial on Architectures and Applications
The paper discusses the use of Large Language Models (LLMs) and Agentic AI systems in smart grids for tasks like forecasting, optimization, and control. It presents a design principle to ensure numerical results come…
Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation
Researchers have developed a system called Natural Language Knowledge Graph Query (NLKGQ) that allows researchers to ask ad-hoc questions about the contents of domain-specific archives using natural language. The…
Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation
Researchers have developed a system called Natural Language Knowledge Graph Query (NLKGQ) that allows them to ask ad-hoc questions about the contents of domain-specific archives using natural language, without…
PEARL: Auditable Repair for Scientific Reasoning Graph Extraction
The paper proposes a new framework called PEARL for extracting reliable scientific reasoning graphs from Large Language Models (LLMs). The LLM outputs often contain errors such as malformed syntax, incorrectly oriented…
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
Researchers propose a simple method to merge large language models with significantly different parameters without training or alignment by using dimensional adaptation and ratio-controlled interpolation. The method…
SGA: Plug&Play Geometric Verification for Educational Video Synthesis
Researchers propose a new module called Symbolic Geometric Agent (SGA) that works with existing animation pipelines to improve spatial correctness and visual legibility for educational videos. The SGA intercepts code…
SGA: Plug&Play Geometric Verification for Educational Video Synthesis
The authors propose the Symbolic Geometric Agent (SGA), a tool for improving the spatial correctness and visual legibility of educational animations generated by Large Language Models (LLMs) using libraries like Manim…
Towards Agentic Agent-based Models: Feasibility, Performance, and Statistical Model Checking
Researchers explore the feasibility, performance, and reliability of using large language models (LLMs) in Agent-based Models (ABMs), specifically with Mesa ABM models. They introduce a hybrid population to test…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The researchers evaluate the performance of language model security agents, focusing on costs rather than just success rates. They use offensive Cybench challenges and defensive Splunk BOTS v1 investigation challenges…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The research focuses on evaluating the efficiency and practicality of language model security agents in both offensive and defensive tasks, rather than just their peak offensive capability. The study uses cost-aware…
Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
Researchers present a multi-agent framework that simulates political coalition formation using Large Language Models (LLMs). The framework combines Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO)…
Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
Researchers have developed a multi-agent framework using Large Language Models (LLMs) to simulate political coalition formation. The framework combines Supervised Fine-Tuning, Direct Preference Optimization, and…
Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
Researchers present a multi-agent framework using Large Language Models (LLMs) to simulate and audit political coalition formation in elections. The framework combines Supervised Fine-Tuning (SFT), Direct Preference…
Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents
Researchers have developed a multi-agent framework using Large Language Models (LLMs) to simulate coalition formation in political elections. The framework combines Supervised Fine-Tuning, Direct Preference…
Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
The authors show that fine-tuning language models on specific datasets can cause broad shifts in ideology across various topics, even when the training data is factually defensible and moderation-passing. This…
Mask-Aware Policy Gradients for Diffusion Language Models
The paper presents a method for improving the performance of large language models in tasks like mathematical reasoning and coding by using reinforcement learning. This method addresses the challenge of extending…
Pretraining Data Can Be Poisoned through Computational Propaganda
The research discusses the potential for malicious content to be injected into large-scale pretraining datasets used for training language models (LMs). The authors propose a novel analysis method, HalfLife, to…
Pretraining Data Can Be Poisoned through Computational Propaganda
The research shows that harmful information can be introduced into language models during their pre-training phase by manipulating web-scale discussion interfaces, such as online forums or comment sections. A new…
SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration
The paper introduces SearchOS, a system designed to improve the efficiency of open-domain information-seeking agents by making their search progress more explicit and persistent. It aims to prevent repetitive loops…
Self-Evolving Human-Centered Framework for Explainable Depression Symptom Annotation
Researchers propose a self-evolving annotation framework to create explainable and reliable datasets for Major Depressive Disorder (MDD) research. The framework uses a large language model (LLM) in combination with…
Self-Evolving Human-Centered Framework for Explainable Depression Symptom Annotation
Researchers have proposed a new framework to improve the quality of annotations in depression-related datasets for artificial intelligence systems in mental health research. The framework uses a large language model…
Show Me How You Reason and I'll Tell You Who You Are: Reasoning Graphs for Robust LLM Authorship Attribution
Researchers propose a graph neural network approach to detect and analyze reasoning structures in large language models (LLMs) generated texts for improved robustness against paraphrasing and obfuscation techniques…
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
The text is about AgentCompass, an open-source evaluation infrastructure designed to evaluate Large Language Models (LLMs) as they evolve into autonomous agents. The infrastructure aims to address the current…
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
The text discusses AgentCompass, an open-source infrastructure designed to evaluate Large Language Models (LLMs) as they transition into autonomous agents. The tool is lightweight, extensible, and organizes the…
AIMO Interpretability Challenge
The text proposes the AIMO Interpretability Challenge, a competition aimed at distinguishing robust from spurious reasoning in advanced mathematical language models. The challenge will provide participants with new…
Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System
The MEDA system is a new framework that uses large language models and symbolic regression to automatically discover ordinary differential equation (ODE) models for biological systems. The system retrieves background…
Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models
Researchers propose a new method called Deep Interaction to correct errors made by large language models (LLMs) during complex, multi-step tasks. The method allows for direct editing of the original response, improving…
Experience Memory Graph: One-Shot Error Correction for Agents
Researchers propose a new framework called Experience Memory Graph (EMG) to help large language model agents recover from failures more effectively in complex, long-horizon tasks. Instead of using prompt-based…
Experience Memory Graph: One-Shot Error Correction for Agents
The researchers propose a new framework called Experience Memory Graph (EMG) to help large language model agents recover from failures more effectively in complex, long-horizon tasks. EMG reformulates failure recovery…
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
The paper proposes a new framework called Memory as a Controlled Process (MemCon) for large language model (LLM) agents that use external memory systems. MemCon learns an online policy to adaptively decide when, what…
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
The paper presents a new framework called Memory as a Controlled Process (MemCon) for large language model (LLM) agents. MemCon aims to address the limitation of existing memory systems that rely on fixed…
Traffic-Aware Randomized Smoothing for LLM-Based Network Intrusion Detection
The paper introduces Traffic-Aware Randomized Smoothing (TA-RS), a defense mechanism for Large Language Model (LLM)-based intrusion detection systems to enhance their robustness against feasible traffic manipulation…
A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism
The research investigates the effectiveness of Reinforcement learning with verifiable rewards and Group Relative Policy Optimization (GRPO) on a small language and vision-language model web agent at scales ranging from…
Atomic Units of X: The Compression Layer of Intelligence
The paper proposes a theoretical framework that views intelligence as a process of compressing complex phenomena into reusable atomic units. This is supported by evidence from various fields, including cognitive…
Bulkhead: Automated Semantic Detection and Remediation of Container Escape Vulnerabilities
Researchers have developed Bulkhead, a framework that uses large language models and formal methods to automatically detect and fix container escape vulnerabilities caused by path traversal. This is done through…
Bulkhead: Automated Semantic Detection and Remediation of Container Escape Vulnerabilities
Researchers present Bulkhead, an automated framework that uses large language models and formal methods to detect and fix container escape vulnerabilities called path traversal (PaTra) vulnerabilities. These…
Can Induced Emotion Bias LLM Behaviors in Sequential Decision Making?
Researchers investigated if emotions induced in large language models (LLMs) could influence their sequential decision-making using the Iowa Gambling Task (IGT). They found that, unlike humans, induced emotion did not…
Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs
The paper introduces EG-VAR, a tool-calling architecture that aims to make the reasoning of a large language model (LLM) more trustworthy by verifying all outputs through tool-attestation axioms and kernel checks. The…
Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models
The study introduces a methodology called PoPE (Popperian Placebo-controlled Evaluation) to measure whether a frozen small code language model (LLM) can use evidence that falsifies its own generated code. The study…
Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Researchers propose a new method called EcoSpec for optimizing the performance of Large Language Models (LLMs) using Sparse Mixture-of-Experts (MoE). This method aims to improve decoding speed by considering the cost…
LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos
The study introduces Elenchos, a framework to evaluate the abductive reasoning capabilities of large language models (LLMs). The research finds that while LLMs can detect when changes have been made in a formal system…
LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos
Researchers introduce Elenchos, a framework to evaluate abductive reasoning (inferring latent hypotheses) of Large Language Models (LLMs). The study finds that LLMs can often detect changes in formal systems but…
PalmClaw: A Native On-Device Agent Framework for Mobile Phones
Researchers have developed a new agent framework called PalmClaw, which allows large language model agents to run natively on mobile phones. This framework aims to overcome the limitations of existing mobile agents…
Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels
The research proposes JADR, a protocol for measuring the internal safety mechanisms of language models before their first response is generated. It uses Jacobian space to record top-k J-space tokens from prompts and…
Tracing Agentic Failure from the Flow of Success
The paper proposes a new method called OAT (One-Class Attribution for Tracing) for identifying errors in language models without relying on expensive or difficult-to-collect data. This method is faster and more…
Win by Silence: Deletion Non-Monotonicity, Autonomous Exploitation, and Typed-State Gating in LLM Plan Evaluation
The study examines how plan evaluators may inadvertently reward less explicit plans generated by Language Model (LLM). The research focuses on a staged expected-value scorer for LLM-produced venture routes and…
Agentic Routing: The Harness-Native Data Flywheel
The authors propose a new method for routing large language model agents called Harness-Native agentic routing. This method selects either a single best-fit model or multiple complementary models based on the full…
Agentic Routing: The Harness-Native Data Flywheel
The authors propose a new method of model selection for large language model agents called Harness-Native agentic routing. This method selects the best-fit model or multiple complementary models based on the full…
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
A research study presents an approach to understanding and mitigating bias in large language models (LLMs) used as judges, focusing on the hidden state of these models rather than just their input-output level. The…
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
The research investigates bias in Large Language Models (LLMs) acting as judges by exploring representation-level accounts, focusing on the hidden state of the LLM, complementing input-output level studies. The study…
MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Models
The paper presents MAGIC, a system that generates connected multi-scene game worlds using large language models. The system overcomes three challenges in single-scene methods: cross-scene consistency, in-scene…
MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Models
The research paper introduces MAGIC, a system that generates multi-scene game worlds using large language models (LLMs). The system overcomes three obstacles in single-scene methods: cross-scene consistency, in-scene…
Metacognition in LLMs: Foundations, Progress, and Opportunities
This paper provides a comprehensive overview of metacognition in Large Language Models (LLMs), highlighting the importance of this aspect for effective learning, problem solving, and communication. It discusses recent…
Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals
The researchers propose a new method called Proxy-guided Update Signal Transfer (PUST) for improving the domain-specific capabilities of large language models (LLMs). Instead of using the primary model for exploration…
Reproducing human biases in route choice using large language models: Toward scalable behavioral modeling
Researchers investigate if large language models (LLMs) can reproduce human biases in decision-making, specifically in route choice, without explicitly specifying parameters for Cumulative Prospect Theory (CPT)…
Reproducing human biases in route choice using large language models: Toward scalable behavioral modeling
This research explores the ability of large language models (LLMs) to reproduce human biases in route choice decisions, traditionally described using Cumulative Prospect Theory (CPT). The study finds that LLMs can…
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
Researchers present G-Frame, an adaptive multi-agent framework that reduces hallucinations in lightweight large language models used in rule-based scientific domains. The framework is based on Bayesian and team game…
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
Researchers present G-Frame, a new framework that uses artificial intelligence (AI) to create a high-quality dataset and train a model called OmniChem. This model performs as well as another AI model (GPT 4o mini) on…
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
Researchers have developed a new framework called G-Frame that helps lightweight Large Language Models perform better in rule-based scientific domains by reducing hallucinations. They created a new model called…
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
The paper proposes SMetric, a new scheduling method for Large Language Models (LLMs) in serving agents. The method aims to balance the session-centric workload of agents and increase Key-Value (KV$) reuse by…
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
The paper proposes a new LLM (Language Model) scheduling method called SMETRIC for serving agents. This method aims to balance the workload between instances by focusing on the first request of each agent session and…
SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets
The authors propose SolarChain-Eval, a benchmark for evaluating the trustworthiness of economic agents in decentralized energy markets. This benchmark ensures that agents make hourly decisions based on a…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors of this paper introduce a new method to evaluate the effects of post-training quantization on large language models, as this technique is commonly used to deploy models in resource-limited settings. The…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors investigate how the quantization process used to deploy large language models in resource-limited settings can lead to unnoticed behavioral changes, even when task performance seems unaffected. They…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
This research paper introduces a new evaluation metric called 'correctness agreement' for large language models that undergo post-training quantization, a method used to deploy these models in resource-constrained…
Towards Precision Therapy in Hepatocellular Carcinoma: A Clinical-Reasoning LLM for Risk Stratification and Treatment Guidance
A large language model named HCC-STAR has been developed to provide personalized treatment recommendations, risk scores, and survival estimates for patients with Hepatocellular Carcinoma (HCC). The model reads through…
Towards Precision Therapy in Hepatocellular Carcinoma: A Clinical-Reasoning LLM for Risk Stratification and Treatment Guidance
Researchers present HCC-STAR, a large language model that reads electronic medical records (EMRs) of patients with Hepatocellular Carcinoma (HCC), a common and deadly type of liver cancer. The model suggests treatments…
Two Axes of LLM Abstention: Answer Correctness and Question Answerability
The paper discusses the issue of language models providing incorrect or unanswerable responses. The author proposes two separate axes for abstention: one for answers the model would get wrong, and another for questions…
Two Axes of LLM Abstention: Answer Correctness and Question Answerability
A study finds that large language models often struggle with two types of errors: answering questions incorrectly and attempting to answer unanswerable or misleading questions. The researchers propose a method to…
WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search
WebSwarm is a framework that uses multiple language model-based search agents to perform complex, deep-and-wide web searches. It aims to overcome the limitations of single ReAct-style agents by dynamically…
WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search
The paper introduces a new framework called WebSwarm, which uses multiple language model (LLM)-based search agents to handle complex, deep, and wide web searches more effectively than single agents or existing…
When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability
The study investigates the inconsistency in Language Model (LLM) as judge scores when different evaluators are used. It compares two methods of upgrading LLMs: scaling Qwen3 dense judges and moving across MiniMax APIs…
Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows
The paper proposes a conceptual model for large language models (LLM) applications that use workflows. This model represents various elements of the workflow, such as definitions, instances, and dependencies, as…
Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows
The paper proposes a conceptual model for large language model (LLM) applications that use explicit workflows. The model is designed to represent various elements of the workflow such as definitions, instances, and…
Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass
The researchers present an approach called Jailbreak that allows analytical workloads to bypass the traditional database driver and read storage files directly, using Large Language Models (LLMs) to regenerate table…
Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass
The paper introduces Jailbreak, an approach that allows bypassing database engines by reading storage files directly and materializing data as in-memory columnar buffers. This method utilizes Large Language Models…
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
Researchers introduce DiaLLM, a system that aims to address the issue of large language models only producing standard English while understanding dialectal English. The study compares three open-weight language model…
From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
Researchers propose a framework called EvoSOP to help large language model (LLM) agents optimize their toolset by synthesizing atomic actions into reusable Standard Operating Procedures (SOPs). This approach aims to…
From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
Researchers propose a method called EvoSOP to improve Large Language Model (LLM) agents' interaction with the real world. The proposed method optimizes the tools used by LLM agents through creating reusable Standard…
Future Confidence Distillation in Large Language Models
The paper investigates how the confidence of large language models evolves during the answering process. It compares pre-solution and post-solution confidence estimates across different models and finds that…
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
The paper proposes a new method called Single-rollout Asynchronous Optimization (SAO) for Reinforcement Learning (RL) in post-training large language models (LLMs). SAO aims to improve stability and task effectiveness…
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
The paper introduces Single-rollout Asynchronous Optimization (SAO), a new approach to reinforcement learning (RL) aimed at post-training large language models (LLMs). SAO addresses stability and off-policy challenges…
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
In this paper, the authors introduce Single-rollout Asynchronous Optimization (SAO), a new approach to reinforcement learning (RL) for post-training large language models (LLMs). SAO aims to improve training stability…
An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery
The researchers propose an experimental design framework to systematically evaluate the autonomous model discovery behavior of large language model coding agents, like Codex and Claude Code. The framework considers…
DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
The research introduces DepthWeave-KV, a new cache compression method for long-context language models. This method adapts to the needs of different tokens by factorizing key and value states across neighboring…
Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
The researchers introduce a method called a recall-controlled probe cascade that predicts early on if an interaction between a large language model (LLM) agent and a task is likely to fail. This prediction allows for…
Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
Researchers propose a method to predict early on if large language models (LLMs) are likely to fail at solving multi-step tasks and abort the process before consuming too much compute. This method uses probes on hidden…
From Application-Layer Simulation to Native Meta-Architecture: Structural Tension as an Endogenous Driver for Heterogeneous AI Evolution
The paper proposes a theoretical framework to integrate higher-order cognitive architectures into a native meta-architecture for large language models (LLMs). This is achieved by introducing three mechanisms…
From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
Researchers present a framework using question-type-specific large language models (LLMs) to improve answer robustness and evidence grounding in biomedical question answering. The framework uses different inference…
From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
A study presents a framework for improving the accuracy and reliability of biomedical question answering by using a large language model (LLM) that adapts its inference strategies based on the type of question (yes/no…
From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b
Researchers have developed a system for answering biomedical questions using large language models (LLMs). The system is designed to improve the robustness of answers by selecting different inference procedures for…
Harnessing Code Agents for Automatic Software Verification
A new method using a general language model code agent and a verification harness proves all targeted lemmas in Iris, Rust's standard libraries, and unfinished Lean 4 port of Iris, achieving full coverage without any…
Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design
Researchers have improved the quality of process models generated by large language models (LLMs) using reinforcement learning (RL). They found that designing the reward function is crucial for optimizing beyond the…
Information Gain-based Rollout Policy Optimization: An Adaptive Tree-Structured Rollout Approach for Multi-Turn LLM Agents
The paper proposes a new method called Information Gain-based Rollout Policy Optimization (IGRPO) for improving large language model (LLM) agents in long-horizon search tasks. The method addresses the issue of existing…
Information Gain-based Rollout Policy Optimization: An Adaptive Tree-Structured Rollout Approach for Multi-Turn LLM Agents
The paper presents Information Gain-based Rollout Policy Optimization (IGRPO), a new framework for improving large language model agents in long-horizon search tasks. IGRPO aims to allocate rollout budget more…
LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability
This research investigates the performance of large language models (LLMs) in collaborative decision-making tasks where information is not fully available to all agents. The study introduces a benchmark for such tasks…
Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows
Researchers have developed Spider 2.0-AIFunc, a benchmark for testing the ability of language models to generate AI-native SQL queries on large cloud data platforms. The benchmark contains 465 verified instances across…
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
The paper presents AgentGym2, a new evaluation framework for language models (LLMs) that are increasingly being used in real-world production environments. Most existing benchmarks evaluate these models in simplified…
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
The paper presents AgentGym2, a new evaluation framework designed to test language models in realistic, complex scenarios that mimic real-world deployments. It highlights that most existing benchmarks do not fully…
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
DSpark is a new speculative decoding framework for Large Language Models (LLMs) that aims to improve inference speed and maintain draft quality. It does this by using a semi-autoregressive architecture, which combines…
LLM-as-a-Verifier: A General-Purpose Verification Framework
The paper introduces LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback to improve the accuracy of agentic tasks. The framework scales across multiple dimensions for better…
LLM-as-a-Verifier: A General-Purpose Verification Framework
Researchers introduce a new framework called LLM-as-a-Verifier to improve the ability of Large Language Models (LLMs) to verify the correctness of solutions for various tasks. The method provides fine-grained feedback…
LLM-Based Test Oracles: Source-of-Authority Taxonomy -- A Systematic Literature Review
A study on Large Language Models (LLMs) used to determine correct behavior in tests, identifies a gap in understanding the source of these models' authority. The study conducts a systematic literature review and finds…
Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing
In this research, the authors propose Localized LoRA-MoE, a framework that combines localized spatial blocking with dynamic, context-conditioned routing for Large Language Models (LLMs) and high-dimensional perception…
MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
MetaSkill-Evolve is a two-timescale framework that allows self-improving language models to improve not only their task skills but also the improvement procedure itself. The authors claim that this recursive approach…
Selective Disclosure Watermarking for Large Language Models
Researchers have proposed a new watermarking method for large language models (LLMs) that allows selective disclosure of embedded metadata. This means that the method can control which parts of the embedded message are…
Toward Trustworthy Large Language Model Agents in Healthcare
A study presents CareConnect, a safety-first conversational agent for automating healthcare appointment scheduling using large language models (LLMs), retrieval-augmented generation (RAG), and safety guardrails. The…
Unified Audio Intelligence Without Regressing on Text Intelligence
The researchers introduce a unified audio-text language model called Audex, which is built on a strong text-only model. Audex allows for strong fusion of audio and text, seamless multimodal generation, and…
Weak-to-Strong Generalization via Direct On-Policy Distillation
Researchers propose a method called Direct On-Policy Distillation (Direct-OPD) for improving stronger language models by leveraging the knowledge gained from reinforcement learning on smaller, weaker models. This…
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
Researchers have identified a new type of attack, called Forged Amplifying Rationale Memory Attack (FARMA), that can manipulate a large language model (LLM) agent's reasoning history by inserting forged traces using…
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
Researchers introduce a new type of attack, called Forged Amplifying Rationale Memory Attack (FARMA), that targets the reasoning history of large language model (LLM) agents. FARMA inserts forged reasoning traces using…
AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
The authors introduce a new bounded contract for long-horizon Large Language Model (LLM) agents in the game Slay the Spire 2. The contract allows decisions to be made from a fresh user message, with no raw…
Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach
The paper evaluates whether four large language models can approximate the judgement of experts when grading Linux/bash command responses from second-year Computer Engineering students. The study uses a four-level…
Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach
This research evaluates the effectiveness of four large language models in grading Linux/bash command responses from second-year Computer Engineering students' exams. The study uses a four-level cognitive taxonomy…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a novel framework for training large language models (LLMs) that addresses the issues of overfitting, suppression of exploration, cross-domain generalization problems, and privileged…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework designed to address issues in on-policy self-distillation (OPSD) for training large language models. The main problems being addressed are overfitting, suppression of…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
DemoPSD is a new framework for training large language models that addresses overfitting, privileged information leakage, and exploration suppression issues in on-policy self-distillation (OPSD). It does this by…
DemoPSD: Disagreement-Modulated Policy Self-Distillation
The paper introduces DemoPSD, a framework for training large language models that addresses the issues of overfitting, suppressed exploration, and privileged information leakage in on-policy self-distillation (OPSD)…
DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models
Researchers introduce DRIFTLENS, a framework to measure changes in the reasoning process used by personalized language models due to user attributes, preferences, and prior context. The study shows that these…
Fast Multi-dimensional Refusal Subspaces via RFM-AGOP
The text discusses a new method for identifying multi-dimensional refusal subspaces in Large Language Models (LLMs), which are used for safety and interpretability. The proposed method adapts the Recursive Feature…
LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning
Researchers introduce LACUNA, a testbed for evaluating the precision of unlearning methods in large language models (LLMs). The testbed allows direct evaluation of whether unlearning targets the weights responsible for…
LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning
The researchers introduce a new testbed called LACUNA for evaluating the precision of post hoc removal methods (unlearning) for LLMs. The testbed focuses on injecting personal identifiable information into specific…
Neuron-Aware Active Few-Shot Learning for LLMs
The paper introduces a new method called NeuFS for Active Few-Shot Learning (AFSL) in Language Models (LLMs). Instead of relying on output-level signals to identify valuable unlabeled samples, it focuses on the…
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
Researchers propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without human annotations in specialized domains where they are costly to obtain. This method…
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
Researchers have developed a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for training large language models without the need for real-world interaction feedback or human-labeled supervision…
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
In this paper, the authors propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD) for annotation-free self-distillation of large language models. This method uses internal neuron activations to…
Online Safety Monitoring for LLMs
Researchers propose a simple online safety monitoring system for Large Language Models (LLMs). The system uses a verifier signal from an external model to decide when safety can no longer be assumed, by setting a…
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Researchers have proposed a new method called Recursive Evidence Replay (RECONTEXT) to improve the ability of large language models to understand and reason over long contexts. This method doesn't require training…
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Researchers propose a new method called Recursive Evidence Replay (RECONTEXT) to improve the ability of large language models (LLMs) to understand and reason over long contexts. This method doesn't require training and…
SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces
The paper proposes SkillFuzz, a method to detect potential unintended objectives (implicit intents) in Large Language Model (LLM)-based agents that use reusable skills from open skill marketplaces. SkillFuzz identifies…
SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces
The paper presents SkillFuzz, a method for discovering unintended objectives (implicit intents) in Large Language Model (LLM)-based agents that use reusable skills from open skill marketplaces. SkillFuzz works by…
Text-Driven 3D Indoor Scene Synthesis in Non-Manhattan Environments
Researchers have developed a new method called SPG-Layout for creating realistic indoor scenes in complex, non-rectangular (non-Manhattan) environments using large language models. This method improves upon existing…
Agentic generation of verifiable rules for deterministic, self-expanding reaction classification
Researchers have developed an automated pipeline using large language models (LLMs) that can classify reactions in chemistry and generate rules for these reactions, expanding a standard taxonomy from 68 to 14,073…
AutoMem: Automated Learning of Memory as a Cognitive Skill
Researchers developed AutoMem, a framework for improving a language model's memory management skills in long-horizon games. By treating memory as a trainable skill, they allowed the model to decide how to manage its…
Autonomous Scientific Discovery via Iterative Meta-Reflection
Researchers present DiscoPER, a framework for autonomous scientific discovery using large language models. It can explore datasets without predefined research objectives, conduct statistical testing to ensure validity…
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
The paper introduces Distill to Detect (D2D), a method that helps identify hidden biases in language models used in high-stakes roles by amplifying these biases through cartridge distillation. The method is designed to…
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Researchers have developed a method called Distill to Detect (D2D) to expose stealth biases in large language models used in high-stakes roles. These biases can favor certain entities, brands, or viewpoints and are…
Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows
A study found that large language models (LLMs) often struggle to operationalize analytical concepts due to a semantic gap between user-level analytical concepts and the information available to workflow-generation…
LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data
A new method called AgentODE has been proposed to discover and refine Ordinary Differential Equations (ODE) structures for modeling complex dynamics, particularly useful in clinical settings. This method works with…
Measuring the Gap Between Human and LLM Research Ideas
A study comparing the research ideas generated by large language models (LLMs) with those of human researchers. The authors build an evaluation framework to measure the gap between them using a two-axis taxonomy to…
MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
The authors propose MemSyco-Bench, a new benchmark to evaluate how memory affects decision-making in language models (LLMs) used in AI agents. The benchmark aims to assess if the agents can handle issues of sycophancy…
MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
Researchers have proposed a new benchmark called MemSyco-Bench to evaluate how memory in large language models (LLMs) can lead to sycophancy, where agents over-align with the user at the cost of factual accuracy or…
SenseWalk: Agent-Based Semantic Trajectory Simulation Powered by Large Language Models in Zoned Environments
The paper proposes SenseWalk, an interactive system that uses large language models (LLMs) to simulate semantic trajectories for human movement analysis. The simulation workflow combines LLMs and the social force model…
Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chains
Researchers introduce Agent Skill Supply Chains (ASSCs) to manage the dependencies of Large Language Model (LLM) agents by characterizing their dependency graphs. The study finds that ASSCs have four structural…
Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chains
Researchers introduce a new method called Agent Skill Supply Chains (ASSCs) to manage dependencies in Large Language Model agents. They develop SkillDepAnalyzer, which extracts dependency graphs and metadata from…
A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems
This research paper discusses the increased role of large language models in various systems beyond just text generation, and how this shift changes the security landscape. The authors systematize the existing…
A Tutorial on Autonomous Fault-Tolerant Control Using Knowledge-Grounded LLM Agents
This research paper proposes a framework for using Large Language Model (LLM) agents to support operators in recovering from faults in process plants. The LLM is used as a constrained supervisory planner, proposing…
ACE: Pluggable Adaptive Context Elasticizer across Agents
Researchers propose a new module called Adaptive Context Elasticizer (ACE) to address the challenges posed by growing trajectory lengths in large language models (LLMs) with fixed context windows. ACE is designed to be…
Adaptive Cluster-First Route-Second Decomposition for Industrial-Scale Vehicle Routing
The researchers propose an adaptive system for solving large-scale capacitated vehicle routing problems (CVRPs). The system uses a large language model (LLM) as a high-level decision maker to analyze the evolving…
Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian
Researchers investigated cross-lingual relation extraction for Romanian using large language models (LLMs) and an English to Romanian translation pipeline. They evaluated a 31B LLM under different configurations…
GR2 Technical Report
Researchers present GR2 (Generative Reasoning Re-Ranker), a framework designed to improve the performance of recommendation systems in an industrial setting. The system combines mid-training on semantic IDs…
Harnessing Textual Refusal Directions for Multimodal Safety
Researchers propose a new approach called Modality-Agnostic Refusal Steering (MARS) to improve safety in Multimodal Large Language Models without requiring unsafe multimodal data. MARS uses textual refusal directions…
Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision
The study examines how language models (LMs) generate self-explanations for their predictions, and finds that when training LMs to explain which aspects of their inputs influenced their behavior, they often produce…
JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering
Researchers introduce JL1-CC&QA, a new benchmark for remote sensing that extends the JL1-CD dataset with two annotation layers: change captioning (CC) and change question answering (QA). The benchmark includes 5,000…
RAISE: LLM-based Automated Heuristic Design with Robust Adversary Instance Search
The paper proposes a framework called Robust Adversary Instance Search (RAISE), which uses Large Language Models (LLMs) to design heuristics that perform well under real-world distributional shifts. The framework…
Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs
Researchers have developed two new mechanisms - Reinforcement Learning with Metacognitive Feedback (RLMF) and Metacognitive Data Selection - to improve the performance of Language Models (LLMs). The goal is to make…
Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
The authors investigate the generation of question-answer (QA) pairs used for training language models and find that this process is not neutral as it can introduce biases due to the selection of evidence and the…
ShopX: A Foundation Model for Intent-to-Item Fulfillment in Agentic Shopping
The paper proposes ShopX, a foundation model aimed at bridging the gap between language understanding and item-space fulfillment in AI shopping applications. It unifies intent understanding, execution planning, and…
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors
The research evaluates the issue of data referencing errors (DREs) made by large language models (LLMs) while performing table tasks. The study presents a systematic evaluation of these errors across different models…
AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
The paper proposes AgentsNet, a new benchmark for multi-agent reasoning. This benchmark is designed to measure the ability of multi-agent systems to collaborate effectively and leverage their network topology. The…
Can LLMs Rank? A Tale of Triads and Triage
The authors discuss the use of Large Language Models (LLMs) in ranking people for scarce resources, such as housing or emergency services. They present two methods for assessing the consistency of LLM's judgments: one…
Collective cooperation without individual fidelity in LLM agents
Researchers compared the behavior of Large Language Models (LLMs) to humans in a networked Prisoner's Dilemma experiment. While LLMs could replicate some collective cooperation patterns, they failed to match human…
Collective cooperation without individual fidelity in LLM agents
Researchers tested large language models (LLMs) in a Prisoner's Dilemma experiment with human participants to evaluate if their behavior can be seen as a faithful representation of human decision-making. The LLM was…
MCP Server Architecture Patterns for LLM-Integrated Applications
This industry experience paper presents five architectural patterns observed in the use of Model Context Protocol (MCP) servers, which connect large language models to external tools, data sources, and services. The…
Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition
Researchers propose a method called ReBA (Retrieval Backward Attention) to enhance the quality of word-level embeddings produced by pretrained language models without affecting sentence-level representations…
Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs
Researchers propose AgentSymbiotic, an iterative framework that enhances both large and small Language Model (LLM) performance in web browsing tasks by combining data synthesis with task-performance. Large LLMs…
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Researchers have collected and released a dataset of 4300 coding agent sessions, containing around 350,000 LLM steps and 430,000 tool calls from their own use of Claude Code and Codex. The analysis shows that…
Translating Natural Language to Strategic Temporal Specifications via LLMs
Researchers have developed a framework for translating natural language descriptions of strategic requirements into well-formed formulas using Large Language Models (LLMs). They created and curated a novel dataset for…
Translating Natural Language to Strategic Temporal Specifications via LLMs
Researchers have developed a framework for translating Natural Language descriptions of strategic requirements into well-formed ATL/ATL* formulas using Large Language Models (LLMs). They created and validated a novel…
Using Large Language Models as Low-Cost Statistical Estimators for Human-Response Data
A study proposes using large language models (LLMs) as a cheaper alternative to human subject experiments for quantitative research in social and behavioral sciences. The LLM is shown to estimate conditional…
Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection
In this research, the authors investigate how Large Language Models (LLMs) used for automated vulnerability detection in code are influenced by cognitive biases similar to those that affect human judgment. They present…
UNKNOWN
The text provided does not contain sufficient information for a summary.
A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models
A new pipeline for creating synthetic clinical notes is introduced to help develop AI systems in healthcare without compromising privacy. The pipeline generates structured, semi-structured, and unstructured clinical…
Application of LLMs to Threat Assessment of Foreign Peacekeeping Missions
The authors propose a new method using Large Language Models (LLMs) to assist in the assessment of threats for foreign peacekeeping missions, using the EU Monitoring Mission in Georgia as a case study.
Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement
Researchers have developed a new system called BINEVAL for evaluating large language models (LLMs). This system breaks down the evaluation criteria into binary questions, making it easier to understand the results and…
Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement
Researchers have developed a new method called BINEVAL to evaluate the output of large language models (LLMs). Instead of relying on human judgment or lexical metrics, which can be expensive and unreliable, BINEVAL…
Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions
This research investigates how different contexts (framings) can affect the responses of large language models used in mental health support tools. The study finds that framing does systematically alter interpretive…
Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions
Researchers investigated how different contexts or framings can affect the responses of large language models (LLMs) used in mental health support tools and psychologically sensitive conversations. They found that…
Autoregressive Boltzmann Generators
A new method called Autoregressive Boltzmann Generators (ArBG) is proposed to efficiently sample molecular systems at thermodynamic equilibrium. This method overcomes limitations of previous methods by using an…
Diagnosing Task Insensitivity in Language Agents
The research discusses a problem with large language models where they struggle to adapt to new tasks outside of their original training data (OOD generalization). The authors call this issue 'task insensitivity'. They…
Diagnosing Task Insensitivity in Language Agents
Researchers identify an issue with large language models where they are unable to adapt to new tasks effectively due to their task insensitivity. The authors propose a lightweight solution called Task-Perturbed NLL…
Einstein World Models
The text discusses a proposed system called Einstein World Models (EWMs) which aim to enhance the reasoning abilities of Language Model (LLM) by incorporating visual-temporal rollouts within the reasoning process…
In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics
The paper proposes a new method called In-Context Model Predictive Generation (ICMPG) for generating human motions based on textual descriptions in digital applications. The method combines language model planning with…
In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics
Researchers propose a new method called In-Context Model Predictive Generation (ICMPG) for generating realistic human motions from textual descriptions. The method combines language-model planning with physical…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
The study reveals that Language Models (LLMs) fine-tuned for security classification can overlook vulnerabilities introduced during fine-tuning itself. These models may learn specific token-level indicators which…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
The study shows that Language Models (LLMs) fine-tuned for security classification may overlook vulnerabilities introduced during fine-tuning itself. These models can learn token-level indicator semantics that preserve…
Joint Learning of Experiential Rules and Policies for Large Language Model Agents
Researchers present a method called Joint Learning of Experiential Rules and Policies for Large Language Model Agents (JERP). This method aims to help LLM agents, which operate in multi-step interactive environments…
Language-Based Digital Twins for Elderly Cognitive Assistance
The paper proposes a language-based digital twin framework that uses large language models to mimic the conversational behavior of elderly individuals, helping in early detection of Mild Cognitive Impairment (MCI). A…
OpenRCA 2.0: From Outcome Labels to Causal Process Supervision
Researchers introduce a new method, called PAVE, to support the rigorous evaluation of large language models (LLMs) in root cause analysis (RCA). The current datasets only label the root cause and not the path…
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
The research investigates the impact of 'prompt injection' on automated resume screening using large language models (LLMs). Prompt injection refers to strategic manipulation in text to influence LLM evaluations. The…
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
The study explores the impact of prompt injection, a self-promotional text strategy used by job applicants to manipulate automated hiring systems, on Large Language Models (LLMs) in résumé screening. The research shows…
ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models
The paper introduces a new method called Reasoning-guided progressive Open Relation Extraction (ReaORE) to improve the extraction of unseen relations between entities in text for real-world applications. Current…
ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models
The paper proposes a new method called Reasoning-guided progressive Open Relation Extraction (ReaORE) to improve the performance of Open Relation Extraction (OpenRE). This method addresses the limitations of current…
Semantic Early-Stopping for Iterative LLM Agent Loops
The research presents a method called semantic early-stopping for multi-agent large language model (LLM) loops, such as a Writer and a Critic. The goal is to halt the loop when the consecutive draft embeddings no…
ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCP
The paper introduces ShareLock, a stealthy multi-tool threshold poisoning framework designed to exploit the interaction between Large Language Models (LLMs) and external tools through the Model Context Protocol (MCP)…
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
The study discusses multi-model language models and how their accuracy is limited by a factor called 'beta', which is the rate at which all models are wrong on the same query. The authors find that across 67 models, a…
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
The text discusses multi-model language model systems and their potential to outperform single-models in terms of accuracy. The study shows that the gain is limited by a quantity called 'beta', which represents the…
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
The study investigates the use of multi-model language models to improve accuracy over single-models. It is found that the improvement is limited by a factor called 'beta', which represents the rate at which all models…
Where Do CoT Training Gains Land in LLM based Agents?
The study investigates how Chain-of-thought (CoT) training in language model agents affects their performance. The authors find that while CoT improves overall, it doesn't significantly widen the advantage of CoT…
Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs
Researchers have found that two open-source language models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, have internal representations (emotion vectors) that can encode emotions such as valence and arousal. The…
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation
This research compares the effectiveness of encoder classifiers (using models like ModernBERT and Ettin) against LLM-based judges in identifying harmful outputs from large language models (LLMs). The study evaluates…
GUI agent: Guided Exploration of User-Sensitive Screens
The text discusses a paper that proposes a solution for a problem in Large Language Model (LLM) agents used within Graphical User Interface (GUI) environments. The current issue is that these agents, fine-tuned to…
Helpful or Harmful? Evaluating LLM-Assisted Vulnerability Patching via a Human Study
Researchers aim to evaluate the effectiveness of Large Language Models (LLMs) in assisting vulnerability patching compared to manual debugging by conducting an empirical experiment on human participants in real-world…
MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
Researchers propose MiniOpt, a reinforcement learning framework that helps large language models solve various optimization problems with limited resources. It breaks down optimization reasoning into structured…
Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents
The authors propose a new method called 'progress advantage' that leverages reinforcement learning (RL) post-training to evaluate Large Language Models (LLMs) at each step during an interaction, eliminating the need…
Probabilistic Agents in Deterministic Audits: Evaluating Multi-Agent Systems for Automated Audits Based on the German IT-Grundschutz
This research paper presents a Multi-Agent System (MAS) architecture with Hybrid Retrieval Augmented Generation (HybridRAG) for partially automating IT-Grundschutz certification, a German standard for IT security. The…
Probabilistic Agents in Deterministic Audits: Evaluating Multi-Agent Systems for Automated Audits Based on the German IT-Grundschutz
This research presents the implementation and evaluation of a Multi-Agent System (MAS) architecture combined with Hybrid Retrieval Augmented Generation (HybridRAG) for partially automating IT-GS certification, which is…
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
A study finds that using low-bit quantization to reduce the cost of large language models can lead to longer chains of thought, which offsets the expected per-token speedup in reasoning tasks. The study introduces a…
SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment
Researchers have proposed a framework called SARA (Semantically Anchored Routing Alignment) to improve the performance of multilingual language models in low-resource languages. The framework aligns the routing…
Weave of Formal Thought
The paper introduces Weave of Formal Thought (WoFT), a new paradigm for large language models that combines syntactic validation with learned structural representations. The proposed approach includes a formal engine…
A specialized reasoning large language model for accelerating rare disease diagnosis: a randomized AI physician assistance trial
Researchers have developed a specialized language model called RaDaR (Rare Disease navigatoR) to assist in the diagnosis of rare diseases. The model, which has 32 billion parameters, was trained on both publicly…
A specialized reasoning large language model for accelerating rare disease diagnosis: a randomized AI physician assistance trial
A new open-source language model called RaDaR (Rare Disease navigatoR) has been developed to aid in the diagnosis of rare diseases. The model was trained on a combination of public free-text cases and synthetic cases…
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
The paper introduces AdversaBench, a red-teaming pipeline for evaluating large language models. The pipeline generates hard inputs using five structured operators and confirms failures through a three-judge panel with…
AI-PAVE-Br: Leveraging Large Language Models for Enhanced Product Attribute Value Extraction through a Golden Set Approach
The paper presents AI-PAVEBr, a system that uses large language models to extract product attribute values from Brazilian e-commerce catalogs. It also introduces the Golden Set, a new dataset in Portuguese for product…
AI-PAVE-Br: Leveraging Large Language Models for Enhanced Product Attribute Value Extraction through a Golden Set Approach
The paper introduces AI-PAVEBr, a system that uses Large Language Models (LLMs) to extract product attributes in Portuguese e-commerce catalogs more accurately than traditional methods. The authors also present a new…
Can Scale Save Us From Plasticity Loss in Large Language Models?
Researchers studied plasticity loss, a challenge in creating neural networks capable of continuous learning, in modern transformer-based language models (LLMs). They found evidence of plasticity loss across various…
Governed Shared Memory for Multi-Agent LLM Systems
The paper proposes a solution for managing shared knowledge in multi-agent language model (LLM) environments. The authors identify four failure modes and introduce four primitives: scoped retrieval, temporal…
Governed Shared Memory for Multi-Agent LLM Systems
The paper discusses the challenges in managing shared knowledge in multi-agent language models (LLMs) environments and proposes solutions to prevent four failure modes: unauthorized leakage, stale propagation…
Grad Detect: Gradient-Based Hallucination Detection in LLMs
The authors present a new method called Grad Detect, which uses the internal gradient structure of Large Language Models (LLMs) during inference to detect hallucinations. This method outperforms confidence-based and…
LaGO: Latent Action Guidance for Online Reinforcement Learning
The paper 'LaGO: Latent Action Guidance for Online Reinforcement Learning' proposes a new framework called LaGO. This framework uses a pre-trained language model (LLM) as a guide to help with online policy optimization…
Large-Language-Model Discovery of Quantum LDPC Codes through Structured Concept Evolution
Researchers introduce Structured Concept Evolution (SCE), a framework that uses a large language model paired with an algebraic mutation grammar to discover new types of quantum low-density parity-check (qLDPC) codes…
Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams
This research explores the use of large language models (LLMs) in detecting scam phone calls in Turkish, a low-resource language where such detection is particularly challenging due to limited annotated data and…
Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams
Researchers introduce a new dataset of Turkish phone call conversations to test large language models' ability to detect scams. They find that using corrected or uncorrected transcripts improves results compared to…
Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity
Researchers propose a new method to enhance large language models while maintaining privacy in sensitive scenarios. The method uses a multi-agent framework that sanitizes retrieved content through semantic rewriting…
Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity
The authors propose a multi-agent framework that uses semantic rewriting to enhance large language models in sensitive scenarios while preserving privacy and contextual fidelity. The framework was tested on two…
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
Researchers have developed SAFARI, a framework that helps diagnose failures in autonomous agents by using a specialized toolbox for reading and searching trajectory segments and a persistent Short-Term Memory (STM) for…
ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling
The paper presents ScaleToT, a method that improves user modeling for low-activity users by learning structured reasoning from a small subset of Large Language Models (LLMs). It constructs typed user-state chains using…
Scaling Laws for Task-Specific LLM Distillation
This research provides empirical scaling laws for compressing Large Language Models (LLMs) specifically for certain domains, focusing on quantitative finance as the application domain. The study investigates the impact…
Abstract representational geometry supports inference in large language models
The study compares human and large language model (LLM) abilities to adapt and infer latent task structures from sparse observations. It finds that while LLMs are less frequent at generalizable reasoning than humans…
Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs
The paper presents a framework for automatically identifying and repairing semantic mistakes in SysML v2 models, which are errors that preserve syntactic validity but violate domain rules. The framework combines a…
Causal Discovery in the Era of Agents
The paper proposes a new approach for causal discovery in which agents help with data analysis, context retrieval, and method explanation, but do not supply causal conclusions or assumptions. This approach aims to…
Distribution-Aware Diffusion-LLM for Robust Ultra-Long-Term Time Series Forecasting
The authors propose a new method called Diffusion-LLM for time series forecasting using Large Language Models (LLMs). The proposed framework integrates a conditional diffusion model into an LLM-based forecasting…
Exposing the Illusion of Erasure in Knowledge Editing for LLMs
This research paper explores the reliability and mechanisms of Knowledge Editing (KE) in Language Models (LLMs), revealing that edited knowledge is not always fully erased but can continue to surface even across…
GIF: Locally Sound Geometric Information Flow Control for LLMs
Researchers present a new framework called Geometric Information Flow (GIF) for tracking information flow from input tokens to outputs in large language models. This framework uses the Jacobian and local output…
GIF: Locally Sound Geometric Information Flow Control for LLMs
Researchers have proposed a new method called Geometric Information Flow (GIF) for controlling information flow in large language models used in agentic systems. GIF uses the model's Jacobian and local output geometry…
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
The paper presents a method called GRINQH (GRaded INput-based Quantization Hierarchy) to accelerate the decoding process of language models during inference, particularly beneficial for edge-computing settings. The…
HOLMES: Evaluating Higher-Order Logical Reasoning in LLMs
The paper introduces HOLMES, a new benchmark for evaluating higher-order symbolic reasoning in Large Language Models (LLMs). This benchmark is designed to test the models' ability to reason over rules, predicates…
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Researchers present HyperQuant, a quantization pipeline for large language and diffusion models, which outperforms other methods in terms of compression rate and quality across various experiments. The method combines…
Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts
The authors present TF-RefusalBench, a multilingual benchmark for assessing over-alignment in legal language models used by the Swiss Federal Supreme Court. They find that over-alignment is complex and influenced by…
Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory
The study identifies a new phenomenon called Memory Contagion, where biased evaluators' influence can affect the experiences of language model agents, causing their memories to become biased as well. This bias is…
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?
The authors question if AdamW, a commonly used optimizer for training large language models, is effective under heavy-tailed noise as its theory only applies to finite-variance regimes. They mention that sign-based…
PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop Refinement
The paper introduces PAPERCLAW, a multi-agent system that automates the research process from start to finish. It uses a large language model as its core, which can drive the entire pipeline and also allows for human…
