research paper
72 notes, newest first.
Inducing Task Models from Computer-Use Traces
Researchers present Task Model Induction (TMI), a method that extracts structured task models from naturalistic computer-use traces. These traces are screenshots and mouse or keyboard actions recorded during work. TMI…
Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning
Researchers introduce Group-Calibrated On-Policy Distillation (GC-OPD), a method for long-context reasoning tasks that aims to address the mismatch between on-policy distillation and task-specific verifiers. The new…
Part of the npj Series
An article titled 'MultiMed-RAG: Leveraging Multi-Source Knowledge and Agent Collaboration for Medical Tasks' published in the npj Series, was written by Yuxing Lu, Sin Yee Goi, and Jinzhuo Wang in 2026. The article…
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
The paper discusses Diagram-MMU, a new benchmark designed to evaluate the ability of Multimodal Large Language Models (MLLMs) in parsing and understanding scientific diagrams. The benchmark includes 3.7k diagrams and…
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Researchers propose Preference Tree Optimization (PTO), a framework that enhances dialogue systems in specialized domains like Motivational Interviewing by using preference data generated through Look-Ahead Simulations…
VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
The paper introduces VICBench, a multi-language benchmark for code vulnerability detection. It was created by human experts and an agentic workflow, featuring 100 verified vulnerabilities across 88 projects in Python…
Two-stage Odd Residual Flows for Mean-Preserving Probabilistic Time Series Forecasting
Researchers propose a new method called Two-stage Odd Residual Flows (TORF) for probabilistic time series forecasting. The method aims to address the trade-off between distributional flexibility and accurate mean…
Defining Decentralization: An Ontological Perspective
This research paper addresses the lack of a universal definition for decentralization in computer science and its related fields like security, AI, cloud infrastructure, IoT, etc. The authors propose a graph-based…
Defining Decentralization: An Ontological Perspective
The authors address the lack of a universally accepted definition of decentralization in computer science, particularly in emerging fields like decentralized AI and machine learning. They propose a graph-based ontology…
From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch
Researchers present a new evaluation framework called 'Grip on LLMs' for Dutch governmental use of large language models. The framework focuses on six dimensions: factuality, honesty, social bias, energy consumption…
Multi-Agent AI Safety as an Institutional Design Problem
In this research paper, Abdullah X discusses the role of institutions in ensuring safety for AI agents within multi-agent systems. The study includes a 5280-episode study suite with various experiments testing…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The paper proposes a new method for document search named Reliable Embedding-free Agentic Document-search (READ), which aims to improve the accuracy of retrieval-augmented generation on specific types of documents such…
Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents
The paper introduces InvestLogicBench, a new benchmark to evaluate the investment logic of large language models (LLMs). The benchmark focuses on personalized financial decision-making and uses real-world data from 151…
FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
Researchers introduce a new benchmark called FinEvo-Bench for testing self-evolving agents in various financial workflows. The benchmark consists of 120 real-world tasks across six domains, with each task having a…
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
Researchers introduce Poli-Bias, a method to measure political bias in large language models (LLMs) by comparing responses to paired prompts where country identities are swapped across various geopolitical…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study compares two access modalities of a large language model (LLM), ChatGPT's chat UI and OpenAI's API, with and without web search enabled. The research shows that the accuracy of responses varies between the…
A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination
Researchers have developed a new framework called A-SR for symbolic regression. This framework uses a self-evolving agentic system that shifts control from expression edits to role-conditioned evidence views. It…
RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists
The text discusses a new benchmark called RepoProbe for evaluating the ability of Large Language Models (LLMs) to understand repository-scale code. The benchmark uses open-ended questions from GitHub Discussions and…
A foundation model of numerical intelligence with cross-disciplinary generalization
Researchers introduce UNICON, a foundation model that exhibits numerical intelligence across various disciplines. This model can infer relationships from graph-based examples within a system and apply them to queries…
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
The paper introduces MANTA, a framework for Multi-Agent Network Topology Adaptation that allows communication structures in large language model-based multi-agent systems to evolve during inference time. This…
ORCA-bench: How Ready Are Language Model Agents for Oncall?
Researchers introduce ORCA-bench, a new benchmark that tests the ability of large language models to perform root cause analysis (RCA) in a production-like setting. The benchmark includes a microservice system with six…
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
The paper proposes a system called PAIChecker to check for misalignment between Pull Requests (PR) and Issues in SWE-bench-like benchmarks used for evaluating Large Language Models' issue resolution capability. The…
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
Researchers introduce a method to study how latent channels in multi-agent systems, which transmit information without using text, impact the receiver's performance. They apply this method to Qwen3-4B and Qwen3-8B on…
MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
MemSecBench is a new benchmark for testing the lifecycle security of agent memory systems. The benchmark contains 310 cases drawn from various contexts, and it follows a controlled protocol to test the persistence…
The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
The paper focuses on improving the multi-turn long-horizon planning for foundation model agents. The authors introduce a controlled environment to study the acquisition, shaping, and integration of planning abilities…
GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
The paper introduces GRADRAG, a framework for improving Retrieval-Augmented Generation (RAG) systems by coordinating improvements across different components of the pipeline. It uses feedback from an Evaluator to…
ICAE-Bench: Evaluating Coding Agents as Interactive Project Builders
The paper introduces a new benchmark called ICAE-Bench for evaluating coding agents in interactive project building settings. This benchmark aims to test the ability of agents to transform incomplete product intent…
Toward Continuous Assurance for the Democratization of AI Agent Creation in Industry
The paper proposes a solution to ensure reliability in AI agents created by non-engineering users within organizations, through democratized low-code, no-code, and conversational development environments. The proposed…
Antigen-specific Antibody Multi-modal Foundation Model for Functional Antibody Design
Researchers introduce a new model called AAMFM (Antigen-specific Antibody Multimodal Foundation Model) for designing functional antibodies. The model learns unified representations of antibody sequences and structures…
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
The paper presents a new benchmark for testing the security of LLM (Language Model) agents against adaptive adversaries in multi-turn interactions. The benchmark, called Adaptive Adversaries, evaluates how defenders…
O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning
Researchers have developed a new method for detecting abnormal objects and events in industrial settings using an object-centric tracking and reasoning approach. This method is designed to overcome the limitations of…
WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting
The paper introduces WorldCupArena, a dynamic benchmark for evaluating language models and deep-research agents in predicting football matches. The 2026 FIFA World Cup was its first evaluation, with systems receiving…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The researchers evaluate the performance of language model security agents, focusing on costs rather than just success rates. They use offensive Cybench challenges and defensive Splunk BOTS v1 investigation challenges…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The researchers evaluated language model security agents based on cost efficiency in addition to their success rates, focusing on offensive CTF challenges and defensive Splunk BOTS v1 investigation challenges. They…
In-Place Tokenizer Expansion for Pre-trained LLMs
Researchers have developed an in-place method called Tokenizer Expansion to upgrade a pre-trained model's tokenizer when language priorities shift, focusing on improving efficiency for users of languages added later…
MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection
The research paper introduces MedFailBench, an open-source benchmark for evaluating the safety of medical AI systems. The benchmark is designed by clinicians and focuses on identifying which safety boundary a system…
MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization
The paper introduces MM-IssueLoc, a new benchmark and evaluation protocol for repository-level issue localization with visual evidence. The benchmark contains over 650 issue-PR instances across multiple languages…
OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios
A research paper titled 'OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios' introduces a new benchmark called OmniaBench, designed to evaluate the capabilities of general AI agents in diverse…
Plover: Steering GUI Agents through Plan-Centric Interaction
The paper introduces Plover, a plan-centric GUI automation system designed to overcome challenges in real-world environments. It allows users to supervise, correct, and inspect the system's behavior by externalizing…
SciDiagramEdit: Learning to Edit Scientific Diagrams from Paper Revisions
The text is about a new system called SciDiagramEdit that automates the editing of scientific diagrams in research papers based on natural language instructions. The system learns from revisions made to figures in…
StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows
Researchers have developed StructureClaw, an artifact-centered workbench that uses LLM (Language Model) agents to operate through governed engineering skills and checks for completion, consistency, and execution of…
Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation
The authors propose a new approach to penetration testing for AI-enabled systems. Instead of focusing on direct compromises of infrastructure, they suggest evaluating if adversaries can manipulate system behavior…
OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning
The paper introduces OOD-RL-Bench, a new framework for testing out-of-distribution (OOD) detection in reinforcement learning (RL). The framework is designed to evaluate the performance of detectors against various…
Heuristic Learning for Active Flow Control Using Coding Agents
A new method for active flow control is proposed in this research paper. Instead of optimizing neural network parameters, the authors use coding agents to search directly for explicit executable feedback laws. The…
Omni-Decision: A Progressive Evidence-State Agent System for Omni-Modal QA
The paper introduces Omni-Decision, a system aimed at answering questions where evidence is spread across multiple modalities like videos, audio, images, web pages, and computation results. The system maintains a…
From Legacy Documentation to OSCAL: An MCP-Based Agent Pipeline for Threat-Informed Continuous Compliance in Critical Infrastructure
The paper proposes an agent pipeline that converts natural language system descriptions into OSCAL format for continuous automated compliance management in critical infrastructure. The pipeline uses MCP-grounded…
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
Researchers have developed a new framework called G-Frame that helps lightweight Large Language Models perform better in rule-based scientific domains by reducing hallucinations. They created a new model called…
GitLake: Git-for-data for the agentic lakehouse
The authors present GitLake, a Git-based data system for an agent-first lakehouse. It allows agents to work on isolated branches while humans review and publish changes. Pipelines run on temporary branches and are…
Swapping Faces, Saving Features: A Dual-Purpose Pipeline for Pedestrian Privacy in ITS
This research focuses on creating a five-stage pipeline to protect pedestrians' privacy in autonomous vehicles (AVs) by using a face-swapping method called Roop, which outperforms Ghost-v2 and strikes a balance between…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors of this paper introduce a new method to evaluate the effects of post-training quantization on large language models, as this technique is commonly used to deploy models in resource-limited settings. The…
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
The authors investigate how the quantization process used to deploy large language models in resource-limited settings can lead to unnoticed behavioral changes, even when task performance seems unaffected. They…
WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search
Researchers propose a new framework called WebSwarm for deep and wide web searches using large language models (LLMs). The framework aims to improve the ability of single agents to handle complexity and coverage…
Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety
The text introduces institutional red-teaming, a new evaluation methodology for testing the impact of deployment rules on multi-agent AI safety. The study uses IABench-CA, a consequence-allocation benchmark, to test…
RSF-GLLM: Bridging the Semantic Gap in Multi-Hop Knowledge Graph QA via Recurrent Soft-Flow and Decoupled LLM Generation
The authors propose a framework called RSF-GLLM to improve Multi-hop Question Answering over Knowledge Graphs. The framework decouples differentiable graph reasoning from answer generation, allowing the system to learn…
QFedAgent: Quantum-Enhanced Personalized Federated Learning for Multi-Agent Activity Recognition
The paper proposes QFedAgent, a hybrid quantum-classical personalized Federated Learning framework for multi-agent activity recognition. It introduces a variational quantum circuit fusion module that models…
Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas
The researchers introduce a large-scale benchmark called DramaSR-532K for speaker recognition in long-form TV dramas and propose a new method, DramaSR-LRM, built on a large reasoning model (LRM). This method aggregates…
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
The paper evaluates three performance-optimization benchmarks (GSO, SWE-Perf, and SWE-efficiency) used to assess coding agents. The authors find that these benchmarks have issues with stability, scoring rules, and the…
AutoMem: Automated Learning of Memory as a Cognitive Skill
Researchers developed AutoMem, a framework to automate the learning of memory as a cognitive skill for large language models (LLMs). This improves the performance of LLMs significantly in long-horizon games by treating…
Self-GC: Self-Governing Context for Long-Horizon LLM Agents
A research paper titled 'Self-GC: Self-Governing Context for Long-Horizon LLM Agents' presents a system called Self-GC that aims to manage the lifecycle of context objects for long-horizon language models (LLMs). The…
Field Order Should Not Matter: Permutation-Invariant Embedding Model Fine-Tuning for Structured Metadata Retrieval
Researchers propose a method called permutation-invariant fine-tuning (PI-FT) to improve structured metadata retrieval. They show that the choice of field order in a record can significantly impact retrieval quality…
MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems
In this research paper titled 'MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems', the authors propose a framework called Mesa to prioritize and secure communication channels in…
AU-Harness: An Open-Source Toolkit for Holistic Evaluation of AudioLLMs
The authors introduce AU-Harness, an efficient and comprehensive evaluation framework for Large Audio Language Models (LALMs). The system addresses limitations in existing toolkits by improving speed, providing…
UNKNOWN
The text does not provide a brief summary.
UNKNOWN
The text does not provide a summary or brief.
UNKNOWN
The text provided does not contain sufficient information for a summary.
A Process Harness for Uplifting Legacy Workflows to Agentic BPM: Design and Realization in CUGA FLO
The authors introduce a new mechanism called 'process harness' for upgrading legacy workflows to an Agentic Business Process Management (Agentic BPM) without replacing the existing engine. The process harness involves…
Bridging Talk and Thought: Understanding Dialogue Dynamics Across Collaborative Problem-Solving Contexts
The authors present a framework for analyzing dialogue in collaborative problem-solving contexts, especially in human-AI and multi-agent collaboration. The framework aims to understand the emerging dynamics of these…
Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities
Researchers have developed Chai, an AI-based system that discovers and validates cryptographic misuse vulnerabilities through naturally occurring signals. Instead of auditing one codebase for many flaws, Chai catalogs…
State Representation Matters in Deep Reinforcement Learning: Application to Energy Trading
Researchers studied the impact of state representation in deep reinforcement learning (DRL) for energy trading decisions, using a fixed Double DQN agent in the HydroDam environment. They found that combining absolute…
Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment
The paper proposes a basic protocol for investigating if AI models exhibit concerning behavior due to malicious intent or benign causes like confusion. The protocol involves analyzing the 'chain of thought' (CoT) to…
Accuracy and Satisfaction in Multi-Turn LLM Dialogues for NFR Assessment
Researchers examined the accuracy of Large Language Model (LLM)-based dialogue assistants in handling Non-Functional Requirements (NFRs) related to Health Insurance Portability and Accountability Act (HIPAA) regulatory…
Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts
The authors present TF-RefusalBench, a multilingual benchmark for assessing over-alignment in legal language models used by the Swiss Federal Supreme Court. They find that over-alignment is complex and influenced by…
