All notes, newest first
2,343 notes. Notes 351 to 400, page 8 of 47.
Benchmarking LLM Judges for Mobile Agent Evaluation
A benchmark study of LLM judges used to evaluate mobile-agent trajectories.
Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models
The study investigates the performance of object-centric world models for visual model-predictive control, which enable agents to accomplish tasks through planning. The researchers find that these models perform better…
Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models
The research focuses on object-centric world models (OCWMs) for artificial intelligence agents. The study compares OCWMs with traditional scene-centric models in terms of representation quality, planning success, and…
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
Researchers present the 'Agentic Self-Improvement' framework to address challenges in professional workflows with modern Image-to-Video (I2V) models. The framework uses a two-stage approach: an iterative prompt…
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
The researchers introduce a new framework called 'Agentic Self-Improvement' to address challenges in professional workflows related to modern black-box Image-to-Video (I2V) models. The framework optimizes the video…
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
Researchers have developed a new framework called 'Agentic Self-Improvement' to improve the performance of Image-to-Video (I2V) models, which are commonly used for automated content creation. The new framework aims to…
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
This research paper provides a comprehensive review of various visual explanation methods used in explainable artificial intelligence, specifically focusing on Class Activation Mapping (CAM). The paper discusses the…
Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers
In this paper, researchers examine a municipal algorithm register used for decision-support tools in welfare benefits eligibility assessments. They find that engaging diverse stakeholders in participatory system…
Constructing Dynamic Master Logic Models as Knowledge Graphs for Complex System Diagnostics Using Retrieval-Augmented Large Language Models
The study introduces a framework to automate the construction of Dynamic Master Logic (DML) models for complex systems using Retrieval-Augmented Generation and Large Language Models. These models, represented as…
Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents
Researchers introduce Convergent Detour Hijacking (CDH), a text-based attack that manipulates third-party skills in LLM agents to make tasks unnecessarily expensive and time-consuming without affecting their final…
Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents
Researchers introduce Convergent Detour Hijacking (CDH), a technique that manipulates Language Learning Model (LLM) agents by using natural language descriptions and instruction bodies to steer the agents onto more…
Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents
Researchers introduce Convergent Detour Hijacking (CDH), an attack that manipulates language model (LLM) agents by exploiting their reliance on third-party skills for task selection and planning. The attack can lead to…
CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations
Researchers have developed a new benchmark called CTBench to evaluate the performance of AI agents in troubleshooting network issues in telecom environments. The benchmark focuses on root cause analysis and path…
CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations
The paper introduces CTBench, a public benchmark for evaluating AI agents' ability to perform troubleshooting tasks in telecom network operations. The focus is on root cause analysis and path restoration. The authors…
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
A new benchmark called Diagram-MMU has been created to test the ability of Multimodal Large Language Models (MLLMs) in parsing and understanding scientific diagrams. The benchmark includes 3,700 curated diagrams and…
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
The paper discusses Diagram-MMU, a new benchmark designed to evaluate the ability of Multimodal Large Language Models (MLLMs) in parsing and understanding scientific diagrams. The benchmark includes 3.7k diagrams and…
Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation
This research introduces a framework to evaluate language models (LLMs) when agents are provided with graded similarity signals. The study finds variations in how different LLMs handle these signals across various…
Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation
Researchers introduce a framework for evaluating how Language Model (LLM) agents cooperate when provided with similarity signals. The study finds that different LLM models exhibit varying behaviors, but some modern…
DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation
Researchers propose DreamFly, a framework for aerial vision-language navigation that addresses challenges like limited historical context, short planning horizons, and unreliable implicit termination. The framework…
DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation
The authors propose DreamFly, a framework for aerial vision-language navigation that uses a causally aligned historical memory to reason about the past without leaking future information. It also uses receding-horizon…
DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation
In this paper, the authors introduce DreamFly, a new framework for aerial vision-language navigation that addresses the challenges of limited historical context, short planning horizons, and unreliable implicit…
From Safety Documentation to Safety Knowledge Support: An Evidence-Grounded LLM Framework for Medical Devices
The paper proposes a framework for medical device safety using Large Language Models (LLMs). The goal is to streamline the process of managing risk-management evidence in medical devices that are becoming increasingly…
From Safety Documentation to Safety Knowledge Support: An Evidence-Grounded LLM Framework for Medical Devices
The paper proposes a framework that uses large language models (LLMs) to support medical device safety work, which is document-based and costly due to the need for safety and domain experts. The framework connects…
Graph-Structured Rubrics: Compiling Rubrics into Typed Evaluation Graphs for LLM Judges
The paper introduces Graph-Structured Rubrics (GSR), a method that converts rubrics into response-independent typed evaluation graphs before observing responses. This allows for more efficient and accurate scoring of…
GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings
The paper introduces GUIDE, a framework designed to automate the creation of enterprise guideline documents. The system uses six specialized agents to parse, extract, check consistency, evaluate, escalate human…
GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings
The paper introduces GUIDE, a framework for generating enterprise guideline documents that are often complex and multimodal (combining text, tables, images). Existing systems struggle with maintaining the structure of…
How Organizations Use AI: Evidence from ChatGPT
The authors study the use of ChatGPT in organizations by analyzing data from ChatGPT Enterprise accounts linked with worker roles, task classifications, and financial data of public companies. They found rapid growth…
Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
The paper proposes a method to integrate robot locomotion and manipulation using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC). The approach generates offline datasets from SMPC…
Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
The authors propose a method to integrate locomotion and manipulation for robots using a combination of Sample-based Model Predictive Control (SMPC) and Reinforcement Learning (RL). By leveraging SMPC, they create an…
MBA: Multimodal Benchmark and Agents for Real-World Business Ideation
A multimodal benchmark and training setup for agents that generate and evaluate business ideas using visual evidence.
Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
Researchers have developed a system called Mechanist that uses AI to autonomously discover the mechanisms underlying AI intelligence. The system is built on an interpretability-focused knowledge graph of around 13,000…
Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
The study introduces Mechanist, an AI system designed to autonomously discover the mechanisms behind AI intelligence. It uses a knowledge graph of approximately 13,000 papers integrated with a multidisciplinary…
NetlistBench: Evaluating LLM Reliability in SPICE Netlist Recognition and Manipulation
Researchers present NetlistBench, a new benchmark for evaluating the reliability of Large Language Models (LLMs) in SPICE netlist recognition and manipulation tasks. The study finds that while LLMs perform well on…
No One to Blame: A Framework of Constitutive AI Unaccountability
The paper 'No One to Blame: A Framework of Constitutive AI Unaccountability' argues that certain configurations of actors, systems, and institutions make AI accountability unachievable, no matter the effort put into…
No One to Blame: A Framework of Constitutive AI Unaccountability
Researchers have proposed a new framework called 'constitutive AI unaccountability' to address challenges in traditional accountability mechanisms with the increasing deployment of autonomous AI systems. This framework…
One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL
The paper discusses a common issue in multi-agent reinforcement learning for human-AI interaction: using a single large language model (LLM) to simulate user behavior often results in the LLM focusing on specific…
One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL
The paper discusses a common issue in multi-agent reinforcement learning (MARL) for human-AI interaction: relying on a single large language model to simulate user behavior often leads to poor generalization due to…
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
The research presents a new framework called Preference Tree Optimization (PTO) for enhancing dialogue systems in specialized domains with limited data. It focuses on Motivational Interviewing (MI), a counseling…
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
Researchers propose Preference Tree Optimization (PTO), a framework that enhances dialogue systems in specialized domains like Motivational Interviewing by using preference data generated through Look-Ahead Simulations…
Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
The research focuses on optimizing the use of GPUs in LLM-agent services by identifying a boundary for concurrent work (ready cohort) and investigating the impact of keeping GPU-computed decisions on device instead of…
Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
The paper discusses a method for determining when it's beneficial to execute small deterministic transitions between model and tool calls on GPUs, and presents a dynamic program that computes the optimal amount of work…
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
A new benchmark called RealisticTritonBench was introduced to test the ability of large language models (LLMs) to generate GPU kernels for real-world AI frameworks. This benchmark addresses limitations in previous…
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
Researchers have introduced a new benchmark called RealisticTritonBench to improve the evaluation of large language models (LLMs) in generating GPU kernels for AI frameworks using Triton. The new benchmark derives…
Redistribution-based Cost Inference Improves Sparse Safe Offline RL
A new framework called Redistribution-based Cost Inference (RCI) is proposed to convert sparse stop-feedback in safe offline reinforcement learning into dense per-step costs. This conversion helps train a constrained…
Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection
The paper presents a framework called BENCH2ROBUST, which converts reliable tool-use benchmarks into controlled stochastic environments with failure scenarios. The authors test two interventions to improve the…
Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection
Researchers present BENCH2ROBUST, a framework for converting reliable tool-use benchmarks into stochastic environments with controlled solvability. This allows study of agent's response to tool failures, which can be…
Structural Silence: When AI Infrastructure Fails Speakers of Underrepresented Languages
The paper discusses how AI tools for education and language support can disadvantage speakers of underrepresented languages before training, using Bengali as a case study. The authors identify four main issues: a web…
Structural Silence: When AI Infrastructure Fails Speakers of Underrepresented Languages
The authors discuss how the underlying infrastructure of AI tools designed for education and language support can disadvantage speakers of underrepresented languages, using Bengali as an example. They highlight four…
VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies
The researchers introduce VAKRA, a benchmark for evaluating the ability of agents to reason across structured APIs and document collections in enterprise settings. The benchmark includes over 8,000 executable APIs…
VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies
The study introduces VAKRA, a benchmark for evaluating the ability of agents to reason across APIs and document collections in enterprise settings. The benchmark includes over 8000 executable APIs across 62 domains…
Page 8 of 47
