Robotics and autonomous systems
65 notes, newest first. Includes the vault tags robotics, autonomous systems, autonomous vehicles, autonomous driving.
BrickBench: Evaluating Agentic Brick Design
The preprint introduces BrickBench, a benchmark for text-conditioned LEGO-set design. Agents must produce assemblies that satisfy semantic and design criteria while remaining physically buildable, selecting parts from…
WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?
The preprint introduces WorldSolver, a benchmark of 168 simulation tasks derived from physical phenomena in 61 classic computer-graphics papers across seven physical domains. Each task provides a code scaffold while…
AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control
AD-WM is a joint-embedding world model designed for counterfactual model predictive control. It combines residual latent dynamics with action-recovery regularization so the model preserves action-dependent differences…
In-Context Robot Learning with VLM Agents
The paper introduces GPT-Policy, a framework combining a context compiler, a vision-language model that proposes robot-tool actions, and a constrained controller that verifies and executes actions. The authors report…
ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation
ENCP rescales conformal-prediction scores for variable-length, dependent navigation episodes and calibrates one maximum score per episode. The abstract reports coverage evaluations across four navigation policies and…
Verifiable abstention makes AI leak diagnosis accountable in water distribution networks
Researchers propose a method for AI in leak localization in water distribution networks that includes verifiable abstention, ensuring accountability when the AI does not act. The method uses a physics-grounded executor…
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
The text discusses the evolution of Large Language Models (LLMs) from text generation models to core components of agents that can perceive environments, invoke tools, and execute tasks. These LLM-driven embodied…
DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
The paper describes a model that, given a starting camera frame, a language instruction, and a planned sequence of robot actions, predicts what the robot's camera would see next, intended to help with robotic…
Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
The paper proposes a method to integrate robot locomotion and manipulation using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC). The approach generates offline datasets from SMPC…
Conversational Orchestration for Organic 6G
The authors propose a lightweight, decentralized conversational orchestration framework using Large Language Model (LLM)-driven domain agents for the future network of networks, Organic 6G. The aim is to provide…
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
Researchers propose a verification layer driven by large language models (LLMs) to evaluate the feasibility and ethics of actions suggested by robot planning models before they are executed. The system aims to reduce…
From Passive Mirrors to Active Agents: Holonic Digital Twins for Physical AI over Networks
The paper proposes a new framework called HDT-Nets to enable real-time physical AI inference for robots and vehicles operating under real-world conditions. This is achieved by creating holonic digital twins (HDTs) that…
From Passive Mirrors to Active Agents: Holonic Digital Twins for Physical AI over Networks
The paper proposes a new framework called HDT-Nets to improve the performance of AI in physical systems like robots and vehicles. This framework uses holonic digital twins (HDTs) which are hierarchical structures that…
NSF-HRPT: Neural Semantic Field meets Hierarchical Risk Perception Tree for Safety-Critical Scenario Assessment
Researchers propose NSF-HRPT, a framework that uses a Neural Semantic Field (NSF) to model scene semantics, trajectory predictions, and probabilistic Time-to-Collision (TTC) distributions from simulation data. During…
NSF-HRPT: Neural Semantic Field meets Hierarchical Risk Perception Tree for Safety-Critical Scenario Assessment
The research introduces NSF-HRPT, a framework that uses neural semantic fields and structured reasoning to accurately assess risks in safety-critical scenarios such as autonomous driving. The method learns to model…
EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE
The authors present a method called EgoGenesis for creating synthetic manipulation videos for embodied AI using egocentric video. The method uses a pretrained video generation prior and introduces two geometry-aware…
BioVLN: A Simulation Platform for Visual Language Navigation in Biomedical Laboratories
Researchers have developed a simulation platform named BioVLN for navigating robotics in biomedical laboratories. The platform focuses on finding the correct position to approach instruments, ensuring safety by…
From Passive Video to Editable Experience: Physically Grounded Experience Synthesis for Embodied Intelligence
Jia Luo introduces Pegasus, a framework to help robots learn from human manipulation videos by translating the human demonstrations into data that can be understood by robots. The method involves creating an…
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
A new approach called Master-Agent Proto-plan System (MAPS) is proposed to improve the coordination of autonomous vehicles at unsignalized intersections. MAPS uses a hierarchical deep reinforcement learning…
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
The paper introduces a new deep reinforcement learning architecture called Master-Agent Proto-plan System (MAPS) for coordinating autonomous vehicles at unsignalized intersections. The MAPS system uses a centralized…
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
Researchers proposed a new system called Master-Agent Proto-plan System (MAPS) for coordinating autonomous vehicles at unsignalized intersections using deep reinforcement learning. MAPS outperformed state-of-the-art…
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
Researchers present MAPS, a deep reinforcement learning architecture for coordinating autonomous vehicles at unsignalized intersections. The system uses a central Master agent that generates a global coordination…
VoLN: Vision-Only Long-Horizon Navigation---Paradigm, Benchmark, and Method
Researchers introduce Vision-Only Long-Horizon Navigation (VoLN), a new paradigm for autonomous agents to navigate using only onboard sensors and in-scene cues, rather than relying on external instructions. They create…
Towards Miniature Humanoid Tele-Loco-Manipulation Using Virtual Reality and Reinforcement Learning
The paper describes a new control stack for miniature humanoids that allows a single operator to see, manipulate, and navigate in distant environments using Virtual Reality (VR) for upper-body teleoperation and…
Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control
The paper introduces Proof-or-Stop Lifecycle Control, a method for controlling transitions in the lifecycle of autonomous coding agents. It ensures that these transitions occur only when supported by fresh evidence…
Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control
The paper presents Proof-or-Stop Lifecycle Control, a method that ensures software work transitions only when supported by verifiable evidence. The method was tested using an open-source implementation and showed…
UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following
The authors introduce the Unified Embodied Seeking and Following Benchmark (UESF-Bench), a new benchmark for embodied agents that allows them to search for and follow language-described targets in dynamic environments…
TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale
TerraZero is a procedural driving simulation developed by researchers to train autonomous driving agents efficiently at scale without the need for real-world data demonstrations. It offers improved speed, realistic…
TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale
TerraZero is a procedural driving simulator and training stack for reinforcement learning at scale, designed to train autonomous driving agents without human demonstrations or fallback planners. The simulator provides…
Unveiling Complex Collective Behaviors from Simple Rewards
Researchers have developed a new method called the Agent Response Map (ARM) to help understand how simple rewards can lead to complex behaviors in multi-agent reinforcement learning (MARL). The ARM reveals that robots…
UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxies
Researchers introduce an offline method called Unsupervised Robotic Value Correction (UR-VC) to correct time-derived progress labels used by modern robot learning systems. The method is based on a regularity in…
Technical Report on the CVPR 2026@AdvML Workshop Challenge
This text is about the CVPR 2026@AdvML Workshop Challenge which focuses on testing the robustness of vision-language agents (VLAs) used in autonomous driving. The challenge involved creating adversarial images and…
Technical Report on the CVPR 2026@AdvML Workshop Challenge
This text is about the CVPR 2026@AdvML Workshop Challenge, focusing on adversarial attacks against Vision-Language Agents (VLAs) used in autonomous driving. The challenge involves creating adversarial images and…
Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning
The paper proposes a method for machine learning that aims to align autonomous agents' behavior with human intent in diverse operational contexts by using a hierarchical machine teaching algorithm for reward learning…
Swapping Faces, Saving Features: A Dual-Purpose Pipeline for Pedestrian Privacy in ITS
This research focuses on creating a five-stage pipeline to protect pedestrians' privacy in autonomous vehicles (AVs) by using a face-swapping method called Roop, which outperforms Ghost-v2 and strikes a balance between…
Swapping Faces, Saving Features: A Dual-Purpose Pipeline for Pedestrian Privacy in ITS
The paper focuses on creating a five-stage pipeline for protecting pedestrians' privacy in AI models used by autonomous vehicles (AVs) while maintaining the image attributes needed to train the models. The proposed…
WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
The research presents WCog-VLA, a novel framework for autonomous driving that bridges semantic world forecasting with generative world evolution. This dual-level model aims to overcome the limitations of existing…
WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving
Researchers propose WCog-VLA, a dual-level model for autonomous driving, which combines semantic world forecasting and generative world evolution to bridge the gap between reactive and proactive driving. The semantic…
Agentic Data Environments
The text discusses early work on Agentic Data Environments (ADE), a system that allows autonomous agents to operate safely and efficiently over a broader data environment. The goal is to increase the benefits of…
CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis
Researchers have proposed a new method called CARLA-GS for generating corner cases in autonomous driving simulations that are essential for safety evaluation. The method decouples the visual representation, semantic…
CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis
The paper proposes CARLA-GS, a modular pipeline for synthesizing corner cases in autonomous driving simulations. The pipeline decouples visual representation, semantic reasoning, and physics-based execution while…
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
The paper introduces a new multi-agent coding harness called Graph-as-Policy (GaP) for improving the reliability of robots in commercial and industrial applications, specifically focusing on variational automation…
Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planning
Researchers have proposed a new planning algorithm called Graph Sparse Sampling (GSS). This algorithm is designed to help autonomous systems plan under uncertainty in continuous domains, which can be computationally…
Grounded autonomous research: a fault-tolerant LLM pipeline from corpus to manuscript in frontier computational physics
The paper presents an autonomous research pipeline that can conduct end-to-end research from a corpus of 11,083 recent condensed matter physics papers to a publication-grade manuscript. The agent is capable of…
Grounded autonomous research: a fault-tolerant LLM pipeline from corpus to manuscript in frontier computational physics
The paper presents a pipeline for an autonomous research agent that can generate publication-grade manuscripts in the field of condensed matter physics. The agent is able to autonomously conceive a research direction…
Hardware-Enforced Semantic Coordination for Safety-Critical Real-Time Autonomous Systems
The paper proposes a hardware-enforced semantic coordination architecture for safety-critical real-time autonomous systems. This approach aims to overcome limitations of software-mediated coordination by implementing…
Hardware-Enforced Semantic Coordination for Safety-Critical Real-Time Autonomous Systems
The authors propose a novel hardware-enforced semantic coordination architecture for safety-critical real-time autonomous systems. This architecture aims to address the limitations of software-mediated coordination by…
Hardware-Enforced Semantic Coordination for Safety-Critical Real-Time Autonomous Systems
The authors propose a hardware-enforced semantic coordination architecture for safety-critical real-time autonomous systems. This new approach aims to address limitations of software-mediated coordination by…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose a new framework called Task-Agnostic Pretraining (TAP) for Vision-Language-Action (VLA) models. The TAP aims to address the scarcity of expert demonstrations by separating two learning objectives…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose Task-Agnostic Pretraining (TAP), a two-stage framework to train Vision-Language-Action (VLA) models. TAP first learns transferable motor priors from unlabeled interaction data using self-supervised…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose Task-Agnostic Pretraining (TAP) as a solution to the scarcity of expert demonstrations in Vision-Language-Action (VLA) models. TAP aims to separate learning physical competence and semantic…
WorldSample: Closed-loop Real-robot RL with World Modelling
The paper presents WorldSample, a framework that uses reinforcement learning (RL) to improve the performance of robots by creating synthetic transitions based on real rollouts and a post-trained world model. This…
A Self-Evolving Agentic System for Automated Generation and Execution of Biological Protocols
The paper presents ProtoPilot, a self-evolving multi-agent system designed to automate the generation and execution of biological protocols. The system achieved high success rates in various tasks compared to…
MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments
The research paper introduces MECoBench, a benchmark for evaluating the collaboration abilities of multimodal agents in visually grounded environments. The study explores three key findings: collaboration generally…
Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
Researchers have developed a new framework called OmniAct for building embodied agents that can operate in unstructured environments. This framework aims to address the challenges of managing both cyber and physical…
Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
The paper presents OmniAct, a framework designed to create persistent embodied agents that can operate autonomously in unstructured environments. It addresses the challenges of integrating cyber (APIs, IoT) and…
Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
Researchers present OmniAct, a framework that aims to advance the development of persistent embodied agents in unstructured environments. The framework integrates a multimodal semantic planner for skill routing, an…
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Researchers have developed a new method called VLM-PBRS for automating potential-based reward shaping in reinforcement learning agents. This method uses vision language models (VLMs) to learn the potential function…
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Researchers have developed a new method for reinforcement learning agents using vision language models (VLMs). The approach called VLM-guided PBRS learns a potential function from VLM feedback to address the challenge…
Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds
Researchers have developed a new camera planning framework called Look-Before-Move for embodied AI operating in dynamic 3D environments. The goal is to enable the camera to actively decide what to observe, not just…
Multi-Agent Goal Recognition with Team- and Goal-Conditioned Reinforcement Learning and Factorized Branch-and-Bound
The paper introduces Multi-Agent Goal Recognition with Team- and Goal-Conditioned Reinforcement Learning and Factorized Branch-and-Bound (MAGR-BB), a method for inferring which agents are working together and what each…
NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation
Researchers have developed a transformer-based multimodal framework named NoContactNoWorries that allows robots to infer binary contact states from RGB-D vision and the robot's proprioception. This approach offers a…
Decentralized Autonomous Traffic Management through Corridor Networks
Researchers propose a decentralized approach to managing traffic in Advanced Air Mobility (AAM) corridors using multi-agent reinforcement learning (MARL). The method is tested on various multi-corridor networks and…
Project Fetch: Phase Two
Anthropic's Frontier Red Team ran a follow-up to their August 2025 "Project Fetch" robotics experiment. Using Claude Opus 4.7 with adaptive thinking (maximum effort) and Claude Code, they measured how fast the model…
Project Fetch: Phase Two
A follow-up to "Project Fetch" (August 2025) in which Anthropic's Frontier Red Team ran Claude Opus 4.7 autonomously on the same robotics tasks previously assigned to human-AI hybrid teams. The model completed those…
