Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
36 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on robotics
36 notes, page 1 of 2
BrickBench: Evaluating Agentic Brick Design
The preprint introduces BrickBench, a benchmark for text-conditioned LEGO-set design. Agents must produce assemblies that satisfy semantic and design criteria while remaining physically buildable, selecting parts from…
WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?
The preprint introduces WorldSolver, a benchmark of 168 simulation tasks derived from physical phenomena in 61 classic computer-graphics papers across seven physical domains. Each task provides a code scaffold while…
AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control
AD-WM is a joint-embedding world model designed for counterfactual model predictive control. It combines residual latent dynamics with action-recovery regularization so the model preserves action-dependent differences…
In-Context Robot Learning with VLM Agents
The paper introduces GPT-Policy, a framework combining a context compiler, a vision-language model that proposes robot-tool actions, and a constrained controller that verifies and executes actions. The authors report…
ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation
ENCP rescales conformal-prediction scores for variable-length, dependent navigation episodes and calibrates one maximum score per episode. The abstract reports coverage evaluations across four navigation policies and…
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
The text discusses the evolution of Large Language Models (LLMs) from text generation models to core components of agents that can perceive environments, invoke tools, and execute tasks. These LLM-driven embodied…
DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
The paper describes a model that, given a starting camera frame, a language instruction, and a planned sequence of robot actions, predicts what the robot's camera would see next, intended to help with robotic…
Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
The paper proposes a method to integrate robot locomotion and manipulation using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC). The approach generates offline datasets from SMPC…
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
Researchers propose a verification layer driven by large language models (LLMs) to evaluate the feasibility and ethics of actions suggested by robot planning models before they are executed. The system aims to reduce…
From Passive Mirrors to Active Agents: Holonic Digital Twins for Physical AI over Networks
The paper proposes a new framework called HDT-Nets to enable real-time physical AI inference for robots and vehicles operating under real-world conditions. This is achieved by creating holonic digital twins (HDTs) that…
From Passive Mirrors to Active Agents: Holonic Digital Twins for Physical AI over Networks
The paper proposes a new framework called HDT-Nets to improve the performance of AI in physical systems like robots and vehicles. This framework uses holonic digital twins (HDTs) which are hierarchical structures that…
EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE
The authors present a method called EgoGenesis for creating synthetic manipulation videos for embodied AI using egocentric video. The method uses a pretrained video generation prior and introduces two geometry-aware…
BioVLN: A Simulation Platform for Visual Language Navigation in Biomedical Laboratories
Researchers have developed a simulation platform named BioVLN for navigating robotics in biomedical laboratories. The platform focuses on finding the correct position to approach instruments, ensuring safety by…
From Passive Video to Editable Experience: Physically Grounded Experience Synthesis for Embodied Intelligence
Jia Luo introduces Pegasus, a framework to help robots learn from human manipulation videos by translating the human demonstrations into data that can be understood by robots. The method involves creating an…
Towards Miniature Humanoid Tele-Loco-Manipulation Using Virtual Reality and Reinforcement Learning
The paper describes a new control stack for miniature humanoids that allows a single operator to see, manipulate, and navigate in distant environments using Virtual Reality (VR) for upper-body teleoperation and…
UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following
The authors introduce the Unified Embodied Seeking and Following Benchmark (UESF-Bench), a new benchmark for embodied agents that allows them to search for and follow language-described targets in dynamic environments…
Unveiling Complex Collective Behaviors from Simple Rewards
Researchers have developed a new method called the Agent Response Map (ARM) to help understand how simple rewards can lead to complex behaviors in multi-agent reinforcement learning (MARL). The ARM reveals that robots…
UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxies
Researchers introduce an offline method called Unsupervised Robotic Value Correction (UR-VC) to correct time-derived progress labels used by modern robot learning systems. The method is based on a regularity in…
Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning
The paper proposes a method for machine learning that aims to align autonomous agents' behavior with human intent in diverse operational contexts by using a hierarchical machine teaching algorithm for reward learning…
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
The paper introduces a new multi-agent coding harness called Graph-as-Policy (GaP) for improving the reliability of robots in commercial and industrial applications, specifically focusing on variational automation…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose a new framework called Task-Agnostic Pretraining (TAP) for Vision-Language-Action (VLA) models. The TAP aims to address the scarcity of expert demonstrations by separating two learning objectives…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose Task-Agnostic Pretraining (TAP), a two-stage framework to train Vision-Language-Action (VLA) models. TAP first learns transferable motor priors from unlabeled interaction data using self-supervised…
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Researchers propose Task-Agnostic Pretraining (TAP) as a solution to the scarcity of expert demonstrations in Vision-Language-Action (VLA) models. TAP aims to separate learning physical competence and semantic…
WorldSample: Closed-loop Real-robot RL with World Modelling
The paper presents WorldSample, a framework that uses reinforcement learning (RL) to improve the performance of robots by creating synthetic transitions based on real rollouts and a post-trained world model. This…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
