Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Field notes

2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.

Every note is in the list below. The web of topics draws here once the page has loaded.

128 of 2,343 notes match.

  • Question answered
  • Source note
  • Topic
  • Shared notes

Notes on reinforcement learning

128 notes, page 1 of 6

  1. Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints

    Source note · Oct 1, 2026 · reinforcement learning, security, infrastructure

    The preprint proposes the Homomorphic Advantage Operator to stabilize deep reinforcement learning when nonlinear operations are approximated under fully homomorphic encryption. It applies a zero-mean centering…

  2. PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents

    Source note · Sep 30, 2026 · reinforcement learning, agents, training

    The technical report studies pivotal mistakes that move an agent farther from task completion during multi-turn interaction. Across three Qwen3 models, the abstract reports that more than half of failed rollouts…

  3. Reinforcing Agentic Creativity in Scientific Ideation with Night Science

    Source note · Sep 28, 2026 · reinforcement learning, agents, creativity

    The preprint introduces AI Night-Scientist, an agentic framework that uses reinforcement learning to shape when and how a model departs from predictable reasoning. It models creativity along action, process, and…

  4. Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

    Source note · Sep 25, 2026 · reinforcement learning, agents

    The preprint studies reusable code-based skills for agents that otherwise must repeatedly choose low-level actions. In NetHack, the authors compare primitive actions, semantic skills, and combinations of both across…

  5. AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control

    Source note · Sep 24, 2026 · reinforcement learning, robotics, planning

    AD-WM is a joint-embedding world model designed for counterfactual model predictive control. It combines residual latent dynamics with action-recovery regularization so the model preserves action-dependent differences…

  6. SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

    Source note · Sep 24, 2026 · reinforcement learning, coding agents, synthetic data

    The preprint describes a framework for synthesizing scientific coding problems by sampling operator chains from graphs built from scientific library interfaces and operation modes. An LLM generates problem statements…

  7. CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

    Source note · Sep 18, 2026 · evaluation, reinforcement learning, software engineering

    CodeMidas converts implemented functionality in open-source codebases into executable reinforcement-learning environments using source code as the only task-specific input. Agents formulate behavioral specifications…

  8. Score Centering Stabilizes Off-policy Reinforcement Learning

    Source note · Sep 17, 2026 · reinforcement learning, optimization, efficiency

    The preprint attributes instability in large-language-model reinforcement learning under training-inference mismatch to persistent drift between engines. It derives an additive score-centering correction that cancels…

  9. Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train

    Source note · Sep 15, 2026 · reinforcement learning, retrieval

    Google Research describes Retrieve-for-Train as an offline reinforcement-learning process that compiles set-level rewards into training data for a lightweight diffusion retriever. The associated arXiv paper describes…

  10. EnvHarness: Awakening Static Worlds for Agent Learning

    Source note · Aug 20, 2026 · ai, machine learning, reinforcement learning

    The paper proposes a new system called Environment Harness (EnvHarness) to improve static environments used in learning by AI agents. The system can adapt the behavior of these environments without modifying their…

  11. EnvHarness: Awakening Static Worlds for Agent Learning

    Source note · Aug 20, 2026 · reinforcement learning

    The authors propose a method called Environment Harness (EnvHarness) to address the issue of static environments in learning agents. This system allows for reshaping behavior without modifying the underlying logic…

  12. SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning

    Source note · Aug 20, 2026 · ai, reinforcement learning

    A new reinforcement learning method called Single-rollout Autoregressive Policy Optimization (SAPO) is proposed to address the memory overhead, computation efficiency, and performance issues of existing methods. SAPO…

  13. MLREF: Efficient Module Reuse for Reward Design in Reinforcement Learning via Large Language Models

    Source note · Aug 19, 2026 · reinforcement learning

    The research proposes a new framework called Module Level Reward Evolution Framework (MLREF) to address the issue of unreliable reward function preservation and reuse in reinforcement learning. MLREF uses a module…

  14. Chronocooked: A Benchmark for Implicit Interval Timing in Reinforcement Learning Agents

    Source note · Aug 17, 2026 · ai, reinforcement learning, benchmark

    The paper introduces Chronocooked, a benchmark suite for studying how reinforcement learning (RL) agents make decisions over time. It's inspired by Overcooked and includes cooking scenarios that require temporal…

  15. ClawGym II: Exploring Black-Box RL on Agent Harness

    Source note · Aug 17, 2026 · reinforcement learning, optimization, long-horizon tasks

    Researchers present a new framework for reinforcement learning (RL) using complex agent harnesses. This framework is designed to optimize RL more stably and efficiently for long-horizon tasks. The researchers' method…

  16. UniRL: A Framework for Unified Multimodal Model Reinforcement Learning

    Source note · Aug 16, 2026 · reinforcement learning, industry, china

    Tencent Hunyuan released UniRL, an open-source framework that applies one shared reinforcement-learning training loop across different kinds of generative models (diffusion/image models, text language models…

  17. Custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge

    Source note · Aug 14, 2026 · reinforcement learning, vendor, engineering-practice

    AWS engineers describe how to design reward functions for training AI models on multi-turn tasks (e.g., a coding assistant that should ask clarifying questions before writing code) using Amazon Nova Forge, and how to…

  18. Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL

    Source note · Aug 12, 2026 · reinforcement learning, robotics, manipulation

    The paper proposes a method to integrate robot locomotion and manipulation using Reinforcement Learning (RL) and Sample-based Model Predictive Control (SMPC). The approach generates offline datasets from SMPC…

  19. One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL

    Source note · Aug 12, 2026 · reinforcement learning, language models, human-ai interaction

    The paper discusses a common issue in multi-agent reinforcement learning for human-AI interaction: using a single large language model (LLM) to simulate user behavior often results in the LLM focusing on specific…

  20. Redistribution-based Cost Inference Improves Sparse Safe Offline RL

    Source note · Aug 12, 2026 · artificial intelligence, machine learning, reinforcement learning

    A new framework called Redistribution-based Cost Inference (RCI) is proposed to convert sparse stop-feedback in safe offline reinforcement learning into dense per-step costs. This conversion helps train a constrained…

  21. MIRA: Medical Image Reflection for Agentic Diagnosis

    Source note · Aug 11, 2026 · reinforcement learning, web search, medical imaging

    Researchers present MIRA, a medical image diagnosis framework that allows an autonomous agent to search for relevant evidence, verify the relevance and consistency of the acquired evidence, and adapt its tool-use…

  22. MIRA: Medical Image Reflection for Agentic Diagnosis

    Source note · Aug 11, 2026 · reinforcement learning, image processing

    The paper introduces MIRA (Medical Image Reflection for Agentic Diagnosis), a medical visual diagnostic framework designed to autonomously search for evidence and verify its relevance and consistency in image…

  23. Parameter Exploration for RLVR via Variational Learning

    Source note · Aug 10, 2026 · large language models, reinforcement learning

    Researchers propose a new approach called Perturbed Parameter Policy Optimization (3PO) to enhance exploration in large language models (LLMs). This method explores the parameter space instead of the action space…

  24. Contextual Information Policy Optimization for Search Agents

    Source note · Aug 6, 2026 · ai, machine learning, reinforcement learning

    The text introduces a new method called Contextual Information Policy Optimization (CIPO), which aims to improve the reliability of search agents by ensuring they use retrieved evidence effectively during multi-step…

An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.