Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Field notes

2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.

Every note is in the list below. The web of topics draws here once the page has loaded.

17 of 2,343 notes match.

  • Question answered
  • Source note
  • Topic
  • Shared notes

Notes on vision-language models

17 notes

  1. Neurosymbolic Embodied Agents

    Source note · Aug 17, 2026 · vision-language models, long-horizon tasks

    The authors propose a neurosymbolic agent that combines vision-language models with symbolic planning for executable long-horizon household tasks. The agent consists of two phases: the first involves a vision-language…

  2. When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understanding

    Source note · Aug 4, 2026 · vision-language models

    Researchers propose EcoFrame, a framework for adaptively scheduling visual evidence selection in long videos for efficient video understanding by vision-language models (VLMs). The method uses the VLM's inference…

  3. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    Source note · Jul 30, 2026 · ai evaluation, vision-language models

    The research introduces OSReward, a new benchmark for evaluating the reliability of Vision-Language Models (VLMs) that judge Computer-Using Agents' (CUAs) trajectories. The study finds that even state-of-the-art VLMs…

  4. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    Source note · Jul 30, 2026 · vision-language models, evaluation benchmark

    The paper introduces OSReward, a benchmark for evaluating the reliability of vision-language models (VLMs) used as judges for computer-using agents (CUAs). The benchmark consists of CUA trajectories from various agent…

  5. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    Source note · Jul 30, 2026 · evaluation, reinforcement learning, vision-language models

    The research introduces OSReward, a new benchmark for evaluating the reliability of vision-language models (VLMs) used to judge computer-using agents' trajectories. The study finds that state-of-the-art VLMs have a…

  6. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    Source note · Jul 30, 2026 · vision-language models, benchmark evaluation

    The research introduces OSReward, a new benchmark for evaluating the reliability of vision-language models (VLMs) used to judge computer-using agents (CUAs). The study finds that even state-of-the-art VLMs have a…

  7. MIRROR: Learning from the Other View for Multi-Modal Reasoning

    Source note · Jul 23, 2026 · vision-language models

    The paper presents MIRROR, a method to improve multi-modal reasoning in AI models by learning from different views (text, image, combined text and image) of the same problems. The study reveals that vision-language…

  8. MIRROR: Learning from the Other View for Multi-Modal Reasoning

    Source note · Jul 23, 2026 · large language models, reinforcement learning, vision-language models

    Researchers have created a new approach called Modality-Informed Reciprocal Reasoning Optimization (MIRROR) to improve the reasoning abilities of both large language models and vision-language models. The approach uses…

  9. OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

    Source note · Jul 9, 2026 · benchmarking, ai in healthcare, vision-language models

    The paper introduces OmniFood-Bench, a new benchmark for evaluating Large Vision-Language Models (VLMs) in the context of personalized healthcare and dietary management. The benchmark is designed to assess the ability…

  10. Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment

    Source note · Jul 7, 2026 · vision-language models, generalization, reward design

    Researchers have developed a new method called VAORA (Visual Action Outcome Reasoning Alignment) to improve the performance of vision-language models in interactive physical reasoning, especially in unseen tasks and…

  11. TreeAgent: A Generalizable Multi-Agent Framework for Automated Bias Labeling in Forestry via Compiled Expert Rules and Vision-Language Models

    Source note · Jun 30, 2026 · multi-agent systems, vision-language models, interpretability

    Researchers propose a multi-agent system (MAS) called TreeAgent to automate bias labeling in forestry via expert decision trees and Vision-Language Models (VLMs). The MAS aims to reduce the need for human annotation…

  12. TreeAgent: A Generalizable Multi-Agent Framework for Automated Bias Labeling in Forestry via Compiled Expert Rules and Vision-Language Models

    Source note · Jun 30, 2026 · vision-language models, interpretability, multi-agent system

    Researchers propose a system called TreeAgent that uses artificial intelligence (AI) to classify tree heights in forestry remote sensing. The system combines decision trees created by experts with Vision-Language…

  13. GROW$^2$: Grounding Which and Where for Robot Tool Use

    Source note · Jun 29, 2026 · vision-language models

    Researchers introduce GROW$^2$, a system that helps robots use tools creatively by grounding which object to use as a tool and where to apply it, even when the object is not intended for that purpose. This system uses…

  14. Automating Potential-based Reward Shaping with Vision Language Model Guidance

    Source note · Jun 25, 2026 · reinforcement learning, automation, vision-language models

    Researchers introduce a new method called VLM-PBRS for automating potential-based reward shaping using vision language model (VLM) feedback. The aim is to improve reinforcement learning agents' ability to learn from…

  15. Automating Potential-based Reward Shaping with Vision Language Model Guidance

    Source note · Jun 25, 2026 · reinforcement learning, robotics, vision-language models

    Researchers have developed a new method called VLM-PBRS for automating potential-based reward shaping in reinforcement learning agents. This method uses vision language models (VLMs) to learn the potential function…

  16. Automating Potential-based Reward Shaping with Vision Language Model Guidance

    Source note · Jun 25, 2026 · reinforcement learning, robotics, vision-language models

    Researchers have developed a new method for reinforcement learning agents using vision language models (VLMs). The approach called VLM-guided PBRS learns a potential function from VLM feedback to address the challenge…

  17. Data Selection Through Iterative Self-Filtering for Vision-Language Settings

    Source note · Jun 22, 2026 · deep learning, vision-language models

    Researchers propose a new method to select data for training neural networks in large scale, noisy datasets. The Self-Filtering method iteratively trains a CLIP model on an evolving dataset, balancing clean samples and…

An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.