Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
18 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on performance evaluation
18 notes
Adaptive Memory and Reflection Multi-Agent System for Medical Question Answering
Researchers have developed an adaptive memory and reflection multi-agent system for medical question answering. This system, which uses specialized agents with dedicated memory and feedback, aims to improve the…
Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis
Researchers introduce a new method called Top-K prompting to train language models for single-step retrosynthesis, a crucial component of computer-aided synthesis planning. They use an ultra-large dataset of verified…
AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design
Researchers have developed a new framework called AutoDesign to improve the process of transforming multimodal sources into structured media outputs, such as converting academic papers into posters. The system uses a…
A Comparative Evaluation of Deep Learning Object Detection Models on a Real-World Multi-Plant Dataset from Africa
A study compares six object detection models on a dataset collected from Nigerian farms, called AgriAISeg, which includes sesame, cabbage, and tomato crops under varying environmental conditions. The models tested are…
ArchAgent v2: A Case Study with the Data Prefetching Championship
In a study published in 2026, researchers presented ArchAgent v2, a framework that automates the search for multi-level data prefetching in computer microarchitecture. The new version of ArchAgent outperformed…
ArchAgent v2: A Case Study with the Data Prefetching Championship
The paper presents ArchAgent v2, a framework that scales automated microarchitecture search to multi-level data prefetching. It outperforms the winning hand-designed solution in the 4th Data Prefetching Championship…
ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment
Researchers have proposed a new method called Answer-Backtracked Credit Assignment (ABC) to train long-horizon search agents more effectively. This method converts sparse trajectory-level outcomes into dense step-level…
When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs
The study investigates how effective relaying key-value caches between multi-agent language models is, particularly when the receiver does not need the sender's private information. The research shows that when the…
When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs
The research examines the claim that multi-agent language models (LLMs) transmit 'latent thoughts' through relayed key-value caches instead of text, and finds significant evidence for this claim when the receiver needs…
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
The authors present a new approach for managing a language model's working memory by treating the decision of which stored items to keep as an estimation problem. They introduce a method called 'fixed-lag smoothing'…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Researchers evaluate the ability of six multimodal large language models to understand scientific visualizations (SciVis) using a standardized test with 49 items based on various visualization techniques and tasks. The…
Concept-Guided Spatial Regularization for World Models in Atari Pong
Researchers evaluate five different visual world-model agents in Atari Pong and find that their rollouts contain errors such as ball disappearance and incorrect motion. They propose Concept-Guided Spatial…
STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle
Researchers introduce a new benchmark called STOCKTAKE for evaluating LLM agents in supply-chain replenishment tasks over 26 weeks. The benchmark separates state estimation and control, measures the agent's ability to…
AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
Researchers introduce a new contract for long-horizon language models (LLMs) called bounded contract. This contract allows each decision to be made from a fresh user message, with no raw cross-decision transcript…
AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
The authors introduce a new bounded contract for long-horizon Large Language Model (LLM) agents in the game Slay the Spire 2. The contract allows decisions to be made from a fresh user message, with no raw…
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
The paper evaluates three repository-level performance optimization benchmarks (GSO, SWE-Perf, and SWE-efficiency) used to measure coding agents. The authors find issues with the stability of runtime for these…
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
The study discusses multi-model language models and how their accuracy is limited by a factor called 'beta', which is the rate at which all models are wrong on the same query. The authors find that across 67 models, a…
POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation
Researchers have developed POTracker, an optimized large language model (LLM) for generating power outage reports that follow strict formatting and structural rules required by energy-sector regulatory bodies in the…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
