Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
104 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on language models
104 notes, page 1 of 5
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
A research paper titled 'Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference' proposes a new method for creating small language models. Instead of adapting large models to CPUs, the authors…
Beyond the Transcript: Detecting Covert Co ordination in Latent Multi-Agent Communication
Researchers have developed Verifiable Latent Alignments (VLA), a framework to monitor and steer private communication channels used by language model agents in multi-agent systems. The framework links the private…
Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis
Researchers introduce a new method called Top-K prompting to train language models for single-step retrosynthesis, a crucial component of computer-aided synthesis planning. They use an ultra-large dataset of verified…
Tuning the Stochastic Machine: A Systems Engineer's Operating Model for Human-AI Engineering
The author, a systems engineer, argues that the persistent correction of errors in LLM (Language Model) assistants is an operations problem rather than a tooling problem. They propose seven principles for an operating…
Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies
The paper introduces Reconstruction, a blind benchmark for testing language models' ability to recover the true research idea of a published paper using only its pre-publication bibliography. The authors use an…
One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL
The paper discusses a common issue in multi-agent reinforcement learning for human-AI interaction: using a single large language model (LLM) to simulate user behavior often results in the LLM focusing on specific…
Most biomedical publications show signs of LLM-assisted writing
A study suggests that by the end of 2025, approximately 89% of biomedical papers on PubMed Central showed signs of LLM-assisted writing, with a higher usage in the Discussion section compared to the Methods section.
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
The paper introduces HarnessOpt-Bench, a benchmark for evaluating the ability of Language Models (LLMs) to optimize the harnesses surrounding them, which includes prompts, tools, control flow, memory, and orchestration…
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Researchers have introduced HarnessOpt-Bench, a benchmark for evaluating the performance of language models (LLMs) in optimizing harnesses (the tools and code surrounding them). The study tests 5 leading LLMs across 4…
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
Researchers introduce Poli-Bias, a method to measure political bias in large language models (LLMs) by comparing responses to paired prompts where country identities are swapped across various geopolitical…
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
The text introduces a new method called Poli-Bias, which is designed to measure bias in large language models (LLMs) when dealing with international political conflicts. This framework compares responses to paired…
Reducing belief in conspiracy theories as they unfold using large language models
Researchers tested if conversations with a large language model (LLM) can reduce belief in emerging conspiracy theories. They conducted experiments in the days following two significant events, and found that…
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
Researchers audit the effectiveness of using visual tools like crop-and-zoom in multimodal language models by creating a causal graph to separate observation-mediated paths from action-induced shortcuts. They find that…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The research examines the limitations of current benchmarks used to evaluate large language models (LLMs) for safety, reliability, and deployment readiness. The study focuses on a popular LLM, ChatGPT, comparing its…
ContextWeave: A Real-World Workflow Benchmark
Researchers introduce ContextWeave, a new benchmark for evaluating the performance of language agents in long-horizon, stateful workflows. The benchmark measures how well an agent can recall and apply past experiences…
Item Response Theory for AI Safety
The paper presents a study using Item Response Theory (IRT) to analyze the safety of language models. The authors find that three factors - refusal strictness, truthfulness, and contextual harm - explain most of the…
Item Response Theory for AI Safety
Researchers used Item Response Theory (IRT), a statistical toolkit, to analyze the safety performance of 192 language models across eight different safety benchmarks. The study found that three key factors - refusal…
Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First
The researchers introduce SuperScout, a cost-effective solution for routing language models to resolve software issues in repositories. SuperScout first explores the repository and produces a structured handoff, which…
When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs
The study investigates how effective relaying key-value caches between multi-agent language models is, particularly when the receiver does not need the sender's private information. The research shows that when the…
Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
The researchers present a study to make knowledge distillation training efficient for small language models deployed under tight constraints. They propose two methods: offline knowledge distillation (caching teacher's…
VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
The text introduces VIBE, a new benchmark for analyzing the emotional tone of language models' outputs. The benchmark measures valence (positive or negative), arousal (exciting or calm), and dominance (powerful or…
InfoOps Bench: A live information operations safety benchmark
The paper introduces a dynamic AI benchmark that measures the integrity of language models against being used for state-backed information operations. The benchmark uses data from over 2,100 information operations from…
InfoOps Bench: A live information operations safety benchmark
The paper introduces an AI benchmark that measures the safety of language models against being manipulated for state-backed information operations. The benchmark uses data from a live monitoring pipeline tracking…
MemHarness: Memory Is Reconstructed, Not Replayed
Researchers propose MemHarness, a framework for enhancing large language models that allows them to adapt and reconstruct past experiences based on the current context, rather than simply replaying them verbatim as…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
