Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
4 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on measurement
4 notes
Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark
The preprint audits whether harmful refusal is a single measurable attribute in HELM Safety. It reports saturation in three candidate datasets and uses multidimensional item-response theory plus differential item…
On the estimation and validity of AI time horizons---a statistical look at the METR plot
The preprint recomputes METR-style 50% AI time horizons over 228 tasks and 26 AIs using splines and item-response theory. The authors report that the fitted spline is nearly flat from roughly 2–30 minutes and close to…
How to connect AI usage to business value
OpenAI describes ChatGPT Work and Codex analytics that combine usage, spend, task insights, and outcome metrics. The post describes task classification, model/reasoning/speed breakdowns, plugin and skills views, Codex…
AI Exposure Scores: what they measure, what they miss, and what comes next
The article discusses the limitations of a set of AI Exposure Scores introduced by Eloundou et al., which measure the share of tasks that can be assisted by large language models. The authors highlight two main gaps…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
