Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
5 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on benchmark evaluation
5 notes
TDD-Agent: Test-Driven Reasoning for Code Generation
Researchers introduce TDD-Agent, a system that uses the test-driven development (TDD) paradigm for code generation. The system first generates tests to clarify expected behaviors before implementation and then refines…
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
The research introduces OSReward, a new benchmark for evaluating the reliability of vision-language models (VLMs) used to judge computer-using agents (CUAs). The study finds that even state-of-the-art VLMs have a…
Multimodal Pretraining for Generalizable EEG Representation Learning
Researchers have developed a multimodal EEG foundation model that can be used for various EEG-related tasks without being limited to specific datasets or tasks. The model uses a raw signal encoder, a time-frequency…
OpenForgeRL: Train Harness-native Agents in Any Environment
OpenForgeRL is an open-source framework for training AI agents using various complex inference harnesses in diverse environments. It allows for end-to-end training with open infrastructure and has been validated across…
Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models
Researchers have developed Blind-Spots-Bench, a new benchmark to evaluate the limitations of AI systems by presenting tasks that are easy for humans but challenging for current AI models. The study reveals significant…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
