Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
4 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on agent evaluation
4 notes
AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
Researchers have developed a tool called AV-AIVAT for evaluating the strength of two agents in imperfect-information games, such as Heads-Up No-Limit Hold'em (HUNL). The tool reduces variance by correcting conditional…
AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
The paper introduces AV-AIVAT, a method for evaluating agents in games like poker, which reduces variance by 54 times using AIVAT and continuously monitored Confidence Sequences (CS). The new method stops evaluations…
AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
The paper introduces AV-AIVAT, a tool for evaluating the strength of two agents in imperfect-information games like Heads-Up No-Limit Hold'em (HUNL). AV-AIVAT reduces variance by using conditional mean-zero corrections…
Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents
The paper introduces InvestLogicBench, a benchmark for evaluating the investment logic of large language models (LLMs). The benchmark measures the logical plausibility and event grounding of LLMs, suggesting that…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
