Field notes
2,348 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
2,348 notes and 295 topics.
- Question answered
- Source note
- Topic
- Shared notes
All notes, newest first
2,348 notes, page 23 of 98
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers have introduced a new framework to evaluate the quality of benchmarks used for task-oriented conversational agents. The framework uses language models (LLMs) to assess the consistency, complexity, and…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The paper introduces a reference-free framework to evaluate the quality of task-oriented conversational agent benchmarks using LLM (Large Language Model) judges. The framework assesses consistency, complexity, and…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The authors present a new framework to evaluate the quality of benchmarks used to test conversational agents. The framework uses language models (LLMs) as judges to assess the consistency, complexity, and policy…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The authors propose a reference-free framework to evaluate the quality of task-oriented conversational agent benchmarks, which are often used to assess these agents. The framework uses Large Language Models (LLMs) as…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
A research paper proposes a new method for document search called Reliable Embedding-free Agentic Document-search (READ), which is designed to overcome limitations in current methods for long documents, particularly…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The authors propose a new method for document search called READ (Reliable Embedding-free Agentic Document-search). This method uses an interpretable agent that performs three operations: normalized lexical search…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The paper introduces a new approach called READ (Reliable Embedding-free Agentic Document-search) for retrieving information from long documents such as financial statements and audit reports. The traditional method of…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The paper proposes a new approach for document search called 'READ (Reliable Embedding-free Agentic Document-search)', which aims to address the issues with current black-box retrieval designs in handling long…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The paper proposes a new method for document search named Reliable Embedding-free Agentic Document-search (READ), which aims to improve the accuracy of retrieval-augmented generation on specific types of documents such…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
The authors propose a new method for document search called Reliable Embedding-free Agentic Document-search (READ). In contrast to the common design of retrieval-augmented generation, READ uses an agent that reads the…
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Researchers used Concept Activation Vectors (CAVs) to analyze the bias of two automatic L2 speaking assessment systems: a text-based BERT grader and a multimodal grader based on Whisper. The study shows that the…
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Researchers analyze the fairness and interpretability of two automatic speaking assessment systems using Concept Activation Vectors (CAVs). They find that the recoverability and sensitivity to certain concepts depend…
Challenges in Evaluating Explanation Methods for Static and Evolving Data
The paper discusses the limitations in evaluating Explainable Artificial Intelligence (XAI) systems, specifically focusing on the DetoxAI image recognition system. It highlights the need for better evaluation methods…
Challenges in Evaluating Explanation Methods for Static and Evolving Data
This research paper highlights the issues in evaluating Explanation Methods for AI systems, using the DetoxAI image recognition system as an example. It discusses methods for adapting explanations to changing data…
Comparative Approaches to Agent Retrieval over Large Skill Libraries
The study compares two systems for deciding which skills to load from a large skill library by an agent. The first system is a hybrid ranker that combines lexical and dense-embedding retrieval, while the second system…
Comparative Approaches to Agent Retrieval over Large Skill Libraries
The study compares two systems for deciding which skills to load from large skill libraries in AI agents. The first system is a hybrid ranker that combines lexical and dense-embedding retrieval for sparse, on-demand…
Comparative Approaches to Agent Retrieval over Large Skill Libraries
The research compares two systems for deciding which skills to load from a large skill library when creating agents. One system combines lexical and dense-embedding retrieval, while the other uses a typed knowledge…
Comparative Approaches to Agent Retrieval over Large Skill Libraries
Researchers compared two methods for deciding which skills to load from large skill libraries in AI agents: a hybrid ranker combining lexical and dense-embedding retrieval, and a typed knowledge graph encoding workflow…
Contextual Information Policy Optimization for Search Agents
The paper proposes a new method called Contextual Information Policy Optimization (CIPO) for search agents. This method aims to improve the efficiency of evidence use in complex or evolving information tasks by…
Contextual Information Policy Optimization for Search Agents
The text introduces a new method called Contextual Information Policy Optimization (CIPO), which aims to improve the reliability of search agents by ensuring they use retrieved evidence effectively during multi-step…
Contextual Information Policy Optimization for Search Agents
The researchers propose a new reinforcement learning framework called Contextual Information Policy Optimization (CIPO) to improve the reliability of search agents for knowledge-intensive tasks. CIPO encourages the use…
Contextual Information Policy Optimization for Search Agents
The paper proposes a new method called Contextual Information Policy Optimization (CIPO) to improve the reliability of search agents for complex or evolving information tasks. CIPO aligns policy optimization with…
Continual Learning in Transition
The paper examines the evolution of Continual Learning (CL), a method that allows models to update and retain knowledge while learning new tasks. The authors suggest a transition from traditional parameter-centric…
Continual Learning in Transition
The text discusses a shift in continual learning, moving away from traditional parameter-centric methods towards system-level adaptation. The authors propose a tri-axial framework to analyze this transition: When, How…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
