Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
13 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on llms
13 notes
PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
PrivDrift is a benchmark for testing whether secrets disclosed during an active conversation remain recoverable after the dialogue shifts to unrelated topics and a later probe uses persuasion. It contains 1,000…
Decomposing Wrong-Consensus Agreement in LLM Self-Consistency: A GPT-4.1 Case Study
The study presents an analysis of the inconsistency in the answers provided by Large Language Models (LLMs) when using majority voting to improve accuracy. The authors propose a quantitative measure called Gamma, which…
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
The text discusses the evolution of Large Language Models (LLMs) from text generation models to core components of agents that can perceive environments, invoke tools, and execute tasks. These LLM-driven embodied…
Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation
Researchers have developed a system called Natural Language Knowledge Graph Query (NLKGQ) that allows researchers to ask ad-hoc questions about the contents of domain-specific archives using natural language. The…
From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
Researchers propose a method called EvoSOP to improve Large Language Model (LLM) agents' interaction with the real world. The proposed method optimizes the tools used by LLM agents through creating reusable Standard…
LLM-as-a-Verifier: A General-Purpose Verification Framework
Researchers introduce a new framework called LLM-as-a-Verifier to improve the ability of Large Language Models (LLMs) to verify the correctness of solutions for various tasks. The method provides fine-grained feedback…
MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
Researchers have proposed a new benchmark called MemSyco-Bench to evaluate how memory in large language models (LLMs) can lead to sycophancy, where agents over-align with the user at the cost of factual accuracy or…
Collective cooperation without individual fidelity in LLM agents
Researchers tested large language models (LLMs) in a Prisoner's Dilemma experiment with human participants to evaluate if their behavior can be seen as a faithful representation of human decision-making. The LLM was…
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Researchers have collected and released a dataset of 4300 coding agent sessions, containing around 350,000 LLM steps and 430,000 tool calls from their own use of Claude Code and Codex. The analysis shows that…
Application of LLMs to Threat Assessment of Foreign Peacekeeping Missions
The authors propose a new method using Large Language Models (LLMs) to assist in the assessment of threats for foreign peacekeeping missions, using the EU Monitoring Mission in Georgia as a case study.
Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs
The paper presents a framework for automatically identifying and repairing semantic mistakes in SysML v2 models, which are errors that preserve syntactic validity but violate domain rules. The framework combines a…
HOLMES: Evaluating Higher-Order Logical Reasoning in LLMs
The paper introduces HOLMES, a new benchmark for evaluating higher-order symbolic reasoning in Large Language Models (LLMs). This benchmark is designed to test the models' ability to reason over rules, predicates…
Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts
The authors present TF-RefusalBench, a multilingual benchmark for assessing over-alignment in legal language models used by the Swiss Federal Supreme Court. They find that over-alignment is complex and influenced by…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
