Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
45 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on nlp
45 notes, page 1 of 2
MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Researchers have introduced a new benchmark called MemTrapBench to test the performance of large language models (LLMs) in terms of cognitive traps caused by their memory components. The study identifies two types of…
Identifying Implicit Premises for Logical Reconstruction of Argument Graphs
The authors propose a neuro-symbolic pipeline to generate implicit premises from natural language text, which can be translated into logical formulae to show the logical relationships between statements (entailment…
Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models
The study explores the ability of large language models (LLMs) to extract contextualized information from research articles efficiently. The authors present four workflows that demonstrate LLMs can perform well in data…
Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models
The paper investigates the performance of large language models (LLMs) in extracting contextualized information from research articles, finding that while LLMs can perform well with expert-curated prompts, they may…
Multiclass Sentiment Analysis for Identifying Political Viewpoints
Researchers developed two machine-learning models to analyze political viewpoints from social media posts. The XGBoost model achieved an F1-score of 0.2835, and the BERT-based model achieved an F1-score of 0.2806 in a…
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?
The paper introduces VibeLifeBench, a new benchmark for evaluating large language model (LLM) agents' ability to handle long-horizon tasks in everyday life scenarios. The benchmark consists of 200 multi-week timeline…
Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
Researchers have created a new method for evaluating text-to-speech (TTS) systems by breaking down the concept of 'naturalness' into ten distinct linguistic dimensions. They found that current TTS evaluation methods do…
Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach
Researchers propose a new method for fine-tuning large language models in a federated learning paradigm using Low-rank adaptation (LoRA). They investigate two strategies, Share-A/Local-B and Share-B/Local-A, and reveal…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers introduce a framework to evaluate the quality of benchmarks used for task-oriented conversational agents, which can lead to unreliable evaluations due to inconsistencies, simplistic scenarios, or limited…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers have introduced a new framework to evaluate the quality of benchmarks used for task-oriented conversational agents. The framework uses language models (LLMs) to assess the consistency, complexity, and…
Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
A research paper proposes a new method for document search called Reliable Embedding-free Agentic Document-search (READ), which is designed to overcome limitations in current methods for long documents, particularly…
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Researchers used Concept Activation Vectors (CAVs) to analyze the bias of two automatic L2 speaking assessment systems: a text-based BERT grader and a multimodal grader based on Whisper. The study shows that the…
Schema-Guided Hierarchical Information Extraction and Semantic Evaluation Using Generative AI
Researchers have developed a framework using generative AI to extract complex, structured information from unstructured text documents. This framework also includes an automated semantic evaluation step to compare the…
OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling
The research presents a new approach called OctoLong for creating longer contexts for language models. It uses an AST parser, a language server backend, and a package manager to retrieve code references from various…
The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections
The paper proposes a new interpretation of Transformers during inference, suggesting they construct prompt-dependent transformations whose parameters are generated during inference. This form of computation is named…
VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
The text introduces VIBE, a new benchmark for analyzing the emotional tone of language models' outputs. The benchmark measures valence (positive or negative), arousal (exciting or calm), and dominance (powerful or…
ORCA-bench: How Ready Are Language Model Agents for Oncall?
The study presents ORCA-bench, a benchmark for evaluating the performance of large language models in oncall root cause analysis (RCA). The benchmark exposes a microservice system with six days of data and 1,079 RCA…
The Tokenizer Tax: Quantifying and Explaining the Cross-Lingual Cost of Subword Tokenization for Indian Languages
The research quantifies the disadvantage faced by many non-English languages in large language models (LLMs) due to subword tokenizers that are primarily trained on English corpora. The study focuses on Indian…
Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
The paper presents a new method for identifying parts of a text generated by Language Model (LLM) at the token level, improving on existing document-level methods. The method is based on smoothing adjacent token scores…
GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
The paper introduces GRADRAG, a framework for improving Retrieval-Augmented Generation (RAG) systems by coordinating improvements across different components of the pipeline. It uses feedback from an Evaluator to…
GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG
The paper introduces GRADRAG, a framework that improves Retrieval-Augmented Generation (RAG) systems by coordinating improvements across the pipeline. Instead of optimizing components in isolation, GRADRAG models the…
Thinkink: 2D Spatial Ink-native Interaction with LLMs
The researchers propose Thinkink, a tool that allows users to interact with large language models (LLMs) using handwritten text or sketches on a shared canvas. The tool uses a semantic tree for ink interpretation and a…
The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
The paper addresses two open questions regarding a method called PortLLM, which adapts large language models after continual pretraining. The authors perform an extensive empirical study to explore the long-term…
Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs
A study investigates the impact of a structured evidence-sufficiency prompt on reducing overconfident answers from four language models in a clinical context. The study finds a significant reduction in unsafe…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
