consistency
4 notes, newest first.
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study compares two access modalities of a large language model (LLM), ChatGPT's chat UI and OpenAI's API, with and without web search enabled. The research shows that the accuracy of responses varies between the…
Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
The paper introduces CoCoEvolve, a method aimed at improving the consistency between chart, table, and code representations in AI systems. The method optimizes models based on agreement between different…
Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations
The paper introduces a method called Reasoning Consistency Scanning to assess the logical consistency between stated reasoning and answers in AI safety evaluation transcripts. The method can be applied to various AI…
From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
A new framework called MAGNET is introduced for generating long-form narratives using multiple agent characters that maintain consistency and coherence in their actions based on a shared world state and evolving story…
