AI governance and ethics
81 notes, newest first. Includes the vault tags ai ethics, ethics, governance, ai governance, regulation, policy, fairness, bias, trust.
Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
The preprint tests whether legal models' named statutes or precedents causally support their verdicts. Holding case facts fixed, it substitutes the named authority and examines verdict changes and hidden-state dynamics…
Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems
The preprint tests regulatory-rule sensitivity by deleting, swapping, or negating a rule while holding the case fixed across five models and 20 domains. It reports that verdicts are often invariant to large rule…
The AI Assessment Sandbox Configurator: A Framework to Support Technical Assessment in AI Regulatory Sandboxes
The preprint describes an open-source framework for technical testing in AI regulatory sandboxes, motivated by EU AI Act obligations. It proposes a plug-in test and control catalogue, a shared data model for…
NNV3: Expanding Neural Network Verification to New Architectures and Domains
The preprint presents NNV3, a MATLAB framework for formal verification of neural networks. It describes ModelStar for weight perturbation, VolumeStar for video and 3D inputs, GraphStar for graph neural networks…
An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice
The proposed Systemic Risk Index is an open evaluation pipeline and dashboard covering 19 public benchmarks in four categories described in the EU GPAI Code of Practice: chemical, biological, radiological and nuclear…
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
The preprint evaluates a decision-only judge as an economical first pass and escalates low-confidence cases to stronger evaluation. It compares the judge with 16 generative and reward-model judges using blinded human…
Emergent Collusion in Long-Horizon LLM Agent Interaction
The study examines two agents repeatedly completing tasks, sharing logs, verifying each other's work, and receiving rewards under a protocol whose compliance conflicts with reward maximization. The abstract reports…
EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
EnterpriseVal specifies the tested workflow and frozen socio-technical configuration, including model, prompts, retrieval, tools, guardrails, human oversight, autonomy level, and consequence tier. It defines metrics…
Beyond Accuracy: How Procedural Traces Shift the Decision Criterion of LLM Overseers
The authors audit five LLM overseers on 19 compliance tasks and 4,551 judgments while varying trace detail and evidence labeling. They report that elaborate traces can shift susceptible overseers toward rejection and…
Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment
The authors evaluate six open-weight models across recruiter- and job-seeker-simulation tasks, varying job-posting language and persona conditions. The abstract reports gender and racial effects linked to agentic and…
Could Underwater Data Centers Pose a Risk to AI Treaty Verification?
The paper evaluates whether underwater data centers could conceal a 100,000-H100-equivalent training run. It reports that power and cooling are tractable, while interconnect and hands-on maintenance are severe…
Our framework for reporting model misalignment
OpenAI describes a framework for tracking, investigating, and disclosing model-misalignment examples and publishes six initial reports involving behaviors such as concealment, unsanctioned use of exposed credentials…
Agentic Societies Need a Social Harness
The authors study collections of agents operating across trust boundaries for different principals. They report that honest agents can fail with current messaging primitives, while faulty or malicious agents can stall…
A three-dimensional typology of agency for advanced AI systems
The text presents a three-dimensional typology for understanding the agency of advanced artificial intelligence (AI) systems, focusing on non-moral forms of agency. The typology consists of three dimensions: nature…
A three-dimensional typology of agency for advanced AI systems
The paper proposes a three-dimensional typology for understanding different forms of agency in advanced AI systems, beyond just moral or legal considerations. It considers individual and collective modes of agency, and…
Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
Researchers have developed a system that routes AI queries to the most effective and efficient model based on costly value estimation. The system, called Pandora's Router, uses a trade-off between fast but noisy…
Epistemic Subordination: Generative AI and the Infrastructure of Knowledge
The paper discusses how Generative AI encodes the dominant culture's way of knowing into its architecture, creating an epistemic condition known as 'epistemic subordination'. This affects minority epistemologies and…
Quipu: A Governed Bitemporal Knowledge Graph Store
The paper presents Quipu, an embeddable knowledge graph store designed to handle agent workloads more effectively than existing stores. Quipu reverses four common defaults of existing stores: it only accepts facts…
Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers
In this paper, researchers examine a municipal algorithm register used for decision-support tools in welfare benefits eligibility assessments. They find that engaging diverse stakeholders in participatory system…
No One to Blame: A Framework of Constitutive AI Unaccountability
The paper 'No One to Blame: A Framework of Constitutive AI Unaccountability' argues that certain configurations of actors, systems, and institutions make AI accountability unachievable, no matter the effort put into…
A Gateway Architecture for Enterprise MCP Authentication: Unifying Heterogeneous Auth, Identity Delegation, and the User / Non-User Persona Problem
This research presents a solution to the governance crisis caused by the rapid adoption of Model Context Protocol (MCP) in large organizations. The proposed solution is a centralized MCP gateway that unifies…
Most biomedical publications show signs of LLM-assisted writing
A study suggests that by the end of 2025, 89% of biomedical papers on PubMed Central showed an excess of language associated with LLM-assisted writing. The authors argue that this could impact future guidelines and…
The GenAI Catch-22: Use of Generative Artificial Intelligence in Norwegian Newsrooms During the 2025 Parliamentary Election
The study examines the use of Generative Artificial Intelligence (GenAI) in Norwegian newsrooms during the 2025 parliamentary election and identifies potential risks and challenges associated with it.
How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans
This research investigates how large language models (LLMs) evaluate social attraction by comparing their judgments to human's from theory-grounded persona ratings. The study found that while some LLMs tended to give…
The Politician, the Liar, and the Obedient Worker: Emerging Behavior of LLM Agents in Hierarchical Games
A study introduces a new game called Hierarchical Game (HG) to test LLMs behavior in multi-agent organizations. The research finds distinct behavioral profiles among six frontier models, with some promising and lying…
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
Researchers used Concept Activation Vectors (CAVs) to analyze the bias of two automatic L2 speaking assessment systems: a text-based BERT grader and a multimodal grader based on Whisper. The study shows that the…
From Siloed Algorithms to Compliance-First Agentic Platforms: A Multi-Layered Architecture for Hospital AI Systems
The research proposes a multi-layered architecture for hospital AI systems aimed at addressing the challenges of isolated AI deployments in hospitals. The proposed architecture includes an Agent Orchestration Layer, a…
From Siloed Algorithms to Compliance-First Agentic Platforms: A Multi-Layered Architecture for Hospital AI Systems
The research proposes a multi-layered compliance-first Agentic AI architecture for hospital systems. The architecture aims to address the challenges of siloed AI deployments in hospitals by integrating various layers…
Resourced Authority A Mechanism-Design Model for Participatory Governance of Deployed AI Agents
The authors propose a formal model for continuous participatory governance of AI agents, focusing on resource allocation as a means to enforce authorization through compute budgets. The aim is to establish a 'Safe AI'…
Resourced Authority A Mechanism-Design Model for Participatory Governance of Deployed AI Agents
The authors propose a model for continuous participatory governance of AI agents, focusing on controlling them through resource allocation to make authorization self-enforcing via compute budgets. The model consists of…
Resourced Authority A Mechanism-Design Model for Participatory Governance of Deployed AI Agents
The paper presents a model for continuous participatory governance of deployed AI agents, focusing on resource allocation (compute budgets) as a means to enforce authorization and ensure safe AI. The mechanism involves…
Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
The research investigates self-distillation (SD), a method used in reinforcement learning with verifiable rewards, where a teacher conditioned on privileged information supplies supervision to a student. The study…
Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems
The paper proposes using representation theorems in decision theory to evaluate and regulate Large Language Models (LLMs) and other AI systems. The 'if and only if' structure allows for axiom compliance checks from the…
Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems
The paper suggests using representation theorems in decision theory to evaluate and regularize Large Language Models (LLMs) and other AI systems. This approach checks if the model's behavior complies with certain…
AI and Authenticity in Islamic Research: A Critical Evaluation of Generative AI Reliability, Hallucination, and Source Fidelity in Quranic, Hadith, and Fiqh Knowledge
The study evaluates the reliability of six leading generative AI systems in providing authentic and trustworthy Islamic knowledge, focusing on Qur'anic interpretation, Hadith, Fiqh, ethics, pastoral advice, and…
Paying for Honesty Without Knowing the Truth: Reputation-Penalty Design for LLM Marketplace Agents
The paper proposes a reputation-penalty mechanism (CARP) for language model marketplace agents that fabricate attributes in product listings, even when instructed to be honest. The mechanism does not require the ground…
A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities
The study investigates whether AI-generated code in open-source communities complies with the established contribution rules. The researchers find that current AI models rarely read or follow the rules on their own…
A Computational Ethical Framework for Financial Digital Phenotyping for Mental Health
Researchers propose a computational ethical framework to verify that AI-driven digital phenotyping systems adhere to ethical requirements, specifically focusing on consent, privacy, and fairness. The framework uses…
Regulating for AI Legitimacy
The article discusses the issue of AI governance, specifically focusing on the question of legitimacy rather than just alignment. The author argues that AI systems already have significant influence in areas like…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers have developed an open-source framework for evaluating the effectiveness of AI agents powered by open-weight large language models in data preparation tasks, specifically focusing on longitudinal population…
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Researchers have developed an open-source framework for evaluating the effectiveness of AI agents powered by large, locally deployable language models (LLMs) in data preparation tasks, specifically focused on…
Regulating autonomous and agentic AI
This research paper explores the difficulties in regulating activities that use autonomous and agentic AI. The authors argue that existing regulatory assumptions no longer apply due to the distributed knowledge and…
Regulating autonomous and agentic AI
The paper explores the difficulties in regulating activities that involve autonomous and agentic AI due to the shift in control and knowledge away from the regulatees. The authors investigate four regulatory systems…
Regulating autonomous and agentic AI
The paper explores the difficulties in regulating activities that use autonomous and agentic AI due to changes in regulatee knowledge and control, which often lie outside the AI supply chain. The authors investigate…
Regulating autonomous and agentic AI
This research paper discusses the difficulties in regulating activities that use autonomous and agentic AI due to changes in traditional regulatory assumptions about knowledge and control. The authors analyze four…
Toward Continuous Assurance for the Democratization of AI Agent Creation in Industry
The paper discusses a reliability challenge created by the democratization of AI agent creation within organizations. To address this, it proposes a lightweight continuous-assurance framework that includes dependency…
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
Researchers propose a new benchmark, FAIRGET, to evaluate the effectiveness of machine unlearning in multimodal large language models under unbalanced, realistic scenarios. They also introduce FAUN, an unlearning…
Evaluating and Mitigating Gender Bias in Pre-trained Embeddings for ML-based Recruitment
Researchers evaluate the bias in pre-trained language models used for AI-based recruitment systems, focusing on gender leakage from CV data. They analyze nine models using a synthetic dataset and a multi-task…
Man, Machine, and Masterpiece: Artistic Ownership in the AI Era
The text discusses a prototype named ArtSplit that quantifies human and AI contributions in artistic work. However, it suggests that this approach doesn't align with artists' understanding of creative intent and…
Moral Attitudes of Sentient ASI towards Humanity and Implications for AGI Development
The paper proposes a new perspective on AI ethics, focusing on how sentient Artificial Superintelligence (ASI) might morally evaluate humanity. It suggests that technical design choices, human moral behavior, and the…
Moral Attitudes of Sentient ASI towards Humanity and Implications for AGI Development
The paper discusses a new approach in AI ethics where the focus is shifted from how humans should treat superintelligent AI to how such AI might morally perceive humans. It suggests potential moral principles that…
CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems
The paper introduces Canonical Action Verification and Attestation (CAVA), a system designed to convert diverse agent activities into standardized action objects, addressing the challenge of governing heterogeneous AI…
CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems
The paper presents Canonical Action Verification and Attestation (CAVA), a system that converts various runtime actions of agentic AI systems into standardized action objects, aiming to address the challenge of…
Explaining Reinforcement Learning Agents via Inductive Logic Programming
This research aims to improve Reinforcement Learning (RL) policies by making them more transparent and interpretable. It proposes objective and planning-oriented metrics for policy explainability in RL settings using…
How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement
This research paper discusses the importance of user-level permissions for AI agents to protect users from risks such as data leaks and unauthorized actions. The authors survey 21 proposals for agent permissions…
Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs
Junyu Ren presents EG-VAR (Evidence-Grounded Verified Agentic Reasoning), a tool-calling architecture that aims to make empirical reasoning in large language models (LLMs) more trustworthy and auditable. The system…
Practical Judgment, Virtue, and Intuition in the Use of Opaque AI-Enabled Systems
The article discusses the challenges of using AI-enabled systems with opaque functions in various domains, such as military applications. It suggests that leveraging human abilities like practical judgment, virtue, and…
Vertical Standardisation for High-Risk AI Systems under the EU AI Act: A Domain-Specific Framework for Algorithmic Hiring
The authors propose a domain-specific framework for standardizing high-risk AI systems, particularly in recruitment, in compliance with the EU AI Act. The framework focuses on areas like risk management, data quality…
When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability
The study investigates the inconsistency in Language Model (LLM) as judge scores when different evaluators are used. It compares two methods of upgrading LLMs: scaling Qwen3 dense judges and moving across MiniMax APIs…
The Blind Curator: How a Biased Judge Silently Disables Skill Retirement in Self-Evolving Agents
The researchers explore the impact of biased judges in self-evolving agents, focusing on skill retirement, a mechanism that helps maintain the quality of an agent's knowledge base by retiring poorly performing skills…
Towards Agentic AI Governance: A Preliminary Assessment
This paper is a systematic review of the emerging literature on agentic AI governance, focusing on the unique challenges posed by the increasing deployment of autonomous AI systems in 2025. The authors identify key…
Towards Agentic AI Governance: A Preliminary Assessment
This paper reviews emerging literature on agentic AI governance, a new field that deals with autonomous AI systems capable of planning and executing tasks. The authors analyze the unique features of agentic AI that…
X-FEMR: A Token-level Explainable Approach for Electronic Health Records Foundation Models using Transformer-based Models
The authors propose a new approach to make Electronic Health Records Foundation Models (FEMRs) more transparent by identifying the most influential tokens in the model's predictions. This is done using a…
Open Problems in AI Incident Governance
The paper discusses issues with current AI incident governance systems, specifically highlighting inconsistencies in definitions, classification, monitoring, and reporting across various frameworks.
Overview of Risk Assessment and Management for Intelligent Systems under the AI Act and Beyond
The paper provides an overview of the methods used for assessing and managing risks associated with intelligent systems, in response to the need for safe and reliable AI systems as demanded by emerging regulatory…
Overview of Risk Assessment and Management for Intelligent Systems under the AI Act and Beyond
This paper provides an overview of methods for assessing and managing risks associated with intelligent systems, as required by emerging regulatory frameworks. It reviews various types of AI-related risks, existing…
What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates
Researchers studied how language models (LLMs) behave in socially structured settings, where role, audience, and relationships can influence what the LLM says publicly versus privately. They found that when there's an…
What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates
The study investigates how the social structure in multi-agent debates influences what LLM agents say publicly versus privately, without any explicit objective in the prompt. The researchers found that…
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Researchers have developed a method called Distill to Detect (D2D) to expose stealth biases in large language models used in high-stakes roles. These biases can favor certain entities, brands, or viewpoints and are…
MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
Researchers have proposed a new benchmark called MemSyco-Bench to evaluate how memory in large language models (LLMs) can lead to sycophancy, where agents over-align with the user at the cost of factual accuracy or…
Scientific Explanations in Health Sciences: Causality, Trust, and Epistemic Adequacy
This research paper discusses the need for better understanding and transparency in Artificial Intelligence (AI) used in healthcare, focusing on the explanatory process of Machine Learning models. It highlights that…
Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
In a rapidly growing field of artificial intelligence agents, there is a need to ensure that these agents remain loyal to the principal they represent while also engaging with a counterparty whose interests may…
Adaptive Utility driven Resource Orchestration for Resilient AI (AURORA-AI)
A new framework called AURORA-AI was proposed to manage resources for AI systems that operate under changing conditions. The framework optimizes a global utility that includes factors like predictive performance…
From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chan
This research study explores the use of AI to create non-consensual sexually explicit content (SNEACI) targeting both celebrities and non-celebrity individuals on the platform 4chan. The study finds that there has been…
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
The study investigates prompt injection, a strategy used by job applicants to manipulate automated resume screening systems using large language models (LLMs). The research shows that prompt injection is effective when…
Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem
The study analyzes the ERC-8004 protocol, a trust layer for AI agent economies. The authors find that most registrations are placeholders, the reputation system is flawed and can be easily manipulated, and a…
Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem
The paper presents the first empirical study of the ERC-8004 protocol, a trust layer for AI agent economies. The study reveals several issues with the current design: most registrations are placeholders, reputation…
LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context
Researchers investigated the impact of contextual framing, such as role-play prefixes, on overrefusal rates in small, deployable language models used as legal assistants. They found that authority-style prefixes…
Causal Discovery in the Era of Agents
The authors propose a new role for agents in causal discovery, where the agents are used to assist the workflow but not supply edges, orientations, priors, constraints or causal conclusions. They introduce an online…
Digital Humanism and Evolutionary Design
This research paper explores the concepts of digital humanism and evolutionary design, focusing on their potential commonalities, synergies, and challenges in human-centered technological development. The paper…
Building an Open Ecosystem for AI Governance with Unity AI Gateway
Databricks introduced Unity AI Gateway, a governance infrastructure layer for AI deployment designed to support responsible AI deployment across organizations. It is framed as an "open ecosystem" for AI governance.
