Challenges in Evaluating Explanation Methods for Static and Evolving Data

Source

Source name: arXiv
Source kind: paper
Authors / organization: Jerzy Stefanowski
Venue: arXiv: cs.AI
Published: 2026-08-06
Collected: 2026-08-07
URL: http://arxiv.org/abs/2608.06351v1
PDF: https://arxiv.org/pdf/2608.06351v1
DOI: UNKNOWN

Plain-Language Brief

This research paper highlights the issues in evaluating Explanation Methods for AI systems, using the DetoxAI image recognition system as an example. It discusses methods for adapting explanations to changing data streams and the challenges of tracking the evolution of data, models, and explanations.

What the Document Says

The paper discusses the limitations of Explainable Artificial Intelligence (XAI) in evaluation.
The DetoxAI image recognition system for bias detection and concept unlearning is used as an example.
A method for human-grounded evaluation of methods for explaining image classification is presented.
Methods for adapting explanations to evolving data streams with concept drift are explored.
Experiences with adapting counterfactuals for this problem are discussed.
The challenges of tracking the co-evolution of data, models, and explanations are addressed.

Key Concepts Mentioned

Explainable Artificial Intelligence (XAI)
DetoxAI
image recognition system
bias detection
concept unlearning
human-grounded evaluation
data streams
concept drift
counterfactuals
co-evolution of data, models, and explanations

Claims Made by the Document

The current evaluation methods for XAI are insufficient.
DetoxAI is an example of a system with issues in bias detection and concept unlearning.

Evidence Used

The paper points out the limitations of XAI in evaluation
The DetoxAI image recognition system is used as an example throughout the text

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

This paper addresses the limitations of Explainable Artificial Intelligence (XAI) with respect to insufficient evaluation. They are illustrated through the DetoxAI image recognition system for bias detection and concept unlearning. Then, an example of a human-grounded evaluation of methods for explaining image classification is presented. The paper further explores methods for adapting explanations to evolving data streams with concept drift. Experiences with adapting counterfactuals for this problem are discussed. Finally it is related to the challenges of tracking the co-evolution of data, models, and explanations.\footnote{This paper has been accepted for a publication in J.Nalepa (ed) Explainable AI in Space. Proceedings of EASi 2026 Workshop at IJCAI-ECAI 2026 Bremen, Springer CCIS vol 3107 (2016).}