extsc{TestifAI}: Tomography-Based Testing for Deep Learning Systems

\textsc{TestifAI}: Tomography-Based Testing for Deep Learning Systems

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Arooj Arif, Tobias Hartung, Elena Botoeva, Alexandros Koliousis
• Venue: arXiv: cs.AI
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.18900v1
• PDF: https://arxiv.org/pdf/2608.18900v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

TestifAI is a deep learning testing framework that efficiently and accurately estimates the robustness of AI models against combinations of perturbations. It allows users to specify operational conditions as structured spaces of semantic input perturbations (such as image blur, brightness, and zoom) and discrete severity levels (low, medium, high). The framework uses partial model tomography, a novel approach to reconstructing model behavior in a multi-perturbation space from tests that apply only a small number of perturbations. TestifAI reduces the number of inferences by 60-80% and has an aggregate robustness estimation error of less than 7% in experiments on five image and language classification tasks.

What the Document Says

• TestifAI is a deep learning testing framework.
• It allows users to specify operational conditions as structured spaces of semantic input perturbations (e.g., image blur, brightness, and zoom) and discrete severity levels (e.g., low, medium, high).
• It introduces partial model tomography for reconstructing model behavior in a multi-perturbation space.
• It trains an auxiliary model to estimate robustness against at least three perturbations without executing an exponential number of tests.
• Experiments on five image and language classification tasks show that TestifAI can predict higher-order (3 and 4 perturbations) test outcomes from low-order (1 and 2 perturbations) observations with an aggregate robustness estimation error of less than 7% and reduces the number of inferences by 60-80%.

Key Concepts Mentioned

• AI systems
• deep learning models
• testing framework
• partial model tomography
• perturbations
• image blur
• brightness
• zoom
• discrete severity levels

Claims Made by the Document

• TestifAI can efficiently and accurately estimate the robustness of deep learning models against combinations of perturbations.
• TestifAI reduces the number of inferences by 60-80%.

Evidence Used

• The text describes TestifAI as a deep learning testing framework that achieves efficiency and accuracy.
• The text mentions that TestifAI reduces the number of inferences by 60-80%.

Methods / Approach

• partial model tomography
• training an auxiliary model

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The text does not mention any limitations of TestifAI.

Open Questions

• What specific AI application domains does the paper focus on?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper focuses on AI systems in safety-critical application domains, but it is not specified which ones exactly. The text only mentions examples such as autonomous driving.

Raw Abstract / Excerpt

As AI systems are increasingly deployed in safety-critical application domains (e.g., autonomous driving), associated risks increase too. Deep learning models underlying modern AI systems, therefore, must undergo thorough testing to ensure their correct behaviour. A single robustness test involves thousands of inferences to empirically verify if a model's outputs remain stable under a bounded perturbation of its inputs. However, existing testing frameworks lack the means to systematically explore and summarise robustness across a combinatorial space of perturbations. We propose TestifAI, a deep learning testing framework for efficient and accurate estimation of robustness against combinations of perturbations. TestifAI enables users to specify operational conditions as structured spaces of semantic input perturbations (e.g., image blur, brightness and zoom) and discrete severity levels (e.g., low, medium and high). Users can query model robustness for any combination (e.g., 'low blur, high brightness, and medium zoom'). To achieve efficiency and accuracy, TestifAI introduces partial model tomography, a novel approach to reconstructing model behaviour in a multi-perturbation space from tests that apply only a small number of perturbations (lower-order projections). To estimate robustness against at least three perturbations, TestifAI trains an auxiliary model on the results of tests involving up to two perturbations only, avoiding execution of an exponential number of tests. Our experiments on five image and language classification tasks show that TestifAI can predict higher-order (3 and 4 perturbations) test outcomes from low-order (1 and 2 perturbations) observations with an aggregate robustness estimation error of less than 7%, while reducing the number of inferences by 60-80%.