An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

Source facts

The proposed Systemic Risk Index is an open evaluation pipeline and dashboard covering 19 public benchmarks in four categories described in the EU GPAI Code of Practice: chemical, biological, radiological and nuclear risks; cyber offense; harmful manipulation; and loss of control. The paper describes harm-preserving perturbations, simulated deployment, average and worst-case aggregation, and traceability across 18 models. It reports worst-case scores falling by 14–37 points under the proposed aggregation and blind-audit agreement measures.

Analysis

The work connects model evaluation to a regulatory evidence workflow: benchmark results are transformed, aggregated, and exposed with provenance rather than reported as a single static score. That could make systemic-risk evidence more inspectable, but the choice of benchmarks, perturbations, aggregation, and simulated deployment remains consequential.

Uncertainty/Doubt

• This is an arXiv preprint submitted to an EACL 2027 Systems Demonstration track. The benchmark transformations, model results, judge agreement, and regulatory fit were not independently verified here.