Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles

Source

arXiv

arxiv.org/abs/2607.08373v1

Read the source

Source

Plain-Language Brief

The paper proposes an online anomaly detection framework for autonomous cyber-physical systems (CPS), specifically connected vehicles, that combines three coordinated mechanisms: a deep Q-network with self-attention to select the best anomaly detector among candidates based on inter-service dependencies; an ensemble of statistical drift detectors that prioritize precision over recall and only raise alarms when all three concur; and a human-in-the-loop retraining mechanism for incorporating expert knowledge while preserving the system's learned response to prior data distributions.

What the Document Says

  • The paper discusses an anomaly detection framework for autonomous CPS in connected vehicles.
  • The framework consists of a factorized deep Q-network, an ensemble of statistical drift detectors, and a human-in-the-loop retraining mechanism.
  • The deep Q-network selects the most suitable detector for each monitored service based on inter-service dependencies.
  • The statistical drift detectors prioritize precision over recall and raise alarms only when all three concur.
  • The human-in-the-loop retraining mechanism allows for incorporation of expert knowledge while preserving the system's learned response to prior data distributions.
  • The framework was evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices.
  • The attention-augmented agent achieved an F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly.
  • After a software update that induces concept drift, the F1 dropped to 0.52; after operator-triggered retraining, performance recovered to 0.65 on the new distribution while remaining at 0.69 on the prior one.

Key Concepts Mentioned

  • anomaly detection
  • connected vehicles
  • cyber-physical systems (CPS)
  • deep Q-network
  • self-attention
  • statistical drift detectors
  • human-in-the-loop retraining

Claims Made by the Document

  • The proposed framework effectively detects anomalies in connected vehicles.
  • The attention-augmented agent outperforms single detectors applied uniformly.

Evidence Used

  • F1 score of 0.69 for the attention-augmented agent compared to at most 0.11 for single detectors

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Connected vehicles are autonomous cyber-physical systems whose behavior must be continuously monitored during operation to detect deviations from normal operation before they propagate into failures. Such evaluation is challenging because the systems themselves evolve: over-the-air updates, configuration changes, and shifting workloads alter the definition of normal behavior, causing static diagnostic methods to degrade silently over time. Existing approaches typically address either automated model adaptation or operator integration in isolation, rather than as a single coordinated supervisory loop. This paper presents an online anomaly detection framework for autonomous CPS that integrates three coordinated mechanisms. A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service, exploiting inter-service dependencies in the microservice topology. An ensemble of three statistical drift detectors monitors the input distribution and raises an alarm only when all three concur, prioritizing precision over recall. A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge while preserving the system's learned response to prior data distributions. The framework is evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices. The attention-augmented agent achieves an F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly. Following a real software update that induces measurable concept drift, F1 drops to 0.52; after operator-triggered retraining, performance recovers to 0.65 on the new distribution while remaining at 0.69 on the prior one, demonstrating sustained adaptation without catastrophic forgetting.