Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Multimodal Pretraining for Generalizable EEG Representation Learning

Source

arXiv

arxiv.org/abs/2607.21384v1

Read the source

Source

Plain-Language Brief

Researchers have developed a multimodal EEG foundation model that can be used for various EEG-related tasks without being limited to specific datasets or tasks. The model uses a raw signal encoder, a time-frequency data encoder, and a lightweight text encoder within a shared embedding space. It was pretrained using several innovative techniques like masked modeling, cross-view contrastive alignment, and temporal consistency losses. The model was evaluated on the CHB-MIT seizure detection benchmark and other seizure detection datasets, demonstrating state-of-the-art performance and robustness across different datasets and evaluation settings.

What the Document Says

  • Researchers developed a multimodal EEG foundation model.
  • The model includes a raw signal encoder, a time-frequency data encoder, and a lightweight text encoder within a shared embedding space.
  • Pretraining techniques used include masked modeling, cross-view contrastive alignment, and temporal consistency losses.
  • The study evaluated the model's effectiveness on the CHB-MIT seizure detection benchmark and additional seizure detection datasets.
  • The best single model achieved an AUROC of 0.874, and an ensemble variant reached 0.878 AUROC on the standard CHB-MIT split.
  • Performance was also evaluated under a leave-one-subject-out (LOSO) protocol with a mean LOSO balanced accuracy of 0.558 across 19 subjects.

Key Concepts Mentioned

  • Electroencephalography (EEG)
  • foundation model
  • multimodal
  • seizure detection
  • AUROC
  • leave-one-subject-out (LOSO)

Claims Made by the Document

  • The developed multimodal EEG foundation model can support a range of EEG related tasks.
  • The pretrained model demonstrates state-of-the-art performance on the CHB-MIT seizure detection benchmark.

Evidence Used

  • The text describes the development and pretraining process of the multimodal EEG foundation model.
  • Performance results are provided for the CHB-MIT seizure detection benchmark and other datasets.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Electroencephalography (EEG) models used for epilepsy are often limited to specific datasets and tasks. This limited approach can make it challenging to apply these models across different datasets or in various situations. However, recent studies in foundation models and self-supervised learning suggest that an adaptable EEG backbone could support a range of EEG related tasks. In this study, we have developed a multimodal EEG foundation model that combines a raw signal encoder based on the Mamba architecture, a Vision Transformer (ViT)-style encoder for time-frequency data, and a lightweight encoder for text, all within a shared embedding space. The pretraining process relies on several innovative techniques, such as masked modeling, cross-view contrastive alignment, and temporal consistency losses. These methods are designed to create rich, seizure-relevant representations without requiring labeled data. To assess the efficacy and generalization of our pretrained model, we fine-tuned it on the canonical CHB-MIT seizure detection benchmark and additional seizure detection datasets, and conducted extensive experiments comparing different model variants. On the standard CHB-MIT split, our best single model achieved an AUROC of 0.874, and an ensemble variant reached 0.878 AUROC, representing state-of-the-art performance on this benchmark. In addition to standard train-test splits, we evaluated performance under a leave-one-subject-out (LOSO) protocol, which is rarely reported in prior EEG seizure modeling work and highlights the difficulty of patient-independent seizure detection, with a mean LOSO balanced accuracy of 0.558 across 19 subjects. Across datasets and evaluation settings, our multimodal foundation model enabled robust seizure detection and straightforward adaptation to new seizure detection scenarios, while also supporting interpretable seizure localization.