Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Anatomy Contextualized Adaption of CT Foundation Models

Source

arXiv

arxiv.org/abs/2607.27154v1

Read the source

Source

Plain-Language Brief

Researchers present Anatomy Contextualized Adaptation (ACA), a framework that modifies pre-trained CT foundation models for improved anatomy-level vision-language alignment while maintaining the global context of whole-volume models. ACA uses anatomical embeddings from TotalSegmentator, a transformer to capture cross-anatomy relationships, and aligns these with text from radiology reports. The method outperforms baselines and existing fine-grained methods in zero-shot finding classification tasks, requiring less training time.

What the Document Says

  • CT vision-language foundation models have shown promising performance but use whole-volume representations that dilute fine-grained anatomical signals.
  • Fine-grained vision-language pre-training aligns visual features with anatomy-specific text, but discards global context provided by whole-volume models.
  • ACA is a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment and enhances global contextualization.
  • ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer capturing cross-anatomy relationships.
  • The transformer-refined embeddings are aligned to both per-anatomy and scan-level text from radiology reports.
  • ACA outperforms the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification.
  • Evaluation of ACA shows it requires less than one hour of training once embeddings are cached.

Key Concepts Mentioned

  • Anatomy Contextualized Adaptation (ACA)
  • CT vision-language foundation models
  • TotalSegmentator
  • radiology reports
  • zero-shot finding classification

Claims Made by the Document

  • CT vision-language foundation models dilute fine-grained anatomical signals with whole-volume representations.
  • Fine-grained vision-language pre-training discards global context provided by whole-volume models.
  • ACA enhances the global contextualization of CT foundation models.
  • ACA outperforms frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification.

Evidence Used

  • The text mentions that whole-volume representations dilute fine-grained anatomical signals, and ACA aims to enhance global contextualization.
  • Results from evaluation of ACA show improved performance compared to the baselines and existing methods.

Methods / Approach

  • Anatomy Contextualized Adaptation (ACA)
  • TotalSegmentator
  • transformer

Limitations Stated by the Authors

  • The authors mention that existing fine-grained approaches train from scratch, making them computationally expensive.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • It is not stated in the source whether ACA has any limitations beyond those mentioned by the authors.

Open Questions

  • What is the specific impact of ACA on other vision-language tasks beyond zero-shot finding classification?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The text does not provide information about how ACA impacts other aspects of anatomy analysis or medical imaging, so it's important to ask about its effects on those areas.

Raw Abstract / Excerpt

CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.