Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Variance-reduced Domain Adaptation using Paired Sampling

Source

arXiv

arxiv.org/abs/2607.20367v1

Read the source

Source

Plain-Language Brief

The paper proposes Paired Sampling for Domain Adaptation (PSDA), a new stochastic variance reduction technique designed for distribution-matching frameworks in unsupervised domain adaptation (UDA) that suffer from high variance and are incompatible with traditional methods. PSDA reduces variance by pairing observations within and across domains to form quadruplets, which are trained together. The results of simulations show reduced variance compared to related methods, and experiments on three domain shift datasets indicate improved target domain accuracy.

What the Document Says

  • Correlation alignment and maximum mean discrepancy are distribution-matching frameworks for unsupervised domain adaptation (UDA).
  • High variance in these losses can undermine their effectiveness in minibatch optimisation settings.
  • These losses lack finite-sum structure, making them incompatible with classical stochastic variance reduction (SVR) methods.
  • The paper introduces Paired Sampling for Domain Adaptation (PSDA), an SVR technique tailored to such objectives.
  • PSDA pairs observations both within and across domains to form quadruplets that are always sampled together during training.
  • The pairings are designed to minimize expected gradient variance.
  • Reduction to solving a set of linear assignment problems occurs in PSDA.
  • Simulations demonstrate reduced variance compared to related methods.
  • Experiments on three domain shift datasets show improved target domain accuracy.

Key Concepts Mentioned

  • unsupervised domain adaptation (UDA)
  • distribution-matching frameworks
  • stochastic variance reduction (SVR)
  • Paired Sampling for Domain Adaptation (PSDA)
  • quadruplets
  • linear assignment problems
  • simulations
  • domain shift datasets

Claims Made by the Document

  • High variance in distribution-matching frameworks can undermine their effectiveness in minibatch optimisation settings.
  • Classical stochastic variance reduction methods are incompatible with these distribution-matching frameworks.
  • PSDA reduces gradient variance.
  • Improved target domain accuracy is achieved by using PSDA.

Evidence Used

  • The text states that high variance in distribution-matching frameworks has been shown to undermine their effectiveness.
  • The text mentions that classical SVR methods are incompatible with these distribution-matching frameworks because they lack finite-sum structure.
  • The text describes how PSDA reduces gradient variance by pairing observations and minimizing expected gradient variance.
  • The text reports improved target domain accuracy achieved through the use of PSDA.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Correlation alignment and the maximum mean discrepancy are two widely used distribution-matching frameworks for unsupervised domain adaptation (UDA). However, high variance in these losses has been shown to undermine their effectiveness in minibatch optimisation settings. Furthermore, the losses lack finite-sum structure, which renders them incompatible with classical stochastic variance reduction (SVR) methods. This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives. PSDA pairs observations both within and across domains, to form quadruplets that are always sampled together during training. The pairings are designed to minimise expected gradient variance, and reduce to solving a set of linear assignment problems. Our simulations demonstrate reduced variance compared to related methods, and experiments on three domain shift datasets show improved target domain accuracy.