Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

RETRIEVALFORMER: TRANSFORMER-QUALITY RECOMMENDATIONS WITH EFFICIENT ANN RETRIEVAL AND COLD-START RESILIENCE

Source

OpenReview

openreview.net/forum

Read the source

Source

Plain-Language Brief

The researchers propose RetrievalFormer, a transformer-based recommendation system that combines high accuracy with efficient Approximate Nearest Neighbor (ANN) retrieval. This system can recommend unseen items during training and achieves competitive results on Amazon and MovieLens benchmarks, even outperforming strong content-based baselines in cold-start experiments.

What the Document Says

  • The authors introduce RetrievalFormer, a transformer-based dual-encoder recommender architecture that uses an attention-based heterogeneous feature encoder and an AttentionFusion module for efficient ANN retrieval.
  • On Amazon and MovieLens benchmarks, RetrievalFormer achieves 86–91% of the Recall@20 of strong transformer-based sequential baselines while enabling up to 288× lower latency via ANN retrieval.
  • RetrievalFormer attains a Recall@20 of 0.337 on MovieLens-1M.
  • In cold-start experiments, RetrievalFormer successfully recommends completely unseen items from their features in a leave-one-out cold (LOOC) protocol with zero item leakage, outperforming a strong content-based baseline.
  • The authors claim that their approach enables practical deployment of efficient recommendations at scale, offering a trade-off between model accuracy and serving efficiency.

Key Concepts Mentioned

  • Recommender System
  • Transformer-Based Architecture
  • Approximate Nearest Neighbor (ANN) Retrieval
  • Cold-Start Resilience

Claims Made by the Document

  • RetrievalFormer achieves competitive recommendation accuracy on Amazon and MovieLens benchmarks.
  • RetrievalFormer enables up to 288× lower latency at a 10M-item scale via ANN retrieval.
  • RetrievalFormer outperforms a strong content-based baseline in cold-start experiments.
  • RetrievalFormer offers a practical deployment of efficient recommendations at scale.

Evidence Used

  • Results on Amazon and MovieLens benchmarks show RetrievalFormer's competitive accuracy and lower latency.
  • The text mentions RetrievalFormer achieving Recall@20 of 0.337 on MovieLens-1M, indicating its performance in recommendation tasks.
  • Cold-start experiments results indicate RetrievalFormer's ability to recommend unseen items and outperforming a strong content-based baseline.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

We propose RetrievalFormer, a transformer-based dual-encoder recommender ar-
chitecture that combines competitive accuracy with strong transformer-based se-
quential baselines, efficient Approximate Nearest Neighbor (ANN) retrieval, and
the ability to score feature-described items that are unseen during training. Our ar-
chitecture uses an attention-based heterogeneous feature encoder that aggregates
item and user attributes via shared embedding tables and an AttentionFusion mod-
ule, so that the resulting user and item representations lie in a shared embedding
space suitable for ANN search. On Amazon and MovieLens benchmarks, Re-
trievalFormer achieves competitive recommendation accuracy, reaching 86–91%
of the Recall@20 of strong transformer-based sequential baselines while enabling
up to 288× lower latency at a 10M-item scale via ANN retrieval. On MovieLens-
1M, RetrievalFormer attains Recall@20 of 0.337. In cold-start experiments where
entire items and all of their interactions are held out during training, Retrieval-
Former successfully recommends completely unseen items from their features in
a leave-one-out cold (LOOC) protocol with zero item leakage between training
and evaluation, in which ID-softmax transformer baselines cannot produce scores
for such items at all, and it outperforms a strong content-based baseline on a 100%
cold-start production dataset. Our approach enables practical deployment of ef-
ficient recommendations at scale, offering a compelling trade-off between model
accuracy and serving efficiency.