FIELD NOTES TOPIC
data preprocessing
2 notes, newest first.
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Researchers propose a framework called DataOrchestra that adapts data processing strategies for each example in Large Language Models (LLMs) pretraining, improving performance across 11 benchmarks.
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
Researchers propose a new method to select data for training neural networks in large scale, noisy datasets. The Self-Filtering method iteratively trains a CLIP model on an evolving dataset, balancing clean samples and…
