🔥 Ultra-FineWeb-L1 is here —— 1T+ tokens of high-quality web data are now available!

High-quality data is the foundation of powerful LLMs. Ultra-FineWeb-L1 is now open-sourced as part of the UltraData ecosystem, providing a carefully processed English web corpus from Common Crawl.

Built for Better Data Quality
1️⃣ Fresh Web Data: Collected from recent Common Crawl snapshots, covering data up to CC-MAIN-2025-51, with 1T+ tokens and ~1.14B documents.
2️⃣ Refined Processing Pipeline: Combines text extraction, language filtering, heuristic filtering, sensitive-field replacement, deduplication, and customized cleaning to improve data quality.

⚡ Highlights:
- 1T+ tokens of filtered English web data
- Enhanced cleaning for noisy web content, encoding issues, and abnormal documents
- Designed as the L1 filtered layer in the UltraData management framework
- Aligned with the latest Ultra-FineWeb updates http://huggingface.co/datasets/openbmb/Ultra-FineWeb

🔗 Resources
🤗 Dataset: http://huggingface.co/datasets/openbmb/Ultra-FineWeb-L1
📄 Paper: http://arxiv.org/abs/2505.05427
🧩 Classifier: http://ultradata.openbmb.cn/