FineData

community

AI & ML interests

We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)

Recent Activity

joelniklaus updated a Space about 22 hours ago

HuggingFaceFW/finephrase

craffel authored a paper 2 days ago

Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models

cfahlgren1 submitted a paper 2 days ago

From AGI to ASI

View all activity

Papers

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

View all Papers

HuggingFaceFW 's Spaces 8

The Synthetic Data Playbook: Generating Trillions of the Finest Tokens

Visualize synthetic‑data experiments as an interactive bookshelf

README

FinePDFs: Liberating 3T of the finest tokens from PDFs

FineWiki Viewer

Viewer to explore the finewiki dataset

FineWeb: decanting the web for the finest text data at scale

Explore and download the FineWeb web‑scale text dataset

Scaling FineWeb to 1000+ languages: Step 1: finding signal in 100s of evaluation tasks

Evaluate multilingual models using FineTasks

Tasks Explorer

Explore and analyze experiment results

Datasets Metrics Explorer

Launch an interactive demo interface