HuggingFaceFW/dclm_100BT
Viewer • Updated • 89.3M • 609
🤏 Smol-Data
Tried and tested mixes for strong pretraining. Inspired by https://huggingface.co/blog/codelion/optimal-dataset-mixing
-
-
HuggingFaceFW/dclm_100BT-shuffled
Viewer • Updated • 89.3M • 873 • 1 -
HuggingFaceFW/finepdfs_100BT
Viewer • Updated • 29.9M • 7.31k -
HuggingFaceFW/finepdfs_100BT-shuffled
Viewer • Updated • 14.6M • 821 -
HuggingFaceFW/finepdfs_edu_100BT
Viewer • Updated • 17.8M • 5.04k -
HuggingFaceFW/finepdfs_edu_100BT-shuffled
Viewer • Updated • 17.8M • 992 -
HuggingFaceFW/fineweb_100BT
Viewer • Updated • 161M • 975 -
HuggingFaceFW/fineweb_100BT-shuffled
Viewer • Updated • 161M • 437 -
HuggingFaceFW/fineweb_edu_100BT
Preview • Updated • 341 -
HuggingFaceFW/fineweb_edu_100BT-shuffled
Viewer • Updated • 102M • 606 -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 56.1M • 51.4k -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 56.1M • 880 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 62.1M • 37.2k • 1 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 62.1M • 982 • 3