mC4
The <strong>mC4 dataset (Multilingual Colossal Cleaned Common Crawl)</strong> is a massive multilingual text corpus, derived from the Common Crawl project and then cleaned by AllenAI. It covers 108 languages, including French, English, English, Arabic, Swahili, etc., and contains hundreds of billions of tokens. This corpus is frequently used as a basis for pre-training or fine-tuning models such as T5, mT5, XLM or BLOOM.
Description
mC4 (Multilingual C4) is a multilingual version of the C4 dataset, itself the result of an advanced Common Crawl cleaning. It is commonly used in the training of multilingual LLMs like mT5. Each language is stored in a separate file, facilitating targeted extraction.
What is this dataset for?
- Pre-train or refine multilingual language models
- Building a raw, realistic, varied web corpus (blogs, forums, web pages)
- Testing the robustness of models through low-resource languages
Can it be enriched or improved?
Yes. Some versions like MC4-FR or MC4-AR have been re-filtered or deduplicated. We can also reduce noise by crossing mC4 with other cleaned corpora (OSCAR, CCnet), or by applying quality filters (langdetect, perplexity).
🔎 In summary
🧠 Recommended for
- AI laboratories
- Multilingual researchers
- LLM open source projects
🔧 Compatible tools
- TensorFlow
- PyTorch
- Hugging Face Datasets
💡 Tip
For French-speaking use, use the filtered and tokenized mc4-fr file
Frequently Asked Questions
Is the dataset balanced between languages?
No, some languages are over-represented (e.g. English), others very weak (e.g. Wolof): a rebalancing is recommended according to your needs.
Is this dataset suitable for training a chatbot?
Yes, as a raw base for pre-training. For a chatbot, you then need a dialogued corpus or supervised data.
What is the difference between mC4 and C4?
C4 is only in English. mC4 is its multilingual version (108 languages), built using the same cleaning method.




