By clicking "Accept", you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. See our Privacy Policy for more information
Text

mC4

The <strong>mC4 dataset (Multilingual Colossal Cleaned Common Crawl)</strong> is a massive multilingual text corpus, derived from the Common Crawl project and then cleaned by AllenAI. It covers 108 languages, including French, English, English, Arabic, Swahili, etc., and contains hundreds of billions of tokens. This corpus is frequently used as a basis for pre-training or fine-tuning models such as T5, mT5, XLM or BLOOM.

Download dataset
Size

Over 1 TB of cleaned text, in JSONL format, for 108 languages

Licence

ODC-BY

Description

‍

mC4 (Multilingual C4) is a multilingual version of the C4 dataset, itself the result of an advanced Common Crawl cleaning. It is commonly used in the training of multilingual LLMs like mT5. Each language is stored in a separate file, facilitating targeted extraction.

‍

‍

What is this dataset for?

‍

  • Pre-train or refine multilingual language models
  • Building a raw, realistic, varied web corpus (blogs, forums, web pages)
  • Testing the robustness of models through low-resource languages

‍

‍

Can it be enriched or improved?

‍

Yes. Some versions like MC4-FR or MC4-AR have been re-filtered or deduplicated. We can also reduce noise by crossing mC4 with other cleaned corpora (OSCAR, CCnet), or by applying quality filters (langdetect, perplexity).

‍

‍

🔎 In summary

Criterion Evaluation
🧩 Ease of use⭐⭐⭐✩✩ (Requires specific preprocessing depending on language)
🧼 Need for cleaning⭐⭐⭐⭐✩ (Medium – relatively clean but may contain web noise)
🏷️ Annotation richness⭐✩✩✩✩ (Low – no annotations, raw text only)
📜 Commercial license⚡ Yes (ODC-BY)
👨‍💻 Beginner friendly⚠️ No – very large, requires technical resources
🔁 Fine-tuning ready🎯 Yes, with preprocessing of text segments
🌍 Cultural diversity🌟 Excellent – 108 languages covered

‍

‍

🧠 Recommended for

  • AI laboratories
  • Multilingual researchers
  • LLM open source projects

‍

‍

🔧 Compatible tools

  • TensorFlow
  • PyTorch
  • Hugging Face Datasets

‍

‍

💡 Tip

For French-speaking use, use the filtered and tokenized mc4-fr file

Frequently Asked Questions

Is the dataset balanced between languages?

No, some languages are over-represented (e.g. English), others very weak (e.g. Wolof): a rebalancing is recommended according to your needs.

Is this dataset suitable for training a chatbot?

Yes, as a raw base for pre-training. For a chatbot, you then need a dialogued corpus or supervised data.

What is the difference between mC4 and C4?

C4 is only in English. mC4 is its multilingual version (108 languages), built using the same cleaning method.

Similar datasets

See more
Category

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique.

Category

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique.

Category

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique.