VietVault: Large-Scale Vietnamese Corpus
VietVault is a massive corpus in Vietnamese built from Common Crawl (2013—2023). After language detection, cleaning, deduplication and filtering of toxic content, it provides a solid basis for pre-training or refining Vietnamese NLP models.
Description
VietVault is an 80GB corpus of Vietnamese texts extracted from over 10 years of Common Crawl dumps. Each text has been filtered to ensure linguistic quality (Vietnamese), diversity of sources, removal of toxic content and removal of duplicates. The Markdown format makes it easy to use directly for model training.
What is this dataset for?
- Pre-train language models in Vietnamese (BERT, RobertA, GPT...)
- Fine-tuning on classification tasks, entity recognition, summary
- Linguistic studies and modeling of the modern Vietnamese language
Can it be enriched or improved?
Yes, VietVault can be crossed with annotated corpora to add supervised labels. It is also possible to inject metadata (dates, themes, source) or to divide it according to the types of content (news, blogs, forums). Its UTF-8 Markdown base simplifies enrichments.
🔎 In summary
🧠 Recommended for
- Computational linguists
- Vietnamese AI projects
- Regional LLM pre-training
🔧 Compatible tools
- Hugging Face Datasets
- PyTorch
- TensorFlow
- FastText
- Tokenizers
💡 Tip
Use a specific Vietnamese tokenizer (like VNCoRenLP) to maximize the quality of pre-workout preprocessing.
Frequently Asked Questions
Does this corpus contain automatically translated texts?
No, the dataset is based on native Vietnamese content identified through language detection, without automatic translation.
Can VietVault be used to train a multilingual model?
Yes, by integrating it with other corpora in different languages, VietVault makes it possible to enrich the linguistic coverage of an LLM.
Is it suitable for industrial use?
Yes, the ODC-by license allows commercial use, provided the sources are properly attributed (VietVault + Common Crawl).
