WMT SQM Human Evaluation
The WMT SQM Human Evaluation dataset contains human evaluations on the quality of machine translations, collected during the WMT 2022 conference. Annotations combine a direct evaluation (raw score) and a scalar metric to stabilize scores between annotators.
Description
This dataset shows human DA+SQM (Direct Assessment + Scalar Quality Metric) annotations for the quality of machine translations from the WMT 2022 task. Each entry includes source text, machine translation, reference, evaluation score, and metadata such as language pair, domain, and year.
What is this dataset for?
- Evaluating the quality of machine translation systems
- Analyze inter-annotator variability and stabilize scores
- Improve and compare MT models across domains and languages
Can it be enriched or improved?
Yes, by adding more language pairs, from various fields, or by combining with other automatic or human evaluation metrics.
🔎 In summary
🧠 Recommended for
- MT researchers
- Automated Assessment Developers
- Linguistic quality analysts
🔧 Compatible tools
- Hugging Face Datasets
- MT assessment tools
- Python
💡 Tip
Use this dataset in conjunction with automatic metrics for a comprehensive assessment.
Frequently Asked Questions
Does this dataset contain machine translation training data?
No, it is only a human evaluation dataset.
What areas are covered by the data?
Mostly “news” texts, but several domains are available.
Can data be filtered by language pair?
Yes, it is possible to filter by source and target language using metadata.
