MathVista
MathVista is a benchmark for mathematical visual reasoning, combining 31 datasets into a single coherent corpus to test multimodal models.
Description
MathVista is an open-source benchmark bringing together 31 datasets, designed to assess the ability of models to solve mathematical problems in various visual contexts. It integrates logical MCQ tests (iQTest), graphical functions (FunctionQA), scientific figures extracted from articles (paperQA), as well as a wide variety of VQA and MathQA tasks from the literature.
What is this dataset for?
- Evaluate the ability of models to reason based on complex visuals (graphs, figures)
- Testing the performance of multimodal models in solving scientific problems
- Refine multimodal LLMs on enriched mathematical tasks
Can it be enriched or improved?
Yes, MathVista can be enriched with additional annotations (e.g. problem typology, difficulty level), or extended with new visual domains (plans, technical diagrams...). It is also possible to translate statements or to add summaries for linguistic simplification.
🔎 In summary
🧠 Recommended for
- Mathematical AI researchers
- VQA scientific projects
- Fine-tuning educational LLMs
🔧 Compatible tools
- Hugging Face Datasets
- PyTorch
- Transformers
- VLLM
💡 Tip
First use the split testmini (1,000 examples) to validate your models before moving on to the full evaluation.
Frequently Asked Questions
Does this dataset have answers for all of the examples?
No, only the 1,000 testmini entries have public answers. The rest of the dataset is intended for standardized evaluation.
Can MathVista be used to train a new model?
Yes, especially for fine-tuning in multimodal mathematical reasoning, with possible adaptation on subsets.
Is this dataset suitable for educational or school use?
It can be used in a research setting, but it requires a certain technical level in image processing and formal reasoning.
