VisualPuzzles — Multimodal Reasoning
Dataset of 1,168 illustrated logic puzzles, designed to test the algorithmic, spatial, and analog reasoning of AI models.
Description
VisualPuzzles is a multimodal benchmark structured around 1,168 visual puzzles classified by types of reasoning: algorithmic, analog, deductive, inductive and spatial. It was designed to test the ability of AI models to reason without relying on specific or encyclopedic knowledge.
What is this dataset for?
- Evaluate the reasoning skills of LLMs or vision-language models
- Testing the impact of “prompt engineering” or visual thought chains
- Comparing human and AI performance on cognitive tasks disconnected from pure knowledge
Can it be enriched or improved?
Yes. The dataset could be extended with cultural variants of the puzzles, other types of representations (audio, animations), or annotations on the expected resolution strategies. It is also relevant to use it as a basis for creating multilingual multimodal reasoning tasks.
🔎 In summary
🧠 Recommended for
- Cognitive AI researchers
- VLM developers
- Computational psychology laboratories
🔧 Compatible tools
- BLIP-2
- GPT-4V
- Lava
- Multimodal transformers
💡 Tip
Use this dataset to assess the logical robustness of a model before exposing it to real world knowledge.
Frequently Asked Questions
What is the particularity of this dataset compared to other benchmarks?
VisualPuzzles minimizes the dependence on encyclopedic knowledge and focuses on pure logic, making it unique for testing raw reasoning.
Is it possible to fine-tune a model with this dataset?
No, the dataset is optimized for evaluation and not for supervised learning. It is not recommended for fine tuning.
Does the dataset require images to work?
Yes, each puzzle is multimodal and involves a visual component that is essential for understanding.
