CV-Bench
CV-bench is a multimodal benchmark designed to test the 2D and 3D visual understanding of models, with accurate annotations from several standard datasets (ADe20k, COCO, OMNI3D). It includes natural language questions to assess spatial perception and in-depth understanding of scenes.
5,276 examples with 2D/3D text annotations, associated images, 810 MB, Parquet and JSONL format
Apache 2.0
Description
CV-Bench offers a set of annotated examples to evaluate multimodal models on classical 2D and 3D vision tasks. Annotations include natural language questions about spatial relationships, object counting, order of depth, and relative distance.
What is this dataset for?
- Evaluate the 2D and 3D capabilities of multimodal models
- Test the understanding of spatial relationships and depth
- Benchmark for vision and multimodality research
Can it be enriched or improved?
The dataset can be enriched by adding new examples, more detailed annotations, or expanding to other types of questions. The addition of dynamic or video data would also be relevant.
🔎 In summary
🧠 Recommended for
- Computer vision researchers
- Multimodal developers
- AI model evaluators
🔧 Compatible tools
- Hugging Face Datasets
- PyTorch
- TensorFlow
- Multimodal assessment tools
💡 Tip
Use 2D or 3D splits depending on the specialization of your model for targeted evaluations.
Frequently Asked Questions
What type of tasks does this dataset evaluate?
It assesses the understanding of spatial relationships, counting objects in 2D, and the perception of depth and distance in 3D.
How many examples does CV-Bench contain?
Approximately 5,276 annotated examples with images and questions in 2D and 3D.
Is this dataset suitable for fine-tuning multimodal models?
Yes, it can be used to fine-tune and evaluate multimodal models in vision and language.
