MMMU-Benchmark/MMMU
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
- Stars
- 598
- Forks
- 56
- Open issues
- 47
- Last push
- 2mo ago
About MMMU
MMMU is a free, open-source AI developer tool by MMMU-Benchmark, written mainly in Python. This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI". It has 598 GitHub stars and 56 forks, and was last updated 2mo ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 598
- Last update
- 2mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
tensorflow201k stars · An Open Source Machine Learning Framework for Everyone
spaCy34k stars · 💫 Industrial-strength Natural Language Processing (NLP) in Python
deepeval19k stars · The LLM Evaluation Framework
mediapipe37k stars · Cross-platform, customizable ML solutions for live and streaming media.
lmms-eval4.5k stars · One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
lm-evaluation-harness14k stars · A framework for few-shot evaluation of language models.
MMMU FAQ
Is MMMU free?
Yes. MMMU is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does MMMU run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does MMMU need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is MMMU still maintained?
It was last updated 2mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to MMMU?
Similar free, open-source tools include tensorflow, spaCy, deepeval, mediapipe.