TIGER-AI-Lab/MMLU-Pro
The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]
- Stars
- 431
- Forks
- 59
- Open issues
- 12
- Last push
- 7mo ago
About MMLU-Pro
MMLU-Pro is a free, open-source AI developer tool by TIGER-AI-Lab, written mainly in Python. The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]. It has 431 GitHub stars and 59 forks, and was last updated 7mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Python
- Stars
- 431
- Last update
- 7mo ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
deepeval19k stars · The LLM Evaluation Framework
helicone6.2k stars · 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
RagaAI-Catalyst16k stars · Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal
giskard-oss5.9k stars · 🐢 Open-Source Evaluation & Testing library for LLM Agents
lm-evaluation-harness14k stars · A framework for few-shot evaluation of language models.
lmms-eval4.5k stars · One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
MMLU-Pro FAQ
Is MMLU-Pro free?
Yes. MMLU-Pro is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does MMLU-Pro run on a Mac?
Yes. According to the project, MMLU-Pro works on macOS.
Does MMLU-Pro need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is MMLU-Pro still maintained?
It was last updated 7mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to MMLU-Pro?
Similar free, open-source tools include deepeval, helicone, RagaAI-Catalyst, giskard-oss.