allenai/olmo-eval
Overview This project provides a unified workbench for evaluating language models throughout the model development loop.
- Stars
- 72
- Forks
- 25
- Open issues
- 43
- Last push
- today
olmo-eval star history
-1 this week
About olmo-eval
olmo-eval is a free, open-source AI developer tool by allenai, written mainly in Python. Overview This project provides a unified workbench for evaluating language models throughout the model development loop. It has 72 GitHub stars and 25 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Commercial use
- Commercial OK
- Language
- Python
- Stars
- 72
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
evals20k stars · Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
DreamLayer-Eval412 stars · DreamLayer Eval: open-source benchmarking for image and video diffusion models. Automate prompts, seeds, and metrics for reproducible results. A research project from DreamLayer AI (dreamlayer.io).
continuous-eval519 stars · Data-Driven Evaluation for LLM-Powered Applications
SkillEvaluator598 stars · Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect a
promptfoo26k stars · Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line
giskard-oss5.9k stars · 🐢 Open-Source Evaluation & Testing library for LLM Agents
olmo-eval FAQ
Can I use olmo-eval commercially?
Yes. Apache-2.0 is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is olmo-eval free?
Yes. olmo-eval is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does olmo-eval run on a Mac?
Yes. According to the project, olmo-eval works on macOS.
Does olmo-eval need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is olmo-eval still maintained?
Yes. It was last updated today and has 43 open issues on GitHub.
What are alternatives to olmo-eval?
Similar free, open-source tools include evals, DreamLayer-Eval, continuous-eval, SkillEvaluator.
Maintain olmo-eval?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/allenai/olmo-eval)