stanford-crfm/helm
Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent e
- Stars
- 3.0k
- Forks
- 426
- Open issues
- 70
- Last push
- 1mo ago
About helm
helm is a free, open-source AI developer tool by stanford-crfm, written mainly in Python. Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent e. It has 3.0k GitHub stars and 426 forks, and was last updated 1mo ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 2,953
- Last update
- 1mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
deepeval19k stars · The LLM Evaluation Framework
lm-evaluation-harness14k stars · A framework for few-shot evaluation of language models.
lmms-eval4.5k stars · One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
helicone6.2k stars · 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
dsh-our-free-model35k stars · 在 dsh 里装上这个插件即可，无需登录、注册或填 API Key，就能使用包括 DeepSeek V4.1 Flash、Kimi K3 在内的前沿模型——完全免费，不限量。 All you do is install this plugin in dsh: no login, no sign-up, no API key — the frontier models are just there,
RagaAI-Catalyst16k stars · Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal
helm FAQ
Is helm free?
Yes. helm is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does helm run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does helm need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is helm still maintained?
It was last updated 1mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to helm?
Similar free, open-source tools include deepeval, lm-evaluation-harness, lmms-eval, helicone.