huggingface/evaluation-guidebook
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
- Stars
- 2.2k
- Forks
- 126
- Open issues
- 4
- Last push
- 10mo ago
About evaluation-guidebook
evaluation-guidebook is a free, open-source LLM chat app by huggingface, written mainly in Jupyter Notebook. Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!. It has 2.2k GitHub stars and 126 forks, and was last updated 10mo ago. GitHub doesn't detect a standard license, so check the repository before using it commercially.
- License
- Not detected
- Language
- Jupyter Notebook
- Stars
- 2,159
- Last update
- 10mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
Hands-On-Large-Language-Models30k stars · Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
FinGPT21k stars · FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.
Qwen22k stars · The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
ml-engineering20k stars · Machine Learning Engineering Open Book
gpt_academic72k stars · 为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss
OpenRLHF10k stars · An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
evaluation-guidebook FAQ
Is evaluation-guidebook free?
Yes. evaluation-guidebook is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does evaluation-guidebook run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does evaluation-guidebook need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is evaluation-guidebook still maintained?
It was last updated 10mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to evaluation-guidebook?
Similar free, open-source tools include Hands-On-Large-Language-Models, FinGPT, Qwen, ml-engineering.