ManuelSLemos/RabbitLLM
Run 70B+ LLMs on a single 4GB GPU — no quantization required.
- Stars
- 87
- Forks
- 18
- Open issues
- 3
- Last push
- 8mo ago
About RabbitLLM
RabbitLLM is a free, open-source LLM chat app by ManuelSLemos, written mainly in Python. Run 70B+ LLMs on a single 4GB GPU — no quantization required. It has 87 GitHub stars and 18 forks, and was last updated 8mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it needs a GPU with at least 4 GB of memory and works on macOS.
- License
- Apache-2.0
- Commercial use
- Commercial OK
- Language
- Python
- Stars
- 87
- Last update
- 8mo ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- 4 GB+ (as stated by the project)
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
self-llm33k stars · 《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程
web-llm19k stars · Runs language models in web browsers for AI chatbots and assistants.
llm_interview_note15k stars · 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题
chinese-llm-benchmark6.5k stars · 非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及step3.5-flash、kimi-k2.
ollama185k stars · Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
RabbitLLM FAQ
Can I use RabbitLLM commercially?
Yes. Apache-2.0 is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is RabbitLLM free?
Yes. RabbitLLM is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does RabbitLLM run on a Mac?
Yes. According to the project, RabbitLLM works on macOS.
Does RabbitLLM need a GPU?
According to the project, RabbitLLM needs a GPU with at least 4 GB of memory.
Is RabbitLLM still maintained?
It was last updated 8mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to RabbitLLM?
Similar free, open-source tools include self-llm, web-llm, llm_interview_note, diy-llm.