modular/llm-inference-handbook
Everything you need to know about LLM inference
- Stars
- 462
- Forks
- 53
- Open issues
- 5
- Last push
- 6d ago
About llm-inference-handbook
llm-inference-handbook is a free, open-source LLM chat app by modular, written mainly in TypeScript. Everything you need to know about LLM inference. It has 462 GitHub stars and 53 forks, and was last updated 6d ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- TypeScript
- Stars
- 462
- Last update
- 6d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
self-llm33k stars · 《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程
Open-LLM-VTuber14k stars · Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D avatar running locally across platforms
open-webui156k stars · User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
omlx23k stars · LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
llm_interview_note15k stars · 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题
chinese-llm-benchmark6.5k stars · 非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及step3.5-flash、kimi-k2.
llm-inference-handbook FAQ
Is llm-inference-handbook free?
Yes. llm-inference-handbook is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does llm-inference-handbook run on a Mac?
Yes. According to the project, llm-inference-handbook works on macOS.
Does llm-inference-handbook need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is llm-inference-handbook still maintained?
Yes. It was last updated 6d ago and has 5 open issues on GitHub.
What are alternatives to llm-inference-handbook?
Similar free, open-source tools include self-llm, Open-LLM-VTuber, open-webui, omlx.