quantumaikr/quant.cpp
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.
- Stars
- 405
- Forks
- 44
- Open issues
- 11
- Last push
- 5mo ago
About quant.cpp
quant.cpp is a free, open-source LLM chat app by quantumaikr, written mainly in C. LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library. It has 405 GitHub stars and 44 forks, and was last updated 5mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- C
- Stars
- 405
- Last update
- 5mo ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
omlx23k stars · LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
llamafile26k stars · Distribute and run LLMs with a single file.
RWKV-LM15k stars · RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN a
self-llm33k stars · 《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程
Open-LLM-VTuber14k stars · Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D avatar running locally across platforms
transformer-explainer9.0k stars · Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization
quant.cpp FAQ
Is quant.cpp free?
Yes. quant.cpp is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does quant.cpp run on a Mac?
Yes. According to the project, quant.cpp works on macOS.
Does quant.cpp need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is quant.cpp still maintained?
It was last updated 5mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to quant.cpp?
Similar free, open-source tools include omlx, llamafile, RWKV-LM, self-llm.