manjunathshiva/turboquant-mlx
Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant)
- Stars
- 91
- Forks
- 23
- Open issues
- 1
- Last push
- 3d ago
turboquant-mlx star history
+0 this week
About turboquant-mlx
turboquant-mlx is a free, open-source LLM chat app by manjunathshiva, written mainly in Python. Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant). It has 91 GitHub stars and 23 forks, and was last updated 3d ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Commercial use
- Commercial OK
- Language
- Python
- Stars
- 91
- Last update
- 3d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
self-llm33k stars · 《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程
vllm-mlx1.6k stars · High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
llm_interview_note15k stars · 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题
react-native-apple-llm346 stars · React Native Apple LLM plugin using Foundation Models
chinese-llm-benchmark6.5k stars · 非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及step3.5-flash、kimi-k2.
mlx-dspark727 stars · Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, tern
turboquant-mlx FAQ
Can I use turboquant-mlx commercially?
Yes. Apache-2.0 is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is turboquant-mlx free?
Yes. turboquant-mlx is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does turboquant-mlx run on a Mac?
Yes. According to the project, turboquant-mlx works on macOS.
Does turboquant-mlx need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is turboquant-mlx still maintained?
Yes. It was last updated 3d ago and has 1 open issues on GitHub.
What are alternatives to turboquant-mlx?
Similar free, open-source tools include self-llm, vllm-mlx, llm_interview_note, react-native-apple-llm.
Maintain turboquant-mlx?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/manjunathshiva/turboquant-mlx)