arozanov/turboquant-mlx
TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.
- Stars
- 113
- Forks
- 19
- Open issues
- 1
- Last push
- 6mo ago
turboquant-mlx star history
+0 this week
About turboquant-mlx
turboquant-mlx is a free, open-source machine learning tool by arozanov, written mainly in Python. TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed. It has 113 GitHub stars and 19 forks, and was last updated 6mo ago. GitHub doesn't detect a standard license, so check the repository before using it commercially. According to the project, it works on macOS.
- License
- Not detected
- Commercial use
- No license
- Language
- Python
- Stars
- 113
- Last update
- 6mo ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
Liger-Kernel6.7k stars · Optimize LLM training with efficient Triton kernels.
pmetal331 stars · PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.
jangq231 stars · JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon
Jackrong-llm-finetuning-guide1.7k stars · An educational, end-to-end open-source knowledge base for LLM fine-tuning, dataset distillation, reinforcement learning, and local deployment.
mlx-vlm5.7k stars · Inference and fine-tuning of Vision Language Models on Mac.
llm-foundry4.5k stars · Train and deploy large language models for inference.
turboquant-mlx FAQ
Can I use turboquant-mlx commercially?
Unclear. No license was detected, which legally means all rights are reserved. Ask the author before using it commercially. This is a plain-English summary, not legal advice.
Is turboquant-mlx free?
Yes. turboquant-mlx is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does turboquant-mlx run on a Mac?
Yes. According to the project, turboquant-mlx works on macOS.
Does turboquant-mlx need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is turboquant-mlx still maintained?
It was last updated 6mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to turboquant-mlx?
Similar free, open-source tools include Liger-Kernel, pmetal, jangq, Jackrong-llm-finetuning-guide.
Maintain turboquant-mlx?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/arozanov/turboquant-mlx)