jy-yuan/KIVI
[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Stars
- 436
- Forks
- 58
- Open issues
- 6
- Last push
- 11mo ago
About KIVI
KIVI is a free, open-source machine learning tool by jy-yuan, written mainly in Python. [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. It has 436 GitHub stars and 58 forks, and was last updated 11mo ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Python
- Stars
- 436
- Last update
- 11mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
pytorch-image-models37k stars · The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT)
ludwig12k stars · Low-code framework for building custom LLMs, neural networks, and other AI models
bitsandbytes8.5k stars · Accessible large language models via k-bit quantization for PyTorch.
align-anything4.7k stars · Align Anything: Training All-modality Model with Feedback
neural-compressor2.7k stars · SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
happy-transformer545 stars · Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.
KIVI FAQ
Is KIVI free?
Yes. KIVI is open source and free to use. It's released under the MIT license, a standard open-source license.
Does KIVI run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does KIVI need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is KIVI still maintained?
It was last updated 11mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to KIVI?
Similar free, open-source tools include pytorch-image-models, ludwig, bitsandbytes, align-anything.