intel/neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- Stars
- 2.7k
- Forks
- 330
- Open issues
- 3
- Last push
- 1d ago
About neural-compressor
neural-compressor is a free, open-source machine learning tool by intel, written mainly in Python. SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime. It has 2.7k GitHub stars and 330 forks, and was last updated 1d ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Python
- Stars
- 2,723
- Last update
- 1d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
LlamaFactory76k stars · Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
pytorch-image-models37k stars · The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT)
align-anything4.7k stars · Align Anything: Training All-modality Model with Feedback
DoRA1.0k stars · [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation
bitsandbytes8.5k stars · Accessible large language models via k-bit quantization for PyTorch.
halo834 stars · Halo is an open-source framework built by White Circle for training large language and multimodal models
neural-compressor FAQ
Is neural-compressor free?
Yes. neural-compressor is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does neural-compressor run on a Mac?
Yes. According to the project, neural-compressor works on macOS.
Does neural-compressor need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is neural-compressor still maintained?
Yes. It was last updated 1d ago and has 3 open issues on GitHub.
What are alternatives to neural-compressor?
Similar free, open-source tools include LlamaFactory, pytorch-image-models, align-anything, DoRA.