huggingface/optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
- Stars
- 3.5k
- Forks
- 711
- Open issues
- 261
- Last push
- 3d ago
About optimum
optimum is a free, open-source AI inference tool by huggingface, written mainly in Python. 🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools. It has 3.5k GitHub stars and 711 forks, and was last updated 3d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 3,508
- Last update
- 3d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
pytorch105k stars · Tensors and Dynamic neural networks in Python with strong GPU acceleration
ort2.6k stars · Fast ML inference & training for ONNX models in Rust
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
pytorch-lightning31k stars · Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
optimum FAQ
Is optimum free?
Yes. optimum is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does optimum run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does optimum need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is optimum still maintained?
Yes. It was last updated 3d ago and has 261 open issues on GitHub.
What are alternatives to optimum?
Similar free, open-source tools include DeepSpeed, vllm, pytorch, ort.