bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
- Stars
- 8.9k
- Forks
- 1.0k
- Open issues
- 151
- Last push
- 2d ago
About BentoML
BentoML is a free, open-source AI inference tool by bentoml, written mainly in Python. The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!. It has 8.9k GitHub stars and 1.0k forks, and was last updated 2d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 8,914
- Last update
- 2d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
Made-With-ML50k stars · Learn how to develop, deploy and iterate on production-grade ML applications.
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
ray44k stars · Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
netron34k stars · Visualizer for neural network, deep learning and machine learning models
ColossalAI41k stars · Making large AI models cheaper, faster and more accessible
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
BentoML FAQ
Is BentoML free?
Yes. BentoML is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does BentoML run on a Mac?
Its README doesn't say. Check the installation guide on GitHub before you start.
Does BentoML need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is BentoML still maintained?
Yes. It was last updated 2d ago and has 151 open issues on GitHub.
What are alternatives to BentoML?
Similar free, open-source tools include Made-With-ML, vllm, ray, netron.