kubeai-project/kubeai
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
- Stars
- 1.3k
- Forks
- 141
- Open issues
- 80
- Last push
- today
About kubeai
kubeai is a free, open-source AI inference tool by kubeai-project, written mainly in Go. AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text. It has 1.3k GitHub stars and 141 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Go
- Stars
- 1,285
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
kong44k stars · 🦍 The API and AI Gateway
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
tyk11k stars · Open Source API and AI Gateway supporting REST, GraphQL, TCP, gRPC and MCP (Model Context Protocol)
kubeai FAQ
Is kubeai free?
Yes. kubeai is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does kubeai run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does kubeai need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is kubeai still maintained?
Yes. It was last updated today and has 80 open issues on GitHub.
What are alternatives to kubeai?
Similar free, open-source tools include kong, vllm, BentoML, TensorRT-LLM.