smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, ch
- Stars
- 661
- Forks
- 199
- Open issues
- 77
- Last push
- today
About smg
smg is a free, open-source AI inference tool by smg-project, written mainly in Rust. Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, ch. It has 661 GitHub stars and 199 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Rust
- Stars
- 661
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
kong44k stars · 🦍 The API and AI Gateway
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
plano7.1k stars · Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
tyk11k stars · Open Source API and AI Gateway supporting REST, GraphQL, TCP, gRPC and MCP (Model Context Protocol)
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
smg FAQ
Is smg free?
Yes. smg is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does smg run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does smg need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is smg still maintained?
Yes. It was last updated today and has 77 open issues on GitHub.
What are alternatives to smg?
Similar free, open-source tools include vllm, kong, TensorRT-LLM, plano.