vllm-project/semantic-router
An open, programmable decision layer for models and compute.
- Stars
- 6.2k
- Forks
- 1.1k
- Open issues
- 486
- Last push
- today
About semantic-router
semantic-router is a free, open-source AI inference tool by vllm-project, written mainly in Go. An open, programmable decision layer for models and compute. It has 6.2k GitHub stars and 1.1k forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Go
- Stars
- 6,248
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
kong44k stars · 🦍 The API and AI Gateway
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
ColossalAI41k stars · Making large AI models cheaper, faster and more accessible
semantic-router FAQ
Is semantic-router free?
Yes. semantic-router is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does semantic-router run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does semantic-router need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is semantic-router still maintained?
Yes. It was last updated today and has 486 open issues on GitHub.
What are alternatives to semantic-router?
Similar free, open-source tools include vllm, kong, BentoML, llm-action.