sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
- Stars
- 38k
- Forks
- 9.8k
- Open issues
- 958
- Last push
- today
About sglang
sglang is a free, open-source AI inference tool by sgl-project, written mainly in Python. SGLang is a high-performance serving framework for large language models and multimodal models. It has 38k GitHub stars and 9.8k forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Python
- Stars
- 37,598
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
llama.cpp132k stars · LLM inference in C/C++
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Discussions
sglang FAQ
Is sglang free?
Yes. sglang is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does sglang run on a Mac?
Yes. According to the project, sglang works on macOS.
Does sglang need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is sglang still maintained?
Yes. It was last updated today and has 958 open issues on GitHub.
What are alternatives to sglang?
Similar free, open-source tools include vllm, llama-cookbook, TensorRT-LLM, llama.cpp.