datawhalechina/zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
- Stars
- 1.6k
- Forks
- 119
- Open issues
- 6
- Last push
- today
About zero-to-sglang
zero-to-sglang is a free, open-source AI inference tool by datawhalechina, written mainly in Python. Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese. It has 1.6k GitHub stars and 119 forks, and was last updated today. GitHub doesn't detect a standard license, so check the repository before using it commercially.
- License
- Not detected
- Language
- Python
- Stars
- 1,558
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
ai-infra-book6.9k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
sglang38k stars · SGLang is a high-performance serving framework for large language models and multimodal models.
zero-to-sglang FAQ
Is zero-to-sglang free?
Yes. zero-to-sglang is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does zero-to-sglang run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does zero-to-sglang need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is zero-to-sglang still maintained?
Yes. It was last updated today and has 6 open issues on GitHub.
What are alternatives to zero-to-sglang?
Similar free, open-source tools include vllm, ai-infra-book, TensorRT-LLM, BentoML.