sgl-project/SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
- Stars
- 1.2k
- Forks
- 371
- Open issues
- 79
- Last push
- today
About SpecForge
SpecForge is a free, open-source AI inference tool by sgl-project, written mainly in Python. Train speculative decoding models effortlessly and port them smoothly to SGLang serving. It has 1.2k GitHub stars and 371 forks, and was last updated today. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Python
- Stars
- 1,231
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
ai-infra-book6.9k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
pytorch105k stars · Tensors and Dynamic neural networks in Python with strong GPU acceleration
SpecForge FAQ
Is SpecForge free?
Yes. SpecForge is open source and free to use. It's released under the MIT license, a standard open-source license.
Does SpecForge run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does SpecForge need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is SpecForge still maintained?
Yes. It was last updated today and has 79 open issues on GitHub.
What are alternatives to SpecForge?
Similar free, open-source tools include vllm, TensorRT-LLM, OpenLLM, BentoML.