FlashML-org/FreeToken
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- Stars
- 15k
- Forks
- 1.5k
- Open issues
- 280
- Last push
- today
About FreeToken
FreeToken is a free, open-source AI inference tool by FlashML-org, written mainly in Python. FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. It has 15k GitHub stars and 1.5k forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 14,707
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
sglang38k stars · SGLang is a high-performance serving framework for large language models and multimodal models.
inference9.6k stars · Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready
kimi-k3-in-c9.2k stars · A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
gpustack5.9k stars · A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
FreeToken FAQ
Is FreeToken free?
Yes. FreeToken is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does FreeToken run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does FreeToken need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is FreeToken still maintained?
Yes. It was last updated today and has 280 open issues on GitHub.
What are alternatives to FreeToken?
Similar free, open-source tools include vllm, sglang, inference, kimi-k3-in-c.