GeeeekExplorer/nano-vllm
Nano vLLM
Really freeMITPython
- Stars
- 16k
- Forks
- 2.8k
- Open issues
- 32
- Last push
- 5mo ago
About nano-vllm
nano-vllm is a free, open-source AI inference tool by GeeeekExplorer, written mainly in Python. Nano vLLM. It has 16k GitHub stars and 2.8k forks, and was last updated 5mo ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Python
- Stars
- 16,060
- Last update
- 5mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
pytorch105k stars · Tensors and Dynamic neural networks in Python with strong GPU acceleration
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
keras64k stars · Deep Learning for humans
pytorch-lightning31k stars · Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
nano-vllm FAQ
Is nano-vllm free?
Yes. nano-vllm is open source and free to use. It's released under the MIT license, a standard open-source license.
Does nano-vllm run on a Mac?
Its README doesn't say. Check the installation guide on GitHub before you start.
Does nano-vllm need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is nano-vllm still maintained?
It was last updated 5mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to nano-vllm?
Similar free, open-source tools include vllm, pytorch, DeepSpeed, BentoML.