microsoft/vidur
Accurate, large-scale, and extensible simulator for LLM inference Systems
- Stars
- 706
- Forks
- 139
- Open issues
- 18
- Last push
- 1mo ago
About vidur
vidur is a free, open-source AI inference tool by microsoft, written mainly in Python. Accurate, large-scale, and extensible simulator for LLM inference Systems. It has 706 GitHub stars and 139 forks, and was last updated 1mo ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Python
- Stars
- 706
- Last update
- 1mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
nano-vllm16k stars · Nano vLLM
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
distributed-llama3.1k stars · Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
vidur FAQ
Is vidur free?
Yes. vidur is open source and free to use. It's released under the MIT license, a standard open-source license.
Does vidur run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does vidur need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is vidur still maintained?
It was last updated 1mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to vidur?
Similar free, open-source tools include vllm, TensorRT-LLM, nano-vllm, BentoML.