vllm-project/vllm-metal
Community maintained hardware plugin for vLLM on Apple Silicon
- Stars
- 1.9k
- Forks
- 317
- Open issues
- 16
- Last push
- today
About vllm-metal
vllm-metal is a free, open-source AI inference tool by vllm-project, written mainly in Python. Community maintained hardware plugin for vLLM on Apple Silicon. It has 1.9k GitHub stars and 317 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Python
- Stars
- 1,898
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
turbo-fieldfare7.0k stars · Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
nano-vllm16k stars · Nano vLLM
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
ai-infra-book6.9k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
vllm-metal FAQ
Is vllm-metal free?
Yes. vllm-metal is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does vllm-metal run on a Mac?
Yes. According to the project, vllm-metal works on macOS.
Does vllm-metal need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is vllm-metal still maintained?
Yes. It was last updated today and has 16 open issues on GitHub.
What are alternatives to vllm-metal?
Similar free, open-source tools include vllm, turbo-fieldfare, TensorRT-LLM, nano-vllm.