waybarrios/vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
- Stars
- 1.6k
- Forks
- 231
- Open issues
- 64
- Last push
- 2d ago
About vllm-mlx
vllm-mlx is a free, open-source LLM chat app by waybarrios, written mainly in Python. High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support. It has 1.6k GitHub stars and 231 forks, and was last updated 2d ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Python
- Stars
- 1,631
- Last update
- 2d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
one-api37k stars · LLM API 管理 & 分发系统，支持 OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 适配，可用于 key 管理与二次分发。单可执行文件，提供 Docker 镜像，一键部署，开箱即用。LLM API management & key
rtk84k stars · CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies
open-webui156k stars · User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
gpt4free67k stars · The official gpt4free repository | various collection of powerful language models | opus 4.6 gpt 5.3 kimi 2.5 deepseek v3.2 gemini 3
omlx23k stars · LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
repomix29k stars · 📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like
vllm-mlx FAQ
Is vllm-mlx free?
Yes. vllm-mlx is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does vllm-mlx run on a Mac?
Yes. According to the project, vllm-mlx works on macOS.
Does vllm-mlx need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is vllm-mlx still maintained?
Yes. It was last updated 2d ago and has 64 open issues on GitHub.
What are alternatives to vllm-mlx?
Similar free, open-source tools include one-api, rtk, open-webui, gpt4free.