runpod-workers/worker-vllm
The Runpod worker template for serving our large language model endpoints. Powered by vLLM.
- Stars
- 473
- Forks
- 394
- Open issues
- 21
- Last push
- today
About worker-vllm
worker-vllm is a free, open-source LLM chat app by runpod-workers, written mainly in Python. The Runpod worker template for serving our large language model endpoints. Powered by vLLM. It has 473 GitHub stars and 394 forks, and was last updated today. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Python
- Stars
- 473
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
Hands-On-Large-Language-Models30k stars · Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
Qwen22k stars · The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
OpenRLHF10k stars · An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
gpt_academic72k stars · 为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss
FinGPT21k stars · FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.
langextract39k stars · A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
worker-vllm FAQ
Is worker-vllm free?
Yes. worker-vllm is open source and free to use. It's released under the MIT license, a standard open-source license.
Does worker-vllm run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does worker-vllm need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is worker-vllm still maintained?
Yes. It was last updated today and has 21 open issues on GitHub.
What are alternatives to worker-vllm?
Similar free, open-source tools include Hands-On-Large-Language-Models, Qwen, OpenRLHF, gpt_academic.