PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
- Stars
- 3.7k
- Forks
- 757
- Open issues
- 246
- Last push
- 28d ago
About FastDeploy
FastDeploy is a free, open-source AI inference tool by PaddlePaddle, written mainly in Python. High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle. It has 3.7k GitHub stars and 757 forks, and was last updated 28d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 3,722
- Last update
- 28d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
nano-vllm16k stars · Nano vLLM
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
FastDeploy FAQ
Is FastDeploy free?
Yes. FastDeploy is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does FastDeploy run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does FastDeploy need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is FastDeploy still maintained?
Yes. It was last updated 28d ago and has 246 open issues on GitHub.
What are alternatives to FastDeploy?
Similar free, open-source tools include vllm, TensorRT-LLM, BentoML, OpenLLM.