flashrt-project/FlashRT
FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Al
- Stars
- 752
- Forks
- 97
- Open issues
- 14
- Last push
- today
About FlashRT
FlashRT is a free, open-source AI inference tool by flashrt-project, written mainly in C++. FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Al. It has 752 GitHub stars and 97 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- C++
- Stars
- 752
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
gpustack5.9k stars · A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
HyperQwen2.2k stars · Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 1
sglang38k stars · SGLang is a high-performance serving framework for large language models and multimodal models.
lucebox2.9k stars · LLM speculative inference server for heterogeneous hardware & consumer GPUs
FlashRT FAQ
Is FlashRT free?
Yes. FlashRT is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does FlashRT run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does FlashRT need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is FlashRT still maintained?
Yes. It was last updated today and has 14 open issues on GitHub.
What are alternatives to FlashRT?
Similar free, open-source tools include vllm, gpustack, BentoML, HyperQwen.