Mattbusel/tokio-prompt-orchestrator
Tokio pipeline for serving LLM requests: bounded five-stage flow with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM. REST, SSE,
- Stars
- 60
- Forks
- 4
- Open issues
- 6
- Last push
- today
tokio-prompt-orchestrator star history
+0 this week
Can I sell what I build with it?
Yes
A guide, not legal advice.
Support tokio-prompt-orchestrator
tokio-prompt-orchestrator is free because people keep building it. If it saves you money, send a little back.
About tokio-prompt-orchestrator
tokio-prompt-orchestrator is a free, open-source AI agent tool by Mattbusel, written mainly in Rust. Tokio pipeline for serving LLM requests: bounded five-stage flow with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM. REST, SSE,. It has 60 GitHub stars and 4 forks, and was last updated today. It's released under the MIT license, a standard open-source license. According to the project, it works on macOS.
- License
- MIT
- Commercial use
- Commercial OK
- Language
- Rust
- Stars
- 60
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
ruflo80k stars · Build and manage multi-agent systems for AI workflows.
AutoGPT188k stars · AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
openai-agents-python30k stars · Build multi-agent workflows with OpenAI Agents SDK for various tasks.
mem068k stars · The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.
planning-with-files28k stars · Persistent file-based planning for AI coding agents and long-running tasks. Crash-proof markdown plans, session recovery after /clear and compaction, per-turn re-injection against context rot, determi
fastmcp28k stars · 🚀 The fast, Pythonic way to build MCP servers and clients.
tokio-prompt-orchestrator FAQ
Can I use tokio-prompt-orchestrator commercially?
Yes. MIT is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is tokio-prompt-orchestrator free?
Yes. tokio-prompt-orchestrator is open source and free to use. It's released under the MIT license, a standard open-source license.
Does tokio-prompt-orchestrator run on a Mac?
Yes. According to the project, tokio-prompt-orchestrator works on macOS.
Does tokio-prompt-orchestrator need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is tokio-prompt-orchestrator still maintained?
Yes. It was last updated today and has 6 open issues on GitHub.
What are alternatives to tokio-prompt-orchestrator?
Similar free, open-source tools include ruflo, AutoGPT, openai-agents-python, mem0.
Maintain tokio-prompt-orchestrator?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/Mattbusel/tokio-prompt-orchestrator)