algorithmicsuperintelligence/optillm
Optimizing inference proxy for LLMs
- Stars
- 4.3k
- Forks
- 393
- Open issues
- 16
- Last push
- 11d ago
About optillm
optillm is a free, open-source AI developer tool by algorithmicsuperintelligence, written mainly in Python. Optimizing inference proxy for LLMs. It has 4.3k GitHub stars and 393 forks, and was last updated 11d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 4,328
- Last update
- 11d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Required
- Self-hostable
- Yes
Similar free tools
litellm62k stars · The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropi
langchain148k stars · The agent engineering platform.
prompt-optimizer39k stars · An AI prompt optimizer for writing better prompts and getting better AI results.
RagaAI-Catalyst16k stars · Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal
dsh-our-free-model35k stars · 在 dsh 里装上这个插件即可，无需登录、注册或填 API Key，就能使用包括 DeepSeek V4.1 Flash、Kimi K3 在内的前沿模型——完全免费，不限量。 All you do is install this plugin in dsh: no login, no sign-up, no API key — the frontier models are just there,
new-api50k stars · A unified AI model hub for aggregation & distribution. It supports cross-converting various LLMs into OpenAI-compatible, Claude-compatible, or Gemini-compatible formats. A centralized gateway for pers
optillm FAQ
Is optillm free?
optillm is open source, but it needs a paid third-party API key to work. It's released under the Apache-2.0 license, a standard open-source license.
Does optillm run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does optillm need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is optillm still maintained?
Yes. It was last updated 11d ago and has 16 open issues on GitHub.
What are alternatives to optillm?
Similar free, open-source tools include litellm, langchain, prompt-optimizer, RagaAI-Catalyst.