messkan/prompt-cache
Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response
- Stars
- 426
- Forks
- 43
- Open issues
- 4
- Last push
- 2mo ago
About prompt-cache
prompt-cache is a free, open-source RAG and AI search tool by messkan, written mainly in Go. Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response. It has 426 GitHub stars and 43 forks, and was last updated 2mo ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- Go
- Stars
- 426
- Last update
- 2mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Required
- Self-hostable
- Yes
Similar free tools
meilisearch60k stars · A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.
milvus46k stars · Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search
onyx33k stars · Open Source AI Platform - AI Chat with advanced features that works with every LLM
ragflow93k stars · RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs
Vane38k stars · Vane is an AI-powered answering engine.
PageIndex39k stars · 📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG
prompt-cache FAQ
Is prompt-cache free?
prompt-cache is open source, but it needs a paid third-party API key to work. It's released under the MIT license, a standard open-source license.
Does prompt-cache run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does prompt-cache need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is prompt-cache still maintained?
It was last updated 2mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to prompt-cache?
Similar free, open-source tools include meilisearch, milvus, onyx, ragflow.