toverainc/willow-inference-server
Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS
- Stars
- 511
- Forks
- 60
- Open issues
- 38
- Last push
- 8mo ago
About willow-inference-server
willow-inference-server is a free, open-source AI voice and audio tool by toverainc, written mainly in Python. Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS. It has 511 GitHub stars and 60 forks, and was last updated 8mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it needs a GPU with at least 6 GB of memory.
- License
- Apache-2.0
- Language
- Python
- Stars
- 511
- Last update
- 8mo ago
- Runs on Mac
- Not stated
- GPU
- 6 GB+ (as stated by the project)
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
transformers168k stars · 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
faster-whisper26k stars · Faster Whisper transcription with CTranslate2
vosk-api15k stars · Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node
Speech19k stars · A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
whisper.cpp55k stars · Port of OpenAI's Whisper model in C/C++
whisper111k stars · Robust Speech Recognition via Large-Scale Weak Supervision
willow-inference-server FAQ
Is willow-inference-server free?
Yes. willow-inference-server is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does willow-inference-server run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does willow-inference-server need a GPU?
According to the project, willow-inference-server needs a GPU with at least 6 GB of memory.
Is willow-inference-server still maintained?
It was last updated 8mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to willow-inference-server?
Similar free, open-source tools include transformers, faster-whisper, vosk-api, Speech.