Nehanth/pooled
Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
- Stars
- 604
- Forks
- 92
- Open issues
- 57
- Last push
- 5d ago
About pooled
pooled is a free, open-source AI inference tool by Nehanth, written mainly in JavaScript. Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents. It has 604 GitHub stars and 92 forks, and was last updated 5d ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- JavaScript
- Stars
- 604
- Last update
- 5d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
distributed-llama3.1k stars · Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
pooled FAQ
Is pooled free?
Yes. pooled is open source and free to use. It's released under the MIT license, a standard open-source license.
Does pooled run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does pooled need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is pooled still maintained?
Yes. It was last updated 5d ago and has 57 open issues on GitHub.
What are alternatives to pooled?
Similar free, open-source tools include distributed-llama, TensorRT-LLM, BentoML, OpenLLM.