ADT109119/llamacpp-distributed-inference
一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。
- Stars
- 95
- Forks
- 20
- Open issues
- 2
- Last push
- 2mo ago
llamacpp-distributed-inference star history
+0 this week
About llamacpp-distributed-inference
llamacpp-distributed-inference is a free, open-source AI inference tool by ADT109119, written mainly in JavaScript. 一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。. It has 95 GitHub stars and 20 forks, and was last updated 2mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Commercial use
- Commercial OK
- Language
- JavaScript
- Stars
- 95
- Last update
- 2mo ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
ai-infra-book7.0k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
TensorRT-LLM15k stars · Optimizes inference for large language models on NVIDIA GPUs.
distributed-llama3.1k stars · Accelerate LLM inference by clustering home devices.
llama.cpp132k stars · LLM inference in C/C++
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
llamacpp-distributed-inference FAQ
Can I use llamacpp-distributed-inference commercially?
Yes. Apache-2.0 is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is llamacpp-distributed-inference free?
Yes. llamacpp-distributed-inference is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does llamacpp-distributed-inference run on a Mac?
Yes. According to the project, llamacpp-distributed-inference works on macOS.
Does llamacpp-distributed-inference need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is llamacpp-distributed-inference still maintained?
It was last updated 2mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to llamacpp-distributed-inference?
Similar free, open-source tools include llm-action, ai-infra-book, TensorRT-LLM, distributed-llama.
Maintain llamacpp-distributed-inference?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/ADT109119/llamacpp-distributed-inference)