raketenkater/ggrun
llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.
- Stars
- 282
- Forks
- 19
- Open issues
- 1
- Last push
- 1d ago
About ggrun
ggrun is a free, open-source AI inference tool by raketenkater, written mainly in Go. llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math. It has 282 GitHub stars and 19 forks, and was last updated 1d ago. It's released under the MIT license, a standard open-source license. According to the project, it works on macOS.
- License
- MIT
- Language
- Go
- Stars
- 282
- Last update
- 1d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
llama.cpp132k stars · LLM inference in C/C++
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
shimmy6.0k stars · ⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
ggrun FAQ
Is ggrun free?
Yes. ggrun is open source and free to use. It's released under the MIT license, a standard open-source license.
Does ggrun run on a Mac?
Yes. According to the project, ggrun works on macOS.
Does ggrun need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is ggrun still maintained?
Yes. It was last updated 1d ago and has 1 open issues on GitHub.
What are alternatives to ggrun?
Similar free, open-source tools include llama.cpp, vllm, TensorRT-LLM, shimmy.