ggml-org/llama.cpp
LLM inference in C/C++
Really freeMITC++
- Stars
- 132k
- Forks
- 25k
- Open issues
- 791
- Last push
- today
About llama.cpp
llama.cpp is a free, open-source AI inference tool by ggml-org, written mainly in C++. LLM inference in C/C++. It has 132k GitHub stars and 25k forks, and was last updated today. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- C++
- Stars
- 131,987
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
OpenLLM13k stars · Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
distributed-llama3.1k stars · Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
llama.cpp FAQ
Is llama.cpp free?
Yes. llama.cpp is open source and free to use. It's released under the MIT license, a standard open-source license.
Does llama.cpp run on a Mac?
Its README doesn't say. Check the installation guide on GitHub before you start.
Does llama.cpp need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is llama.cpp still maintained?
Yes. It was last updated today and has 791 open issues on GitHub.
What are alternatives to llama.cpp?
Similar free, open-source tools include llama-cookbook, OpenLLM, vllm, TensorRT-LLM.