abetlen/llama-cpp-python
Python bindings for llama.cpp
- Stars
- 11k
- Forks
- 1.5k
- Open issues
- 597
- Last push
- 21d ago
About llama-cpp-python
llama-cpp-python is a free, open-source AI inference tool by abetlen, written mainly in Python. Python bindings for llama.cpp. It has 11k GitHub stars and 1.5k forks, and was last updated 21d ago. It's released under the MIT license, a standard open-source license. According to its README, it works on macOS.
- License
- MIT
- Language
- Python
- Stars
- 10,653
- Last update
- 21d ago
- Runs on Mac
- Yes (per README)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
llama.cpp132k stars · LLM inference in C/C++
node-llama-cpp2.2k stars · Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
llama-swap6.0k stars · Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
llama.rn1.1k stars · React Native binding of llama.cpp
scalene14k stars · Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals
llama-cpp-python FAQ
Is llama-cpp-python free?
Yes. llama-cpp-python is open source and free to use. It's released under the MIT license, a standard open-source license.
Does llama-cpp-python run on a Mac?
Yes. According to its README, llama-cpp-python works on macOS.
Does llama-cpp-python need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is llama-cpp-python still maintained?
Yes. It was last updated 21d ago and has 597 open issues on GitHub.
What are alternatives to llama-cpp-python?
Similar free, open-source tools include llama-cookbook, llama.cpp, node-llama-cpp, llama-swap.