foldl/chatllm.cpp
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
- Stars
- 934
- Forks
- 76
- Open issues
- 11
- Last push
- 1d ago
About chatllm.cpp
chatllm.cpp is a free, open-source AI inference tool by foldl, written mainly in C++. Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU). It has 934 GitHub stars and 76 forks, and was last updated 1d ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- C++
- Stars
- 934
- Last update
- 1d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
llama.cpp132k stars · LLM inference in C/C++
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
distributed-llama3.1k stars · Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
chatllm.cpp FAQ
Is chatllm.cpp free?
Yes. chatllm.cpp is open source and free to use. It's released under the MIT license, a standard open-source license.
Does chatllm.cpp run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does chatllm.cpp need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is chatllm.cpp still maintained?
Yes. It was last updated 1d ago and has 11 open issues on GitHub.
What are alternatives to chatllm.cpp?
Similar free, open-source tools include llama.cpp, TensorRT-LLM, BentoML, OpenLLM.