thu-pacman/chitu
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
- Stars
- 3.0k
- Forks
- 249
- Open issues
- 55
- Last push
- 1d ago
About chitu
chitu is a free, open-source AI inference tool by thu-pacman, written mainly in Python. High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability. It has 3.0k GitHub stars and 249 forks, and was last updated 1d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 2,990
- Last update
- 1d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
pytorch105k stars · Tensors and Dynamic neural networks in Python with strong GPU acceleration
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
chitu FAQ
Is chitu free?
Yes. chitu is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does chitu run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does chitu need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is chitu still maintained?
Yes. It was last updated 1d ago and has 55 open issues on GitHub.
What are alternatives to chitu?
Similar free, open-source tools include vllm, TensorRT-LLM, llm-action, pytorch.