bojieli/ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
- Stars
- 6.9k
- Forks
- 538
- Open issues
- 5
- Last push
- 21d ago
About ai-infra-book
ai-infra-book is a free, open-source AI inference tool by bojieli, written mainly in Python. 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验. It has 6.9k GitHub stars and 538 forks, and was last updated 21d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 6,855
- Last update
- 21d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
ray44k stars · Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
FedML4.1k stars · FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on a
ai-infra-book FAQ
Is ai-infra-book free?
Yes. ai-infra-book is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does ai-infra-book run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does ai-infra-book need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is ai-infra-book still maintained?
Yes. It was last updated 21d ago and has 5 open issues on GitHub.
What are alternatives to ai-infra-book?
Similar free, open-source tools include vllm, BentoML, TensorRT-LLM, DeepSpeed.