helasaoudi/llm-inspector
The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.
- Stars
- 76
- Forks
- 6
- Open issues
- 0
- Last push
- 28d ago
llm-inspector star history
+0 this week
About llm-inspector
llm-inspector is a free, open-source AI inference tool by helasaoudi, written mainly in Python. The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings. It has 76 GitHub stars and 6 forks, and was last updated 28d ago. It's released under the MIT license, a standard open-source license. According to the project, it works on macOS.
- License
- MIT
- Commercial use
- Commercial OK
- Language
- Python
- Stars
- 76
- Last update
- 28d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
vllm95k stars · High-throughput inference engine for LLMs.
scalene14k stars · Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
nano-vllm16k stars · A fast, lightweight inference engine for language models.
TensorRT-LLM15k stars · Optimizes inference for large language models on NVIDIA GPUs.
BitNet41k stars · Optimized inference framework for 1-bit LLMs on CPU and GPU.
llm-inspector FAQ
Can I use llm-inspector commercially?
Yes. MIT is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is llm-inspector free?
Yes. llm-inspector is open source and free to use. It's released under the MIT license, a standard open-source license.
Does llm-inspector run on a Mac?
Yes. According to the project, llm-inspector works on macOS.
Does llm-inspector need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is llm-inspector still maintained?
Yes. It was last updated 28d ago and has 0 open issues on GitHub.
What are alternatives to llm-inspector?
Similar free, open-source tools include vllm, scalene, llm-action, nano-vllm.
Maintain llm-inspector?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/helasaoudi/llm-inspector)