jitllm/jitllm
Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory
- Stars
- 58
- Forks
- 5
- Open issues
- 3
- Last push
- today
jitllm star history
+8 this week
About jitllm
jitllm is a free, open-source AI inference tool by jitllm, written mainly in Go. Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory. It has 58 GitHub stars and 5 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Commercial use
- Commercial OK
- Language
- Go
- Stars
- 58
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Not stated
Similar free tools
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
vllm95k stars · High-throughput inference engine for LLMs.
mlc-llm23k stars · Compile and deploy large language models on various platforms.
ai-infra-book7.0k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
TensorRT-LLM15k stars · Optimizes inference for large language models on NVIDIA GPUs.
LMCache12k stars · Accelerates LLM inference by managing KV cache.
jitllm FAQ
Can I use jitllm commercially?
Yes. Apache-2.0 is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.
Is jitllm free?
Yes. jitllm is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does jitllm run on a Mac?
Yes. According to the project, jitllm works on macOS.
Does jitllm need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is jitllm still maintained?
Yes. It was last updated today and has 3 open issues on GitHub.
What are alternatives to jitllm?
Similar free, open-source tools include llm-action, vllm, mlc-llm, ai-infra-book.
Maintain jitllm?
Add this badge to your README to show it's featured on FreeAIRepo.
[](https://www.freeairepo.com/repo/jitllm/jitllm)