FreeAIRepo
Inference & serving

beehive-lab/jitllm

Java Inference Tornado toolkit: High performance LLM inference & serving for the JVM on GPUs

Really freeCommercial OK · MITRuns on MacJava
Open on GitHub Website
Stars
290
Forks
40
Open issues
20
Last push
1d ago

About jitllm

jitllm is a free, open-source AI inference tool by beehive-lab, written mainly in Java. Java Inference Tornado toolkit: High performance LLM inference & serving for the JVM on GPUs. It has 290 GitHub stars and 40 forks, and was last updated 1d ago. It's released under the MIT license, a standard open-source license. According to the project, it works on macOS.

License
MIT
Commercial use
Commercial OK
Language
Java
Stars
290
Last update
1d ago
Runs on Mac
Yes (as stated by the project)
GPU
Not stated
Paid API key
Not required
Self-hostable
Yes

Similar free tools

All Inference & serving tools →

jitllm FAQ

Can I use jitllm commercially?

Yes. MIT is a permissive license: commercial use is allowed as long as you keep the license notice. This is a plain-English summary, not legal advice.

Is jitllm free?

Yes. jitllm is open source and free to use. It's released under the MIT license, a standard open-source license.

Does jitllm run on a Mac?

Yes. According to the project, jitllm works on macOS.

Does jitllm need a GPU?

The project doesn't state hardware needs. AI models generally run much faster with a GPU.

Is jitllm still maintained?

Yes. It was last updated 1d ago and has 20 open issues on GitHub.

What are alternatives to jitllm?

Similar free, open-source tools include llm-action, TensorRT-LLM, mlc-llm, ai-infra-book.

Related topics

acceleratorscompilersgpujavajava21llama3nvidiatornadovm