Luce-Org/lucebox
LLM speculative inference server for heterogeneous hardware & consumer GPUs
- Stars
- 2.9k
- Forks
- 285
- Open issues
- 30
- Last push
- today
About lucebox
lucebox is a free, open-source AI inference tool by Luce-Org, written mainly in C++. LLM speculative inference server for heterogeneous hardware & consumer GPUs. It has 2.9k GitHub stars and 285 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- C++
- Stars
- 2,948
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
llama.cpp132k stars · LLM inference in C/C++
node-llama-cpp2.2k stars · Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
skypilot11k stars · The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
llama-cpp-python11k stars · Python bindings for llama.cpp
ColossalAI41k stars · Making large AI models cheaper, faster and more accessible
lucebox FAQ
Is lucebox free?
Yes. lucebox is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does lucebox run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does lucebox need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is lucebox still maintained?
Yes. It was last updated today and has 30 open issues on GitHub.
What are alternatives to lucebox?
Similar free, open-source tools include llama-cookbook, llama.cpp, node-llama-cpp, skypilot.