brontoguana/krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
- Stars
- 522
- Forks
- 33
- Open issues
- 11
- Last push
- today
About krasis
krasis is a free, open-source AI inference tool by brontoguana, written mainly in C++. Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware. It has 522 GitHub stars and 33 forks, and was last updated today. GitHub doesn't detect a standard license, so check the repository before using it commercially.
- License
- Not detected
- Language
- C++
- Stars
- 522
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
llama.cpp132k stars · LLM inference in C/C++
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
sglang38k stars · SGLang is a high-performance serving framework for large language models and multimodal models.
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
scalene14k stars · Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals
krasis FAQ
Is krasis free?
Yes. krasis is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does krasis run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does krasis need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is krasis still maintained?
Yes. It was last updated today and has 11 open issues on GitHub.
What are alternatives to krasis?
Similar free, open-source tools include vllm, llama.cpp, TensorRT-LLM, sglang.