RustingStudio/RustingStrata
Pure-Rust MoE inference engine for Qwen3.5/3.6/3.8 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp prefill and decode on a 12 GB RTX 3060.
Rising: +33 stars in the last 2 days, created 28d ago
- 130stars
- 16forks
- 7open issues
- updated today
About RustingStrata
RustingStrata is a free, open-source AI inference tool by RustingStudio, written mainly in Rust. Pure-Rust MoE inference engine for Qwen3.5/3.6/3.8 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.cpp prefill and decode on a 12 GB RTX 3060. It has 130 GitHub stars and 16 forks, and was last updated today. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Similar free tools
llama.cpp134k stars · LLM inference in C/C++ with minimal setup and state-of-the-art performance.
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
vllm96k stars · High-throughput inference engine for LLMs.
OpenLLM13k stars · Self-host any open-source LLM as an OpenAI-compatible API server.
shimmy6.0k stars · Rust-based OpenAI-compatible inference server for GGUF models.
ai-infra-book7.4k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
RustingStrata FAQ
Can I use RustingStrata commercially?
Unclear. No license was detected, which legally means all rights are reserved. Ask the author before using it commercially. This is a plain-English summary, not legal advice.
Is RustingStrata free?
Yes. RustingStrata is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does RustingStrata run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does RustingStrata need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is RustingStrata still maintained?
Yes. It was last updated today and has 7 open issues on GitHub.
What are alternatives to RustingStrata?
Similar free, open-source tools include llama.cpp, llm-action, vllm, OpenLLM.
