The database of free AI tools on GitHub
22,947 open-source AI repos. Find a free alternative to any paid AI tool, or see what developers are talking about.
127 repos · showing 121–127
Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration
Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
Self-hosted, OpenAI-compatible inference for the agentic era: reasoning LLMs, universal tool calling, and the Responses API alongside embeddings, speech, and image models — many models sharing your GP
Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.
edge-dit.cpp — a native C/C++ inference engine for Diffusion Transformers (DiT), designed for local and resource-constrained devices with automatic VRAM-aware precision, placement, and offloading.
Streamline AI Workloads on Kubernetes: Universal Python SDK Guide 2026
A web-based calculator for estimating GPU memory requirements and maximum concurrent requests for self-hosted LLM inference.