The database of free AI tools on GitHub

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Ultra-lightweight (600KB) Face Anti-Spoofing classifier. Optimized MiniFASNetV2-SE implementation validated on 70k+ samples with ~98% accuracy for edge devices.

A PyTorch native library for training speculative decoding models

Fast local TTS inference engine in C# with ONNX runtime. Multi-speaker, multi-platform and multilingual. Integrate on your .NET projects using a plug-and-play NuGet package, complete with all voices.

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, mo

The inference engine the open-source world built for itself.

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places att

Graphical management tool for Kubernetes on desktop and mobile.

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

AMD MIVisionX is a computer vision toolkit built around a highly optimized, conformant open-source implementation of the Khronos OpenVX™ 1.3.2 specification. As of the 4.0.0 release, MIVisionX ships t

⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.

DevOps cockpit in one window: terminal with local AI, Kubernetes, SSH, databases, IDE with git, alerts, notes and KeePass. Free, open source, Linux/Windows/macOS. Рабочее место DevOps-инженера в одном

OnnxTR a docTR (Document Text Recognition) library Onnx pipeline wrapper - for seamless, high-performing & accessible OCR

Snap any image, screenshot, or webpage into plaintext. No GPU. No cloud. One command.

Stable diffusion pipeline in Java using ONNX Runtime

Unbagrnd - Free, Fast & Open-source AI-powered background remover for Windows, macOS, Linux and Android.

llama.cpp tutorial on Android phone

teleskopio is a lightweight kubernetes web client

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Live truecolor terminal dashboard for a locally running LLM: tok/s, TTFT, GPUs, context, MTP acceptance, layer and real expert routing (llama.cpp)

Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

Accelerating Long Context LLM Inference with Accuracy-Preserving Context Optimization in SGLang, vLLM, llama.cpp, OpenClaw, RAG, and Agentic AI.

User friendly GUI for configuring and launching llama.cpp

lcpp is a dart implementation of llama.cpp used by the mobile artificial intelligence distribution (maid)

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 2

AI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric API for seamless local execution and cloud provider routing

A tiny neural-network framework in pure Go with AVX2 SIMD kernels (GOEXPERIMENT=simd)

Gemma 3 pure inference in C

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, one-click llama.cpp updates, several models at once, and Ember

Lightweight OCI container runtime for NixOS