The database of free AI tools on GitHub

一款简单易用和高性能的AI部署框架 | An Easy-to-Use and High-Performance AI Deployment Framework

Large-scale LLM inference engine

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

Nvidia GPU exporter for prometheus using nvidia-smi binary OR using NVML

Chat with your favourite LLaMA models in a native macOS app

LLM Inference Engine for Metal and CUDA.

Real-time monitoring, slowlog analysis, and audit trails for Valkey and Redis

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC

React Native binding of llama.cpp

Run Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime

A scalable inference server for models optimized with OpenVINO™

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster t

Anything — 파일 이름이 아니라 본문으로 찾는 문서 검색기. HWP·PDF·Office 전문 검색을 오프라인으로. Windows·macOS | Offline content search across Korean documents (HWP, PDF, Office). Windows and macOS

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behin

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)

Accelerate your Stable Diffusion inference with the library's universal C/C++ framework design, powered by ONNXRuntime & across platforms.

A Rust library integrated with ONNXRuntime, providing a collection of Computer Vison and Vision-Language models such as YOLO, FastVLM, and more.

Open-source local AI workspace — advancing on-device inference.

search every screenshot you've ever taken by the text inside it. native linux, offline, PaddleOCR + sqlite + gpui

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

ClearML Agent - MLOps/LLMOps made easy. MLOps/LLMOps scheduler & orchestration solution

lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.

Java Inference Tornado toolkit: High performance LLM inference & serving for the JVM on GPUs