The database of free AI tools on GitHub

High-throughput inference engine for LLMs.

Distributed AI compute engine for ML and RL workloads.

Optimized inference framework for 1-bit LLMs on CPU and GPU.

Automatically recommends LLM models based on hardware capabilities.

Serves large language and multimodal models for inference and RL.

Local NVR with AI object detection for IP cameras.

Run LLMs and speech-to-text models with a single-file executable.

Compile and deploy large language models on various platforms.

A fast, low-memory vector index for Python and Rust.

A high-performance inference engine for deploying LLMs and Edge AI models.

A fast, in-process vector database for similarity search.

Builds knowledge-based local AI applications with small, specialized models.

Builds context and knowledge graphs for AI systems with full decision provenance.

AI observability and evaluation platform for LLMs.

Optimize and deploy deep learning models for various platforms.

Provides Python bindings for llama.cpp inference engine.

Serve and deploy AI models with a single command.

Runs a 2.78-trillion-parameter model on a single CPU with 8 GB of RAM.

Build and serve AI models as APIs and pipelines.

Automates Kubernetes cluster management with AI diagnostics.

AI-powered real-time observability system for metrics and logs.

AI search and vector database platform for large-scale data.

Enforces policies to ensure reliable AI agent operations.

Edge AI inference engine for mobile devices and wearables.

Manage and switch between multiple local AI models easily.

Run local AI apps and models on your GPU and NPU.

Converts models between various AI frameworks for deployment.

A C++ AI toolkit for object detection, segmentation, and face fusion.

A lightweight Python framework for LLM inference and serving.

Manage AI gateways, monitor requests, and track costs locally.