facebookincubator/AITemplate
AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
- Stars
- 4.7k
- Forks
- 387
- Open issues
- 110
- Last push
- 15d ago
About AITemplate
AITemplate is a free, open-source AI inference tool by facebookincubator, written mainly in Python. AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference. It has 4.7k GitHub stars and 387 forks, and was last updated 15d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 4,730
- Last update
- 15d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
kimi-k3-in-c9.2k stars · A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
krasis522 stars · Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
GenerativeAIExamples4.2k stars · Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
kvcached1.9k stars · Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
text-embeddings-inference5.1k stars · A blazing fast inference solution for text embeddings models
AITemplate FAQ
Is AITemplate free?
Yes. AITemplate is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does AITemplate run on a Mac?
Its README doesn't say. Check the installation guide on GitHub before you start.
Does AITemplate need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is AITemplate still maintained?
Yes. It was last updated 15d ago and has 110 open issues on GitHub.
What are alternatives to AITemplate?
Similar free, open-source tools include kimi-k3-in-c, BentoML, krasis, GenerativeAIExamples.