ashhart/TensorFold
LLM Inference Engine for Metal and CUDA.
- Stars
- 1.5k
- Forks
- 284
- Open issues
- 64
- Last push
- today
About TensorFold
TensorFold is a free, open-source AI inference tool by ashhart, written mainly in Zig. LLM Inference Engine for Metal and CUDA. It has 1.5k GitHub stars and 284 forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it works on macOS.
- License
- Apache-2.0
- Language
- Zig
- Stars
- 1,454
- Last update
- today
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not required
- Self-hostable
- Yes
Similar free tools
llm-action25k stars · 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
ai-infra-book6.9k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
plano7.1k stars · Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
TensorFold FAQ
Is TensorFold free?
Yes. TensorFold is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does TensorFold run on a Mac?
Yes. According to the project, TensorFold works on macOS.
Does TensorFold need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is TensorFold still maintained?
Yes. It was last updated today and has 64 open issues on GitHub.
What are alternatives to TensorFold?
Similar free, open-source tools include llm-action, BentoML, ai-infra-book, TensorRT-LLM.