vitoplantamura/OnnxStream
Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-
- Stars
- 2.1k
- Forks
- 99
- Open issues
- 67
- Last push
- 4mo ago
About OnnxStream
OnnxStream is a free, open-source AI inference tool by vitoplantamura, written mainly in C++. Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-. It has 2.1k GitHub stars and 99 forks, and was last updated 4mo ago. GitHub doesn't detect a standard license, so check the repository before using it commercially.
- License
- Not detected
- Language
- C++
- Stars
- 2,088
- Last update
- 4mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
Similar free tools
llama-cookbook19k stars · Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model f
netron34k stars · Visualizer for neural network, deep learning and machine learning models
ray44k stars · Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
keras64k stars · Deep Learning for humans
Made-With-ML50k stars · Learn how to develop, deploy and iterate on production-grade ML applications.
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
OnnxStream FAQ
Is OnnxStream free?
Yes. OnnxStream is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does OnnxStream run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does OnnxStream need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is OnnxStream still maintained?
It was last updated 4mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to OnnxStream?
Similar free, open-source tools include llama-cookbook, netron, ray, keras.