LeanModels/DFloat11
DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference
- Stars
- 662
- Forks
- 40
- Open issues
- 24
- Last push
- 11mo ago
About DFloat11
DFloat11 is a free, open-source AI inference tool by LeanModels, written mainly in Python. DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference. It has 662 GitHub stars and 40 forks, and was last updated 11mo ago. It's released under the Apache-2.0 license, a standard open-source license. According to the project, it needs a GPU with at least 24 GB of memory.
- License
- Apache-2.0
- Language
- Python
- Stars
- 662
- Last update
- 11mo ago
- Runs on Mac
- Not stated
- GPU
- 24 GB+ (as stated by the project)
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
PowerInfer9.8k stars · High-speed Large Language Model Serving for Local Deployment
DeepSpeed43k stars · DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
DFloat11 FAQ
Is DFloat11 free?
Yes. DFloat11 is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does DFloat11 run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does DFloat11 need a GPU?
According to the project, DFloat11 needs a GPU with at least 24 GB of memory.
Is DFloat11 still maintained?
It was last updated 11mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to DFloat11?
Similar free, open-source tools include BentoML, TensorRT-LLM, vllm, OpenLLM.