FreeAIRepo
Inference & serving

uccl-project/uccl

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Apache-2.0C++
Open on GitHub Website
Stars
1.6k
Forks
185
Open issues
65
Last push
1d ago

About uccl

uccl is a free, open-source AI inference tool by uccl-project, written mainly in C++. UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven). It has 1.6k GitHub stars and 185 forks, and was last updated 1d ago. It's released under the Apache-2.0 license, a standard open-source license.

License
Apache-2.0
Language
C++
Stars
1,550
Last update
1d ago
Runs on Mac
Not stated
GPU
Not stated
Paid API key
Not stated
Self-hostable
Not stated

Similar free tools

All Inference & serving tools →

uccl FAQ

Is uccl free?

Yes. uccl is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.

Does uccl run on a Mac?

The project doesn't say. Check its installation guide before you start.

Does uccl need a GPU?

The project doesn't state hardware needs. AI models generally run much faster with a GPU.

Is uccl still maintained?

Yes. It was last updated 1d ago and has 65 open issues on GitHub.

What are alternatives to uccl?

Similar free, open-source tools include TensorRT-LLM, ai-infra-book, vllm, kvcached.

Related topics

aiallreduceamdbroadcomcollectivecudagpuhpc