containers/ramalama
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of con
- Stars
- 3.1k
- Forks
- 382
- Open issues
- 72
- Last push
- 1d ago
About ramalama
ramalama is a free, open-source AI inference tool by containers, written mainly in Python. RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of con. It has 3.1k GitHub stars and 382 forks, and was last updated 1d ago. It's released under the MIT license, a standard open-source license. According to the project, it works on macOS.
- License
- MIT
- Language
- Python
- Stars
- 3,088
- Last update
- 1d ago
- Runs on Mac
- Yes (as stated by the project)
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
vllm95k stars · A high-throughput and memory-efficient inference and serving engine for LLMs
BentoML8.9k stars · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
TensorRT-LLM15k stars · TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT
ai-infra-book6.9k stars · 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
plano7.1k stars · Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
ramalama FAQ
Is ramalama free?
Yes. ramalama is open source and free to use. It's released under the MIT license, a standard open-source license.
Does ramalama run on a Mac?
Yes. According to the project, ramalama works on macOS.
Does ramalama need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is ramalama still maintained?
Yes. It was last updated 1d ago and has 72 open issues on GitHub.
What are alternatives to ramalama?
Similar free, open-source tools include vllm, BentoML, TensorRT-LLM, ai-infra-book.