facebookresearch/LayerSkip
Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
- Stars
- 380
- Forks
- 46
- Open issues
- 7
- Last push
- 3mo ago
About LayerSkip
LayerSkip is a free, open-source LLM chat app by facebookresearch, written mainly in Python. Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024. It has 380 GitHub stars and 46 forks, and was last updated 3mo ago. GitHub doesn't detect a standard license, so check the repository before using it commercially.
- License
- Not detected
- Language
- Python
- Stars
- 380
- Last update
- 3mo ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
self-llm33k stars · 《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程
Open-LLM-VTuber14k stars · Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D avatar running locally across platforms
transformer-explainer9.0k stars · Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization
open-webui156k stars · User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
omlx23k stars · LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
llm_interview_note15k stars · 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题
LayerSkip FAQ
Is LayerSkip free?
Yes. LayerSkip is open source and free to use. GitHub doesn't detect a standard license, so check the repository before using it commercially.
Does LayerSkip run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does LayerSkip need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is LayerSkip still maintained?
It was last updated 3mo ago. Check recent issues on GitHub to judge how active it is.
What are alternatives to LayerSkip?
Similar free, open-source tools include self-llm, Open-LLM-VTuber, transformer-explainer, open-webui.