niieani/gpt-tokenizer
The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.
- Stars
- 856
- Forks
- 63
- Open issues
- 1
- Last push
- 4d ago
About gpt-tokenizer
gpt-tokenizer is a free, open-source AI voice and audio tool by niieani, written mainly in TypeScript. The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features. It has 856 GitHub stars and 63 forks, and was last updated 4d ago. It's released under the MIT license, a standard open-source license.
- License
- MIT
- Language
- TypeScript
- Stars
- 856
- Last update
- 4d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
transformers168k stars · 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
go-openai11k stars · OpenAI, GPT 6.1, GPT-Image-2, Whisper API clients for Go. Zero-dependency.
ChatGPT-Next-Web-Pro802 stars · 基于chatgpt-next-web，增加了midjourney绘画功能，支持mj-plus的ai换脸和局部重绘，接入了stable-diffusion，支持oss，支持接入fastgpt知识库，支持suno，支持luma。支持dall-e-3、gpt-4-vision-preview、whisper、tts等多模态模型，支持gpt-4-all，支持GPTs商店。新增加了支持后台管理的版本，包括登
nightingale1.5k stars · Machine learning powered Karaoke app (with scores!)
GPT-SoVITS63k stars · 1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Matcha-TTS1.4k stars · [ICASSP 2024] 🍵 Matcha-TTS: A fast TTS architecture with conditional flow matching
gpt-tokenizer FAQ
Is gpt-tokenizer free?
Yes. gpt-tokenizer is open source and free to use. It's released under the MIT license, a standard open-source license.
Does gpt-tokenizer run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does gpt-tokenizer need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is gpt-tokenizer still maintained?
Yes. It was last updated 4d ago and has 1 open issues on GitHub.
What are alternatives to gpt-tokenizer?
Similar free, open-source tools include transformers, go-openai, ChatGPT-Next-Web-Pro, nightingale.