huggingface/datatrove
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
- Stars
- 3.4k
- Forks
- 321
- Open issues
- 84
- Last push
- 15d ago
About datatrove
datatrove is a free, open-source machine learning tool by huggingface, written mainly in Python. Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks. It has 3.4k GitHub stars and 321 forks, and was last updated 15d ago. It's released under the Apache-2.0 license, a standard open-source license.
- License
- Apache-2.0
- Language
- Python
- Stars
- 3,397
- Last update
- 15d ago
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Not stated
datatrove FAQ
Is datatrove free?
Yes. datatrove is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does datatrove run on a Mac?
Its README doesn't say. Check the installation guide on GitHub before you start.
Does datatrove need a GPU?
Its README doesn't state hardware needs. AI models generally run much faster with a GPU.
Is datatrove still maintained?
Yes. It was last updated 15d ago and has 84 open issues on GitHub.