apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o
- Stars
- 26k
- Forks
- 1.7k
- Open issues
- 99
- Last push
- today
About crawlee
crawlee is a free, open-source AI developer tool by apify, written mainly in TypeScript. Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o. It has 26k GitHub stars and 1.7k forks, and was last updated today. It's released under the Apache-2.0 license, a standard open-source license. People use it as a free alternative to Context.dev, Firecrawl.
- License
- Apache-2.0
- Language
- TypeScript
- Stars
- 26,224
- Last update
- today
- Runs on Mac
- Not stated
- GPU
- Not stated
- Paid API key
- Not stated
- Self-hostable
- Yes
Similar free tools
Scrapling89k stars · 🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: http
Scrapegraph-ai32k stars · Python scraper based on AI
spider2.8k stars · Foundational low latency web data collecting in Rust
trafilatura7.0k stars · Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
vendure8.5k stars · Open-source headless commerce platform built with TypeScript, NestJS, React, and GraphQL
typescript-sdk14k stars · The official TypeScript SDK for Model Context Protocol servers and clients
crawlee FAQ
Is crawlee free?
Yes. crawlee is open source and free to use. It's released under the Apache-2.0 license, a standard open-source license.
Does crawlee run on a Mac?
The project doesn't say. Check its installation guide before you start.
Does crawlee need a GPU?
The project doesn't state hardware needs. AI models generally run much faster with a GPU.
Is crawlee still maintained?
Yes. It was last updated today and has 99 open issues on GitHub.
What are alternatives to crawlee?
Similar free, open-source tools include Scrapling, Scrapegraph-ai, spider, trafilatura. crawlee itself is a free alternative to Context.dev, Firecrawl.