crawler.sh
Plans
| Plan | Billed and limits | Price |
|---|---|---|
| Free | Up to 50 pages per session · SEO analysis with CSV/TXT export · JSON and Sitemap XML export · No account required | Free |
| CLI Pro | $99 / year · Up to 10,000 pages per session · NDJSON, JSON, and Sitemap XML output · Markdown content extraction · 16-category SEO analysis with CSV/TXT export | $99 a year |
| Desktop Pro | $99 / year · Visual dashboard with 8 interactive cards · Real-time crawl feed · SEO issues panel with per-URL details · Markdown content archive export | $99 a year |
As crawler.sh lists them, read 12 Oct 2026. The maker's own page has the final word.
Facts from crawler.sh's own pages
- AI content
- It extracts article content as clean Markdown with page metadata such as word count, author byline, language, and excerpt for RAG, fine-tuning, or agent context. · crawler.sh, 12 Oct 2026
- Cloud API
- Crawler Cloud is described as a hosted crawling API with scheduled crawls, webhooks, and a dashboard, marked as coming soon. · crawler.sh, 12 Oct 2026
- Crawl behavior
- The crawler respects robots.txt and adapts per-host pacing after 429 and 403 responses. · crawler.sh, 12 Oct 2026
- Data handling
- The maker says crawl data, results, and exported files stay on the user's computer and that it does not collect crawl targets, results, or extracted website content. · crawler.sh, 12 Oct 2026
- Free plan
- Yes · crawler.sh, 21 Sept 2026
- Integrations
- The documentation describes an MCP server for connecting crawler.sh output to tools and lists Claude Code, Claude Desktop, Cursor, OpenCode, and Zed as clients. · crawler.sh, 12 Oct 2026
- Interfaces
- The product includes a CLI for automation and scripting and a desktop app with a visual dashboard; both use the same crawling engine. · crawler.sh, 12 Oct 2026
- JavaScript rendering
- A built-in JavaScript engine renders React, Vue, Next, and similar single-page apps without headless Chrome. · crawler.sh, 12 Oct 2026
- Product
- crawler.sh is a local web crawler that extracts website content, analyzes SEO issues, and exports results in standard formats. · crawler.sh, 12 Oct 2026
- Security
- The privacy policy says device-to-API communication uses HTTPS/TLS and authentication tokens are generated with secure random values and expire after a set period. · crawler.sh, 12 Oct 2026
- SEO checks
- The SEO analysis runs 24 automated checks on successful HTML pages and can export findings as CSV or TXT. · crawler.sh, 12 Oct 2026
- Site audit
- Yes · crawler.sh, 21 Sept 2026
- Support
- Support is offered by email, with typical replies within one business day; the maker says it does not offer live chat, phone support, ticket SLAs, or dedicated account managers. · crawler.sh, 12 Oct 2026
- Windows support
- A native Windows desktop build is in progress, and the maker says its Linux CLI binary can run on Windows through WSL2. · crawler.sh, 12 Oct 2026
Where crawler.sh stands on Freedom251
- SEO tools#105 of 245