LibreCrawl

LibreCrawl is a web-based crawler for technical SEO audits, with a self-hosting option and an API for automating crawl workflows. It suits SEO teams and developers who want a focused crawler they can run on their own terms. Its core appeal is an unlimited-URL, MIT-licensed free plan; it is not a broad SEO research suite.

Overview

LibreCrawl examines technical site issues, including meta tags, schema markup, hreflang, social tags and PageSpeed metrics. That focus makes it useful for auditing a site’s technical health, but it does not cover keyword research, rank tracking, backlink analysis or competitor research. Teams that need those capabilities will need separate tools.

The MIT license allows self-hosting, a meaningful option for teams that want control over their crawling environment. The maker says the software does not track users, collect analytics or send crawl data home. Bug reports and feature requests go to GitHub Issues, while GitHub Discussions are the place for conversations.

Key features

JavaScript crawling

Playwright rendering lets LibreCrawl crawl sites built with React, Vue, Angular and Next.js, in headed or headless mode. That makes it a stronger fit than a crawler limited to static page content when important site elements depend on browser rendering.

The maker says memory profiling and virtual scrolling support stable crawling beyond one million URLs. That stated capacity is attractive for large sites, though teams should treat it as a product claim rather than a guarantee for every deployment.

Exports and API

Crawl data can be exported as CSV, JSON or XML, with custom field selection and no stated export restrictions. The REST API supports crawl control, settings and exports, giving developers a route to build crawling into their workflows.

API use has an important caveat: every endpoint requires a session, and authentication is session-based. Guest users are limited to three crawls per 24 hours, tracked by IP address. That suits occasional evaluation, but teams relying on regular API access must manage sessions and should not assume the guest allowance will cover ongoing use.

Pricing

LibreCrawl: free. The free, open-source plan includes unlimited URLs, carries no subscription, and is available under the MIT license. There are no paid tiers, so there is no higher plan to lift the guest API limit; the three-crawl-per-24-hour cap applies to guest use.

There is no free trial because the product is free. For teams able to self-host, unlimited URLs and no subscription make the plan suitable for sustained crawling without a per-URL charge. The trade-off is a focused tool rather than a paid suite of research features.

Platforms

LibreCrawl is available on the web and supports API use, Linux, macOS, Windows and self-hosting. That combination gives teams options to work through the web interface or run the software in their own environment.

Who it's for

SEO consultants, agency developers, enterprise SEO teams and open-source contributors are the maker’s intended users. Consultants and SEO teams can use it for technical audits; developers gain JavaScript rendering, configurable exports and API endpoints to support workflows. Self-hosting makes it particularly relevant to privacy-conscious teams. It is a weaker fit for anyone seeking keyword, backlink, ranking or competitor analysis in the same product.

Pros and cons

• Pros: Unlimited URLs on a free, MIT-licensed plan make sustained crawling accessible without a subscription.
• Pros: Playwright rendering covers several common JavaScript frameworks, while CSV, JSON and XML exports allow flexible handling of crawl data.
• Pros: Self-hosting and the maker’s stated no-tracking, no-analytics and no-phone-home practices appeal to users who want control over crawl privacy.
• Cons: Guest API use stops at three crawls per 24 hours, and all endpoints require session-based authentication.
• Cons: The product does not include keyword research, rank tracking, backlink analysis or competitor research, limiting its usefulness as a standalone SEO toolkit.

Alternatives

SEO Crawler Software and SEO tools are broader directories for comparing options in those categories.

• Open SEO Crawler is another free, MIT-licensed, self-hosted crawler with unlimited URLs and no account, API key or per-URL billing. Choose it if avoiding accounts and API keys is more important than LibreCrawl’s documented session-based API.
• SpiderNow offers a free plan with 10 single-page scans per month, five scan details and issue views, and five page metric scores. Choose it if that limited scan allowance suits your needs and its Android, iOS or browser-extension platforms matter.
• WebCrawler Pro has free guest and signed-in request allowances of 50 and 100 requests per day, respectively. Consider it if those request-based tiers suit your crawling needs.
• A1 Website Analyzer offers a free mode after a 30-day evaluation period, with 500 page URLs per website and 500 imported URLs. It may suit users whose sites and imports fit those caps and who prefer macOS or Windows software.
• Beam Us Up is a free crawler with no limits stated. Consider it as another free option.
• Crab is open-source software with source code available on GitHub. Consider it if that is the priority.
• FreeCrawl offers unlimited URLs on a free plan. Consider it as another no-cost crawler.
• Netpeak Spider has a free plan capped at 500 URLs, with five threads and two devices per user, for Windows and macOS. It may suit smaller crawls on those platforms.

Verdict

Choose LibreCrawl for free technical SEO audits when JavaScript rendering, flexible exports and the option to self-host matter. Its unlimited-URL plan is compelling for teams focused on crawling, but the session-required API and narrow scope make it a poor substitute for a broader SEO platform or a guest API workflow that needs more than three crawls a day.