DRKSpiderJava

What DRKSpiderJava is

DRKSpiderJava is a free website crawler for finding broken links and examining how a site is structured. It can restrict crawls by external links, a maximum depth, excluded URLs, and an optional robots.txt setting. Results can be exported as sitemaps, all links, or broken links in XML, plain text, and CSV formats. Link searches can narrow results by tag, URL, anchor text, nofollow status, internal or external status, and error status; node searches filter page metadata. The crawler supports 1 to 15 threads, though the maker says using three or more may put an aggressive load on the target server. It also offers basic matching of titles, descriptions, and keywords against page content, but does not identify over-optimization and is not intended as an SEO tool. Other features include saved crawl trees for later review, regex search, redirect handling, cookies, and basic HTTP authentication. It runs on Linux, macOS, and Windows and requires a Java SE virtual machine. Listed downloads are dated March 7, 2015, and the maker reports few to no updates. It is open source and free.

Who it suits

It suits people who need to map site links or locate broken links on Linux, macOS, or Windows. Users should be comfortable with a Java SE virtual machine and note the maker’s limited update activity.

Limits

The free plan requires a Java SE virtual machine. Basic title, description, and keyword matching does not detect over-optimization, and the maker says the project receives few to no updates.

Getting started

1. Visit the DRKSpiderJava website.
2. Choose a listed ZIP or TAR.GZ binary package, or the source archive.
3. Use Linux, macOS, or Windows with a Java SE virtual machine.
4. Set crawl constraints and choose a thread count from 1 to 15.

Verdict

DRKSpiderJava combines link crawling, filtering, and export options in a free, open-source tool. Its dated downloads and limited maintenance are important considerations.