Free tools Windows power users keep installed
One-click scans. No signup required.
The most practical wget workflow is a recursive spider crawl that writes every result to a log, followed by a search for HTTP errors and connection failures. Start with wget --spider -r -nd -nv -w 2 -o crawl.log https://yoursite.example, then inspect the log with grep -E "broken link|failed|404|403" crawl.log. Use a depth limit, keep the crawl on your own host by default, and verify suspicious results with a normal GET request before changing a link.
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
What wget can—and cannot—tell you about broken links
GNU Wget is a command-line network downloader with a spider mode. In spider mode it checks URLs without saving the response bodies, while recursive mode follows links discovered in pages. That combination makes wget useful for a site-wide first pass: it can discover internal links, request their targets, and record failures in one log.
As an Amazon Associate I earn from qualifying purchases.
A “broken link” is not limited to a 404 response. A missing page commonly returns 404 Not Found, but a link can also fail because its hostname cannot be resolved, a server refuses access, or redirects loop indefinitely. Conversely, a 403 Forbidden response means the server refused the request; it does not prove that the destination is missing. Treat the log as a triage list, not an automatic list of pages to delete.
wget has important boundaries. It follows links present in the HTML it receives, but it does not render a browser page or execute client-side JavaScript. Links created only after a script runs can therefore be absent from the crawl. Authentication, bot protection, rate limits, robots rules, and servers that handle HEAD and GET differently can also make a valid page look unavailable to the spider.
#1 Best Overall
- [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
- [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
- [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
- [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
- [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.
Prerequisites and a safe starting point
Install and identify wget
Use the wget package supplied by your operating system, then confirm that the command is available:
wget --version
wget --helpThe GNU Wget manual indexed for this workflow is labelled version 1.25.0, but distributions can ship another release or patch level. Read the help output on the machine that will run the crawl, especially if a flag behaves differently than expected.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesChoose the starting URL
Give wget a canonical page on the site you own, normally the homepage:
https://yoursite.example/Use the final HTTPS address if the HTTP version immediately redirects. A crawl begins with the URL you specify and discovers additional URLs from the pages it can retrieve. If the site has separate sections or subdomains, decide whether each is in scope before running the command.
Run during a quiet, authorized window
A recursive crawl sends many requests. Run it only against a property you control or have explicit permission to test. Keep a delay between requests, avoid launching multiple crawls at once, and monitor server load. A crawl that ignores robots rules can access areas the site owner intended to exclude, so do not disable robots handling casually.
The basic internal-link crawl
Run this command from a directory where you want the log file:
wget --spider -r -nd -nv -w 2 -o crawl.log https://yoursite.exampleEach option has a separate job:
| Option | Meaning in this workflow | Why it matters |
|---|---|---|
--spider |
Check URLs without downloading page bodies. | Reduces disk use and makes the run a link-validation crawl rather than a site mirror. |
-r or --recursive |
Follow links found on retrieved pages. | Turns a single URL check into a crawl of discoverable pages. |
-nd or --no-directories |
Do not create a directory tree for retrieved resources. | Keeps a spider run from building a mirror-like filesystem structure. |
-nv |
Use a less verbose progress format. | Leaves the log easier to scan than a fully verbose transfer trace. |
-w 2 |
Wait two seconds between requests. | Slows the crawl to reduce request bursts. Confirm the exact behavior with wget --help on your system. |
-o crawl.log |
Write output to crawl.log. |
Creates a durable record that you can search, archive, and compare with a later run. |
The short options are commonly documented this way, but the exact definitions of -nv, -w, and -o were not independently checked against every current build. If your system’s help text differs, follow that local documentation.
Rank #2
- 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
- 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
- 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
- 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
- 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.
Limit the crawl before it becomes too large
Set an explicit recursion depth
Recursive wget uses a default maximum depth of 5. Depth is measured from the starting URL, so a page several navigation hops away may not be checked unless you raise the limit. Set it deliberately instead of relying on the default:
wget --spider --recursive --level=3 --no-directories --no-verbose --wait=2
--output-file=crawl.log https://yoursite.example/Use a smaller level for a quick smoke test and a larger level when the site’s important content is deeply nested. An unlimited or very high crawl can create a large queue and consume substantial memory because wget tracks URLs it has seen. Test the scope on a staging host or a small section before expanding it.
Keep the crawl on the original host
By default, recursive retrieval refuses to visit hosts other than the one named on the command line. That is the safest starting point for an internal-link audit. A page may contain links to an image CDN, a documentation subdomain, a payment provider, or another external site; those hosts are not automatically crawled.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Do not add --span-hosts unless checking other hosts is an explicit requirement. Span-hosts allows wget to leave the starting host and can widen the crawl dramatically because foreign pages often link to still more hosts. If you must use it, pair it with a depth limit and a domain allow-list:
wget --spider --recursive --span-hosts --domains=yoursite.example,docs.yoursite.example
--level=2 --no-directories --no-verbose --wait=2
--output-file=external-crawl.log https://yoursite.example/Review the domain list carefully. A subdomain is a separate host for scope purposes, and a domain allow-list does not automatically make every third-party destination safe or desirable to crawl.
Run separate passes for different questions
One crawl should answer one scope question. Use an internal-only pass to find links your site controls, then a deliberately restricted external pass if you need to check outbound references. Separate logs prevent a third-party timeout from being confused with an internal outage.
Read the log for actionable failures
After the crawl finishes, search the log for the signatures most often associated with broken or unreachable destinations:
Recommended Free Tools
Rank #3
- [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
- [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
- [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
- [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
- [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by
grep -E "broken link|failed|404|403" crawl.logThe exact capitalization and wording can vary by wget version, so search broadly when a simple expression returns nothing:
grep -Ei "broken link|failed|404|403|redirect|name or service not known" crawl.logSave the unfiltered log as well. A filtered view is convenient for triage, but the surrounding request and redirect lines often contain the URL and the final status that explain what happened.
404 Not Found
A 404 normally means the requested resource is missing at that URL. Check whether the URL contains a typo, a stale slug, an incorrect case, or an old file extension. If the page was intentionally moved, update the referring page to the new canonical URL or create an appropriate redirect. If the content was intentionally removed, remove the link or replace it with a useful destination.
403 Forbidden
A 403 means the server refused the request. It may be caused by permissions, an IP policy, authentication, a web application firewall, or a rule that blocks automated clients. Do not classify every 403 as a broken link. Open the URL in an authorized browser session or test it with a normal GET request, then inspect server or firewall logs before changing site content.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11“Failed: Name or service not known”
This message indicates that the hostname could not be resolved or reached through the configured name service. Check the spelling, DNS records, and whether the failure is temporary. A typo in a domain is a genuine broken outbound link; a short-lived DNS outage needs a later recheck before you edit links throughout the site.
“Too many redirects”
A redirect loop or an excessive redirect chain prevents wget from reaching a final resource. Common causes include conflicting HTTP-to-HTTPS rules, a host that redirects between www and non-www forms, or an application redirecting an unauthenticated request repeatedly. Resolve the redirect configuration or update the link to the final stable URL.
Successful responses and warnings
A URL that returns a successful response is not necessarily ideal. It may lead to an unexpected page, a login screen, or a soft-404 page that returns success while displaying “not found.” Use the crawl to locate candidates, then inspect important destinations and your application’s analytics or server logs for a complete audit.
Rank #4
- Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
- Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
- All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
- Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
- Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.
Verify suspicious results with GET
Spider checks may use HEAD requests, and some servers answer HEAD differently from GET. A URL that appears failed in spider mode can therefore work when a browser requests the full page. Conversely, a server may return a misleading success to HEAD while the actual GET is unavailable.
Recheck a questionable URL with a normal wget request:
wget --server-response --max-redirect=10 -O /dev/null https://yoursite.example/suspectThis command requests the resource while discarding the body and prints response headers, including status and redirect information. If you need to inspect the returned HTML, omit -O /dev/null or save to a temporary file. Use the same authentication, cookies, user-agent, and network location that a legitimate visitor requires; otherwise the test may reproduce a block rather than the real user experience.
For a repeatable investigation, record the URL, timestamp, response status, redirect chain, whether the test used HEAD or GET, and the action taken. Re-run transient failures later instead of making a permanent edit after one network error.
Robots rules, rate limits, and site protection
wget normally respects robots rules during recursive retrieval. Some examples use -e robots=off to disable that handling, but doing so ignores the site’s robots.txt instructions. Use that option only on a site you own and only when you understand which paths will be requested. It does not grant permission to access private areas.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Wait settings reduce request bursts but do not guarantee that a crawl is harmless. A large site can still receive thousands of requests, and a web application firewall may block the crawler after a threshold. If responses suddenly become 403 or 429, stop the run, check the server’s policy, increase the delay, reduce depth, or crawl a smaller path range. Do not attempt to evade a protection system on a site you do not control.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What wget misses in modern websites
JavaScript-generated links
wget parses the documents it receives; it does not run a browser’s JavaScript. Links inserted by a client-side router, fetched from an API, or generated after a user interaction may never enter the crawl queue. Include server-rendered navigation and sitemap URLs in your audit, and use a browser-based crawler or application-specific test when JavaScript-only routes are critical.
Authenticated and personalized content
A public crawl cannot validate links behind a login, in a customer dashboard, or in content shown only to a particular role. Supplying credentials to an automated command can expose sensitive data in process lists or logs. If private areas must be tested, use a controlled test account, a non-production environment, and your organization’s approved secret-handling method.
Non-HTML resources and alternate URL schemes
Documents can link to PDFs, images, feeds, mailto addresses, telephone links, API endpoints, and files that require special headers. A failure in one of these categories may need a different test than a normal page request. Classify results by URL type before assigning them to an editorial or engineering queue.
A repeatable broken-link audit procedure
- Define ownership and scope. Write down the starting host, subdomains that belong to the site, maximum depth, and whether outbound links are included.
- Check the local build. Run
wget --versionandwget --help. Confirm that the options you plan to use are available and note the version in your audit record. - Run a small smoke test. Use a low
--levelvalue and a short path to verify that DNS, HTTPS, robots handling, and logging work before starting a larger crawl. - Run the internal crawl. Use
--spider -r -nd -nv -w 2 -o crawl.logwith an explicit--levelsuited to the site’s size. - Preserve the original log. Do not overwrite it while filtering. Copy it to an audit directory with the date and scope.
- Extract candidates. Search for 404, 403, failed name resolution, redirect loops, and other errors visible in your wget version.
- Recheck candidates with GET. Confirm whether a failure is persistent and whether the response differs from spider mode.
- Classify each result. Mark it as a broken internal link, intentional access control, transient network issue, redirect configuration problem, external outage, or false positive.
- Fix at the source. Update the page that contains the link, correct the destination, add a redirect where appropriate, or document why access is intentionally restricted.
- Run the crawl again. Use the same scope and settings so the before-and-after logs are comparable, then verify that the corrected URLs no longer produce the original errors.
Practical command variants
Readable long-option form
wget --spider --recursive --no-directories --no-verbose --wait=2
--output-file=crawl.log --level=5 https://yoursite.example/This makes the intent visible in scripts and runbooks. The explicit level matches wget’s documented default maximum recursion depth of 5; setting it in the command prevents a future default change from silently altering the audit.
Quick homepage and first-level check
wget --spider --recursive --level=1 --no-directories --no-verbose
--wait=2 --output-file=quick-crawl.log https://yoursite.example/Use this after a deployment when you need a fast check of the homepage and directly linked pages. It is not a substitute for a deeper crawl of a large information architecture.
Internal crawl with a dated log name
wget --spider --recursive --level=4 --no-directories --no-verbose
--wait=2 --output-file="crawl-2026-10-14.log" https://yoursite.example/A dated filename helps you compare runs and identify which deployment or content change preceded a new failure. Use your actual run date rather than copying the example unchanged.
Common mistakes and their fixes
- Running without
--spider: wget may save a site copy and consume disk space. Add spider mode when validation, not downloading, is the goal. - Assuming one homepage request checks the whole site: Without recursive mode, links discovered on other pages are not followed. Add
-rand set a deliberate level. - Expecting external links to be tested automatically: Default recursion stays on the starting host. Use a carefully restricted
--span-hostspass only when outbound checking is authorized. - Interpreting every 403 as a missing page: Verify with GET and inspect access controls; 403 is a refusal, not proof of absence.
- Ignoring the depth limit: A page deeper than level 5 can be missed by the default. Increase
--levelor crawl that section separately. - Disabling robots rules by habit:
-e robots=offchanges the safety boundary. Use it only for an owned site with a documented reason. - Trusting a single failed request: DNS, rate limits, and transient outages can produce temporary errors. Re-run and compare a normal GET.
- Expecting JavaScript routes to appear: wget does not render client-side code. Pair it with a browser-capable or application-level test for those URLs.
- Replacing links before examining context: A redirect, login wall, or intentional permission rule may be correct. Review the referring page and destination behavior first.
How to turn a log into a maintenance queue
For each candidate, capture the referring page, target URL, observed status or error text, test method, timestamp, and owner. Group duplicates by target URL so one outage does not create dozens of separate tickets. Prioritize internal 404s on high-traffic pages, redirect loops, and hostname failures that affect many references. Handle access-controlled resources with the security or platform owner rather than asking editors to weaken permissions.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Keep the raw log with the processed list. The raw record lets you distinguish a crawler configuration problem from a site problem and provides evidence when a server team investigates. After corrections, run the same command and compare the remaining failures; a lower count is useful only when the scope and settings stayed consistent.
Bottom line
For an authorized internal audit, start with wget --spider -r -nd -nv -w 2 -o crawl.log, set --level deliberately, and leave --span-hosts off unless external checking is intentional. Search the log for 404, 403, DNS, and redirect failures, then verify each important candidate with GET. wget is a strong lightweight crawler for server-visible links, but JavaScript-only, authenticated, and intentionally restricted destinations require additional checks.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
