Selenium can test whether a link works as part of a real browser journey, but it is not the right tool for crawling every link across a site. For a user-facing test, click or navigate to the destination and assert that the expected page appears. For a site-wide link inventory, use an HTTP crawler; Selenium’s own guidance recommends alternatives such as curl or BeautifulSoup because starting a browser and traversing the DOM adds overhead.
What Selenium can—and cannot—tell you about a broken link
Selenium WebDriver drives a browser in a way that represents a user interacting with a website. That makes it useful for checking rendered pages, navigation and visible error states. It does not include a built-in broken-link checker, and Selenium advises against using WebDriver to spider links across a site.
As an Amazon Associate I earn from qualifying purchases.
The distinction is the test’s purpose:
- User-journey test: verify that a particular link takes a user to the expected page or that a failure produces an understandable error page. Selenium is a good fit.
- Site-wide inventory: discover and request many links, including links users may never click in a test flow. Use a crawler or HTTP-based approach instead.
Selenium’s link-spidering guidance specifically suggests curl or BeautifulSoup for link discovery and checking. Browser automation can still be used to test selected links that matter to a user journey.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Test a link as part of a real user journey
For a functional test, begin at the page a user would visit, follow the link, and assert a meaningful feature of the destination. Prefer a stable heading, page title, or other reliable element over merely checking that the browser changed URLs: a destination can load and still show an error.
#1 Best Overall
- Open the page containing the link.
- Wait for the link to be present and interactable if the page renders it asynchronously.
- Click the link and wait for the destination or its identifying content.
- Assert that the expected destination content is visible. If the destination is an error page, assert its title or a reliable element such as its H1.
Selenium’s guidance on HTTP response codes notes that, for functional testing, the user steps preceding a failure are usually more important than the status code alone. Checking the rendered result tests what the user experiences; it does not establish the HTTP response status or enumerate other links on the site.
Python example
This example uses Selenium’s Python API. Replace the page URL, link locator and expected heading with values from your application. It assumes a Selenium WebDriver setup for the browser you want to test.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
start_url = "https://example.com/help"
expected_heading = "Getting started"
# Configure the driver for the browser used by your test environment.
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
try:
driver.get(start_url)
help_link = wait.until(
EC.element_to_be_clickable((By.LINK_TEXT, "Getting started"))
)
help_link.click()
heading = wait.until(
EC.visibility_of_element_located((By.TAG_NAME, "h1"))
)
assert heading.text.strip() == expected_heading, (
f"Unexpected destination heading: {heading.text!r}"
)
finally:
driver.quit()The example checks one known journey, not every anchor on the starting page. If the destination contains multiple H1 elements or the application uses a more stable test identifier, use a locator specific to the expected destination instead of the generic H1 locator.
Rank #2
Wait for JavaScript-generated links and content
A browser reporting that the initial document is ready does not mean a JavaScript application has finished changing the page. Links may be inserted later, and a destination’s content may arrive after navigation. Selenium’s waiting strategies explain why tests should wait for the condition they need rather than assume page readiness means all scripts are done.
- Use an explicit wait for the link to become present or clickable before clicking it.
- After navigation, wait for the expected destination element to become visible.
- Choose a timeout appropriate to the application and test environment; a timeout is a limit for the wait, not proof that the link is broken.
- Avoid adding arbitrary sleeps unless there is a specific timing reason; condition-based waits express what the test actually needs.
When HTTP status codes matter
A visible page assertion answers whether the user reached the expected experience. It does not tell you whether the response was, for example, an HTTP 404 or whether a redirect occurred along the way. Selenium documents a proxy as an advanced way to capture response information during browser testing, while noting that browser support for exposing response codes varies. Use that route only when status-code evidence is a real requirement of the test, and verify that the chosen browser and proxy setup expose the responses you need.
WebDriver BiDi can stream browser events, including network requests, console messages and JavaScript errors. Those events can help investigate a browser flow, but they are observability capabilities—not a documented, one-step recipe for crawling and validating every link on a site.
For a site-wide broken-link inventory, use an HTTP crawler
If the goal is to find links across many pages, use a crawler or HTTP-based workflow rather than launching a browser for every link. At a high level, a crawler can collect links from pages, normalize their URLs, request targets and report failures. Decisions such as which status codes count as failures, how to handle redirects, and how to limit crawling belong to the crawler’s design and should be defined for your site.
HTTP crawling and Selenium answer different questions. A crawler is suited to broad link discovery and requests without browser startup and DOM traversal. Selenium is suited to verifying what happens when a user follows selected links, including content created or changed in the browser. If links only appear after client-side interaction, a crawler may not discover them from the initial HTML; test those important flows in Selenium or choose a crawling approach that renders pages.
Performance, reliability and scope
- Runtime overhead: WebDriver starts and drives a browser and traverses the DOM, which Selenium identifies as a reason not to spider links with it. The documentation provides no general speed-up figure; actual runtime depends on the pages and test setup.
- Coverage: A user-flow test covers only the paths and links it exercises. It is not a complete site inventory.
- Dynamic pages: JavaScript timing can make a link or its destination content appear later, so condition-based waits are part of a reliable test.
- Failure diagnosis: A failed assertion may reflect a broken destination, a synchronization problem, or an underlying browser-driver issue. Isolate the cause before treating every failure as a broken link.
Troubleshoot a failing Selenium link test
The link cannot be found
It may not exist in the initial document yet, or the locator may not match the rendered element. Wait for the expected element, then verify the locator against the page’s actual DOM and accessible link text.
Rank #4
- Used Book in Good Condition
The test times out after clicking
The destination may be slow, its expected element may differ from the assertion, or JavaScript may still be updating the page. Wait for a destination-specific condition and confirm that the test is checking the right element rather than assuming navigation alone is success.
The browser shows an error, but the cause is unclear
Check the rendered title or a reliable error-page element to distinguish a user-visible failure from a successful destination. If you specifically require the HTTP response status, consider the documented proxy approach and account for browser support differences.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Failures vary by browser or driver
Selenium’s troubleshooting guidance cautions that synchronization and the underlying browser driver can both cause failures. Reproduce the issue with another supported browser where practical to help isolate a browser- or driver-specific problem from a site defect.
Best Value
Or skip the browser setup
If you need a screenshot of the destination as part of a check or report, ScreenshotNeo is a website screenshot API and MCP server. A single GET request returns a screenshot or PDF; its clean-shot workflow removes known consent banners, newsletter popups and chat widgets before capture. Only clean shots are billed: bot checks, blank pages, timeouts, failed loads and cache hits cost nothing, and responses identify the page verdict and billing status. Its MCP server lets AI agents take screenshots, and the free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000.
For example, save a WebP screenshot of the destination URL:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/help -o shot.webpSee the ScreenshotNeo API documentation for parameters and response details. A screenshot can help inspect a rendered page, but it does not replace Selenium assertions or a site-wide link crawler. Sign up for 1,000 free screenshots a month with no card.
Frequently Asked Questions
Does Selenium have a built-in broken-link checker?
No. WebDriver automates browser interactions; Selenium advises against using it to spider links across a site.
Can Selenium tell me a link’s HTTP status code?
Not as a simple, universally supported built-in check. Selenium documents a proxy as an advanced option, with browser support varying.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
