Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
A “navigation block” is an observation, not a diagnosis. Your automation may not have started a request, may have been redirected or canceled, may have committed a document without reaching the readiness condition you chose, or may have hit DNS, network, URL, TLS, server, or download behavior. First preserve the evidence and identify the stage that failed. Only call it an access block when the browser actually displays a denial or challenge.
This guide uses Playwright terminology, but the diagnostic method applies to Selenium and other browser drivers. The official Playwright Page API and navigation guide define the lifecycle and failure categories used below.
What a navigation block can mean
Navigation and document loading are separate stages. A click, form submission, or page.goto() can create navigation intent. The browser then requests a main resource, follows redirects, commits response headers and session history, parses the document, runs scripts, and fires lifecycle events. Any stage can fail or remain incomplete.
- Before the request: the URL may be malformed, an interaction may not have fired, or a script may have canceled navigation.
- Request or response: DNS, connectivity, TLS, an unreachable server, or a failed main resource can stop the operation.
- Redirect: the destination may differ from the URL you supplied, and the final page may have different readiness behavior.
- Commit: response headers may arrive and the document may start, but scripts or resources can still fail.
- After commit: the page may load successfully while your chosen DOM assertion,
loadevent, or network-idle condition never occurs. - Download: navigation intent can become a file download instead of a document.
Playwright documents that goto can throw for invalid URLs, SSL errors, exceeded navigation timeouts, unreachable or nonresponsive servers, and failed main-resource loads. Those categories are a starting point for logging, not an exhaustive list of browser failures.
#1 Best Overall
- Compact Mouse: With a comfortable and contoured shape, this Logitech ambidextrous wireless mouse feels great in either right or left hand and is far superior to a touchpad
- Durable and Reliable: This USB wireless mouse features a line-by-line scroll wheel, up to 1 year of battery life (2) thanks to a smart sleep mode function, and comes with the included AA battery
- Universal Compatibility: Your Logitech mouse works with your Windows PC, Mac, or laptop, so no matter what type of computer you own today or buy tomorrow your mouse will be compatible
- Plug and Play Simplicity: Just plug in the tiny nano USB receiver and start working in seconds with a strong, reliable connection to your wireless computer mouse up to 33 feet / 10 m (5)
- Better than touchpad: Get more done by adding M185 to your laptop; according to a recent study, laptop users who chose this mouse over a touchpad were 50% more productive (3) and worked 30% faster (4)
How can I tell whether a navigation actually finished?
Record enough information to distinguish a transport failure from a synchronization mistake. For every failure, save:
- the exact target URL and the eventual URL;
- the action that initiated navigation;
- the full exception text and stack;
- the main-resource response, status, and headers when available;
- browser, driver, Playwright/Selenium, and operating-system versions;
- whether the same URL works outside automation on the same network, account, proxy, and DNS path.
Then answer four questions in order:
- Did the URL change? Inspect
page.url()after a failure and capture redirect events if your framework exposes them. - Did a main-document response arrive? A response status, even an error status, is different evidence from a timeout before any response.
- Did the document commit? In Playwright,
waitUntil: 'commit'represents parsed response headers and updated session history. - Did the required state appear? An assertion such as a heading, account control, or table row is stronger evidence of readiness than elapsed time.
A timeout means only that an operation exceeded its configured limit. It does not prove that a site detected or blocked automation.
Why is Playwright navigation timing out?
Use the milestone that matches the test
Playwright supports these waitUntil milestones:
| Milestone | What it tells you | When to use it |
|---|---|---|
commit |
Response headers were parsed and the document began. | Testing response receipt or an early document boundary. |
domcontentloaded |
The initial HTML was parsed and the event fired. | Pages whose required controls exist after HTML parsing. |
load |
The page’s load event fired. | When the test genuinely depends on load-complete resources. |
networkidle |
No network connections for at least 500 ms. | Rare cases where quiet networking is a product requirement; the docs discourage it for tests. |
The Page API explicitly says networkidle is discouraged for testing and recommends web assertions instead. Analytics, polling, advertisements, and long-lived sockets can keep a page non-idle forever. A semantic assertion verifies the state your user actually needs.
A diagnostic Playwright example
import { test, expect } from '@playwright/test';
test('diagnose the destination', async ({ page }) => {
const target = 'https://example.com/account';
page.on('requestfailed', request => {
console.log('request failed', request.url(), request.failure());
});
page.on('response', response => {
if (response.request().isNavigationRequest() && response.request().frame() === page.mainFrame()) {
console.log('main response', response.status(), response.url());
}
});
try {
const response = await page.goto(target, {
waitUntil: 'domcontentloaded',
timeout: 45_000
});
console.log({
suppliedUrl: target,
finalUrl: page.url(),
status: response?.status() ?? null
});
await expect(page.getByRole('heading', { name: /account/i })).toBeVisible({ timeout: 15_000 });
} catch (error) {
console.error('navigation failed', { target, finalUrl: page.url(), error });
throw error;
}
});Use a shorter commit wait while isolating transport problems, then add the assertion that defines success. Do not simply increase the timeout before knowing which stage is late.
Rank #2
- The next-generation optical HERO sensor delivers incredible performance and up to 10x the power efficiency over previous generations, with 400 IPS precision and up to 12,000 DPI sensitivity
- Ultra-fast LIGHTSPEED wireless technology gives you a lag-free gaming experience, delivering incredible responsiveness and reliability with 1 ms report rate for competition-level performance
- G305 wireless mouse boasts an incredible 250 hours of continuous gameplay on just 1 AA battery; switch to Endurance mode via Logitech G HUB software and extend battery life up to 9 months
- Wireless does not have to mean heavy, G305 lightweight mouse provides high maneuverability coming in at only 3.4 oz thanks to efficient lightweight mechanical design and ultra-efficient battery usage
- The durable, compact design with built-in nano receiver storage makes G305 not just a great portable desktop mouse, but also a great laptop travel companion, use with a gaming laptop and play anywhere
Timeout settings are binding-specific
Timeouts can be set at the page, browser-context, or project/configuration level. The current JavaScript Page API describes a goto navigation-timeout default of 0, meaning no limit at that API level; other languages and test runners can apply different defaults. Check the documentation for the binding and version installed in your project instead of copying a value from another language. Set a deliberate navigation timeout and a separate assertion timeout so a slow server is not confused with a missing UI state.
Why does page.goto hang or redirect?
Redirects
A server or client-side script can redirect before the page reaches load. Playwright waits for the redirected page’s selected milestone, so the original URL is not necessarily the final destination. Always inspect page.url() and the response you received. A successful redirect to a login page, regional host, consent route, or error document may look like a hang if the test only checks the original URL.
For a navigation triggered by a click, use a URL expectation around the action:
Recommended Free Tools
await Promise.all([
page.waitForURL('**/reports/**', { waitUntil: 'domcontentloaded' }),
page.getByRole('link', { name: 'Reports' }).click()
]);
await expect(page.getByRole('heading', { name: 'Reports' })).toBeVisible();Playwright marks waitForNavigation as inherently racy and recommends waitForURL. When redirects occur, the navigation response returned by waitForNavigation corresponds to the last redirect response, while goto describes the first non-redirect response. Log the final URL and status rather than assuming those objects represent the same hop.
Rank #3
- Compact Mouse: With a comfortable and contoured shape, this Logitech ambidextrous wireless mouse feels great in either right or left hand and is far superior to a touchpad
- Durable and Reliable: This USB wireless mouse features a line-by-line scroll wheel, up to 1 year of battery life (2) thanks to a smart sleep mode function, and comes with the included AA battery
- Universal Compatibility: Your Logitech mouse works with your Windows PC, Mac, or laptop, so no matter what type of computer you own today or buy tomorrow your mouse will be compatible
- Plug and Play Simplicity: Just plug in the tiny nano USB receiver and start working in seconds with a strong, reliable connection to your wireless computer mouse up to 33 feet / 10 m (5)
- Better than touchpad: Get more done by adding M185 to your laptop; according to a recent study, laptop users who chose this mouse over a touchpad were 50% more productive (3) and worked 30% faster (4)
Back and forward navigation
Back/forward cache (BFCache) restoration can bypass network lifecycle events that Playwright’s internal page state expects. The result can be a timeout or desynchronization even though the browser visibly restored a page. Reproduce the flow with a normal link or direct navigation, and check the installed framework version’s current limitations before changing application code.
Downloads and canceled navigations
A response with a download disposition is not a document to wait for. Handle it as a download event and verify the file. Navigation can also be canceled, including by unresolved DNS or page code. A missing load event in either case is not evidence of an anti-bot system.
How do I handle a blocked page in browser automation?
First distinguish denial from failure
Capture a screenshot, page text, final URL, response status, and browser console output. An explicit “access denied,” sign-in requirement, CAPTCHA, or challenge page is an observed denial. A blank page, DNS error, SSL exception, timeout, or failed assertion is a different class of failure. Do not label a site as blocking automation from timing alone.
Use an authorized route
If the destination explicitly denies access or presents a challenge, use an approved account, test environment, documented API, IP allowlist, or a route the site owner has authorized. Ask the owner to permit your test traffic or provide stable fixtures. Do not attempt to defeat CAPTCHA, fingerprint checks, rate limits, or other access controls.
Rank #4
- Computer mouse for easily navigating a computer interface; click, scroll, and more
- USB-A wired connection; if existing device only supports USB-C, an additional adapter will be required
- High-definition (1000 dpi) optical tracking ensures responsive cursor control for precise tracking and easy text selection
- 3 buttons offer effortless fingertip control
- Plug-and-go ready for instant use
Check ordinary causes before access policy
- Validate and refresh the URL, including encoding and trailing paths.
- Resolve the hostname from the same machine, container, proxy, and DNS configuration used by the test.
- Check certificate validity, corporate TLS interception, clock skew, and trust stores.
- Verify browser/driver compatibility against the project versions. Selenium’s supported-browser documentation is the appropriate starting point for Selenium setups.
- Confirm that authentication, cookies, headers, geolocation, and timezone match the approved test scenario.
- Check whether a redirect ends at a download, login wall, regional page, or maintenance response.
A repeatable repair workflow
- Preserve evidence. Save the target, final URL, exception, response/status, versions, trace, screenshot, and reproduction context.
- Classify the stage. Decide whether failure occurred before request, during response/redirect, at commit, during document loading, or during a post-load assertion.
- Reproduce outside automation. Use the same network, credentials, proxy, browser family, and destination. A difference narrows the cause; identical failure points toward infrastructure or authorization.
- Select a milestone. Use
commit,domcontentloaded, orloadonly when that lifecycle point matters. Otherwise wait for the specific control or content with a web assertion. - Synchronize the initiating action. Install
waitForURLbefore clicking or submitting, then assert the destination and meaningful content. - Apply a targeted fix. Correct the URL, DNS/TLS, credentials, timeout, selector, browser version, or authorized access route indicated by the evidence.
- Retest with tracing. Keep the trace and logs for regressions; remove only diagnostic noise, not the assertion that defines success.
Performance, reliability, and cost considerations
Short waits expose failures quickly, while semantic assertions prevent unnecessary delays on pages that remain busy in the background. Reuse authenticated contexts only when isolation and security requirements allow it. Keep retries bounded: retrying DNS or TLS failures without changing conditions adds load and obscures the first useful error. For flaky third-party pages, record response timing and status separately from assertion timing so capacity problems are not mistaken for selector bugs.
For visual evidence, a screenshot taken after the page reaches the intended assertion is more useful than one captured at an arbitrary delay. If your own browser setup is the bottleneck, ScreenshotNeo provides a separate URL-to-image and PDF path.
Or skip the browser setup
ScreenshotNeo is a website screenshot API and MCP server. It accepts consent banners before capture and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each step can be disabled. Only clean shots are billed: bot checks/CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, with the result identified by X-Page-Verdict and X-Billed headers. AI agents can call its MCP tools—take_screenshot, get_page_info, and capture_pdf—from Claude, Cursor, or another MCP client.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The one-call cURL form is:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo documentation for all 63 options, including full-page and element capture, device presets, retina scale, PDF page ranges, custom CSS/JavaScript, click and wait rules, request blocking, headers/cookies/user agents, timezone and geolocation, transparent backgrounds, resizing, TTL caching, signed links, asynchronous webhooks, bulk capture of up to 100 URLs, usage reporting, and OpenAPI compatibility.
Python:
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)Node.js:
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);The Free plan includes 1,000 shots per month with no card. Paid plans start at $5 for 3,000 shots; every feature is on every plan. Create a free ScreenshotNeo account.
Best Value
- 【Plug and Play for Home/Office/School】The wireless computer mouse features 2.4GHz connectivity, delivering a stable, interference-free connection up to 32ft. Designed for 𝐦𝐞𝐝𝐢𝐮𝐦 𝐭𝐨 𝐥𝐚𝐫𝐠𝐞 𝐬𝐢𝐳𝐞𝐝 𝐡𝐚𝐧𝐝𝐬, it ensures comfortable use all day. Simply plug in the USB-A receiver for instant pairing—no drivers needed. 📌📌 If the mouse isn’t suitable, place the USB receiver in the battery compartment and return both.
- 【3 Levels Adjustable DPI】This travel USB mouse offers 3 adjustable DPI settings (800, 1200, 1600), allowing you to customize sensitivity for precise design work. Effortlessly switch to match your task and elevate your productivity. 📌 Please remove the film at the bottom of the mouse before use.
- 【Effortless Browsing】Equipped with forward and backward buttons, this computer mice streamlines your workflow, making it easy to navigate through web pages and files with a simple click. 📌Side button does not work on Mac.
- 【Visible Indicator Light】 The pc mouse features a visual indicator for DPI levels and low battery alerts. The red light flashes once for 800 DPI, twice for 1200 DPI, and three times for 1600 DPI. When the battery level is below 10%, the light flashes red until the mouse is completely out of power.
- 【Click to Wake】With smart sleep mode, it saves power by standby after 10 inactive minutes, just 2-3 clicks to wake. This efficient design delivers 3x longer battery life than motion-wake mice. Engineered for durability, its buttons and scroll wheel are tested for 10 million clicks, ensuring long-term reliability and consistent performance.
Frequently asked questions
Frequently Asked Questions
Does a timeout prove that a website is blocking my bot?
No. A timeout can result from DNS, TLS, an unreachable server, a redirect, a download, an unsuitable wait condition, or a test synchronization error. Treat an explicit denial or challenge page as evidence of access policy.
Should I always increase Playwright’s navigation timeout?
No. First identify whether the request, response, commit, lifecycle event, or assertion is late. Then set a deliberate navigation timeout and a separate assertion timeout appropriate to your binding and environment.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →What should I use instead of waitForNavigation?
For an interaction that changes the URL, install Playwright’s waitForURL expectation before the action and follow it with an assertion on the destination’s meaningful content.
The Bottom Line
Handle a navigation block by proving which stage failed, selecting the readiness condition your test actually needs, and using an authorized access path when the destination explicitly denies automation. A timeout alone is not a diagnosis.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
