Use one Selenium WebDriver session to visit each URL, save a PNG into a folder named for its hostname, and log failures without stopping the rest of the batch. The example below uses Python, creates folders as needed, sets a consistent viewport and page-load timeout, and always closes the browser.
Set up Selenium and prepare your URL list
Install Selenium in the Python environment that will run the batch:
As an Amazon Associate I earn from qualifying purchases.
python -m pip install seleniumInstall or configure a compatible browser and WebDriver for your environment as well. Selenium’s driver.get(url) expects a URL with a scheme such as https://. It waits for the page-load event, but that event alone does not prove that a client-rendered app or its asynchronous assets are ready. Selenium WebDriver API documentation
Choose what “by domain” means for your folders. This script uses the exact hostname, so shop.example.com and www.example.com go into separate folders. Combining subdomains under a registrable domain such as example.com requires public-suffix-aware parsing; splitting on dots is not reliable for domains such as example.co.uk.
#1 Best Overall
Capture the batch into hostname folders
Save this as capture_batch.py. Replace the sample URLs and, for dynamic pages, replace the general readiness check with a condition tied to the content you need to capture.
from pathlib import Path
from urllib.parse import urlsplit
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
urls = [
"https://example.com/",
"https://www.example.org/products",
]
out = Path("screenshots")
out.mkdir(parents=True, exist_ok=True)
# This pattern assumes Chrome is installed and Selenium can obtain/use its driver.
driver = webdriver.Chrome()
driver.set_window_size(1440, 1000)
driver.set_page_load_timeout(30)
try:
for index, url in enumerate(urls, start=1):
parts = urlsplit(url)
if parts.scheme not in {"http", "https"} or not parts.hostname:
print(f"Skipped invalid or unsupported URL: {url}")
continue
host = parts.hostname.lower()
safe_host = "".join(ch if ch.isalnum() or ch in ".-" else "_" for ch in host)
domain_dir = out / safe_host
domain_dir.mkdir(parents=True, exist_ok=True)
filename = domain_dir / f"page-{index:04d}.png"
try:
driver.get(url)
WebDriverWait(driver, 10).until(
lambda d: d.execute_script("return document.readyState") == "complete"
)
saved = driver.get_screenshot_as_file(str(filename))
if not saved:
print(f"Screenshot write failed: {url} -> {filename}")
except Exception as exc:
print(f"Capture failed: {url}: {exc}")
finally:
driver.quit()Run it with python capture_batch.py. The script writes files under screenshots/<hostname>/, for example screenshots/example.com/page-0001.png. Its increasing index avoids overwriting images within one run, but the name does not identify the page by itself; use a manifest if you need an unambiguous URL-to-file record.
What the browser settings do
set_window_size(1440, 1000)makes the viewport consistent across captures. Selenium provides window-size controls; use the dimensions appropriate to your comparison or test. Selenium WebDriver API documentationset_page_load_timeout(30)bounds navigation waiting so a slow page does not hold up the batch indefinitely. Adjust the limit to suit the sites and network conditions.WebDriverWaitwaits for a condition rather than adding an arbitrary sleep. Selenium recommends choosing waits that fit the page’s behavior. Thedocument.readyStatecondition is only a general example; a site may still be fetching or rendering the specific data you need. Selenium waiting strategiesget_screenshot_as_file()saves a PNG of the current browser window and returnsFalseif an I/O error occurs. The parent folder must exist, which is why the script creates it first. Use a path ending in.png. Selenium WebDriver API documentation- The
finallyblock callsquit()even if a capture fails; Selenium documents this as closing the browser and driver executable. Selenium WebDriver API documentation
Wait for the content that matters
Selenium’s navigation call returns after the page’s load event, but modern pages can populate content afterward. The document-ready wait in the example is not a universal “page is finished” test. For a known site, wait for a meaningful element or state that must be present in the screenshot:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 15).until(
EC.visibility_of_element_located((By.CSS_SELECTOR, "main article"))
)Replace main article with a selector that represents the content you actually need. If it is absent on some URLs, that should be treated as a page-specific failure or handled with an alternate condition—not silently replaced with a delay that may be too short or waste time on already-ready pages. Selenium waiting strategies
Keep filenames and failure records useful
Domain folders organize the output, but a domain-only filename can collide when multiple pages share a hostname. The sample’s per-run index prevents same-run overwrites. To identify captures later, derive a short safe filename from the URL path or a stable hash, and keep the original URL in a CSV or JSON manifest alongside the capture status and output path.
Decide how query strings should behave: two URLs with different query parameters may represent distinct pages even when the hostname and path match. If so, include a safe representation or hash of the full URL in the filename or manifest. Avoid putting raw URL paths or query text directly into filesystem names without sanitizing them.
Rank #3
The sample reports failures to the console. For unattended batches, append one record per URL—including invalid URLs, navigation exceptions, and screenshot-write failures—to a manifest so the run can be reviewed or retried without losing which pages failed.
Choose capture scope and session behavior
Viewport or full page
The standard WebDriver screenshot method captures the current browser window; it does not promise a full-page image. This makes it suitable when a consistent viewport is the goal. Firefox exposes separate full-document screenshot methods, but full-page behavior should be verified for the chosen browser and Selenium version rather than assumed to work identically across browsers. Selenium WebDriver API documentation
One reused session or a fresh session
Reusing a single session sequentially is straightforward and avoids repeatedly creating a browser. Use separate sessions when the captures require different login states, cookies, or stronger separation between sites. The right choice depends on the state and privacy requirements of the pages; the script’s one-session pattern does not isolate them.
Exact hostname or broader domain grouping
The included parser groups by exact hostname. If your intended grouping combines subdomains, use a public-suffix-aware approach and define how internationalized hostnames should be represented. The Selenium API does not decide this folder policy.
Troubleshoot common batch failures
- Navigation rejects the URL: provide an absolute URL with
http://orhttps://, and check that it has a hostname before navigating. Selenium’s navigation API requires the scheme. Selenium WebDriver API documentation - A page times out: the configured page-load timeout has been reached. Check whether the site is slow or unreachable, then choose a timeout appropriate to the batch. The per-URL exception handler allows later URLs to continue.
- The screenshot is blank or missing expected content: the load event or document-ready state may have occurred before the relevant app content appeared. Wait for a site-specific element or state using Selenium’s wait strategies. Selenium waiting strategies
- The PNG was not written: check that the output path is writable and that its parent folder exists. Inspect the boolean return from
get_screenshot_as_file();Falseindicates an I/O error. Selenium WebDriver API documentation - Images overwrite or are hard to identify: use URL-aware names or a manifest. Foldering by hostname alone cannot distinguish many pages on the same site.
- The browser remains open after an error: keep browser creation and the loop inside a
try/finallyand calldriver.quit()in the cleanup block. Selenium WebDriver API documentation
Performance, reliability, and handling sensitive pages
A sequential loop is easy to reason about, and reusing a session avoids browser startup for every URL. Its throughput still depends on navigation, the waits you choose, and each target site’s behavior. A fixed delay can make every capture slower while still failing to wait long enough on a slow page; a relevant explicit condition avoids that mismatch.
Set a navigation timeout, handle exceptions per URL, record outcomes, and close the driver in finally. For large batches, persist a manifest as the run progresses so interruption does not erase the record of completed and failed URLs. Do not assume a screenshot is safe to share: authenticated pages and other sensitive content are persisted to disk, so restrict output access and retention according to your requirements.
Best Value
Or skip the browser setup
If you do not want to maintain a browser and WebDriver loop, ScreenshotNeo can return a screenshot or PDF from one GET request. See the ScreenshotNeo API documentation for the available parameters.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webpScreenshotNeo removes cookie/consent banners, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers indicate the page verdict and billing status. Its MCP server provides screenshot tools for AI agents. The free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000 shots.
Sign up for ScreenshotNeo’s free plan: 1,000 screenshots a month, no card required.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsFAQ
Can this script capture a list of URLs from a text file?
Yes. Read one URL per line into the urls list before the loop, while retaining the same URL validation and per-page error handling.
Does the example capture the entire page?
No. It uses the standard current-window screenshot method, so the result reflects the configured viewport.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
