Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsClean PCRecommendedOne scan can reveal what keeps slowing WindowsLook for cleanup and repair opportunities.Run ScanHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › browser automation

How to Take Bulk Screenshots of URLs with Different Wait Conditions

By Freedom251 Team · · 6 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Store each page as a capture record containing its URL, its own readiness rule, and a unique output filename. Then loop over the records with Playwright: navigate using the chosen event, wait for a page-specific selector when needed, capture, and log failures per URL so one timeout does not stop the batch.

Model the batch as per-URL capture records

Different pages become screenshot-ready at different times. A server-rendered page may be usable once its DOM is loaded; a client-rendered app may need a particular component to appear. Keep those rules with the URL rather than applying one delay to the entire batch.

As an Amazon Associate I earn from qualifying purchases.

A JSON array is convenient for a script. Each record can include url, waitUntil, optional waitFor, and output. Use a navigation event when it is sufficient, a selector for asynchronous content, or a fixed delay only when no observable readiness signal exists.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Take the screenshots with Playwright

Install Playwright and its browser binaries according to the Playwright installation guide. Save this as an ES module, for example capture.mjs. Create the output directory before running it, then use a JSON file such as captures.json for the per-URL configuration.

import { chromium } from 'playwright';
import { readFile, mkdir, appendFile } from 'node:fs/promises';

const captures = JSON.parse(await readFile('captures.json', 'utf8'));
await mkdir('screenshots', { recursive: true });
const browser = await chromium.launch();
const results = [];

try {
  for (const [index, item] of captures.entries()) {
    const output = item.output ?? `screenshots/${String(index + 1).padStart(3, '0')}.png`;
    const page = await browser.newPage();
    try {
      await page.goto(item.url, {
        waitUntil: item.waitUntil ?? 'load',
        timeout: item.timeoutMs ?? 30_000,
      });
      if (item.waitFor) {
        await page.locator(item.waitFor).waitFor({
          state: 'visible',
          timeout: item.timeoutMs ?? 30_000,
        });
      }
      await page.screenshot({ path: output, fullPage: item.fullPage ?? true });
      results.push({ url: item.url, output, status: 'ok' });
      console.log(`OK ${item.url} -> ${output}`);
    } catch (error) {
      results.push({ url: item.url, output, status: 'error', error: String(error) });
      console.error(`FAILED ${item.url}: ${error.message}`);
    } finally {
      await page.close();
    }
  }
} finally {
  await browser.close();
  await appendFile('screenshots/results.json', JSON.stringify(results, null, 2));
}

Example captures.json:

[
  {
    "url": "https://example.com/",
    "waitUntil": "domcontentloaded",
    "output": "screenshots/example-home.png"
  },
  {
    "url": "https://example.org/app",
    "waitUntil": "load",
    "waitFor": "[data-ready="true"]",
    "output": "screenshots/example-app.png"
  }
]

Run it with node capture.mjs. The script launches one browser for the batch, creates and closes a page per record, and records each result independently. Use stable, distinct filenames: if two records write to the same path, the later image replaces the earlier one. The loop and result report are orchestration around Playwright’s documented navigation, locator, screenshot, and timeout primitives; tune selector syntax and options for the Playwright version you install. See the Playwright Page API.

Choose a readiness condition that matches the page

Navigation events

Playwright supports commit, domcontentloaded, load, and networkidle for navigation completion. commit means the response arrived and document loading began. domcontentloaded waits for the DOM content-loaded event; load waits for the load event. These are different stages, not interchangeable guarantees that every dynamic widget is ready. Choose the earliest event that reliably precedes the content you need.

Rank #2
Free Fling File Transfer Software for Windows [PC Download]
  • Intuitive interface of a conventional FTP client
  • Easy and Reliable FTP Site Maintenance.
  • FTP Automation and Synchronization

Selectors for asynchronous content

When a known component signals readiness, wait for it directly, such as [data-ready="true"] or a visible chart container. This is usually more meaningful than waiting the same number of milliseconds on every page. The shot-scraper documentation also demonstrates configuration-driven URL captures with selector-based waiting.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Fixed delay and network idle

A fixed delay is a heuristic: it may be too short under a slow run and waste time under a fast one. Use it only when the page offers no reliable observable condition, and set a finite upper bound elsewhere in the capture process.

Playwright defines networkidle as no network connections for at least 500 ms, and its Page API discourages using that condition for tests in favor of checking meaningful page state. Pages with polling or long-lived connections may not become idle at the time you expect. Prefer a selector or application-specific assertion when possible. See Playwright’s Page API guidance.

Keep batch output useful and recoverable

  • Make names unique: use a stable index or sanitized hostname plus path. Avoid deriving filenames from raw URLs without sanitizing characters.
  • Handle each URL independently: catch navigation, selector, and screenshot errors inside the loop, and preserve a report so successful captures remain useful when other pages fail.
  • Set finite timeouts: a stalled page should produce a recorded failure rather than blocking the batch indefinitely. Keep the timeout in the record if some pages need longer than others.
  • Retry selectively: use the results report to rerun only timed-out or failed records instead of recapturing successful URLs.
  • Limit concurrency deliberately: the example is sequential, which is simple to debug and avoids launching an uncontrolled number of pages. If you add parallel workers, cap the number of simultaneous pages according to your machine and target sites.

Make visual captures more consistent

A screenshot can differ even when the URL and wait rule are unchanged. Playwright’s visual comparison guidance notes that browser, platform, fonts, settings, hardware, power state, and headless mode can affect rendering. Keep the browser and operating environment consistent when comparing batches. If motion or changing content makes images unstable, consider disabling animations or masking volatile regions for visual regression work; a screenshot match alone does not prove that the page was semantically ready. See Playwright visual comparisons and screenshot assertion options.

Common failures and fixes

  • Navigation timeout: the chosen event did not occur before the timeout, or the site is slow. Try a less restrictive event if it still captures the needed content, increase the per-record timeout for that site, and retain the failed URL in the report.
  • Selector wait timeout: the selector may be wrong, hidden, or not present on that page. Inspect the page’s actual markup and choose a selector that represents visible readiness; do not silently remove the wait if the component is essential to the image.
  • Blank or incomplete screenshot: the selected navigation event may precede the client-rendered content. Add a selector wait for the relevant component instead of simply slowing every capture with a large delay.
  • Overwritten images: multiple records share an output path. Assign unique paths or generate them from a stable index and hostname.
  • Batch stops after one bad URL: ensure error handling is inside the per-record loop, with page cleanup in finally, so the next record can proceed.
  • Images vary between runs: standardize browser and environment settings, and control animations or volatile elements where repeatability matters.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Or skip the browser setup

ScreenshotNeo takes screenshots through a single GET request, with output options including PNG, JPEG, WebP, or PDF. For example, save a WebP response for a URL:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

See the ScreenshotNeo API documentation for request options. ScreenshotNeo accepts cookie or consent banners and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers identify the page verdict and billing status. Its MCP server offers take_screenshot, get_page_info, and capture_pdf for AI agents. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 shots.

Sign up for ScreenshotNeo’s free plan to try 1,000 screenshots a month with no card.

Frequently Asked Questions

Can every URL in a batch use a different readiness rule?

Yes. Put the navigation event or selector wait on each URL’s capture record and apply that record’s rule during the loop.

Does waiting for network idle prove that the page is ready?

No. Network idle describes a period without network connections, not whether the content you need is present; a page-specific selector is often a clearer readiness signal.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Can I use a command-line tool instead of writing the loop?

Yes. shot-scraper documents configuration-based URL captures and selector waits; a custom Playwright script gives you direct control over filenames and per-URL reporting.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

browser automationPlaywrightScreenshotNeoscreenshotsWeb development

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.