Driver FixRecommendedSound, Wi-Fi or graphics acting up? Check drivers firstFind missing or outdated drivers fast.Check DriversHalloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix Now×
FREEDOM251

Freedom251 › Blog › Automation

Best Way to Bulk Screenshot Hundreds of URLs with Playwright

By Freedom251 Team · · 8 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

For hundreds of URLs, launch one Playwright browser and process a fixed number of pages at a time. Reuse the browser, choose a readiness condition that fits the sites, save each result under a collision-safe name, and log failures per URL so one bad page does not stop the batch. There is no universally correct concurrency setting: tune it against your pages and machine rather than assuming more workers are always faster.

Use a bounded queue, not hundreds of simultaneous pages

A custom Node.js script is a practical choice when your input is a URL list and you want a predictable output directory. The example below reuses one browser process, opens a new page for each job, limits the number of active jobs, and writes a JSONL record for every URL. It uses load as the default navigation condition; change that to an application-specific selector or another condition when it better represents a usable page.

As an Amazon Associate I earn from qualifying purchases.

Install Playwright and its Chromium browser in your project with npm install playwright followed by npx playwright install chromium. Save the following as bulk-screenshots.mjs. Put one URL per line in urls.txt, then run node bulk-screenshots.mjs.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
import { chromium } from 'playwright';
import { createHash } from 'node:crypto';
import { appendFile, mkdir, readFile } from 'node:fs/promises';
import path from 'node:path';

const inputPath = process.argv[2] ?? 'urls.txt';
const outputDir = process.argv[3] ?? 'screenshots';
const concurrency = Number(process.env.CONCURRENCY ?? 4);
const navigationTimeoutMs = Number(process.env.NAVIGATION_TIMEOUT_MS ?? 30000);
const readinessSelector = process.env.READINESS_SELECTOR;

if (!Number.isInteger(concurrency) || concurrency < 1) {
  throw new Error('CONCURRENCY must be a positive integer');
}

const urls = (await readFile(inputPath, 'utf8'))
  .split(/r?n/)
  .map(line => line.trim())
  .filter(line => line && !line.startsWith('#'));

await mkdir(outputDir, { recursive: true });
const resultsPath = path.join(outputDir, 'results.jsonl');
const browser = await chromium.launch();
let nextIndex = 0;

async function capture(url, index) {
  let page;
  const filename = `${String(index + 1).padStart(4, '0')}-${createHash('sha256').update(url).digest('hex').slice(0, 12)}.png`;
  const outputPath = path.join(outputDir, filename);
  const result = { index, url, outputPath, status: 'failed' };

  try {
    page = await browser.newPage();
    page.setDefaultNavigationTimeout(navigationTimeoutMs);
    const response = await page.goto(url, { waitUntil: 'load' });
    if (readinessSelector) {
      await page.locator(readinessSelector).waitFor({ state: 'visible', timeout: navigationTimeoutMs });
    }
    await page.screenshot({ path: outputPath, fullPage: true });
    result.status = 'ok';
    result.httpStatus = response?.status() ?? null;
  } catch (error) {
    result.error = error instanceof Error ? error.message : String(error);
  } finally {
    if (page) await page.close().catch(() => {});
    await appendFile(resultsPath, `${JSON.stringify(result)}n`);
    console.log(`${result.status}: ${url}`);
  }
}

async function worker() {
  while (true) {
    const index = nextIndex++;
    if (index >= urls.length) return;
    await capture(urls[index], index);
  }
}

try {
  await Promise.all(Array.from({ length: Math.min(concurrency, urls.length) }, () => worker()));
} finally {
  await browser.close();
}

Output names combine the input order and a short URL hash rather than using raw URLs as filenames. The index keeps duplicate URLs distinct; the hash makes names less opaque while avoiding collisions from common URL characters. The JSONL file records the URL, status, output path, and any error, so you can inspect or retry failed rows without losing successful captures. If you rerun the script, remove or rotate the old results file first if you want a fresh log.

#1 Best Overall
Lavsoul 4K Webcam with Microphone for PC & Streaming Computer Camera | PDAF autofocus, dual noise-canceling mics & low-light correction. Best usb hd webcam for mac, laptop and smooth streaming.
  • ULTRA HD 4K CLARITY: Stand out in every video call with breathtaking 4K video at 30fps or smooth 1080p at 60fps. Powered by a premium 1/2.5" CMOS sensor and a wide f/1.78 aperture, this webcam captures every detail with vibrant color and stunning low-light performance-so you always look your best
  • FAST AUTOFOCUS & SMART LIGHT CORRECTION: No more blurry moments with this webcam for PC. Advanced Phase Detection Auto Focus (PDAF) locks onto your face instantly and keeps you sharp-even when you move. Built-in light correction adapts to your environment, balancing brightness and contrast for a flawless image in dim rooms or bright spaces
  • DUAL NOISE-CANCELING MICS: Speak with confidence using this webcam with microphones. Dual microphones with intelligent noise-canceling tech isolate your voice and reduce background noise-suitable for webinars, live streams, team meetings, and virtual interviews
  • WIDE-ANGLE LENS & FLEXIBLE MOUNTING OPTIONS: Capture more of your world with an 80 field of view and full 360 swivel rotation. Whether this streaming webcam is mounted on a laptop, monitor, or tripod, it allows you to find the right angle for any setup
  • BUILT-IN PRIVACY COVER & PLUG-AND-PLAY SIMPLICITY: Protect your privacy with a secure sliding lens cover that blocks the camera when not in use. Setup is a breeze-just plug into any USB-A port and start streaming, chatting, or recording instantly. The USB webcam is compatible with Zoom, Microsoft Teams, Skype, OBS Studio, and all major platforms across Windows, macOS, and Linux

Choose concurrency based on the actual batch

Set the number of simultaneous page jobs with the CONCURRENCY environment variable. The example defaults to four, but that is only a starting setting, not a Playwright recommendation or throughput guarantee. Start low, run a representative sample, then adjust while watching memory, CPU, target-site rate limits, and failure rates.

Pages are lighter-weight concurrency units than Playwright Test workers: Test workers are separate OS processes and each starts its own browser. The Test runner is useful when each URL should be treated as a test with runner reporting, retries, and test configuration; explicitly set its worker count, for example npx playwright test --workers 4. In configuration, the documented pattern includes workers: process.env.CI ? 2 : undefined. Test files run in parallel by default while tests in one file run in order, unless a fully parallel mode is selected. Workers cannot share module globals or state, so ensure outputs and mutable data do not collide. See Playwright Test parallelism.

For a sequential run, use CONCURRENCY=1 node bulk-screenshots.mjs. It reduces parallel load and preserves input order of execution, but does not guarantee a particular total duration: remote response times and page behavior dominate. The custom queue is generally simpler when the job is just “capture these URLs”; the Test runner is a better fit when test-style reporting and test organization matter.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
10.1 Inch Mini Netbook, Quad-Core Processor Laptop Computer, 2GB Memory 64GB Storage Android 12 Portable Notebook Built-in Webcam, WiFi & Bluetooth Keyboard & Mouse for Home Schooling & Office Work
  • 【Efficient Quad-Core Performance】 Powered by a 1.8GHz Quad-Core processor, this mini laptop ensures smooth multitasking. With 2GB RAM and 64GB ROM (expandable to 1TB), it handles daily work and online tasks with ease.
  • 【10.1" HD IPS Display & GMS Support】 Featuring a 1280x800 HD IPS screen, this cheap laptop delivers vibrant visuals. Pre-installed with Android OS and GMS, you get direct access to the Google Play Store for apps.
  • 【Ultra-Portable & Lightweight Design】 Weighing only 1.76 lbs, this Blue computer is designed for mobility. Its compact form makes it an ideal companion for students and professionals for home schooling or trips.
  • 【Versatile Connectivity Options】 Stay productive with dual USB 2.0 ports, a headphone jack, and a TF card slot. This computer for kids and adults features built-in Wi-Fi and Bluetooth for stable connections.
  • 【Complete All-in-One Bundle】 This kid laptop kit includes the laptop, carrying bag, mouse, mouse pad, and power adapter. It is the perfect ready-to-use set for online classes, remote work, and entertainment.

Wait for the right page state

page.goto() supports commit, domcontentloaded, load, and networkidle. The example uses load, the navigation default. For known sites, a selector that appears when the relevant content is ready is often a better signal; set READINESS_SELECTOR to a selector used on every page in the batch. For heterogeneous sites, use a configurable strategy or accept that no one readiness signal will perfectly fit all pages.

Playwright’s Page API explicitly discourages networkidle as a general testing readiness signal: it considers navigation finished after at least 500 ms without network connections, but the documentation says to rely on web assertions to assess readiness instead. Analytics, polling, ads, and other long-lived requests can make network idleness misleading or unattainable. See the Page API navigation reference.

A successful navigation response is not necessarily a successful application page. A server can return an error status while still rendering an error document; decide whether your batch should retain that screenshot, classify it as a failure, or both. The example records the HTTP status but treats successful screenshot creation as an ok capture.

Rank #3
Sale
Logitech C920x HD Pro Webcam, Full HD 1080p/30fps - Black w/Blue Yeti USB Microphone - Blackout
  • Webcam comes with a 3-month XSplit VCam license and no privacy shutter. XSplit VCam lets you remove, replace and blur your background without a Green Screen.
  • Full HD 1080p video calling and recording at 30 fps - You’ll make a strong impression when it counts with crisp, clearly detailed and vibrantly colored video.
  • Stereo audio with dual mics - Capture natural sound on calls and recorded videos.
  • Custom three-capsule array: This professional USB mic produces clear, powerful, broadcast-quality sound for YouTube videos, Twitch game streaming, podcasting, Zoom meetings, music recording and more
  • Blue VOICE software: Elevate your streamings and recordings with clear broadcast vocal sound and entertain your audience with enhanced effects, advanced modulation and HD audio samples

Choose viewport, full-page, and image settings deliberately

By default, page.screenshot() captures the current viewport. Set fullPage: true to include the full scrollable document, as in the script above. Full-page images can be substantially larger and slower to produce than viewport captures. For a specific element, use page.locator('selector').screenshot({ path: outputPath }). You can also omit path and use the returned buffer for post-processing. The Playwright Screenshots guide covers viewport, full-page, and element screenshots.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Set the browser context viewport and device settings when captures need a known screen size or emulation. A context can contain multiple pages, and its viewport, locale, and other emulation settings apply to those pages; use separate contexts when cookies or storage must not leak between jobs. See Pages and contexts.
  • Choose PNG for lossless output, or JPEG/WebP and a quality setting when smaller files matter more than exact fidelity. The Page screenshot API documents format, quality, and scale options; verify option availability against the Playwright version installed in your project.
  • CSS scale produces one image pixel per CSS pixel; device scale captures device pixels and can produce larger files. Pick based on whether you need compact page previews or high-DPI artifacts.
  • For very long pages, decide whether full-page capture is worth the memory, time, and output size. If not, capture the viewport or a relevant element.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Make the batch repeatable and recoverable

For visual comparisons, keep the Playwright/browser version, operating system, headless mode, viewport, and other relevant settings consistent. Rendering can vary with browser and machine environment. Mask or hide dynamic content only when doing so does not remove information the screenshot is meant to preserve; custom screenshot styling and locator-based capture can help, but should not conceal meaningful differences. See Playwright visual comparisons.

Bound navigation with timeouts, log each URL independently, and retry only failures likely to be transient. A timeout caused by a permanently broken URL or an unsuitable readiness condition will not be fixed by repeated retries. Keep the original input-to-result mapping, and consider recording timestamps and retry counts if you need an auditable batch. These are operational practices, not guarantees of a particular retry policy or run time.

Rank #4
Webcam Cover for Logitech C920 C930e c922x Lens Privacy Shutter Slider
  • Compatible with Logitech C920x HD Pro Webcam, Full HD 1080p/30fps Video Calling. Compatible with Logitech C920 Hd Pro Webcam. Compatible with Logitech HD Pro Webcam C920 Widescreen Video Calling and Recording Webcam.
  • Compatible with Logitech C930e Webcam. Compatible with Logitech C922 Pro Stream Webcam 1080P Camera for HD Video Streaming. Compatible with Logitech Privacy Cover for C920 and C930e.
  • This webcam cover conveniently blocks your camera cover to protect your privacy.
  • This also compatible with other popular webcams. This is also known as webcam lid, webcam cap, webcam protector, web camera privacy cover.
  • ienza is a registered trademark and a registered Amazon brand. Use of the ienza trademark without the prior written consent of ienza, LLC. may constitute trademark infringement and unfair competition in violation of federal and state laws. ienza products are developed as cost-effective alternatives to OEM parts. They are not necessarily endorsed by the OEMs

Troubleshoot common batch failures

  • Navigation timeout: The site may be slow, the timeout too short, or the chosen wait condition too strict. Inspect the page and network behavior, adjust the timeout cautiously, or wait for a meaningful selector instead of network idleness.
  • Screenshot is blank or incomplete: The page may still be rendering application content after the selected navigation event. Wait for a content-specific selector or bounded delay, and verify that the page is not showing a bot check or an error state.
  • Memory use climbs or the process exits: Reduce concurrency, close pages after capture as the example does, and consider viewport rather than full-page screenshots. Avoid creating a page for every URL at once.
  • Files overwrite one another: Do not derive a filename from only a hostname or a sanitized URL. Preserve an index or unique hash, and ensure multiple workers cannot write the same path.
  • Cookies or account state appear in another capture: Reuse a context only when shared state is intended. Use separate contexts for jobs that require isolated cookies or storage; browser contexts provide that isolation boundary.
  • Visual diffs vary between runs: Standardize the browser version, OS/environment, headless setting, viewport, scale, and page state. Remote content and dynamic elements can still change independently.
  • Browser executable is missing: Install the browser binary for the Playwright package in use with npx playwright install chromium, and check the release notes for version-specific changes at Playwright release notes.

Or skip the browser setup

If you do not want to operate a browser queue, ScreenshotNeo is a website screenshot API and MCP server. One GET request returns an image or PDF, and its capture workflow accepts cookie/consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Only clean shots are billed: bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, with verdict and billing information in response headers. AI agents can use its MCP tools, including take_screenshot, get_page_info, and capture_pdf.

For a quick capture, create an API key and substitute it and your target URL in this cURL request. See the ScreenshotNeo API documentation for request options.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up for free.

Frequently Asked Questions

Can one Playwright browser process handle multiple pages at once?

Yes. A browser can host multiple pages; the practical limit depends on the workload and machine, so measure rather than treating a fixed page count as universal.

Does Playwright publish a recommended number of pages per batch?

The cited Playwright documentation explains worker configuration but does not establish a universal concurrency setting or throughput benchmark for arbitrary screenshot jobs.

Does a full-page screenshot include content loaded only after scrolling?

It captures the full scrollable document, but pages that load content lazily may need additional scrolling or a page-specific readiness step before capture.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

AutomationNode.jsPlaywrightscreenshotsWeb development

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.