The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →To capture infinite-scroll search results with Puppeteer, first load results in a bounded loop: scroll the element that actually owns scrolling, wait for a result-count increase or an end marker, and stop when the feed ends or makes no progress. Then call page.screenshot(). fullPage: true captures what is already rendered; it does not load the rest of the feed for you.
What you need to identify before writing the loop
Infinite-scroll behavior is site-specific. Before automating it, inspect the search page and determine the result selector, the scroll owner, and a signal that means either new results arrived or the feed is finished. There is no universal search-results selector or completion rule.
As an Amazon Associate I earn from qualifying purchases.
- Results: a selector whose matching elements represent individual rendered results.
- Scroll owner: either the document or a nested results panel. Scrolling the wrong element may do nothing.
- Progress or completion: preferably a changed result count or a visible end-of-results marker. A loading indicator transition or a site-specific response can also help, if it is directly tied to rendered results.
- Bounds: a maximum number of rounds, a timeout, and a no-progress exit to prevent an endless loop if the feed, selector, or network fails.
Load results, then take a screenshot
The following illustrative ES module shows the control flow. Replace the example URL and selectors with values verified on the target site. It is not tested against a particular search page, and the five-second wait is only a fallback—not proof that the feed is complete.
Recommended Free Tools
import puppeteer from 'puppeteer';
const searchUrl = 'https://example.com/search?q=puppeteer';
const resultSelector = '.search-result';
const endMarkerSelector = '.end-of-results';
const scrollContainerSelector = null; // null means the document
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
await page.goto(searchUrl, { waitUntil: 'domcontentloaded' });
let stalledRounds = 0;
const maxStalledRounds = 3;
const maxRounds = 40;
for (let round = 0; round < maxRounds && stalledRounds < maxStalledRounds; round++) {
const before = await page.evaluate(selector =>
document.querySelectorAll(selector).length,
resultSelector,
);
if (await page.evaluate(selector => Boolean(document.querySelector(selector)), endMarkerSelector)) {
break;
}
await page.evaluate(containerSelector => {
const container = containerSelector
? document.querySelector(containerSelector)
: document.scrollingElement;
if (!container) throw new Error('Scroll container not found');
container.scrollBy(0, Math.max(300, container.clientHeight * 0.8));
}, scrollContainerSelector);
try {
await page.waitForFunction(
(resultSel, endSel, previousCount) =>
document.querySelectorAll(resultSel).length > previousCount ||
Boolean(document.querySelector(endSel)),
{ timeout: 5000 },
resultSelector,
endMarkerSelector,
before,
);
} catch {
// A timeout is not confirmation that all results have loaded.
}
const after = await page.evaluate(selector =>
document.querySelectorAll(selector).length,
resultSelector,
);
const ended = await page.evaluate(selector => Boolean(document.querySelector(selector)), endMarkerSelector);
if (ended) break;
stalledRounds = after > before ? 0 : stalledRounds + 1;
}
await page.screenshot({ path: 'search-results.png', fullPage: true });
} finally {
await browser.close();
}The loop checks for an end marker before scrolling, then waits for a result-count increase or that marker. It rechecks progress after the wait and stops after three consecutive rounds without additional results, or after 40 total rounds. Those bounds are example choices, not universal settings: adjust them to the target feed and the amount of content you intend to capture. If the page exposes no reliable end marker, a no-progress limit can keep the script bounded, but it cannot prove that every possible result has loaded.
#1 Best Overall
- [INTEL POWERED CONTENT] - Built with a 8th Generation Hexa-Core Intel i5 and 32GB of DDR4 RAM; Modern, Windows 11 ready, with 4K support, Executive multitasking, media streaming and smooth, multi-tab web browsing; Perfect as an all-purpose multimedia computer; built for content creators; Plenty of RAM and Mass storage for photo and video editing powered by Intel HD 630
- [LATEST WIRELESS TECH] - This Dell Desktop Computer easily connects to the internet through the Built In WiFi / Bluetooth
- [SOLID STATE STORAGE] - This Dell Computer setup comes with an ultra-fast 1TB Solid State Drive (SSD); Setup as the primary boot device; Boot and load programs with lightning speed ; Additional expansion available
- [BUY & OWN WITH CONFIDENCE] - From the world's largest Microsoft Authorized Refurbisher; Quality Guarantee and Free Tech Support; Award-winning Customer Service; | Support Sustainable Business
- [MODERN HI-SPEED PORTS] - USB 3.0 (x4) | USB 2.0 (x4) | DisplayPort (x1) | HDMI Port (x1) | Audio Combo Jack (x1) | Audio Out (x1) | RJ-45 Ethernet (x1) | Internal SATA (x3)
Use a nested results panel when that is what scrolls
If results live inside a panel with its own scrollbar, set scrollContainerSelector to that panel’s selector. Scrolling document.scrollingElement will not trigger a panel-driven feed. Some pages listen specifically for user-like wheel events rather than changes to scroll position; Puppeteer’s interaction guide describes locator scrolling through mouse-wheel events, which can be a better fit for those pages. See the Puppeteer page-interactions guide.
Choose the right wait condition
page.waitForFunction() waits until a function evaluated in the page context returns a truthy value. Its predicate should observe the thing you care about, such as a result count becoming larger or an end marker appearing. Puppeteer documents that page.evaluate() runs code in the page context and waits for a returned Promise to resolve; that makes it useful for browser-side checks and scrolling logic. See the Page.evaluate() API and Page.waitForFunction() API.
Rank #2
- Model: Dell OptiPlex 7050 Small Form Factor (SFF)
- Processor: Intel Core i7-7700 3.60 GHz
- Memory: 32GB DDR4 Ram
- Storage: 1TB Solid State Drive (SSD) Fast Boot + Storage
- Operating System: Windows 11 Pro (64-bit)
Network quietness is not a semantic completion signal. A feed can pause between batches, and a request can finish before newly fetched results are rendered. Puppeteer’s waitForNetworkIdle() waits for network activity to be idle for at least its configured idle interval; combine it with a content predicate if it helps settle a page, but do not treat it alone as proof that all search results are present. See the Page.waitForNetworkIdle() API.
A timeout means the expected predicate did not become true within the allotted time. Recheck the selector and current count; do not silently equate a timeout with reaching the end. The example catches the timeout only so it can inspect progress and apply its bounded stop condition.
Rank #3
- IMMERSIVE 24 INCH DISPLAY: Experience stunning clarity on a Full HD IPS screen with ultra-thin bezels, offering a 90% screen-to-body ratio that makes everything from spreadsheets to streaming come alive with vibrant colors and crisp details.
- POWERFUL INTEL PROCESSING: Tackle demanding tasks with ease thanks to the Intel processor and 16GB of high-speed memory, delivering smooth performance whether you're multitasking between applications or running productivity software.
- GENEROUS STORAGE: Store all your important files, photos, and programs with blazing-fast solid state drive technology that ensures quick boot times, rapid file access, and plenty of space for your digital life.
- ENHANCED PRIVACY AND COLLABORATION: Work confidently with the pop-up privacy camera that tucks away when not in use, plus dual microphones with noise reduction for crystal-clear video calls that keep you connected professionally.
- ECO-CONSCIOUS DESIGN: Feel good about your purchase with an EPEAT Gold registered and ENERGY STAR certified computer that combines premium performance with responsible environmental manufacturing practices.
Capture a full page, a panel, or separate segments
page.screenshot() captures the page, and ElementHandle.screenshot() can capture a particular element. A full-page screenshot requests an image of the document’s full page height, but it only includes content present when the capture happens. Puppeteer’s documented fullPage default is false. See the Page.screenshot() API and ElementHandle.screenshot() API.
- One tall image: use
await page.screenshot({ path: 'results.png', fullPage: true })after loading the intended results. - One results element: locate the relevant element and use its screenshot method when you want the panel rather than the entire page.
- Very long feed: capture deliberate viewport-sized segments if a single tall image is unwieldy. Keep track of each segment and its scroll position to avoid accidental gaps or duplicated content.
Troubleshoot common failures
| Symptom | Likely cause | What to check or change |
|---|---|---|
| The screenshot contains only the first batch | fullPage was mistaken for a feed-loading command, or the loop did not scroll the actual owner. |
Load results before capture; inspect whether the document or a nested panel scrolls, and target that element. |
| The loop stops before later results appear | The selector does not match each result, the wait predicate watches the wrong signal, or a batch takes longer than the fallback wait. | Verify the selector against the live DOM and wait for an observable count change or site-provided completion state. Treat timeouts as inconclusive. |
| The loop never finishes | The feed has no end marker, the selected marker never appears, or results keep arriving. | Keep a maximum-round and no-progress limit. If the feed is continuously changing, define a capture endpoint rather than trying to exhaust it. |
| Scrolling has no visible effect | The wrong element is being scrolled or the page reacts to wheel events rather than programmatic scroll changes. | Identify the actual scroll container and consider Puppeteer’s locator-based wheel scrolling described in its interaction guide. |
| Network idle occurs but results are missing | Network quietness was treated as feed completion. | Wait for rendered-result progress or an explicit end marker; network idle can only supplement that content signal. |
| The final image is unexpectedly huge | A full-page capture includes the full rendered document height. | Capture the results element or use viewport-sized segments instead of one image. |
Or skip the browser setup
ScreenshotNeo provides a screenshot API and MCP server. One request returns an image or PDF; its cleanup steps can accept cookie or consent banners and remove known consent platforms, newsletter popups, and chat widgets before capture. Those steps can be turned off. Bot checks/CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP tools—take_screenshot, get_page_info, and capture_pdf—let AI agents use the service.
For a basic capture, save this as capture.sh and replace the URL and key. See the ScreenshotNeo API documentation for request options. This is a one-call capture, not an infinite-feed completion loop: it does not establish that a particular search site has loaded every result.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Rank #4
- This Certified Refurbished product is tested and certified to look and work like new. The refurbishing process includes functionality testing, basic cleaning, inspection, and repackaging. The product ships with all relevant accessories, a minimum 90-day warranty, and may arrive in a generic box. Only select sellers who maintain a high-performance bar may offer Certified Refurbished products on Amazon.com.
- Dell Optiplex 3050 SFF Desktop computer PC, Intel Quad Core i5-6500 up to 3.6GHz, 16GB DDR4, 256GB SSD
- Includes: USB Keyboard & Mouse, USB WiFi adapter, Microsoft office 30 days free trail.
- Port: Front: USB 3.0(2), USB 2.0(2); Rear: DP, HDMI, USB 3.0(2), USB 2.0(2), RJ-45.
- Support 4K (3840x2160) Dual display, makes it easy to connect two monitors at the same time, and you can expand working Windows, mirror content, or expand a single window across multiple monitors.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/search?q=puppeteer -o shot.webpThere is a free allowance of 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Sign up for ScreenshotNeo’s free plan.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Frequently Asked Questions
Does Puppeteer have a built-in option to load every infinite-scroll result?
No. The script must interact with the target page and decide from its behavior when to stop.
Best Value
- Connectivity: Includes WiFi, Bluetooth, and LAN for wireless and wired connections
- Memory: Features 16GB DDR4 RAM for smooth multitasking and performance
- Storage: Combines 500GB SSD and 1TB HDD for ample storage space
- Graphics: Integrated Intel UHD Graphics 630 for crisp visuals and video playback
- Design: Sleek desktop tower with black color and slim profile for modern look
Can a full-page screenshot include content inside a scrollable results panel?
It may not capture the panel’s entire scrollable contents as one complete list; capture the panel deliberately or scroll and record segments when needed.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
