To capture an infinite-scroll page without repeated sections, first make the page load the content you want, then verify that its rendered items are unique, and only then call page.screenshot({ fullPage: true }). Puppeteer’s full-page option captures the page’s current rendered extent; it does not scroll through an infinite feed or guarantee that the feed has finished loading.
How to capture an infinite-scroll page with Puppeteer
The right sequence is: navigate, wait for a page-specific starting signal, scroll the actual feed container in bounded steps, check whether new unique items arrived, and take one screenshot after the content and layout settle. Puppeteer provides the browser-control primitives; the selectors, scroll logic, and stopping condition must match the target site.
As an Amazon Associate I earn from qualifying purchases.
Puppeteer’s screenshot guide recommends Page.screenshot() for screenshots. The ScreenshotOptions API defines fullPage: true as capturing the full page; the default is false. Neither setting means “scroll until the infinite feed ends.”
Runnable control-flow example
Install Puppeteer in your project with npm install puppeteer. Save this as an ES module, such as screenshot.mjs, then run node screenshot.mjs https://example.com/feed. Replace the example selectors and ID attribute with those used by the page. The bounded loop is a defensive pattern, not a universal or officially prescribed infinite-scroll algorithm.
#1 Best Overall
import puppeteer from 'puppeteer';
const url = process.argv[2];
if (!url) throw new Error('Usage: node screenshot.mjs <url>');
const ITEM = '[data-feed-item]';
const MAX_SCROLLS = 30;
const STABLE_ROUNDS_REQUIRED = 3;
const SETTLE_MS = 800;
const browser = await puppeteer.launch({ headless: true });
try {
const page = await browser.newPage();
await page.goto(url, { waitUntil: 'networkidle2' });
await page.waitForSelector(ITEM);
let previousUniqueCount = 0;
let stableRounds = 0;
for (let i = 0; i < MAX_SCROLLS && stableRounds < STABLE_ROUNDS_REQUIRED; i++) {
await page.evaluate(() => {
window.scrollTo(0, document.documentElement.scrollHeight);
});
// Replace this with a page-specific wait if the feed exposes one.
await new Promise(resolve => setTimeout(resolve, SETTLE_MS));
const uniqueCount = await page.$$eval(ITEM, items => {
const keys = items.map(item =>
item.getAttribute('data-id') || item.textContent.trim()
);
return new Set(keys).size;
});
stableRounds = uniqueCount === previousUniqueCount ? stableRounds + 1 : 0;
previousUniqueCount = uniqueCount;
}
const counts = await page.$$eval(ITEM, items => {
const keys = items.map(item =>
item.getAttribute('data-id') || item.textContent.trim()
);
return { rendered: keys.length, unique: new Set(keys).size };
});
console.log('Feed items:', counts);
await page.screenshot({ path: 'page.png', fullPage: true });
} finally {
await browser.close();
}This example scrolls the document, waits a fixed settling interval, and stops after three rounds without an increase in unique items or after 30 scrolls. Those values are adjustable safeguards, not proof of completion. Prefer a site’s explicit end marker or a specific content-change condition when available. Puppeteer documents evaluate(), waitForFunction(), and network-idle waiting as page APIs, but does not prescribe this loop or guarantee that it detects a finished feed (Page API).
Adapt the scroll and completion checks to the page
Find the real scroll container
Some feeds scroll the document; others load content inside a nested element with its own scrollbar. If the feed is nested, scrolling window may never trigger its loading handler. Identify the element whose scrollTop changes as you scroll, then update that element in page.evaluate() and measure its scrollHeight or feed-item count. The page’s own intersection observers or event handlers need an opportunity to run.
Use a meaningful stopping condition
- End marker: Stop when a page-specific “end of results” element appears, if the site exposes one.
- Unique-item count: Compare stable item identifiers across rounds. A count of DOM nodes alone can rise because duplicates were appended.
- No-change threshold: If there is no explicit marker, stop after a chosen number of rounds without new unique items, while retaining a maximum-scroll limit.
- Network idle: Use it as a possible settling signal, not as proof the feed is complete. Another scroll may be needed to trigger the next request.
For a page-specific predicate, Puppeteer’s waitForFunction() can wait until an observable condition becomes true. Avoid waiting only for “at least one item”: once the first item exists, that condition may already be true before the next batch arrives. The relevant tools are documented in the Page API.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
Why a screenshot may repeat sections
Check the rendered page before changing screenshot settings. If the DOM already contains repeated item IDs or content, the site’s feed-rendering behavior is producing duplicates; fullPage does not remove them. If the DOM is correct but the image appears repeated, investigate whether the feed had finished updating, whether you scrolled the right container, and whether sticky or fixed page elements are being mistaken for repeated content.
Puppeteer’s documentation defines screenshot capture and full-page extent, but does not establish a universal cause of repeated sections or a built-in deduplication option (ScreenshotOptions; Page API). Compare the item identifiers in the DOM with the screenshot to isolate whether repetition starts in the page or in the capture/layout outcome.
Virtualized feeds and choosing the output
A virtualized feed may remove off-screen items from the DOM as new ones appear. In that case, reaching the end does not mean earlier items remain available for one full-page capture. A full-page screenshot should not be assumed to contain content the application no longer renders.
- If the full feed remains in the DOM, scrolling to completion and taking one full-page capture may suit the task.
- If the feed is virtualized, consider capturing viewport-sized screenshots as you scroll and combining them only if the page layout makes that reliable, or use the site’s underlying paginated data/export path where permitted.
- If content is loaded by cursor or “load more” controls, use the site’s actual progression mechanism and verify each new batch rather than relying on document height alone.
Capture after content and layout stabilize
Once the feed has reached the stopping condition and the item IDs have been checked, take the screenshot with:
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Rank #3
await page.screenshot({ path: 'page.png', fullPage: true });That call captures the page as rendered at that point. For screenshot operations generally, Puppeteer documents coordination intended to prevent certain BrowserContext operations from interfering with a screenshot in progress (Page.screenshot() API); it does not define how a site stores or updates its feed records.
Troubleshooting
The script captures only the first batch
Confirm that the correct scroll container is being moved and that the site loads content from scrolling rather than a button or cursor request. Replace generic waits with a page-specific condition that can change after the next batch arrives.
The loop stops while more content exists
A temporary pause may leave the unique-item count unchanged even though a request is still in flight. Increase the settling interval or wait for a known loading indicator to disappear or a known item count to increase. Do not treat network inactivity as a universal end-of-feed signal.
The loop never stops
Keep a hard maximum iteration count. Infinite feeds may continuously append content, mutate identifiers, or show changing elements that make an overly broad condition appear to change forever. Prefer stable item IDs and a real end marker.
The duplicate count is misleading
Text content may not uniquely identify records: two legitimate posts can have identical text, while one record may change its text after rendering. Prefer a stable data ID or link URL. If the site exposes no stable key, treat text-based uniqueness as a diagnostic approximation, not a definitive deduplication test.
The screenshot omits earlier content
Check whether the application virtualizes the feed and removes off-screen items from the DOM. A full-page screenshot cannot capture records that are no longer rendered. Use a viewport-by-viewport capture workflow or another page-supported export strategy where appropriate.
Best Value
The screenshot looks repeated although the DOM does not
Inspect the actual rendered image and review fixed headers, sticky elements, and capture timing. Puppeteer’s documented fullPage option sets capture extent; the documentation does not promise to normalize sticky elements or diagnose repeated visual bands.
Or skip the browser setup
ScreenshotNeo is a website screenshot API and MCP server. For a single request, it accepts a URL and returns an image or PDF. Example cURL request:
Free tools Windows power users keep installed
One-click scans. No signup required.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/feed -o shot.webpSee the ScreenshotNeo API documentation for request options. The API can capture full pages and can wait for a selector, a delay, or network idle, but an infinite feed still needs page-specific handling and validation: a full-page request should not be taken as proof that every item has loaded or that the site has no duplicate records.
- Cookie banners are accepted and more than 60 known consent platforms, newsletter popups, and chat widgets are removed before capture; each step can be turned off.
- Bot checks/CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing; response headers identify the page verdict and billing status.
- An MCP server provides
take_screenshot,get_page_info, andcapture_pdftools for AI agents and MCP clients. - The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000.
Sign up for ScreenshotNeo’s free plan to get 1,000 screenshots a month with no card.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
