Recommended Free Tools
To capture infinite-scroll results without duplicates in Puppeteer, collect each rendered batch, derive a stable key for every result, and keep a persistent Set in Node.js. After each batch, scroll the page’s actual results container and wait for a page-specific progress signal; stop at an end marker or after a bounded number of rounds with no new results. The selectors and wait condition must be adapted to the site you are automating.
Build the collection loop around progress
Infinite-scroll pages may retain earlier cards as new ones load, so taking a final snapshot can include overlap. Others virtualize their lists by removing cards that have scrolled out of view. Collect and persist each batch as you go, and keep deduplication state outside the page snapshot.
As an Amazon Associate I earn from qualifying purchases.
- Inspect the page: identify the result-card selector, fields to extract, stable ID if available, actual scroll container, loading indicator, and any explicit end marker or pagination cursor.
- Read the currently rendered cards into plain data such as ID, title, and URL.
- Normalize a key consistently: prefer a stable result ID supplied by the page; otherwise use a canonicalized URL and check whether distinct results can share it.
- Append only unseen records: store keys in a persistent
Setand records in an array. - Trigger the next batch: scroll the document or the nested results panel, or use the site’s load-more control.
- Wait for an observable change: for example, an increased card count, changed cursor, cleared loading indicator, or end marker.
- Bound the loop: stop at a reliable end marker, after a chosen number of no-new-item rounds, or when a maximum round or duration limit is reached.
This is a general pattern, not a tested script for a particular search site. Its selectors, key, scrolling mechanics, and progress signal need to match the target page.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsExtract and deduplicate each batch
Puppeteer’s page.$$eval() runs a function over all elements matching a selector and is convenient for extracting multiple result cards. The example below assumes the page exposes data-result-id; replace every selector and attribute with ones verified on the target site.
#1 Best Overall
- Plug-and-Play USB Camera Module: Experience ultimate convenience with our plug-and-play USB camera module. This 8MP camera is instantly recognized by Windows, Linux, Android, and macOS without any extra drivers. Just connect the USB and immediately start capturing crisp images, making it a perfect mini USB camera for rapid deployment in any project
- AI Resolution for Advanced Applications: Leverage multiple preset AI image resolutions to train and deploy your models seamlessly. This USB webcam and 3D printer camera eliminates the need for manual image cropping, delivering ready-to-process data straight from the sensor. It’s an ideal vision solution for developers and makers
- Autofocus & High-Definition Clarity: Equipped with a premium autofocus lens, this 4K mini camera automatically adjusts to maintain sharpness at various distances. Whether you’re using it as a lightburn camera for laser engraver or for detailed inspection, it delivers consistently clear and professional USB camera 4K quality video
- Robust & Reliable USB Security Camera: Built for durability and performance, this USB security camera offers steadfast monitoring with high-resolution imaging. Its versatile mounting and plug-and-play operation make it suitable for both home security setups and professional surveillance systems
- Upgraded Option with HDR: The enhanced model includes High Dynamic Range (HDR), an autofocus lens, and a rugged metal case. This upgraded USB camera module is especially suited for demanding applications like laser engraving with LightBurn or as a high-end 3D printer camera
const RESULT_SELECTOR = '.search-result';
const TITLE_SELECTOR = '.result-title';
const END_SELECTOR = '.end-of-results';
const seen = new Set();
const results = [];
function canonicalize(rawUrl) {
if (!rawUrl) return '';
try {
const url = new URL(rawUrl);
url.hash = '';
// Remove tracking parameters only when they are known not to identify results.
for (const key of [...url.searchParams.keys()]) {
if (/^(utm_|gclid$|fbclid$)/i.test(key)) url.searchParams.delete(key);
}
url.searchParams.sort();
return url.href;
} catch {
return '';
}
}
async function collectBatch(page) {
return page.$$eval(RESULT_SELECTOR, cards => cards.map(card => ({
id: card.getAttribute('data-result-id')?.trim() || '',
title: card.querySelector('.result-title')?.textContent?.trim() || '',
url: card.querySelector('a[href]')?.href || '',
})));
}
function addUnseen(batch) {
let added = 0;
for (const item of batch) {
const key = item.id || canonicalize(item.url);
if (!key || seen.has(key)) continue;
seen.add(key);
results.push(item);
added++;
}
return added;
}
Keep the Set and result array alive across all rounds. A title is usually a weaker key because different results can have the same title. A URL can also be insufficient if one URL represents multiple records or if meaningful query parameters are discarded. Validate the key against the target site before relying on it.
Wait for a real change before continuing
Fixed sleeps can be useful as a fallback, but they do not prove that new results arrived: slow pages may need longer, while fast pages waste the remaining delay. Puppeteer’s page.waitForFunction() resolves when its predicate becomes truthy; its polling can use animation frames, DOM mutations, or a numeric interval. The documented default timeout is 30 seconds and can be changed. Choose a signal that actually indicates progress on this page.
Rank #2
- ENGINEERED FOR AI & MOBILITY - Meet the Dell Pro 16 Plus, the AI-enhanced evolution of the Latitude 5550. Engineered for on-the-go productivity, it features a slim and lightweight design, delivers up to 11.9 hours of battery life, and supports ExpressCharge capability to keep you efficient. Boasting a durable aluminum chassis and having passed MIL-STD 810H tests, it offers robust reliability for professionals on the move, from the office to demanding field environments
- POWERFUL PERFORMANCE – The Dell Pro 16 Plus delivers power-efficient performance for demanding workloads with an AI PC powered by the AMD Ryzen AI 7 PRO 350 processor (up to 5.0GHz) and integrated Radeon 860M Graphics. Equipped with 32GB LPDDR5x RAM and 2TB M.2 NVMe PCIE SSD, enabling smooth multitasking and fast loading across a wide range of applications
- COPILOT+ PC AI POWERHOUSE - The dedicated NPU delivers 50 TOPS for local AI processing without relying on the cloud. It enables Recall (effortless retrieval of past actions and content), Cocreate (AI image tools), Windows Studio Effects (auto-framing/background blur for video calls), and Live Captions (real-time translation). It redefines productivity and creativity with seamless, offline AI acceleration
- IMMERSIVE DISPLAY - Features a 16-inch WUXGA (1920x1200) display with narrow borders, 300 nits brightness, and anti-glare coating to maximize screen real estate and reduce eye strain during extended use. Expand your workspace by connecting up to 3 external monitors via HDMI or Thunderbolt 4, with a max resolution of up to 4K@60Hz without docking station
- ADVANCED CONNECTIVITY -With Thunderbolt 4, USB-A, and HDMI 2.1, MicroSD card reader, Global Headset Jack and RJ45 Ethernet port, you can easily connect external displays, storage devices, and essential peripherals. Stay fast and reliable on the go with Wi-Fi 7 and Bluetooth 5.4, perfect for video calls, cloud work, and wireless devices without lag. The 1080p IR camera with temporal noise reduction ensures crisp video calls in any lighting and secure facial recognition login. Plus, the backlit keyboard enables precise typing in low-light environments
const previousCount = await page.$$eval(RESULT_SELECTOR, cards => cards.length);
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
await page.waitForFunction(
({ selector, previous }) =>
document.querySelectorAll(selector).length > previous ||
document.querySelector('.end-of-results') !== null,
{ timeout: 15000 },
{ selector: RESULT_SELECTOR, previous: previousCount },
);
The example waits for the rendered count to increase or for an end marker to appear. It works only if the page retains old cards and those selectors represent meaningful state. For virtualized lists, count may stay constant while visible records change; wait for a changed cursor, last visible key, or another site-specific marker instead. A timeout should lead to a deliberate retry or stop decision, not an unbounded loop.
Free tools Windows power users keep installed
One-click scans. No signup required.
Use a bounded loop and the correct scroll target
This loop illustrates the control flow for a page whose document is the scrolling surface and whose result count increases as cards are appended. Adjust its selectors, wait signal, maximum rounds, and no-progress policy to the page’s behavior.
Rank #3
- Size: 5" x 4.6"
- Al weather vinyl sticker
- Phone sticker, laptop sticker, car sticker, water bottle sticker, and so many more applications!
- Peel & stick, simple application, reusable
- Made in the USA
const MAX_ROUNDS = 100;
const MAX_NO_PROGRESS = 3;
let noProgressRounds = 0;
for (let round = 0; round < MAX_ROUNDS; round++) {
const batch = await collectBatch(page);
const added = addUnseen(batch);
noProgressRounds = added ? 0 : noProgressRounds + 1;
if (await page.$(END_SELECTOR)) break;
if (noProgressRounds >= MAX_NO_PROGRESS) break;
const previousCount = batch.length;
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
try {
await page.waitForFunction(
({ selector, previous, endSelector }) =>
document.querySelectorAll(selector).length > previous ||
document.querySelector(endSelector) !== null,
{ timeout: 15000 },
{ selector: RESULT_SELECTOR, previous: previousCount, endSelector: END_SELECTOR },
);
} catch (error) {
// A timeout is not proof of completion. Let the bounded no-progress rule decide.
if (error.name !== 'TimeoutError') throw error;
}
}
console.log(`Collected ${results.length} unique results`);
For a nested scrolling panel, scroll that element instead of the window. One option is to find and scroll the verified panel in the page context:
await page.evaluate(() => {
const panel = document.querySelector('.results-panel');
if (!panel) throw new Error('Results panel not found');
panel.scrollTop = panel.scrollHeight;
});If the page uses a load-more button, Puppeteer recommends locators for selection and interaction because locator actions wait for the element to be present and ready. Replace the scroll step with a locator for the inspected button, then wait for a page-specific change.
Rank #4
- For programmers and web developers who have a sense of gothic macabre about them. Perfect for coding meetups, gaming sessions, or casual outings. Do you live for code? Are you a programmer, IT professional or developer who is constantly coding?
- Web Developer Coding Skeleton In Front of Laptop Halloween. Perfect for dark mode developers, programmers, software engineers, anyone in tech with a dark side who lives at their computer. Great for Halloween or the rest of the year.
- Lightweight, Classic fit, Double-needle sleeve and bottom hem
Choose APIs and signals for the page’s behavior
- Read many cards: use
page.$$eval()to map all currently matching elements to plain data. - Inspect broader page state: use
page.evaluate(), which executes in the page context and can await a returned promise. - Interact with controls: prefer Puppeteer locators for actions such as clicking a load-more button. A lower-level
waitForSelector()waits for a selector but does not automatically retry a failed action. - Wait on a condition: use
waitForFunction()when a meaningful predicate can describe progress, and set a timeout appropriate to the page. - Pick a progress marker: count, cursor, visible last-result key, loading state, or end marker can work, but only if that page updates it reliably.
The official Puppeteer guide says, “Locators is the recommended way to select an element and interact with it.” See the Page interactions guide, Page.evaluate() reference, Page class reference, waitForFunction options, and Locator class reference. The reviewed reference pages identify Puppeteer versions 25.12.0 and 25.3.0; check the current API reference if you use a later version.
Handle common failure modes
- Repeated records: batches overlap because earlier cards remain in the DOM. Deduplicate every batch against the same persistent key set.
- Unstable or colliding keys: titles may repeat and URLs may carry tracking parameters—or may not uniquely identify an item. Prefer a site-provided ID and validate URL normalization against real records.
- Scrolling appears to do nothing: the page may use an inner overflow container. Inspect scrollable ancestors and operate on the results panel rather than
window. - Loop stops too early: a delayed response can resemble completion. Wait for a page-specific loading or progress condition and give it a bounded retry window; do not treat one timeout alone as an end marker.
- Loop never stops: require an explicit end condition where available and enforce maximum rounds or elapsed time plus a no-new-unique-results limit.
- Final scrape misses earlier results: the page may virtualize rows and remove them from the DOM. Persist each extracted batch immediately instead of scraping only at the end.
- Wait times out despite new items: the chosen signal may be unsuitable—for example, a virtualized list may keep a constant card count. Wait for a cursor or visible-item key change instead.
Or skip the browser setup
If your goal is a screenshot rather than extracting structured search results, ScreenshotNeo is a website screenshot API with a single GET request. It accepts consent banners like a visitor and removes 60+ known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, and failed loads are not billed, and cache hits cost nothing. Responses report the page verdict and billing status in headers. Its MCP server gives AI agents tools to take screenshots, get page information, and capture PDFs.
For example, the call below captures a screenshot of the target URL; it does not extract a structured list of search-result records.
Best Value
- The Ultimate Developer Humor: Celebrate the fuel behind your best lines of code with this "I Turn Coffee Into Code" sticker. It is a must-have accessory for software engineers, web developers, data scientists, and computer science students.
- Premium Waterproof & Heat-Resistant: Crafted from high-quality, durable vinyl that is 100% waterproof and heat-resistant. Perfect for sticking on high-performance laptops, coffee tumblers, or office water bottles without worrying about peeling or fading.
- Sleek Professional Design: Featuring a bold black and white aesthetic with a clean coffee cup icon, this die-cut decal looks professional and stylish on MacBooks, PC cases, and office monitors.
- Easy Application, Zero Residue: Equipped with a strong adhesive that stays put through daily wear. If you upgrade your hardware, it peels off cleanly without leaving any sticky mess or residue behind on your expensive electronics.
- Perfect Tech Gift: Looking for a great gift for a programmer, IT professional, or coding student? This decal makes an excellent stocking stuffer, "new job" gift, or secret santa present for your tech-savvy coworkers.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo API documentation for request options. The Free plan includes 1,000 shots a month with no card; paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month with no card.
Frequently Asked Questions
Does deduplicating in Puppeteer remove duplicate results from the website?
No. It prevents your collector from appending the same keyed record more than once; it does not change the page’s results.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchCan I use this loop unchanged on any infinite-scroll site?
No. The result selectors, stable key, scroll target, progress signal, and end condition must be identified for the specific page.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
