Playwright se infinite-scroll catalog ki poori product list capture karne ke liye ek product card ko locator se identify karein, page ya catalog ke asli scroll container ko baar-baar scroll karein, aur har round mein dikh rahe products ko DOM se nikaal kar stable ID ya product URL se deduplicate karein. Explicit “end of results” marker ya known total mile to use completion signal banayein; warna site ke hisaab se bounded no-growth rule lagayein aur natije ko verify karein. Sirf ek baar DOM padh lena poori list milne ka saboot nahin hai.
Kaunsi strategy kaam karti hai?
Infinite scroll mein agle products scroll karne par load ho sakte hain. Isliye capture ko rounds mein chalayein: scroll karein, site ka loading signal poora hone ka wait karein, visible cards se fields extract karein, aur unhe memory mein jama karein. Agle round mein phir wahi karein. Playwright ke scrolling primitives aur locators is workflow ke liye hain; selectors aur sahi completion signal har site ke markup par nirbhar karte hain.
As an Amazon Associate I earn from qualifying purchases.
Virtualized catalog mein purane cards scroll karte hi DOM se hat sakte hain. Aise mein end par ek hi baar DOM padhne se pehle dekhe gaye products chhoot sakte hain. Har round ke records ko DOM se alag jama karein.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsJavaScript mein products ko rounds mein capture karein
Neeche ka code ek general pattern hai, kisi khaas store par test kiya hua script nahin. data-testid selectors, end-marker text, aur wait logic ko target page inspect karke badlein.
#1 Best Overall
const cards = page.locator('[data-testid="product-card"]');
const seen = new Map();
let unchangedRounds = 0;
const maxUnchangedRounds = 3; // Site ke loading behavior ke mutabik tune karein
for (let round = 0; round < 100 && unchangedRounds < maxUnchangedRounds; round++) {
// Agar catalog apne panel mein scroll karta hai, window ke bajay us panel ko scroll karein.
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
// Yeh bounded fallback hai. Jahan mumkin ho, site-specific signal ka wait karein.
await page.waitForTimeout(500);
const batch = await cards.evaluateAll(nodes => nodes.map(card => ({
title: card.querySelector('[data-testid="product-title"]')?.textContent?.trim() ?? '',
price: card.querySelector('[data-testid="price"]')?.textContent?.trim() ?? '',
url: card.querySelector('a')?.href ?? ''
})));
const before = seen.size;
for (const item of batch) {
const key = item.url || item.title;
if (key) seen.set(key, item);
}
unchangedRounds = seen.size === before ? unchangedRounds + 1 : 0;
// Agar site explicit end marker deti hai, to use fallback se behtar signal banayein.
// if (await page.getByText('End of results').isVisible()) break;
}
const products = [...seen.values()];
console.log(`Captured ${products.length} unique products`);
console.log(products);Selectors ko target page se match karein
[data-testid="product-card"] aur andar ke title aur price selectors illustrative hain. Browser inspector ya Playwright locator se sabse chhota, stable card locator dhoondhein; phir title, price, image aur product link ko us card ke andar scope karein. Site agar accessible roles ya explicit test contracts deti hai, to unse bane locators ko tarjeeh dein.
Har card ke andar ka pehla a hamesha product page nahin hota. Sahi link selector chun kar URL ko canonicalize karna pad sakta hai. Product URL ya product ID aam taur par title se behtar deduplication key hai. Sirf title available ho to alag variants ek jaise naam rakh sakte hain; SKU ya variant ko key mein shamil karein.
Wait ko signal ke saath jodein
Fixed waitForTimeout(500) sirf bounded fallback hai: slow response mein yeh chhota pad sakta hai aur fast response mein bekaar waqt leta hai. Behtar hai us site ka signal observe karein—jaise loading spinner ka gayab hona, card count badhna, “load more” control ka badalna, ya end marker dikhna. Sirf networkidle ko universal proof na samjhein; Playwright actions aam taur par auto-wait karte hain aur general readiness ke liye web assertions ko tarjeeh di jaati hai.
Free tools Windows power users keep installed
One-click scans. No signup required.
Completion rule ko soch-samajhkar chunein
- Explicit end marker: “End of results” jaise visible indicator par rukna achha signal hai, lekin pehle confirm karein ki woh sach mein catalog ke ant ko darshata hai.
- Known total: Agar page total result count batata hai, unique records ko usse milayein. Filters, region ya variants ke kaaran total alag ho sakta hai.
- No-growth fallback: Code mein teen unchanged rounds aur 100-round cap sirf examples hain, universal values nahin. Apne catalog ke loading behavior ke mutabik tune karein. Count na badhna stalled request, rate limit, ya virtualization ki wajah se bhi ho sakta hai—yeh apne aap mein end ka saboot nahin.
Page scroll karein ya catalog container?
Wahi element scroll karein jiska scroll position catalog ko aage badhata hai. Agar poora document move hota hai, window scroll karein. Agar catalog ek fixed-height panel ke andar hai, us panel ko scroll karna hoga; window scroll karne se naye cards load nahin honge.
Panel ka selector dekh kar uske scroll position ko badla ja sakta hai:
const catalog = page.locator('[data-testid="catalog-scroll-container"]');
await catalog.evaluate(el => { el.scrollTop = el.scrollHeight; });Selector ko inspect karke badlein. Playwright ka doosra documented tareeqa panel par hover karke mouse-wheel input bhejna hai:
await page.getByTestId('catalog-scroll-container').hover();
await page.mouse.wheel(0, 900);Agar scroll ke baad card count ya position nahin badalta, confirm karein ki sahi container target hua aur page par consent overlay ya koi doosra blocking layer to nahin hai.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Records ki completeness kaise jaanchein
- Har round mein visible card count, unique-record count, scroll position aur loading/end signal log karein. Isse premature stop ko pehchanna aasan hota hai.
- Records ko har round mein memory mein jama karein; yeh maan kar na chalein ki pehle dekhe cards aakhir tak DOM mein rahenge.
- Duplicates ko stable key se merge karein. Agar product URL ke saath tracking query parameters aate hain, deduplication ke liye unhe normalize karna upyogi ho sakta hai—magar asli record mein zaroori variant ya region jankari na hataayein.
- Natije ko known total ya site ke visible count se milayein, agar aisa count diya gaya ho. Sirf timeout ya unchanged count ko exhaustiveness ki guarantee na samjhein.
- Catalog mein pagination, filters, regional availability ya “load more” control bhi ho sakta hai. Infinite scroll ka hona yeh guarantee nahin deta ki har product ek hi listing state mein mil jayega.
Rendered cards ya site ka data endpoint?
Rendered cards browser mein dikhne wali jankari capture karne ka seedha rasta hain, lekin markup badalne par selectors toot sakte hain aur virtualized list mein records ko scrolling ke dauran jama karna padta hai. Agar site documented API ya export deti hai aur uska istemal site ki terms ke mutabik allowed hai, to woh bade catalog ke liye zyada seedha ya bharosemand ho sakta hai. Kisi unnamed store ke liye specific endpoint ki sifarish nahin ki ja sakti; undocumented endpoint ko documented ya permitted maan kar na chalein.
Or skip the browser setup
ScreenshotNeo product-card data extractor nahin hai: iska API page ka screenshot ya PDF deta hai, product records ki structured list nahin. Agar aapko rendered page ka visual capture chahiye, ek GET request yeh kar sakti hai; ScreenshotNeo API documentation mein options dekhein.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpCookie banners, popups aur chat widgets capture se pehle hataye jaate hain; bot checks, blank pages aur failed loads bill nahin hote. AI agents ke liye MCP server mein screenshot aur PDF capture tools hain. Free plan mein bina card ke 1,000 screenshots mahina milte hain; paid plans $5 mein 3,000 se shuru hote hain. ScreenshotNeo ke baare mein adhik jaanein, ya free account banayein.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Aam dikkat aur unke hal
Scroll ke baad naye products nahin aate
Page ke bajay nested panel scroll ho raha ho sakta hai, ya ulta. Asli scrolling element identify karein, usi par scroll input bhejein, aur dekhein ki uska scroll position badla ya nahin. Consent overlay, filters, ya “load more” control bhi loading rok sakte hain.
Script jaldi ruk jaata hai
No-growth rounds ka limit bahut kam ho sakta hai, ya wait site ke response se chhota ho. Fixed sleep badhane se pehle spinner, count growth, control state, ya end marker jaisa site-specific signal use karein. Request failure aur rate-limit jaise errors bhi jaanchein.
List mein products chhoot rahe hain ya duplicates hain
DOM ko sirf aakhir mein padhne ke bajay har round ka batch jama karein. Deduplication ke liye title ke badle product ID ya URL ka istemal karein; agar variants alag entries hain to unki pehchan key mein rakhein.
Best Value
Count nahin badhta, lekin end marker bhi nahin dikhta
Isse completion na maan lein. Yeh stalled request, rate limiting, galat scroll target, ya virtualized list ka asar ho sakta hai. Round logs dekhein, loading signal aur page errors inspect karein, aur total ya alag pagination control check karein.
locator.all() se adhuri list milti hai
locator.all() us waqt maujood elements ka turant snapshot deta hai; badalti list ke settle hone ka intezar nahin karta aur changing list par natije unpredictable ho sakte hain. Pehle site-specific condition se list stable hone ka wait karein, phir batch extract karein. Scrolling ke dauran records lena zaroori ho to har round mein extraction karein.
FAQ
Kya ek DOM snapshot poori product list ke liye kaafi hai?
Nahin. Lazy loading mein kuch products abhi load nahin hue honge; virtualization mein pehle dekhe cards DOM se nikal chuke ho sakte hain. Scroll ke saath records jama karein.
No-growth ke kitne rounds rakhne chahiye?
Iska universal number nahin hai. Site ki latency aur loading behavior ke mutabik threshold chunein, use bounded rakhein, aur unchanged count ko end ka pakka saboot na samjhein.
Kya timeout ka matlab catalog khatam ho gaya?
Nahin. Timeout failure ya slow response ka signal hai, completeness ka nahin. Errors aur site ke end ya total signal ko verify karein.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →
