A JavaScript-rendering scraping API loads a page in a browser engine, runs its client-side scripts, waits for the content you need, and returns the resulting HTML (or structured data, where supported). Use ordinary HTTP first; turn rendering on only when the target data is absent from the initial response. In ScraperAPI’s documented syntax, that means adding render=true to the request.
What JavaScript rendering changes
A normal HTTP scraper downloads the server response. If a page sends an empty application shell and JavaScript later fetches products, prices, comments, or account data, a normal request cannot see those values. A rendering API routes the URL through a headless browser, executes scripts, and returns the post-render page.
Rendering is not automatically better. Browser startup, script execution, network calls, and anti-bot checks add latency. ScraperAPI says rendered requests can reduce success rates and throughput, so enable the feature only for pages that need it.
When to render
- Render when the required element is inserted or changed after page load.
- Do not render when the value is already present in the ordinary response HTML or an accessible JSON endpoint.
- Use a browser-rendered request for pages that require clicks, scrolling, typing, or other user-like actions before the data appears.
A complete ScraperAPI request
ScraperAPI documents this request shape:
GET https://api.scraperapi.com?api_key=API_KEY&render=true&url=https://example.com/Use a real API key in an environment variable or secret store, and URL-encode the target URL. Never commit keys to source control or print full request URLs in application logs.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →#1 Best Overall
cURL
export SCRAPER_API_KEY='your-key'
curl -G 'https://api.scraperapi.com'
--data-urlencode "api_key=$SCRAPER_API_KEY"
--data-urlencode 'render=true'
--data-urlencode 'url=https://example.com/'
-o rendered.htmlOpen rendered.html and check that the exact field you need exists. A successful HTTP status only proves that a response was returned; it does not prove that the application finished loading your data.
Python
import os
import requests
params = {
"api_key": os.environ["SCRAPER_API_KEY"],
"render": "true",
"url": "https://example.com/",
}
response = requests.get(
"https://api.scraperapi.com",
params=params,
timeout=90,
)
response.raise_for_status()
with open("rendered.html", "w", encoding="utf-8") as file:
file.write(response.text)
print("bytes:", len(response.content))Node.js
const key = process.env.SCRAPER_API_KEY;
const query = new URLSearchParams({
api_key: key,
render: 'true',
url: 'https://example.com/'
});
const response = await fetch(`https://api.scraperapi.com?${query}`);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const html = await response.text();
console.log('characters:', html.length);Wait for the data, not merely the page
Many single-page applications continue making requests after the first paint. ScraperAPI documents wait_for_selector for this case, and it requires render=true. Choose a selector that represents the data you will parse, such as a results container, rather than an arbitrary sleep.
curl -G 'https://api.scraperapi.com'
--data-urlencode 'api_key=API_KEY'
--data-urlencode 'render=true'
--data-urlencode 'wait_for_selector=.product-card'
--data-urlencode 'url=https://example.com/catalog'
-o catalog.htmlMake the selector specific enough to mean “the requested data is ready,” but not so specific that a harmless class-name change breaks every job. If the site uses a loading placeholder with the same selector, wait for a descendant containing the final value or use an interaction sequence that ends at a result selector.
Fixed delays and browser events
A short fixed wait can help when no stable selector exists. Browser-event waits are another option in ScraperAPI’s instruction set. Both should be bounded: unnecessary waiting increases cost and latency, while an unbounded or excessive wait creates timeouts. Prefer a selector whenever one reliably marks readiness.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallRank #2
- HTML CSS Design and Build Web Sites
- Comes with secure packaging
- It can be a gift option
Click, type, scroll, then extract
Some pages expose the target only after an interaction. ScraperAPI’s instruction-set feature supports entering text, clicking, scrolling, waiting for browser events, and waiting for selectors. Keep a sequence to roughly three or four actions, as the provider recommends; every extra action adds execution time and another possible failure.
- Open the URL with rendering enabled.
- Enter text into the search or filter field.
- Click the submit, filter, or “load more” control.
- Wait for the result selector, then parse the returned HTML.
The exact instruction syntax is provider-specific, so validate it against the current provider documentation before deploying. Treat each action as fallible: a cookie dialog may intercept a click, a selector may change, or a request may time out.
Scrolling and lazy content
Scroll only when the site loads additional records as the viewport approaches the bottom. Limit the distance and number of scrolls to what is required. If the page exposes a documented data endpoint, calling that endpoint may be faster and cheaper than simulating a long scroll, subject to the site’s terms and access controls.
Parse the returned response
After the browser step, parse the HTML just as you would any other response. Select the smallest stable element that contains the field, normalize whitespace, and validate missing values. If your chosen provider offers structured extraction, it may return fields directly; ZenRows advertises CSS extraction and automatic parsing, but those are product capabilities, not independent proof of accuracy.
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #3
- Brand: Wiley
- Set of 2 Volumes
- A handy two-book set that uniquely combines related technologies Highly visual format and accessible language makes these books highly effective learning tools Perfect for beginning web designers and front-end developers
from bs4 import BeautifulSoup
soup = BeautifulSoup(response.text, "html.parser")
items = []
for card in soup.select(".product-card"):
name = card.select_one(".name")
price = card.select_one(".price")
if name and price:
items.append({
"name": name.get_text(" ", strip=True),
"price": price.get_text(" ", strip=True),
})
print(items)Store the request URL, render setting, wait condition, response status, elapsed time, parser version, and a redacted failure reason. This makes a later empty result distinguishable from a page redesign, timeout, or blocked request.
Cost, latency, and throughput decisions
| ScraperAPI mode | Documented credit use | Use it when |
|---|---|---|
| JavaScript rendering | 10 API credits per request | The target value is created or changed by client-side JavaScript. |
| Rendering with premium proxies | 25 API credits per request | Your permitted workload needs that proxy tier. |
| Rendering with ultra-premium proxies | 75 API credits per request | Your permitted workload specifically requires the higher proxy tier. |
These are ScraperAPI’s documented, provider-specific values and can change. They are not industry benchmarks. The provider also documents a default rendering burst limit of 10 requests per second. Design a queue with backoff rather than assuming every API supports that rate.
Reduce unnecessary browser work
- Probe a normal request first and render only when the target is missing.
- Wait for a selector instead of sleeping for a worst-case duration.
- Keep interaction instructions short.
- Cache results when freshness permits, and avoid recapturing unchanged pages.
- Measure the same permitted URLs with the same parsing rules before changing providers.
Rendering can increase latency and reduce success rates, which lowers the number of jobs your system completes per hour. Set client timeouts long enough for browser work, but impose a job deadline and retry only transient failures. Repeating a deterministic selector failure will not fix it.
Choosing a provider without misleading comparisons
Compare capabilities against your workload, not marketing claims. Ask whether an API returns rendered HTML or structured data, supports selector waits and browser instructions, provides screenshots if you need visual evidence, offers the proxy or geographic controls required by your use case, and exposes predictable credit accounting.
| Requirement | What to verify |
|---|---|
| Asynchronous data | Selector wait, event wait, or a documented equivalent. |
| Interactions | Typing, clicking, scrolling, and a bounded instruction sequence. |
| Extraction | Raw rendered HTML versus provider-side CSS or structured extraction. |
| Visual output | Screenshot format, viewport controls, and whether it is separate from HTML scraping. |
| Reliability | Your own success and latency measurements on an allowed, representative workload. |
ZenRows advertises rendering, selector waits, CSS extraction, automatic parsing, and screenshots. Bright Data describes browser rendering and interactions such as clicking and screenshots. Those pages establish advertised features, not a neutral ranking. Run a controlled evaluation and obey each site’s terms, authentication requirements, and rate limits.
Robots.txt, permission, and responsible collection
RFC 9309 defines robots.txt as crawler guidance and explicitly says: “These rules are not a form of access authorization.” A permissive file does not grant legal, contractual, or authenticated permission to collect data. Check applicable law, the site’s terms, account permissions, privacy obligations, and technical access controls. Do not use rendering to defeat a CAPTCHA, bot check, paywall, or authentication boundary you are not authorized to cross.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Common failures and fixes
The response contains an empty app shell
Cause: rendering is disabled or the page has not finished its client-side request. Fix: set render=true, then wait for a selector representing the final data.
wait_for_selector has no effect
Cause: the request is not rendered, or the selector never appears. Fix: include both rendering and the selector; inspect the page manually and choose a stable, post-load element.
A click times out
Cause: a consent dialog covers the control, the selector changed, or the page is still loading. Fix: handle the dialog where permitted, wait for the control, shorten the sequence, and capture diagnostic HTML on failure.
Results are incomplete
Cause: lazy loading, pagination, a premature wait, or an application request that failed in the browser. Fix: wait for the result container, perform only the necessary scroll or pagination actions, and log response size and elapsed time.
Requests are slow or exceed the burst limit
Cause: browser startup and interactions are expensive, or concurrency exceeds the provider’s documented default. Fix: remove rendering from static pages, reduce actions, queue jobs, add exponential backoff, and measure throughput after each change.
Or skip the browser setup
If your deliverable is a clean visual capture rather than parsed HTML, ScreenshotNeo is the first alternative to try: it removes cookie banners, newsletter popups, and chat widgets before capture; bot checks, blank pages, failed loads, and timeouts are not billed; and its MCP server lets Claude, Cursor, or another MCP client take screenshots. Every plan includes the features, with 1,000 screenshots a month free without a card and paid plans starting at $5 for 3,000.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →One GET request returns PNG, JPEG, WebP, or PDF:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo API documentation for rendering options such as full-page capture, selector targeting, waits, custom JavaScript, headers, cookies, device presets, PDFs, caching, and asynchronous jobs. Sign up free at ScreenshotNeo.
FAQ
Is JavaScript rendering the same as using Selenium?
Both execute page scripts in a browser context, but a hosted API manages browser infrastructure and returns the response through an HTTP interface. The exact controls and limits depend on the provider.
Should I always wait for network idle?
No. Analytics, advertisements, and long-lived connections can prevent network-idle conditions. A selector tied to the required data is usually more deterministic.
Can robots.txt alone make a scrape permitted?
No. Robots.txt communicates crawler preferences; it is not access authorization. Obtain the permissions your use case requires.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
