Recommended Free Tools
Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
To send a POST as part of a Pyppeteer page, enable request interception, handle the request event, and call request.continue_() with the documented method, postData, and headers overrides. Every intercepted request must be continued, fulfilled, or aborted; otherwise page loading can stall. The exact URL, body encoding, cookies, authentication, and CSRF values come from the website you are automating.
What Pyppeteer interception actually does
Pyppeteer is an unofficial Python port of Puppeteer. Its request-interception API changes a request that the browser page is already making (or is about to make). It does not turn Pyppeteer into a general-purpose HTTP client by itself.
When interception is enabled with Page.setRequestInterception(True), Pyppeteer pauses requests and emits a request event. Your handler decides what happens next. For the target request, call Request.continue_() with overrides. For all other requests, call Request.continue_() without overrides. The documented body key is camel-case postData, not post_data.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteThe API reference used for this pattern is Pyppeteer 0.0.25-era documentation. Treat package compatibility and browser-download behavior as version-sensitive, and verify the API against the version installed in your environment.
#1 Best Overall
Before you write the handler
Identify the request the site expects
Find the form or network call in the site’s own documentation or browser developer tools. Record its exact URL, HTTP method, content type, field names, authentication mechanism, cookies, and any CSRF token. A generic payload such as key=value is only an illustration; it will not be valid for an arbitrary website.
Choose the body encoding
- URL encoded: send a string such as
name=Ada&role=adminand setContent-Typetoapplication/x-www-form-urlencoded. - JSON: serialize an object with
json.dumps()and setContent-Typetoapplication/json. - Multipart: browser forms that upload files usually need multipart boundaries and browser-managed streams. Reproducing those manually is error-prone; use the page’s form or a client library designed for multipart requests.
Know whether browser state matters
A page navigation can establish cookies, execute JavaScript, obtain a CSRF token, or add an authorization header. A standalone HTTP client will not automatically reproduce that state. Interception is appropriate when the POST must occur in that browser context.
Complete Pyppeteer interception example
The following script intercepts one illustrative endpoint while allowing every other request to proceed. Replace the URL, payload, and headers with values required by the target site. The example navigates to a page to create browser state, then triggers a request with a button click; adapt the trigger to the page you control.
Free tools Windows power users keep installed
One-click scans. No signup required.
import asyncio
import json
from pyppeteer import launch
TARGET_URL = "https://example.com/endpoint"
async def main():
browser = await launch()
page = await browser.newPage()
await page.setRequestInterception(True)
post_body = json.dumps({"key": "value"})
async def handle_request(request):
if request.url == TARGET_URL and request.method == "GET":
await request.continue_({
"method": "POST",
"postData": post_body,
"headers": {
**request.headers,
"Content-Type": "application/json",
},
})
else:
await request.continue_()
page.on("request", lambda request: asyncio.ensure_future(handle_request(request)))
response = await page.goto("https://example.com")
if response:
print("Navigation status:", response.status)
# Replace this with the page action that causes the matching request.
# await page.click("button#submit")
await asyncio.sleep(2)
await browser.close()
asyncio.run(main())The conditional includes request.method == "GET" so that an already-POST request is not rewritten accidentally. If your page action itself creates the POST, match the URL and the original method or another unmistakable property, then let the action run once. A broad URL-only match can rewrite analytics, retries, or multiple requests.
URL-encoded variant
form_body = "key=value&another=two"
await request.continue_({
"method": "POST",
"postData": form_body,
"headers": {
**request.headers,
"Content-Type": "application/x-www-form-urlencoded",
},
})Keep useful original headers by starting with request.headers and overriding only what must change. Some servers require an Origin, Referer, authorization header, or a site-generated CSRF token. Do not invent those values; obtain them through the site’s supported flow.
Triggering and observing the POST
Trigger it from page JavaScript
If the page has a form or button, navigate first, wait for the relevant selector, and perform the action. This preserves cookies and JavaScript state.
await page.goto("https://example.com/form", {"waitUntil": "networkidle2"})
await page.waitForSelector("form#checkout")
await page.click("form#checkout button[type=submit]")Use a one-time guard when retries are possible:
matched = False
async def handle_request(request):
nonlocal matched
if (not matched and request.url == TARGET_URL
and request.method == "GET"):
matched = True
await request.continue_({
"method": "POST",
"postData": "key=value",
"headers": {
**request.headers,
"Content-Type": "application/x-www-form-urlencoded",
},
})
else:
await request.continue_()Read the response
Pyppeteer’s page events include response, requestfinished, and requestfailed. Capture the response matching the endpoint and inspect its status and body. A server returning HTTP 400 or 500 is still a completed HTTP response; it is not necessarily a transport-level failure.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchasync def log_response(response):
if response.url == TARGET_URL:
print("POST response:", response.status)
print((await response.text())[:1000])
page.on("response", lambda response: asyncio.ensure_future(log_response(response)))Register the listener before triggering the request. For structured JSON, parse the text only after checking the content type and status your endpoint documents.
Why every intercepted request needs a decision
After interception is enabled, requests can remain paused until you continue, respond to, or abort them. The browser-cache completion exception described in Puppeteer documentation does not remove the need for a normal continuation path. A handler that only handles the intended POST and does nothing for images, scripts, fonts, or the main document can make navigation appear to hang.
- Use
await request.continue_()for requests you are not changing. - Use
await request.abort()only when intentionally blocking a resource. - Use a response override only when you deliberately want to synthesize the response.
- Return promptly; slow interception handlers delay every matching browser request.
Because the event callback is asynchronous, the asyncio.ensure_future() pattern schedules the coroutine. In production, add logging and make sure exceptions inside the handler are surfaced rather than silently leaving a request unresolved.
Interception versus a direct POST
If you simply need to call an API and do not need a page, interception adds browser startup and page traffic that provide no benefit. Choose the smallest tool that preserves the behavior you require.
| Approach | Use it when | Important trade-off |
|---|---|---|
| Pyppeteer request interception | The POST must happen inside page activity, with browser cookies, JavaScript, or navigation state. | Browser startup is heavier, and every intercepted request needs a resolution. |
Playwright APIRequestContext.post |
You want a direct HTTP context with documented JSON, URL-encoded form, multipart, and cookie sharing within that context. | It does not reproduce arbitrary page JavaScript unless you also use a browser page. |
Requests requests.post |
You need a conventional Python HTTP call with data or json. |
You must manage cookies, tokens, redirects, and browser-only behavior yourself. |
For a direct Requests call, the basic shape is:
import requests
r = requests.post(
"https://example.com/endpoint",
json={"key": "value"},
timeout=30,
)
r.raise_for_status()
print(r.text)Use Playwright’s API request context when its shared request/browser context model is valuable. Use Pyppeteer interception when changing or observing traffic generated by a real page is the requirement.
Installation and browser considerations
Install Pyppeteer in the environment that will run the script, then launch it with the Chromium executable available to that environment. Project documentation for the historical API describes a first-run Chromium download and the pyppeteer-install command. In restricted CI or production images, preinstalling a compatible browser and passing its executable path can make deployments more predictable. The supplied API material does not establish current maintenance or compatibility guarantees, so pin and test the package and browser versions you choose.
Security and data handling
- Keep access keys and authorization values out of source control; use environment variables or a secret manager.
- Do not log cookies, bearer tokens, or full personal-data payloads.
- Respect the target site’s terms, authentication rules, rate limits, and applicable law.
- Validate redirects and final URLs if credentials are attached to requests.
Common failures and fixes
The page hangs after enabling interception
Cause: at least one request path never calls continue_(), abort(), or a response method. Fix: add an unconditional continuation branch and ensure the handler cannot exit early on an exception.
The server says the body is invalid
Cause: the encoding or field names do not match the endpoint. Fix: send JSON only with application/json; send URL-encoded fields with application/x-www-form-urlencoded; reproduce required multipart behavior through the page or a suitable HTTP client.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →The request is unauthorized or rejected for CSRF
Cause: missing cookies, authorization, origin, referer, or a short-lived CSRF token. Fix: navigate through the site’s supported login/form flow, inspect the actual request, and preserve or generate the required values. Do not hard-code a token that expires.
The handler changes the wrong request
Cause: the URL is shared by retries, analytics, or multiple methods. Fix: match URL plus method, query parameters, resource type, or a one-time state flag, and remove the interception after the intended request if your workflow permits.
The script reports failure but the server processed the POST
Cause: a timeout or disconnected browser can occur after the server receives data. Fix: inspect server-side idempotency support, record a request identifier when available, and retry only when the operation is safe to repeat.
Navigation succeeds but no POST appears
Cause: the page action was never triggered, the selector was wrong, or the condition did not match the actual URL/method. Fix: wait for the selector, log every request URL and method temporarily, and compare the observed request with your predicate.
Pyppeteer cannot launch Chromium
Cause: the browser binary is absent, incompatible, or blocked by the runtime. Fix: install the documented browser dependency or provide a known executable path, then verify the package/browser pair in the same environment as the script.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Timing, reliability, and cost notes
Interception adds work to every page request, so narrow the predicate and keep the callback fast. Prefer explicit waits for the selector or response that proves the POST completed over arbitrary long sleeps. Set navigation and operation timeouts appropriate to your environment, and close the browser in a finally block in long-running services so crashed jobs do not leak processes.
Retries require special care. A POST may have side effects even when the browser reports a timeout. Use an idempotency key if the endpoint supports one, or reconcile the result before retrying. Capture status, response text (with secrets removed), and request-failure events to diagnose intermittent network problems.
Pyppeteer itself does not define the target site’s pricing or request limits. Any costs, quotas, authentication requirements, and rate limits come from that website or API.
Or skip the browser setup
If your actual goal is to obtain a clean image or PDF of a page rather than submit a page-generated POST, ScreenshotNeo provides a one-call website screenshot API and MCP server. It accepts a URL and can return PNG, JPEG, WebP, or PDF. Cookie and consent banners, newsletter popups, and chat widgets are removed before capture; bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers identify the page verdict and billing status.
For a screenshot of Stripe, using the documented API parameters:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpPython:
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)Node.js:
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);See the ScreenshotNeo API documentation for the full option set, including full-page and element capture, device and retina settings, custom CSS and JavaScript, waits, headers and cookies, blocking rules, caching, signed links, PDF controls, asynchronous jobs, bulk capture, and usage reporting. Its MCP server exposes take_screenshot, get_page_info, and capture_pdf to Claude, Cursor, and other MCP clients.
The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 shots, and every feature is available on every plan. Create a free ScreenshotNeo account to try it.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →FAQ
Is postData case-sensitive?
Yes. Use the documented camel-case override key postData in the dictionary passed to continue_().
Can interception submit a POST without loading a page?
It is designed to modify requests made by a page. For an independent HTTP call, use a direct client such as Requests or Playwright’s API request context.
Will an HTTP 500 trigger requestfailed?
Not necessarily. An HTTP error is still an HTTP response; transport failures and server status codes are different conditions. Inspect the response status as well as failure events.
Frequently Asked Questions
What does setRequestInterception(True) change?
It pauses page requests and emits them to your request handler so each can be continued, modified, fulfilled, or aborted.
How do I send JSON instead of form data?
Serialize the object with json.dumps(), pass the resulting string as postData, and set Content-Type to application/json.
Why should I use Requests instead of Pyppeteer?
Requests is simpler for a standalone HTTP POST when browser cookies, JavaScript execution, and page activity are unnecessary.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
