Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Cloudflare’s Pay Per Crawl flow uses HTTP 402 to tell a crawler the price when it has not supplied payment intent; a crawler that supplies the required intent can receive the content with HTTP 200. A separately configured custom 402 response is just a message for a blocked crawler—it does not, by itself, process payment. Pay Per Crawl is documented as a closed beta as of October 4, 2026, so first confirm that your account is eligible.
Check eligibility and make the zone visible
Cloudflare currently describes Pay Per Crawl as a closed beta. Interested site owners can use Cloudflare’s signup route or contact an Enterprise account executive, but access is not guaranteed.
As an Amazon Associate I earn from qualifying purchases.
Before its controls can be configured, your Cloudflare account must have domains added, the domain must use Cloudflare nameservers or have DNS records managed by Cloudflare, and an Administrator or Super Administrator must make the domain visible to the account’s domain administrators.
- In the Cloudflare dashboard, open Manage Account > Settings > Pay Per Crawl.
- Set the domain or domains you want to configure to Visible.
Cloudflare says visibility exposes configuration controls; it does not change the domain’s security rules.
Set the default price and crawler actions
For each zone, first turn on Pay Per Crawl and set a default price. Cloudflare’s price-setting guide states a minimum of USD 0.001 per crawl (Cloudflare, 2026). That price applies to each successful content retrieval with HTTP 200, not to every incoming request.
- Open AI Crawl Control > Payments.
- Enable Pay Per Crawl, enter the default per-crawl price, and save.
- Open AI Crawl Control > Security and select an action for each crawler.
| Action | Effect |
|---|---|
| Charge | The crawler follows the paid-access flow and is charged the configured price for successful access. |
| Allow | The crawler can access the content without a Pay Per Crawl charge. |
| Block | The crawler is denied access. |
There is one zone-default price for all crawlers assigned Charge; Cloudflare’s FAQ does not describe a separate default price for each crawler. For crawler- or content-specific pricing, use dynamic pricing instead.
Make selected URI patterns free
A Configuration Rule can disable Pay Per Crawl for matching requests while leaving it active elsewhere. Cloudflare documents the following dashboard sequence:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Go to Rules > Overview and select Create rule > Configuration Rule.
- Set the matching field to URI Full, choose the wildcard operator, and enter the URI pattern to exempt.
- Add the Disable Pay Per Crawl setting and deploy the rule.
Cloudflare’s examples include exempting a public-directory wildcard or making the homepage free. It also says these paths are always free to crawl: /robots.txt, /sitemap.xml, /security.txt, /.well-known/security.txt, and /crawlers.json.
Use dynamic pricing for different content
The zone price is the default. To quote a different price based on a request or content, enable dynamic pricing and have your origin return a crawler-price response header. For example, Cloudflare documents the value crawler-price: USD 3.14. Cloudflare uses that value instead of the zone default and forwards the header to the crawler only when asking it to pay with HTTP 402.
Cloudflare adds a cf-pay-per-crawl header to origin requests to identify the active pricing mode. Its documented values are:
| Value | Meaning |
|---|---|
zone-default |
Use the zone’s configured default price. |
in-band |
Use the request-specific price supplied in-band through crawler-price. |
bypass |
Bypass Pay Per Crawl for the request. |
A Cloudflare Worker can set crawler-price using the URL path, request properties, or content. Cloudflare says Snippets can read the cf-pay-per-crawl request header, but cannot choose crawler-price because Snippets run too late in request processing. Cloudflare’s changelog dates the availability of URI exclusions and dynamic-pricing configuration to June 16, 2026.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteDistinguish paid-access 402 from a custom block response
HTTP 402 can appear in two different situations in Cloudflare’s controls. The status code alone does not prove that a payment was processed.
| Response | What it means | What it does not do |
|---|---|---|
| Pay Per Crawl 402 | Cloudflare can return pricing when a crawler has not supplied payment intent. A crawler that supplies the required payment intent can receive successful access with HTTP 200. | It is not simply a custom message for a blocked crawler. Cloudflare acts as Merchant of Record for Pay Per Crawl. |
| Custom 402 block response | A configured response body tells a blocked crawler that payment is required or that it should contact the site owner. | The message alone is not a billing or payment integration. |
Configure a custom 402 message for blocked crawlers
This setting is separate from Pay Per Crawl negotiation. Cloudflare documents it for paid plans when at least one AI crawler is blocked.
- Open AI Crawl Control > Settings > Block response.
- Choose 402 Payment Required and enter the response body as plain text. Cloudflare also permits 403 Forbidden.
If the associated AI Crawl Control WAF rule has been manually changed to use a response code other than 402 or 403, Cloudflare says the selected-code control may be unavailable or blank.
Account for rule precedence, billing, and search indexing
- Cloudflare says WAF or Bot Management blocks override a Pay Per Crawl Charge action. A crawler blocked by either does not reach the zone through the paid-crawl flow.
- Cloudflare’s FAQ says error responses are not billed, while each successful recrawl incurs the configured cost. Repeated successful retrievals therefore incur repeated charges.
- Cloudflare warns that charging or blocking search-engine crawlers may prevent proper indexing and negatively affect SEO. Review crawler actions with that trade-off in mind.
Cloudflare describes Pay Per Crawl as a feature of AI Crawl Control that lets site owners set a per-zone price to control and monetize AI crawler access. The controls determine which crawlers may access content and on what pricing basis; an HTTP 402 response must be interpreted in the context of which of the two configurations produced it.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
