Hardware FixRecommendedDevice not working? Your driver may be the problemCheck updates for common hardware issues.Fix DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PC×
Skip to content
Cloudflare error 1015

Error 1015: How to Solve Rate Limiting When Web Scraping

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Cloudflare Error 1015 means the website owner’s rate-limit rule has temporarily blocked your requests. Stop the scraper, honor any Retry-After value, reduce request pressure when you have permission to resume, and contact the site owner or use an authorized API if the block continues. Changing IP addresses or trying to evade anti-bot controls is not the documented solution.

What Error 1015 means

Cloudflare’s Error 1015 page describes a site that has received too many requests and has temporarily blocked the visitor. The limit is configured by the website owner; Cloudflare is returning the error on that owner’s behalf. A block therefore says nothing about a universal “safe” scraping speed. Different sites can apply different thresholds, windows, request attributes and response conditions.

Treat 1015 as a stop signal, not as an invitation to retry faster. Repeated attempts during a short block can prolong the denial. The correct response depends on whether you are a visitor or scraper operator, or the owner responsible for the Cloudflare zone.

What to do immediately as a scraper operator

  1. Stop the affected job. Cancel workers and disable automatic retries for that host. Do not keep refreshing the URL in a browser or loop over the same endpoint.
  2. Inspect the response. Save the status code, response headers and body. Look specifically for Retry-After, request identifiers and any message identifying the protected host.
  3. Honor retry guidance. If Retry-After is present, wait at least that long before considering a permitted retry. Cloudflare’s error reference lists retry_after: 30 for Error 1015, but a dynamic value supplied by a WAF rule takes precedence. Thirty seconds is not a guarantee that access will resume.
  4. Confirm that access is authorized. Check the site’s terms, published API documentation, account permissions and any written agreement. If you do not have a valid basis to collect the data, stop rather than trying to work around the control.
  5. Resume conservatively only when justified. Lower concurrency and total request volume, avoid bursty retries, cache responses and request only what you need. The available guidance does not establish a universal requests-per-second number.
  6. Escalate when necessary. Contact the site owner or support team if the limit persists, request an approved access method, or switch to an official or licensed data source.

Handling Retry-After correctly

HTTP servers may send Retry-After as either a delay in seconds or an HTTP date. Your client should parse both forms and avoid issuing requests until the indicated time. A missing header does not mean immediate access is safe; pause and reduce pressure based on the owner’s documented policy.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Example: a bounded Python retry policy

import email.utils
import time
from datetime import datetime, timezone


def retry_after_seconds(value):
    if not value:
        return None
    try:
        return max(0, int(value))
    except ValueError:
        try:
            target = email.utils.parsedate_to_datetime(value)
            if target.tzinfo is None:
                target = target.replace(tzinfo=timezone.utc)
            return max(0, int((target - datetime.now(timezone.utc)).total_seconds()))
        except (TypeError, ValueError, OverflowError):
            return None


def should_retry(response):
    if response.status_code not in (429, 1015):
        return False
    delay = retry_after_seconds(response.headers.get("Retry-After"))
    if delay is None:
        return False  # require an explicit policy or human review
    time.sleep(delay)
    return True

This example deliberately refuses to guess a delay when the server provides no usable value. Add a maximum-attempt limit and a circuit breaker in production so a persistent block ends the job instead of creating a retry storm.

HTTP 429 versus Cloudflare 1015

Signal What it tells you Correct response
HTTP 429 The server received too many requests in a specified period. The response may include Retry-After. Pause, honor the header, reduce permitted traffic and check the service’s policy.
Cloudflare Error 1015 A Cloudflare-protected site owner’s configured rate-limit rule temporarily blocked the visitor. Stop rapid retries, wait, then seek authorized access or contact the owner if it persists.

A 429 is not proof that you exceeded a provider-wide quota. Cloudflare documents rate-limit headers such as Ratelimit and Ratelimit-Policy for some services, but numerical quotas in Cloudflare’s own API documentation apply to that API, not to unrelated websites.

Lowering request pressure without bypassing controls

Use a queue and bounded concurrency

Place URLs in a durable queue and process a small, configurable number of workers. Add jitter so every worker does not wake at the same instant. When one response indicates a limit, pause the host-wide queue rather than only the individual URL.

Cache and deduplicate

Store successful responses with an appropriate freshness period, normalize URLs before enqueueing, and remove duplicate links. Conditional requests such as If-None-Match or If-Modified-Since can reduce transferred data when the site supports them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Request less

Prefer an official API, feeds, sitemaps or exports. Fetch only the pages and fields required for your use case, and avoid repeatedly downloading unchanged assets. Identify your application honestly where the site’s policy asks for a user agent or contact address.

Do not rotate around the block

Changing IP addresses, disguising a bot, defeating a CAPTCHA or using a proxy service to evade a configured limit does not fix the underlying authorization problem. It can violate the site’s terms and trigger stronger defenses. The documented remedy is to wait, change behavior within an authorized arrangement, or ask the owner for access.

Robots.txt is guidance, not permission

RFC 9309 defines the Robots Exclusion Protocol. A crawler that successfully retrieves a parseable robots.txt should follow its rules. The same RFC expressly says: “These rules are not a form of access authorization.” A permissive robots file does not override authentication, contractual terms, a rate limit or a Cloudflare block; a restrictive file is an important signal to stop or seek clarification.

When the problem is on the site-owner side

If you operate the Cloudflare zone, inspect the rate-limiting rule that matched: its period, threshold, expression, counted requests and response action. Cloudflare’s best-practices guidance describes rate limiting as a way to protect operations, including against content scraping, and its rules can use request attributes and response patterns.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Tune the rule to the real workload

  • Measure normal traffic for the specific route before changing thresholds.
  • Separate expensive write or search operations from inexpensive static requests.
  • Use a longer evaluation period when a very short window creates needless bursts; Cloudflare gives changing an example one-second period to ten seconds as owner-side guidance.
  • Provide a clear support path so legitimate clients can report a false positive.
  • Return an accurate status and, where appropriate, a dynamic Retry-After value.

These settings protect your service; they do not define a safe interval for every external crawler.

Diagnosing other causes that look like 1015

  • Authentication failure: a 401 or 403 generally requires credentials or permission, not slower requests.
  • Bot or CAPTCHA challenge: solve it through the site’s normal visitor flow or request an approved integration; do not automate evasion.
  • Timeout or origin failure: a 5xx response can indicate an overloaded or unavailable origin rather than a rate limit.
  • Local throttling: connection-pool limits, DNS failures and client timeouts can occur before a request reaches Cloudflare.
  • Account quota: an API may impose a documented monthly or per-minute allowance distinct from website traffic controls.

Record the status, headers, timestamp, URL pattern and request identity for each failure. That evidence helps the owner distinguish a true rate rule from another access problem.

Operational checklist

  • Have you stopped all rapid retries for the affected host?
  • Did you parse and honor Retry-After, including an HTTP-date form?
  • Is your collection authorized by the owner, terms or an official API?
  • Are concurrency, bursts, duplicate URLs and unnecessary assets controlled?
  • Do you cache results and enforce a circuit breaker?
  • Have you avoided IP rotation and anti-bot evasion?
  • Do you know whom to contact if the block remains?
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Or skip the browser setup

If your task is simply to obtain a clean screenshot rather than crawl pages, ScreenshotNeo makes one authorized request to its screenshot API. It accepts consent banners before capture and removes more than 60 known consent platforms, newsletter popups and chat widgets; each step can be disabled. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads and cache hits are not billed, and response headers identify the page verdict and billing status. Its MCP server lets Claude, Cursor and other MCP clients call take_screenshot, get_page_info and capture_pdf.

Use the documented API parameters and respect the target site’s access rules. The following cURL request captures Stripe as a WebP image:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python:

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

See the complete parameter reference in the ScreenshotNeo documentation. Every plan includes the same features, including full-page and selector capture, device presets, custom headers and cookies, waits, blocking rules, PDFs, async webhooks, bulk capture and signed links. The Free plan includes 1,000 screenshots each month without a card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

FAQ

How long should I wait after Error 1015?

Use the response’s Retry-After value when supplied. Cloudflare lists 30 seconds as a default reference for 1015, but a dynamic WAF value overrides it and no delay guarantees access.

Is there a safe requests-per-second number?

No universal number is established. The site owner controls the rule, so use the owner’s published limits or obtain permission for a tailored allowance.

Can I scrape after robots.txt allows my crawler?

Robots.txt rules should be honored, but RFC 9309 says they are not access authorization. You still need permission and must respect rate limits and authentication.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Should I switch proxies when 1015 appears?

Not to evade the rule. Pause, reduce authorized traffic or contact the owner; bypass attempts can violate terms and worsen blocking.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Read next

Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.