October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk6 min

How to Set Rate Limits and Bot Rules Without Blocking Real Users

A practical guide to rate limits and bot rules: target risky actions, measure normal traffic first, and use graduated responses that reduce false positives.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

To limit abusive traffic without disrupting legitimate visitors, target the risky action—not every request—then observe real traffic in preview or count mode before enforcing a rule. Choose a counting identity that fits your application, challenge uncertain traffic, and reserve blocks for repeated or high-confidence abuse. Treat search crawlers, APIs, webhooks, partner integrations, and mobile apps as traffic to review deliberately, not as automatic exceptions or automatic threats.

Start with the risky action, not a site-wide request ceiling

A rate limit works best when it protects a specific operation: for example, POST requests to a login route or requests that validate one-time passwords (OTPs). A broad rule can penalize ordinary page loads, app activity, or users behind a shared network even when they are not causing the risk.

Confirm the actual hostname, path, and method receiving the unwanted traffic in your analytics or logs. A rule that targets the wrong path can miss the abusive requests altogether. Cloudflare’s rate-limiting best practices emphasize matching the route involved; its rate-limiting rules documentation also describes rule expressions and counting characteristics.

Choose what counts as one user or client

IP address is a simple counting key, but it is not always a person. Offices, schools, mobile carriers, and other shared networks can put many legitimate users behind one public IP address. A strict per-IP ceiling may therefore affect a crowd of unrelated users.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

If your platform and application support reliable identity signals, consider whether a session, account, token, cookie, or operation-based key better represents the activity you need to control. Available counting fields and aggregation options differ by provider and plan. If traffic passes through a CDN or reverse proxy, verify that the rule sees the originating client address correctly; otherwise, many visitors may appear to share the proxy’s address. Cloudflare documents provider-specific counting behavior and limitations in its rate-limiting rules guide.

Measure normal traffic before you enforce a threshold

Look at request patterns before setting a limit. Include ordinary peaks, password-manager retries, batch jobs, partner integrations, and the workflows users follow when something goes wrong. Start in a logging, preview, or count mode when available; inspect which requests a rule would match and look for legitimate traffic that would be caught.

Rank #2
FORTINET | FG-100E | FortiGate-100E Network Security Appliance
  • Protects against known exploits, malware and malicious websites; detects unknown attacks; identify thousands of applications

Google Cloud Armor recommends previewing a new rate-limiting policy and describes using an observed per-IP traffic percentile, such as the 99th percentile, as one way to inform a threshold. That is a tuning method, not a universal safe limit. AWS likewise recommends deploying Bot Control in count mode first and examining its labels in logs before blocking traffic. See Google’s Cloud Armor best practices and AWS’s Bot Control configuration guidance.

When the application exposes failed authentication responses, a login rule can count failures rather than every submission. Cloudflare illustrates counting 401 or 403 responses on login and OTP routes so a successful sign-in does not consume the same failure budget. If valid and invalid OTP submissions both return 200, that distinction is unavailable from the response code; Cloudflare’s guidance instead describes using a lower request-based threshold. Its figures are examples, not copy-and-paste defaults. See Cloudflare’s rate-limiting best practices.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #3
Fortinet Web Application Firewall - Virtual Appliance for All Supported Platforms. Supports up to 1 x vCPU core FWB-VM01
  • Fortinet Web Application Firewall - virtual appliance for all supported platforms. Supports up to 1 x vCPU core
  • Fortinet HW FWB-VM01
  • Manufacturer Part: FWB-VM01

Use graduated responses for uncertain traffic

Do not make every threshold breach an immediate permanent-feeling denial. A throttle or challenge can slow or verify uncertain traffic while allowing a real visitor to continue. Escalate to a temporary block when activity repeatedly exceeds limits or there is stronger evidence of automation.

  • Observe: Log or count matches while you check scope and false positives.
  • Challenge: Ask a suspicious client to verify before continuing, where your provider supports it.
  • Throttle: Slow excessive activity without denying every request outright.
  • Block: Use for repeated excess or high-confidence abuse, with a duration and recovery path appropriate to the risk.

AWS describes using Bot Control labels in the application for step-up verification such as multi-factor authentication (MFA), rather than relying only on an edge decision. If users encounter a challenge or denial, explain what happened and provide a support path. Provider behavior and available actions vary; AWS’s Bot Control guidance and Cloudflare’s bot-challenge documentation describe their respective options.

Example: stage a rule for a login endpoint

  1. Match the exact operation. Scope the rule to the correct hostname, the /login path, and the POST method. Confirm the route against actual traffic rather than assuming the application’s visible login URL is the endpoint receiving requests.
  2. Run it in preview or logging mode. Observe normal retries, password-manager behavior, shared-NAT traffic, and any integrations that use the same route.
  3. Count failures if the response makes that possible. If failed attempts return 401 or 403, consider counting those responses instead of all login submissions. If successful and failed attempts have indistinguishable responses, use request-based counting with a carefully observed threshold.
  4. Begin with a proportionate action. Challenge or throttle elevated activity; reserve blocking for repeat excess or stronger evidence.
  5. Handle trusted automation by identity or verified signals. Do not treat a user-agent string alone as proof that a request comes from a trusted service.
  6. Review impact after rollout. Check logs and user-impact signals, then adjust the rule’s scope, counting key, or threshold if legitimate traffic is caught.

Cloudflare documents an illustrative staged login policy: a managed challenge after four failed attempts in a minute, another challenge after ten failures in ten minutes, and a one-day block after twenty failures in an hour. Its documentation says this example requires Business or higher. These are vendor examples, not general recommendations for a safe login limit. Cloudflare also gives a five-failed-OTP-attempts-per-minute example before a ten-minute block. See Cloudflare’s best practices.

Keep legitimate bots and special clients working

Review verified search crawlers, monitoring services, payment callbacks, webhooks, partner APIs, and mobile-app traffic before enabling broad bot blocking. These clients may have distinct routes or authentication and operational needs. Excluding an API path can be appropriate when its protections are handled separately, but an exclusion should be deliberate and monitored.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Do not allow a client merely because its user-agent header claims to be Googlebot or another known service; headers can be spoofed. Prefer provider-verified bot signals or authenticated application identities where available. Cloudflare notes that bot detection can be more sensitive to mobile traffic and illustrates excluding API paths; AWS says verified bots are permitted by default in Bot Control and supports using labels in application logic. These behaviors are product-specific; consult Cloudflare’s bot-challenge guidance and AWS’s Bot Control use cases.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Check rule order, deployment scope, and provider limits

A rule’s effective behavior depends on where it runs and what happens before or after it. Cloudflare rules execute in order, and some actions stop later evaluation. Confirm rule precedence so a challenge, exception, or earlier rule does not produce an unexpected result.

Cloud Armor applies configured thresholds independently across regions. In a multi-region deployment, the aggregate rate across regions can therefore be higher than the threshold configured for any one region. Rate limits may also be approximate rather than exact: Cloudflare documents that counter updates can lag by seconds, allowing some excess requests to reach the origin before mitigation takes effect. Review plan prerequisites, available counting fields, aggregation behavior, and logging before relying on a feature. See Cloudflare rate-limiting rules and Google’s Cloud Armor rate-limiting overview.

Review the rule after launch

Monitor allowed, challenged, throttled, and blocked requests alongside customer reports, successful conversions, and origin load. Revisit the baseline and rule scope when campaigns, product releases, user geography, or abuse patterns change. If real users are caught, determine whether the cause is an overly broad route, a shared-IP counting key, an underestimated normal peak, or a client that needs a verified exception. Cloudflare also cautions that limiting verified bots can affect SEO; consider that impact before applying a limit to them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

How the provider examples differ

These are examples of product-specific capabilities, not a ranking or endorsement. Compare the counting keys and bot signals your plan supports, whether preview or count modes are available, what challenges and logs you can use, how rules are ordered, and how multi-region deployments behave.

Provider Relevant documented behavior What to verify
Cloudflare Rule expressions, counting characteristics, example response-based counting, managed challenges, and bot-score matching. Rate counters can lag by seconds. Available fields and aggregation options vary by plan; the staged login example requires Business or higher. See rate-limiting rules and best practices.
AWS WAF Bot Control can count and label traffic before enforcement; labels can inform application-level step-up verification. Inspect labels in WAF logs before blocking and confirm originating-client identification behind a CDN. See Bot Control use cases.
Google Cloud Armor Supports throttling and rate-based bans, with preview-mode tuning guidance. Validate application-specific WAF behavior; configured thresholds are independent across regions. See the rate-limiting overview and best practices.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.