Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
API rate limiting controls how quickly a caller can consume an API. It protects backends from overload, allocates capacity more fairly, and gives clients a predictable signal when they must slow down. The most reliable design separates a sustained rate from a burst allowance, chooses an identity and scope that match the fairness goal, and decides where counters live across replicas and regions. A limit is not automatically a perfectly hard global ceiling: Amazon API Gateway describes its throttles as best-effort targets, and Microsoft cautions that distributed throttling is never completely accurate. This guide explains the algorithms, configuration choices, HTTP 429 behavior, provider-specific caveats, and production checks needed to make a limit behave as intended.
What is API rate limiting and why does it matter?
API rate limiting is a policy that admits, delays, or rejects requests according to a measured allowance over time. The allowance may belong to an account, API key, subscription, tenant, route, operation, or caller IP. A gateway, service, or middleware component evaluates the request before expensive backend work begins, then records the decision in a counter or bucket.
As an Amazon Associate I earn from qualifying purchases.
Free tools Windows power users keep installed
One-click scans. No signup required.
Good limits solve two different problems:
- Protection: they keep a sudden spike from exhausting connection pools, CPU, memory, database capacity, or downstream quotas.
- Fairness: they prevent one noisy caller from consuming capacity intended for other tenants or operations.
Those goals can conflict. A single gateway-wide limit is simple, but a small group of busy callers can consume it and make unrelated users fail. Per-tenant or per-subscription limits isolate that impact, although they require more keys and more accounting. A limit also does not replace authentication, authorization, payload validation, caching, queueing, or a strategy for handling malicious traffic; it is one control in the request path.
Think of a policy as a contract rather than a magic number. The contract should state who shares an allowance, which operations count, whether capacity is regional or global, how bursts are treated, what response a rejected request receives, and how clients are expected to retry.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
How does API rate limiting work on a request path?
At a high level, a limiter identifies the caller, reads the relevant state, applies an algorithm, and either forwards or rejects the request. A typical flow is:
- Authenticate or identify the request. Derive a stable key such as a tenant ID, subscription, API key, authenticated account, route, or IP address. Do not use an identity that changes between retries or replicas.
- Select the policy. Match the key and operation to a configured rate, burst capacity, window, queue, or combination of policies. A route-specific rule can protect an expensive operation more tightly than a cheap read.
- Read and update state. The limiter checks a token bucket, window counter, queue, or shared counter. The placement of this state determines whether multiple replicas see the same traffic.
- Make the admission decision. If capacity remains, the request proceeds and the state is updated. If it does not, the service can reject immediately or, with a leaky-bucket-style design, queue work for later emission.
- Return a useful signal. HTTP 429 is a common response when a caller exceeds a configured rate. Amazon API Gateway documents 429 responses after throttling, and Azure API Management documents the same response for its rate-limit policy. AWS API Gateway throttling documentation and Azure rate-limit policy reference
- Record telemetry. Capture accepted requests, throttles, latency, backend errors, and the identity and operation dimensions needed to explain a decision without logging secrets.
Each step can introduce approximation. Clock differences, concurrent updates, replication delay, queue latency, and requests already in flight mean that an observed count may not match a configured value exactly. The closer a policy gets to a globally shared hard cap, the more coordination it needs.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallWhich rate-limiting algorithm should you choose?
Choose based on the traffic shape you want to permit. Token bucket is a strong default when short spikes are acceptable but sustained throughput must be bounded. Leaky bucket is better when the backend needs a smooth output rate. Fixed windows minimize implementation complexity, while sliding windows reduce bursts at interval boundaries at the cost of additional state.
| Algorithm | Behavior | Useful when | Tradeoff |
|---|---|---|---|
| Token bucket | Allows bursts up to a configured capacity and replenishes at a steady rate. | Short spikes should be absorbed while sustained throughput stays bounded. | A burst can exceed the steady rate temporarily; implementation counters may be approximate. |
| Leaky bucket | Emits work at a constant rate. | A backend needs smoother, more predictable ingress. | Smoothing can add queuing delay or require a policy for excess traffic. |
| Fixed window | Counts calls within fixed intervals. | Simplicity is important. | Requests at adjacent window boundaries can create a short burst. |
| Sliding window | Tracks a rolling interval to smooth boundary effects. | Boundary bursts are undesirable. | More state is needed than for a simple fixed window. |
These behaviors and tradeoffs are summarized in the Microsoft Azure Architecture Center throttling pattern.
When is a token bucket the right fit?
A token bucket starts with a capacity and refills at a configured rate. A request consumes tokens; when the bucket is empty, the request is throttled or delayed. Because unused tokens accumulate up to the capacity, a quiet period can fund a short burst without changing the long-term average.
Capacity and refill rate are distinct settings. For an illustrative, non-universal policy, a refill rate of 100 requests per minute with a capacity of 20 permits up to 20 immediately available requests, then sustains about 100 per minute as tokens return. Calling both values “the limit” hides the behavior that clients will actually experience.
When does a leaky bucket improve backend stability?
A leaky bucket removes work at a fixed rate, so arrivals are converted into a smoother stream. This is useful when a database, third-party service, or single-threaded worker cannot tolerate bursts even if the average volume is acceptable. You must define what happens when the queue is full: reject excess work, shed the oldest work, or apply a separate priority policy. Queueing also means latency can rise before a request is rejected.
What are the boundary effects of fixed and sliding windows?
A fixed window resets all counts at a boundary such as the start of a minute. A caller can send its allowance at the end of one window and again at the start of the next, creating a short burst that is larger than the nominal per-minute figure. A sliding window evaluates a rolling interval, which reduces that edge effect, but it stores more timestamps or weighted counts and therefore costs more memory and coordination.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
How do token-bucket rate and burst settings differ?
The rate is the long-term refill speed, expressed in units such as requests per second or calls per minute. The burst is the maximum accumulated capacity available for an immediate spike. A policy that lists only one number cannot tell an integrator whether ten requests may arrive at once or must be evenly spaced.
Document both values with their scope and units. State whether the burst is per key, route, stage, region, or account, and whether rejected requests consume any capacity. Keep the refill rate aligned with measured backend capacity; set burst capacity according to the amount of short-lived work the backend can absorb without queue growth. The values are policy decisions, not industry benchmarks.
Amazon API Gateway exposes separate rate and burst targets for account and route or stage throttling. Its documentation also warns that these are best-effort targets rather than guaranteed ceilings, so a configuration should not be advertised as an exact global cap. Read the provider’s HTTP API throttling description.
What identity and scope should a limit use?
Use the narrowest key that matches the protection or fairness objective. A tenant-fairness policy should usually charge each tenant separately; a backend-protection policy may need an aggregate limit as well. Combining a local limit with a broader safety limit is often more useful than forcing every caller into one shared bucket.
Account, tenant, or subscription
Per-account, per-tenant, or per-subscription keys prevent a busy customer from consuming another customer’s allowance. They require reliable identity extraction and a policy for accounts that use multiple API keys. Azure API Management documents subscription, API, and operation scopes for its rate-limit policy, but the exact counting behavior depends on the gateway deployment.
API key or authenticated principal
API-key limits are straightforward for partner integrations, provided keys are not shared by many independent users. An authenticated principal gives better attribution in multi-user applications. Rotate keys without accidentally merging the old and new identities during a traffic spike, and avoid putting secret key material into metric labels.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Route or operation
Apply a tighter policy to expensive operations such as report generation, search, or writes while allowing more headroom for inexpensive reads. Count the operation consistently across aliases and versioned paths, or a caller may bypass the intended rule by changing the URL form.
Caller IP
IP-based limits are a useful unauthenticated fallback, but NAT, mobile networks, proxies, and corporate gateways can put many legitimate users behind one address. IPv6 formatting and trusted-proxy configuration must be normalized before deriving a key. Do not treat an IP limit as a substitute for user or tenant identity when those identities are available.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
A broad gateway-level limit can create collateral damage when only a small caller group is noisy. Start with the fairness dimension you can explain to a customer, then add aggregate protection for the backend itself.
How do distributed counters change accuracy?
In a single process, a counter or bucket can be updated locally with little coordination. In a replicated service, each node may see only part of the traffic. Microsoft’s guidance describes local counters as fast but capable of undercounting traffic spread across replicas, while shared counters observe more requests at the cost of added decision latency. Azure’s throttling pattern discusses local and centralized counters.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesLocal counters
Local state minimizes network calls and keeps the limiter available when a shared store is unhealthy. Its allowance is effectively multiplied across replicas unless traffic is consistently routed to the same node. That can be acceptable for a soft protection threshold, but it is unsuitable when the contract promises one precise allowance for all nodes.
Shared counters
A centralized store or coordinated service gives multiple replicas a common view. Atomic updates are important: a read followed by a separate write can admit concurrent requests that should have competed for the same token. The shared decision path adds latency and creates another dependency, so define fail-open and fail-closed behavior before an outage occurs.
Multiple regions
Decide whether each region receives an independent allowance or all regions share one global budget. Gateway-local behavior does not magically aggregate counts across a deployment, and network delay makes globally synchronized decisions harder. If a global cap is a contractual requirement, use an architecture that provides the required consistency and document the residual accuracy limits; otherwise describe the allowance as regional or best effort.
Azure API Management specifically notes that distributed throttling is not completely accurate and that actual counts can vary with request volume, backend latency, and other factors. Its documented policy also tracks counts independently at each gateway rather than aggregating them across the entire instance. See the Azure rate-limit policy limitations.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How should clients handle HTTP 429?
Treat 429 as a signal to reduce pressure, not as an invitation to retry immediately in a tight loop. A client should stop or defer work, apply a bounded retry policy, and add jitter so many callers do not resume at the same instant. Amazon’s API Gateway guidance recommends graceful throttling handling, and its FAQ covers throttling behavior for callers. Amazon API Gateway FAQs
- Classify the response as retryable only when the service contract says the operation is safe to repeat. Non-idempotent writes need an idempotency key or another deduplication mechanism before automatic retry.
- Use exponential backoff with a maximum delay and a maximum attempt count. The cap prevents a single request from consuming resources indefinitely.
- Add random jitter to each delay. This spreads retries from synchronized clients across time.
- Honor any retry timing information the service provides, while retaining a client-side maximum so a broken value cannot stall the application forever.
- Stop retrying when the operation’s deadline, user request, or queue budget expires, and surface a clear error to the caller.
Servers should make 429 responses cheap, include enough request or policy context for support diagnostics, and avoid doing expensive backend work before rejecting. Clients should also limit concurrency: backoff alone does not help if hundreds of workers continue starting new requests.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
What do AWS and Azure provider implementations actually guarantee?
Provider settings are examples of implementation behavior, not universal quotas or recommendations. Read the service’s current contract before publishing a number or promising exact enforcement.
Amazon API Gateway: best-effort token-bucket throttling
Amazon API Gateway documents token-bucket throttling with separate rate and burst targets at account and route or stage levels. It states that throttling is a best-effort target rather than a guaranteed ceiling, and callers can receive HTTP 429 after limits are exceeded. The practical implication is to size downstream capacity for the possibility of temporary variation instead of assuming every request beyond a threshold will be rejected instantly. Amazon API Gateway HTTP API throttling
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Amazon EC2: separate request and resource buckets
Amazon EC2 illustrates that a service may meter more than raw request count. Its API throttling model uses request-rate buckets and separate resource-rate buckets for some operations, each with burst capacity and refill behavior. This is a service-specific design; do not copy its bucket dimensions into a different API without an equivalent workload model. Amazon EC2 request throttling
Azure API Management: policy scopes and gateway-local behavior
Azure API Management’s rate-limit policy returns 429 when the configured call rate is exceeded. The documentation warns that distributed throttling is not completely accurate, explains that counts can vary with request volume, backend latency, and other factors, and notes that counters are tracked independently at each gateway for the described policy. Treat its scope and accuracy notes as provider-specific characteristics, not as a definition of rate limiting everywhere. Azure API Management rate-limit policy
How do you design and roll out limits in production?
Use a staged process so that a limit protects capacity without surprising legitimate callers.
- Define the protected resource. List the backend, dependency, queue, database, or operation whose capacity you are protecting. Identify whether the risk is sustained load, bursts, concurrency, expensive individual requests, or unfair tenant consumption.
- Choose the fairness key. Select tenant, subscription, account, API key, principal, route, operation, IP, or an explicit combination. Write down which callers share a bucket and how anonymous traffic is grouped.
- Select the algorithm. Use token bucket for controlled bursts, leaky bucket for smooth output, fixed window when simplicity outweighs boundary bursts, or sliding window when those boundaries are unacceptable.
- Set separate rate and burst values. Express units clearly and record whether they are per second, minute, request, resource, region, or another dimension. Do not present a provider example as a general recommendation.
- Choose counter placement. Decide whether local undercounting is acceptable or whether shared state and its latency are required. For shared state, define atomicity, expiration, clock handling, and behavior if the store is unavailable.
- Specify regional semantics. State whether allowances are independent per region or intended to be global. If the architecture cannot make a global decision exact, describe the policy as best effort and set customer expectations accordingly.
- Define rejection and retry behavior. Return 429 for rate exhaustion where appropriate, document safe retry conditions, and provide bounded backoff guidance. For queued work, document queue limits and what happens when the queue is full.
- Launch in observation mode. Measure what would have been throttled before enforcing a new policy. Compare projected rejects with tenant activity, backend saturation, and known batch jobs.
- Roll out gradually. Apply the policy to a small tenant set or a percentage of traffic, watch error and latency changes, then expand. Keep a tested rollback or temporary exemption path for migrations and incident response.
- Revisit after workload changes. New endpoints, client retries, regional expansion, and backend redesigns can invalidate an old limit. Version the contract and announce material changes rather than silently changing behavior.
What should you monitor and test?
A rate limiter is working only when its decisions and the protected system can be explained together. Monitor accepted request rate, throttled request rate, active keys, burst consumption, queue depth if applicable, backend latency, saturation indicators, and 4xx and 5xx errors. Break these metrics down by policy scope and operation; an aggregate success rate can hide one tenant that is constantly rejected.
Test traffic patterns, not just a steady stream:
- Steady load: verify that a caller near the refill rate remains admitted without oscillation.
- Short burst: confirm that the configured capacity is available and that sustained traffic settles at the refill rate.
- Window boundary: for fixed-window policies, test calls on both sides of a reset and decide whether the resulting burst is acceptable.
- Replica spread: send the same key through multiple nodes to measure local undercounting or shared-counter latency.
- Regional split: direct traffic to more than one region and compare the observed allowance with the documented regional contract.
- Concurrent updates: issue simultaneous requests for one key to detect non-atomic check-and-increment behavior.
- Dependency failure: disable or delay the counter store and verify the chosen fail-open or fail-closed behavior.
- Retry storm: return 429 to many clients and confirm that bounded backoff and jitter reduce, rather than amplify, load.
Use production-like identities and payload costs. A request counter can look healthy while a small number of expensive operations exhaust backend capacity, which is why some services meter resources as well as requests.
What are common rate-limiting failure modes?
A limit is exceeded even though the dashboard shows spare capacity
Check whether counters are local to replicas, whether the dashboard aggregates a different time interval, and whether multiple routes share the same key. Local counters can undercount distributed traffic; shared counters can also report delayed observations.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Legitimate tenants receive 429 during another tenant’s spike
The policy is probably scoped too broadly. Split the allowance by tenant, subscription, account, or another fairness identity, then retain a separate aggregate safety limit for the backend.
Clients create more load after throttling begins
Look for immediate retries, unlimited attempts, synchronized worker schedules, or missing concurrency caps. Enforce bounded exponential backoff with jitter and make non-idempotent operations safe to retry before enabling automation.
Traffic surges at every interval reset
This is the fixed-window boundary effect. Move to a sliding window or token bucket if smoother admission matters, or stagger client schedules and set a burst policy that explicitly accounts for the edge.
Backend latency rises although request rate is within the limit
The rate may ignore request cost, concurrency, payload size, or a constrained downstream resource. Add operation-specific policies, a queue with a defined capacity, or a resource-based dimension rather than increasing a request-count threshold blindly.
The published quota sounds exact but observed counts vary
Qualify the contract as best effort when the provider or distributed architecture cannot guarantee a hard global ceiling. Document replica and region behavior, then size backend safety margins for in-flight and coordination effects.
How should a rate-limit contract be documented?
Publish a concise policy reference alongside the API specification. Include:
Recommended Free Tools
- the identity key and every scope that shares an allowance;
- the algorithm and whether it admits bursts, queues work, or resets windows;
- the sustained rate, burst capacity, units, and whether values are recurring or temporary;
- which routes, operations, resource types, and regions are covered;
- whether counters are local, shared, or provider-managed, and the resulting accuracy expectation;
- the status and response shape for throttled calls, including safe retry guidance;
- idempotency requirements for retried writes;
- how clients can request a limit review and how policy changes are versioned.
That documentation prevents a common integration error: treating a nominal per-minute number as an exact, globally synchronized promise when it is actually a best-effort, burst-capable policy applied independently at several gateways.
Frequently Asked Questions
Can API rate limiting prevent a denial-of-service attack by itself?
No. It can reduce pressure from identified callers and protect application capacity, but a volumetric attack may saturate links or edge infrastructure before the API limiter runs. Pair rate limiting with network-level filtering, authentication controls, caching, and an incident response plan appropriate to the threat.
Should I limit requests by IP address or by API key?
Use an authenticated identity such as a tenant, subscription, or API key when you need customer-level fairness. IP limits are useful for anonymous traffic, but shared networks can combine many legitimate users and make one address look abusive.
Do rejected requests count against a client’s quota?
There is no universal answer; it depends on the implementation and provider contract. State explicitly whether a rejected attempt consumes tokens or window count, because clients and retry logic behave differently under each rule.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How can I change a limit without breaking batch jobs?
Version the policy, announce the effective time, and provide a migration window in which clients can observe throttling before enforcement tightens. Schedule large jobs with bounded concurrency and resumable checkpoints so a temporary 429 does not restart the entire batch.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
