Free tools Windows power users keep installed
One-click scans. No signup required.
Choose a token bucket when clients need controlled bursts, a leaky-bucket shaper when downstream work must leave at a steady pace, and a sliding window counter when you want a low-state approximation of a rolling quota. The names alone are not enough: leaky buckets can reject or queue, and a sliding counter estimates rather than records the exact time of every request.
How does API rate limiting work?
A rate limiter decides whether to admit work according to a policy, usually scoped to an identity such as an account, API key, user, IP address, route, or method. Its algorithm determines how it accounts for requests over time and what it does when the policy is exceeded: admit a burst, reject excess work, or delay it in a queue.
As an Amazon Associate I earn from qualifying purchases.
Three separate questions help make a useful comparison: how much burst traffic is allowed, how smoothly admitted work reaches a downstream service, and how closely the limiter enforces a quota over a rolling interval. State and deployment behavior matter too: a limiter that is precise on one server can overshoot when several workers update shared state concurrently.
Token bucket: allow bursts, control the sustained rate
A token bucket has a capacity B and a refill rate r tokens per second. It holds no more than B tokens. As time passes, tokens accrue at rate r, up to capacity; additional refill is discarded. A request with cost c is admitted if the bucket has at least c tokens, then consumes that amount. The bucket may start full or at a configured level.
#1 Best Overall
- API Design Patterns
- ABIS BOOK
- Manning Publications
The two settings serve different purposes: capacity bounds the accumulated burst allowance, while refill rate governs the long-term sustained throughput. After depletion, the bucket can admit more work as tokens arrive. This is not a rule that allows a fixed number of requests in every aligned one-second interval.
For example, a policy with capacity 30 and refill of 5 tokens per second could admit up to 30 one-token requests at once if full. Once depleted, it would replenish at 5 tokens per second. Those are illustrative settings, not a provider default. If requests have different costs, charge according to expected resource use rather than treating every request as equivalent.
AWS documents token-bucket throttling for Elastic Load Balancing, EC2, and API Gateway. The exact scope and enforcement semantics depend on the AWS service and policy; API Gateway, in particular, describes configured throttling values as best-effort targets rather than guaranteed ceilings.
Recommended Free Tools
Leaky bucket: distinguish policing from shaping
“Leaky bucket” describes related designs, not one universal API behavior. The key distinction is whether excess work is rejected immediately or held for later.
Rank #2
Policing rejects work over the threshold
In the model described by RFC 7415 for SIP rate control, bucket content drains continuously and increases by an increment for each forwarded request. If content exceeds a tolerance threshold, the request is rejected. This provides a way to control the admitted rate while allowing a configured amount of tolerance. RFC 7415 is a formal model for SIP overload control; it should not be read as a specification for every API gateway.
Shaping queues work for a steady output
A shaping implementation queues excess requests and releases them at a controlled pace. That can protect a downstream service from a sudden surge, but changes rejection into delay. It therefore needs a bounded queue, an overflow policy, and a plan for latency and work that cannot be completed in time. Not every leaky-bucket limiter queues; the RFC model above rejects over-threshold requests.
Sliding window counter: estimate a rolling quota with low state
A common sliding window counter uses two fixed-window counters: one for the current interval and one for the immediately preceding interval. If e is the fraction of the current window that has elapsed, it estimates the count in the rolling interval as:
estimated count = current count + previous count × (1 − e)
The previous interval contributes less as the current one progresses. The limiter compares the estimate with its configured limit and admits or rejects accordingly. Because it blends adjacent windows, this avoids the sharp reset boundary of a simple fixed-window counter, while keeping a constant number of counters per identity.
The trade-off is precision: the counter does not retain each request’s timestamp, so its estimate can differ slightly from the exact number of events in the rolling interval. Redis’s tutorial presents this as a low-memory implementation pattern, not a standard or guarantee of exact enforcement.
Compare the three algorithms
| Model | Burst behavior | Output pace | Rolling-quota precision | State and overload behavior |
|---|---|---|---|---|
| Token bucket | Explicit burst allowance up to capacity B | Requests can arrive in bursts; refill rate r controls sustained admission over time | Not an exact count in every aligned or rolling interval | Tracks token balance and time; normally admits or rejects, rather than queueing |
| Leaky-bucket policing | Allows configured tolerance before the threshold is exceeded | Controls admitted rate; the cited RFC model does not smooth output with a queue | Depends on the specific policing policy; it is not automatically an exact rolling request count | Tracks bucket content; rejects requests above threshold |
| Leaky-bucket shaping | Can absorb a burst up to available queue capacity | Releases queued work at a configured pace | Not inherently a rolling quota | Requires a bounded queue and overflow behavior; excess work is delayed or dropped |
| Sliding window counter | Reduces the adjacent-window spike of a fixed counter | Does not itself guarantee evenly spaced output | Approximate rolling count using weighted neighboring windows | Uses two counters per identity in the common pattern; rejects or admits based on the estimate |
How to choose a rate-limiting model
- Permit controlled bursts while bounding sustained throughput: start with a token bucket. Capacity and refill rate map directly to those two goals.
- Protect a downstream service by smoothing eligible work: use a shaping queue if delayed execution is acceptable; set queue and overflow limits to match the service’s latency budget.
- Reject overload without queueing: use policing when its threshold semantics fit the contract, or a sliding counter when the desired policy is an approximate rolling request quota.
- Avoid obvious fixed-window boundary spikes at low state cost: use a sliding window counter, accepting that it is an estimate.
- Require an exact rolling-window count: consider a sliding-window log that retains request timestamps. Redis’s tutorial contrasts timestamp logs with counters: timestamps provide event-level information but add per-request state and the work of pruning and counting.
- Defer work rather than reject it: a queue or stream may be more appropriate when processing is asynchronous. AWS recommends queues or streams to smooth workloads that can be buffered; this is a workload-architecture choice, not another name for a rate-limit algorithm.
Why fixed-window counters can admit boundary bursts
A fixed-window counter resets at a boundary, so requests near the end of one interval and requests near the start of the next can consume two quotas close together. Cloudflare AI Gateway illustrates this with a limit of ten requests per ten minutes: ten requests at 12:09 and another ten at 12:11 pass under adjacent fixed windows, although more than ten occurred in the rolling ten-minute interval. Its sliding-window example rejects the second set because the earlier requests still count.
A sliding counter reduces this specific discontinuity by carrying a weighted share of the preceding window forward. It does not make its estimate an exact timestamp log; if the contract requires exact rolling counts, use an approach that stores events or select a different explicit policy.
Rank #4
Provider examples are not universal defaults
These figures describe provider-specific documentation, not recommended starting values for arbitrary APIs. AWS documentation accessed in 2026 gives the following examples:
- Elastic Load Balancing: an account-level bucket with capacity 40 tokens and refill of 10 request tokens per second; for its non-mutating request category, capacity 200 and refill 50 per second.
- EC2: a
DescribeHostsrequest bucket with capacity 100 tokens and refill of 20 per second; for the resource-rate bucket example,RunInstanceshas capacity 1,000 tokens and refill of 2 per second.
Cloudflare’s API limits documentation in 2026 lists a global client API limit of 1,200 requests per five-minute period per user and a client API limit of 200 per second per IP. It also lists a GraphQL maximum of 320 requests per five minutes, with limits varying by query cost. These are Cloudflare-specific limits, not a general rate-limit recipe; provider policies can change.
Managed enforcement also has semantics beyond the underlying algorithm. Amazon API Gateway says its throttles and quotas are applied on a best-effort basis and should be treated as targets rather than guaranteed request ceilings; it notes that requests may receive HTTP 429 responses when throttled. A mathematically exact limiter in your own service does not make a separate provider’s managed limit a hard global ceiling.
Implementation details that determine whether the policy works
Choose the key and scope
Define exactly who or what shares a limit: account, API key, user, IP, route, method, resource, or a combination. Also decide whether a rule is global, regional, stage-level, route-level, or client-specific. AWS API Gateway documents account/Region, stage or method, and usage-plan/client scopes; EC2 documents per-account and per-Region behavior alongside per-API token buckets. Layered policies can protect different things, but operators need to know which layer rejected a request.
Best Value
Set request cost to reflect the work
One token per request assumes operations impose roughly equal load. For APIs with materially different costs, use weighted charges, separate buckets, or resource-based quotas. EC2 documents resource token buckets for actions such as RunInstances and TerminateInstances, in addition to request throttling.
Make shared updates atomic
In a multi-instance deployment, workers that independently read and update a shared counter can all observe available capacity and collectively overshoot it. Redis’s sliding-counter example uses an atomic Lua script to read counters, calculate the weighted estimate, and conditionally increment them as one operation. Its key scheme and code are an implementation example; review the datastore’s consistency and failover behavior, hot-key load, and cluster-slot constraints for your own deployment.
Expose the policy that actually rejected the request
When several limits can apply, observability should identify the relevant key and policy, remaining capacity where meaningful, and retry guidance where available. This makes it possible to distinguish an account-wide ceiling from a route-level guard or a per-client quota, rather than treating every throttle as the same incident.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsHow to handle 429 Too Many Requests
A 429 response means the request was throttled; retrying immediately can add load without changing the decision. Clients should follow any provider-supplied retry timing, use exponential backoff with jitter when appropriate, and keep retries bounded and rate-limited. Cloudflare documents rate-limit headers and Retry-After for its REST APIs; exact headers and behavior vary by provider, so clients should follow the API’s documented contract.
For service owners, return a documented throttling response and communicate retry timing when the platform supports it. AWS recommends handling throttling gracefully and testing intended limits before raising them. For queued shaping, make the delay and queue-overflow outcome explicit rather than allowing unbounded work to accumulate.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
