Rate limiting is the kind of feature that feels optional until the day someone runs a script against your API without thinking about it, and your database CPU goes to 100%. At that point it stops being optional.

The good news is that rate limiting is well-understood, the algorithms are straightforward, and there’s no reason to implement the storage layer yourself.

Why Rate Limiting Exists

An API without rate limiting is vulnerable to several failure modes:

Accidental overload: a well-intentioned client has a bug - a missing backoff in a retry loop, a misconfigured cron job, a forgotten while(true). Without rate limiting, their bug becomes your incident.

Intentional abuse: scrapers, brute force login attempts, DDoS. Rate limiting doesn’t stop sophisticated attackers, but it raises the cost significantly for the opportunistic ones.

Resource fairness: one client consuming 90% of your API capacity degrades service for everyone else. Rate limits protect shared infrastructure.

Cost control: if your API calls cost money (LLM inference, third-party data lookups, outbound SMS), unlimited API calls means unlimited cost exposure.

The Algorithms

Fixed window counter: count requests in a fixed time window (e.g., the current minute). When the count exceeds the limit, reject requests until the window resets.

Window: 12:00:00 - 12:01:00
Limit: 100 requests per minute
Count: 87

Request at 12:00:45: allowed (87 < 100), count becomes 88
Request at 12:01:01: allowed (new window, count resets to 1)

Simple to implement. The problem: a burst can effectively double the limit at window boundaries. A client can send 100 requests at 12:00:59 and another 100 at 12:01:01 - 200 requests in 2 seconds, both within their “per minute” limits.

Sliding window log: store the timestamp of every request. On each new request, count how many requests fall within the window. Reject if over the limit.

More accurate but expensive: requires storing and querying a log of all request timestamps. Memory usage scales with request rate.

Sliding window counter: approximate the sliding window without storing individual timestamps. Blend the current window’s count with a weighted portion of the previous window’s count:

current_count + previous_count * ((window_size - elapsed) / window_size)

This gives you most of the accuracy of the sliding window log with a much lower memory footprint. It’s the most common production algorithm.

Token bucket: a bucket holds up to N tokens. Tokens refill at a constant rate. Each request consumes one token. If the bucket is empty, the request is rejected.

The token bucket allows bursts up to the bucket capacity, then enforces an average rate. Useful for APIs where short bursts are acceptable but sustained high rate isn’t.

Leaky bucket: requests enter a queue (the “bucket”) which processes them at a constant rate. If the queue is full, requests are dropped. This enforces a strict constant output rate, useful for smoothing traffic before passing to downstream services.

For most API rate limiting, the sliding window counter or token bucket are the right choices.

What to Limit By

The granularity of rate limiting matters as much as the algorithm.

By IP address: the simplest default. Doesn’t require authentication. Problems: legitimate users behind NAT share an IP, so a rate limit per IP can unfairly block multiple users; sophisticated abusers use IP rotation.

By API key / authenticated user: more accurate - each client gets their own limit. Requires authentication. This is the right approach for developer APIs.

By user account: for user-facing features (login attempts, form submissions). Rate limiting login attempts is one of the most effective mitigations against credential stuffing.

By endpoint: different endpoints have different costs. GET /users/:id is cheap; POST /reports/generate might be expensive. Apply tighter limits to expensive endpoints.

In practice, combine these: global IP-based limits as a first defense, authenticated limits per client for precise control.

Implementing It

Redis is the standard backing store for rate limiting counters in distributed systems. All instances of your service share the same counter, preventing a client from bypassing per-instance limits by hitting different servers.

A sliding window counter in Redis (Lua script ensures atomicity):

local key = KEYS[1]
local window = tonumber(ARGV[1])  -- window size in seconds
local limit = tonumber(ARGV[2])
local now = tonumber(ARGV[3])     -- current timestamp in ms

-- Remove old entries
redis.call('ZREMRANGEBYSCORE', key, 0, now - window * 1000)

-- Count current requests
local count = redis.call('ZCARD', key)

if count < limit then
    -- Add current request
    redis.call('ZADD', key, now, now)
    redis.call('EXPIRE', key, window)
    return 1  -- allowed
else
    return 0  -- rejected
end

import redis
import time

r = redis.Redis()

def is_rate_limited(client_id: str, limit: int = 100, window: int = 60) -> bool:
    key = f"ratelimit:{client_id}"
    now = int(time.time() * 1000)
    result = r.eval(RATE_LIMIT_SCRIPT, 1, key, window, limit, now)
    return result == 0

Don’t reinvent this. Libraries like limits (Python), express-rate-limit (Node.js), and rack-attack (Ruby) implement common algorithms with Redis backing. Use them.

For infrastructure-level rate limiting, API gateways (Kong, AWS API Gateway, NGINX with limit_req) handle rate limiting before requests reach your application. This is more efficient and doesn’t require changes to your service code.

Response Headers

Tell clients their limit and remaining quota in response headers. This lets clients self-govern and avoid hitting limits:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 47
X-RateLimit-Reset: 1713485200
Retry-After: 45

Retry-After is critical on 429 (Too Many Requests) responses. Clients that respect it will back off. Clients that don’t will hammer your API anyway - but you’ve done your part.

Return 429 Too Many Requests (not 503) when rate limiting. The distinction matters: 503 means the server is unavailable; 429 means the client is sending too many requests.

Avoiding False Positives

Rate limiting shared IPs incorrectly blocks legitimate users. Common scenarios:

• Office networks where thousands of employees share a few IP addresses
• Mobile networks with carrier-grade NAT
• Tor exit nodes legitimately used for privacy

Apply IP-based rate limits generously (high enough to not trigger for normal legitimate use) and use them as a last resort, with per-user limits as the primary mechanism for authenticated traffic.

For login rate limiting, rate limit by username (or email), not just by IP. An attacker using IP rotation still triggers the per-username limit.

Burst Traffic and Fairness

Not all high-frequency access is abuse. A developer integrating your API might legitimately need to make 500 calls in a short period for a one-time data migration. A mobile app might batch-sync data on first launch. Rate limiting that treats all burst traffic as abuse blocks legitimate users at the worst possible moment - when they’re most engaged.

The token bucket algorithm handles this better than fixed windows because it allows short bursts up to the bucket capacity. A developer with a limit of 1000 requests/minute who hasn’t made a request in 10 minutes has accumulated credit they can spend on a burst. This is more reflective of real usage patterns than a hard window that resets every 60 seconds.

For APIs with diverse client types, consider tiered limits: a free tier with a lower limit and stricter burst control, a paid tier with higher limits and more burst capacity. The tier difference is both a product boundary and an abuse boundary.

Fairness means that one client’s burst doesn’t degrade service for others. Per-client limits enforce this. Global limits (total requests per second across all clients) protect infrastructure but don’t enforce fairness between clients.

Abuse and the Client Experience

Rate limiting without communication produces a bad experience for legitimate clients and ineffective protection against malicious ones.

Malicious clients don’t respect Retry-After headers or back off when they see 429s. They’re designed to push through limits. Rate limiting slows them down but doesn’t stop determined attackers - it raises the cost. Combine rate limiting with anomaly detection (unusual request patterns, unusual error rates) for actual abuse prevention.

For legitimate clients, the experience matters. A 429 with no explanation and no retry guidance leaves the developer confused and the integration broken. A 429 with Retry-After, X-RateLimit-Remaining, and a link to documentation about limits gives the developer what they need to build a resilient client.

The best client SDKs handle rate limits transparently: they read the headers, back off when needed, and retry automatically. If you publish a client library, build this in. If you don’t, document the expected behavior clearly enough that developers can build it themselves.

One pattern that helps both legitimate clients and abuse detection: differentiate between clients who are rate-limited occasionally (normal behavior near a limit) and clients who are rate-limited constantly (either the limit is wrong, or they’re hammering). The former might need a higher tier. The latter might need a conversation or a block.