Rate Limiting: Token Buckets and Sliding Windows in Practice
Published on · In Software Engineeringby Steven Brown
A rate limiter is a component that caps how many requests a client can make within a period of time. Every incoming request is checked against that client's recent usage: requests inside the allowance proceed, and requests beyond it are rejected, usually with an HTTP 429. Services rate limit to protect shared capacity from overload, to keep one heavy client from degrading service for everyone else, and to put a ceiling on cost.
The design question inside every rate limiter is how "recent usage" is measured, and the standard answers form a neat progression. This post builds the four of them in order: the fixed window, the sliding window log, the sliding window counter, and the token bucket. Each one fixes a specific weakness of the one before it, and each is a few dozen lines of TypeScript with an injected clock, the same testing habit we used in the LRU cache post. The closing sections cover what to send back with a rejection and what changes when the limiter runs on more than one server.
