Backpressure: Bounding Queues Before They Bound You
Backpressure is the practice of letting a slow stage in a system push back on the stage feeding it, so that work cannot pile up faster than it can be finished. The most common form is a bounded queue: the buffer in front of a worker is given a fixed maximum size, and once it is full the system refuses new work instead of accepting it. That refusal is the backpressure, and it travels upstream, telling the producer to slow down, retry later, or give up.
The alternative is an unbounded queue, which is what you get by default from an in-memory array, an unbuffered channel left to grow, or a message consumer that accepts everything offered. An unbounded queue looks safe because it never rejects anything, and it is the single most common way a system that was merely slow becomes a system that is down. In the simulation below, a service running 30% over capacity with an unbounded queue reaches a six-minute request latency and keeps climbing; the same service with a bounded queue holds latency at 168 milliseconds by refusing the 23% of requests it was never going to serve anyway.
In the full article
What it covers
- The Unbounded Queue Is a Latency Bomb
- Little's Law
- Bounding the Queue
- A Bounded Work Queue in TypeScript
- Choosing the Limits
- Shedding Load Well
- Backpressure Across a System
- When You Do Not Need It
- Key Takeaways
The full article is on Bliztek, LLC Blog, my web development company's blog. This page is the summary.
- Read time
- 10 min
- Sections
- 9
