Foundations / 2 / 3
Latency and throughput
How long one request takes and how many requests fit into a second are two different numbers. A search API grows from a handful of users to a rush hour: why the average hides the pain, and why a server at 90% is already in trouble.
- 11 min
- Beginner
- 10 steps
- Throughput
- 5 /s
- Latency p50
- 41 ms
- Latency p99
- 60 ms
- Errors
- 0 %
- request
- reply
- redelivery
- duplicate
- lost
Numbers are a model scenario, not measurements
The real request
Users → Search APIRequest
GET /search?q=lamp HTTP/1.1
Host: api.example.com
X-Request-Id1: req_4410
Response
HTTP/1.1 200 OK
Server-Timing2: app;dur=29, db;dur=12
X-Request-Id— Ties this request to its log lines, so its time can be looked up later.Server-Timing— The server's own breakdown: 29 ms of app code and 12 ms of database.
What happens if…?
Put in your own numbers: the result and the formula update at once.
Requests finished per second.
How long one request spends in the server.
Most requests one server works on at once.
How busy you plan to run the servers.
Result
- In flight L
- 8.2
- Servers needed
- 1
- Load per server
- 13%
Working
L = λ × W = 200/s × 0.041 s = 8.2
N = ⌈L / (c × u)⌉ = ⌈8.2 / (64 × 0.70)⌉ = 1
load = L / (N × c) = 8.2 / (1 × 64) = 13%A simplified teaching model: it shows direction and scale, not a benchmark.
Build it yourselfCheat sheetCompare lessons
Course: 74 lessons
01 Foundations
02 Scaling
03 Data & storage
04 Communication
05 Reliability
06 Architecture
07 Security
08 Shipping
09 Whole systems
10 Business processes
11 Case studies