Foundations / 2 / 3
Latency and throughput
How long one request takes and how many requests fit into a second are two different numbers. A search API grows from a handful of users to a rush hour: why the average hides the pain, and why a server at 90% is already in trouble.
- 11 min
- Beginner
- 10 steps
- Throughput
- 5 /s
- Latency p50
- 41 ms
- Latency p99
- 60 ms
- Errors
- 0 %
- request
- reply
- redelivery
- duplicate
- lost
Numbers are a model scenario, not measurements
The real request
Users → Search APIRequest
GET /search?q=lamp HTTP/1.1
Host: api.example.com
X-Request-Id1: req_4410
Response
HTTP/1.1 200 OK
Server-Timing2: app;dur=29, db;dur=12
X-Request-Id— Ties this request to its log lines, so its time can be looked up later.Server-Timing— The server's own breakdown: 29 ms of app code and 12 ms of database.
Course: 42 lessons
01 Foundations
02 Scaling
03 Data & storage
04 Communication
05 Reliability
06 Architecture
07 Security
08 Shipping
09 Whole systems
10 Business processes