mufassall

Foundations / 2 / 3

Latency and throughput

How long one request takes and how many requests fit into a second are two different numbers. A search API grows from a handful of users to a rush hour: why the average hides the pain, and why a server at 90% is already in trouble.

  • 11 min
  • Beginner
  • 10 steps
Throughput
5 /s
Latency p50
41 ms
Latency p99
60 ms
Errors
0 %
Open in Lab
5%
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Users → Search API

Request

GET /search?q=lamp HTTP/1.1
Host: api.example.com
X-Request-Id1: req_4410

Response

HTTP/1.1 200 OK
Server-Timing2: app;dur=29, db;dur=12
  1. X-Request-Id — Ties this request to its log lines, so its time can be looked up later.
  2. Server-Timing — The server's own breakdown: 29 ms of app code and 12 ms of database.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway