mufassall

Scaling / 1 / 5

Scaling up vs scaling out

A feed API outgrows its server. First it moves to a bigger box, until there is no bigger box and the one box falls over. Then it spreads over several smaller ones, and learns why that needs a load balancer and an app that remembers nothing.

  • 11 min
  • Beginner
  • 10 steps
Served
300 /s
Latency p95
80 ms
Errors
0 %
Cost per month
160 $
Open in Lab
45%15%
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Users → App server 1

Request

GET /feed HTTP/1.1
Host: api.example.com

Response

HTTP/1.1 200 OK
X-Served-By1: app-1
Server-Timing2: app;dur=38
  1. X-Served-By — Which server answered. Right now there is only one.
  2. Server-Timing — 38 ms of app work. The rest is network.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway