mufassall

Scaling / 2 / 5

Load balancing

One balancer, three app servers. How the balancer chooses where each request goes, why the wrong choice leaves one server burning while the others idle, how it notices a dead server, and what sticky sessions really cost.

  • 13 min
  • Intermediate
  • 12 steps
Requests
900 /s
Latency p99
140 ms
Errors
0 %
Busiest server
35 %
Open in Lab
35%35%35%
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Load balancer → Server B

Request

TCP 198.51.100.7:52114 -> 192.0.2.12:84431
handshake: SYN, SYN-ACK, ACK
payload: TLS records, unreadable here2
backend chosen once per connection3
  1. TCP — The balancer's whole view: who connects to which port.
  2. TLS — The traffic is encrypted end to end. The balancer cannot see paths or headers.
  3. per connection — One decision per connection. Every request on that connection goes to the same server.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway