mufassall

Data & storage / 9 / 11

Consensus and leader election

Two copies of a nightly payout job must agree which one runs it. Pinging each other fails, one lock server is a single point of failure. How a five-node cluster elects a leader, keeps a replicated log and avoids split brain by counting to a majority.

  • 15 min
  • Advanced
  • 10 steps
Paid twice
48 000 $
No lock available
0 s
Lock write time
—
Open in Lab
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Payout worker 2 → Payouts DB

Request

INSERT INTO payouts (run, merchant, amount)1
SELECT '2026-11-18', merchant, balance
FROM balances WHERE balance > 0;

Response

INSERT 0 3122
  1. run — The night being paid. Both workers wrote the same night.
  2. INSERT 0 312 — 312 payout rows from worker 2, on top of the 312 from worker 1.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway