Data & storage / 9 / 11
Consensus and leader election
Two copies of a nightly payout job must agree which one runs it. Pinging each other fails, one lock server is a single point of failure. How a five-node cluster elects a leader, keeps a replicated log and avoids split brain by counting to a majority.
- 15 min
- Advanced
- 10 steps
- Paid twice
- 48 000 $
- No lock available
- 0 s
- Lock write time
- —
- request
- reply
- redelivery
- duplicate
- lost
Numbers are a model scenario, not measurements
The real request
Payout worker 2 → Payouts DBRequest
INSERT INTO payouts (run, merchant, amount)1
SELECT '2026-11-18', merchant, balance
FROM balances WHERE balance > 0;
Response
INSERT 0 3122
run— The night being paid. Both workers wrote the same night.INSERT 0 312— 312 payout rows from worker 2, on top of the 312 from worker 1.
Course: 42 lessons
01 Foundations
02 Scaling
03 Data & storage
04 Communication
05 Reliability
06 Architecture
07 Security
08 Shipping
09 Whole systems
10 Business processes