mufassall

Scaling / 5 / 6

Cache invalidation

An account API keeps each account's plan in a cache in front of the database. A TTL alone keeps a customer who just paid on the old plan for up to ten minutes. Deleting the key after the write fixes the usual case but leaves a race in which an old value is written back after the delete. Delete twice, leases and invalidation from the database change log close the gaps one by one, and on the read side a hot key is served stale while one request refreshes it, and rows that do not exist are cached too.

  • 14 min
  • Advanced
  • 12 steps
Stale cache fills
0 /day
Longest stale window
0 s
DB reads
900 /s
Cache hit ratio
94 %
p99 latency
6 ms
Open in Lab

  • request
  • reply
  • redelivery
  • duplicate
  • lost
  • call: the reply returns on the same line

Numbers are a model scenario, not measurements

The real request

Web app → Account API

Request

GET /v1/accounts/4821/plan HTTP/1.13
Host: api.example.com
Authorization: hidden

Response

HTTP/1.1 200 OK
X-Cache1: HIT
Age2: 212

{
  "account_id": 4821,
  "plan": "free",
  "seats": 43
}
  1. X-Cache — This answer came from the cache, not from the database.
  2. Age — The cached copy is 212 s old. It can be up to 600 s old.
  3. plan — The value that decides which features the customer gets.
  4. seats — Cached together with the plan, so it goes stale together with it.

Build it yourselfCheat sheetCompare lessons

Course: 74 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. Cache invalidation
    6. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Geo search: who is nearby
    4. Full-text search
    5. Vector search
    6. Storage engines: B-tree and LSM
    7. Bloom filters
    8. Database replication
    9. Sharding
    10. Consistent hashing
    11. Unique IDs
    12. CAP: when the link breaks
    13. Consensus and leader election
    14. Transactions
    15. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
    6. Sending webhooks reliably
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
    5. SLOs and error budgets
    6. Backpressure
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Change data capture (CDC)
    5. Transactional outbox
    6. Distributed locks
    7. Batch vs stream
    8. Multi-tenancy
    9. Event sourcing
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
    4. Secrets management
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
    3. Zero-downtime migrations
    4. Feature flags
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
    5. Design a video streaming site
    6. Design search autocomplete
    7. Design a job scheduler
  10. 10 Business processes

    1. Payment gateway
    2. Money and rounding
    3. Double-entry ledger
    4. Payment reconciliation
    5. Subscription billing
    6. Electronic digital signature
    7. Document lifecycle
    8. Audit log
    9. Notification delivery
    10. Bulk import
    11. Refunds and chargebacks
    12. Gapless document numbering
  11. 11 Case studies

    1. Case: how Discord stores messages
    2. Case: Figma multiplayer editing