mufassall

Reliability / 2 / 4

Rate limiting

One noisy client can slow an API down for everyone. How to cap each client fairly: windows, token buckets, 429 answers and a counter shared by all servers.

  • 13 min
  • Intermediate
  • 12 steps
Partner p95
2 400 ms
Honest errors
9 %
Over the limit
90 x
Open in Lab
100%95%
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Script → API

Request

GET /v1/products?page=8812 HTTP/1.12
Host: api.example.com
X-Api-Key1: hidden

Response

HTTP/1.1 200 OK
X-Response-Time3: 2380 ms
  1. X-Api-Key · hidden — The script has a valid key. The requests are allowed, just far too many.
  2. page — Page 8812: nobody reads that far. It is a crawl, not a user.
  3. X-Response-Time — Even this request waited 2.4 s in a queue behind its own siblings.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway