mufassall

Communication / 3 / 5

Pub/sub and event logs

One event, many teams that need it. How a topic decouples them, and what offsets, consumer groups, keys and schemas add.

  • 13 min
  • Intermediate
  • 12 steps
Checkout p95
900 ms
Failed orders
0 %
Lag (events)
0
Out of order
0 /min
Open in Lab
  • request
  • reply
  • redelivery
  • duplicate
  • lost

Numbers are a model scenario, not measurements

The real request

Orders → Email

Request

POST /send-receipt HTTP/1.1
Host: email.internal

{
  "order_id": 1"ord_9051",
  "user_id": "u_42"
}

Response

HTTP/1.1 200 OK
X-Response-Time2: 410 ms
  1. order_id — The same few fields are sent to every service in a different request.
  2. X-Response-Time — The receipt alone takes 410 ms. Three calls in a row add up to almost a second.
Course: 42 lessons
  1. 01 Foundations

    1. The life of a request
    2. Latency and throughput
    3. Napkin math
  2. 02 Scaling

    1. Scaling up vs scaling out
    2. Load balancing
    3. Stateless servers
    4. Caching
    5. CDN and edge caching
  3. 03 Data & storage

    1. SQL and NoSQL
    2. Database indexes
    3. Storage engines: B-tree and LSM
    4. Database replication
    5. Sharding
    6. Consistent hashing
    7. Unique IDs
    8. CAP: when the link breaks
    9. Consensus and leader election
    10. Transactions
    11. File uploads
  4. 04 Communication

    1. API styles
    2. Message queues
    3. Pub/sub and event logs
    4. Real-time updates
    5. Reliable callbacks
  5. 05 Reliability

    1. Redundancy and failover
    2. Rate limiting
    3. Timeouts and retries
    4. Observability
  6. 06 Architecture

    1. Microservices
    2. Service discovery
    3. Sagas and the outbox
    4. Batch vs stream
  7. 07 Security

    1. Authentication
    2. Authorization
    3. Encryption and secrets
  8. 08 Shipping

    1. Testing
    2. CI/CD: from commit to production
  9. 09 Whole systems

    1. From requirements to design
    2. Design a URL shortener
    3. Design a news feed
    4. Design a chat
  10. 10 Business processes

    1. Payment gateway