mufassall

Cheat sheet · Whole systems

Design a chat

15 min · Advanced · mufassall.dtwin.uz/en/l/design-chat/

← Open the lesson

The problem

Alice writes to Bob, and the message should appear on his screen in a fraction of a second, in the right order, even if his phone was asleep. We build the chat step by step: WebSocket gateways, routing, storage by conversation, sequence numbers, receipts, push, groups and presence.

What you learn

  • Why a chat keeps a WebSocket open and how a message finds the right gateway
  • Why messages are partitioned by conversation, not by user
  • Per-conversation sequence numbers instead of clocks
  • Receipts, push for offline users, group fan-out and presence with TTL
Final scene
30%60%65%30%40%25%

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_alice(["Alice's app"])
  n_bob(["Bob's app"])
  n_chat["Chat service"]
  n_msgs[("Message store")]
  n_gw_a["Gateway A"]
  n_gw_b["Gateway B"]
  n_sessions{{"Session registry"}}
  n_notify("Push worker")
  n_push>"Push service"]
  n_chat --> n_msgs
  n_alice --> n_gw_a
  n_gw_a --> n_chat
  n_gw_b --> n_sessions
  n_chat --> n_sessions
  n_chat --> n_gw_b
  n_chat -.-> n_gw_a
  n_gw_a -.-> n_alice
  n_chat --> n_notify
  n_notify --> n_push
  n_push --> n_bob
  n_bob --> n_gw_b
  n_gw_b --> n_bob

Mermaid: request sequence

sequenceDiagram
  participant n_bob as Bob's app
  participant n_chat as Chat service
  participant n_gw_b as Gateway B
  participant n_sessions as Session registry
  participant n_msgs as Message store
  participant n_notify as Push worker
  participant n_push as Push service
  Note over n_bob,n_chat: Asking every two seconds: anything new?
  n_bob->>n_chat: GET /v1/messages?since=1795400100 HTTP/1.1
  n_chat-->>n_bob: HTTP/1.1 200 OK
  Note over n_bob,n_gw_b: One open connection per device
  n_bob->>n_gw_b: GET /ws HTTP/1.1
  n_gw_b-->>n_bob: HTTP/1.1 101 Switching Protocols
  Note over n_chat,n_sessions: Finding Bob's gateway
  n_chat->>n_sessions: SMEMBERS conn:u_bob
  n_sessions-->>n_chat: 1) #34;gw-b/c_8812#34;
  Note over n_chat,n_msgs: Store by conversation, not by user
  n_chat->>n_msgs: INSERT INTO messages
  n_msgs-->>n_chat: INSERT 0 1 -- shard 12 of 64
  Note over n_gw_b,n_bob: Two quick messages arrive swapped
  n_gw_b->>n_bob: u_bob history, sorted by sent_at:
  Note over n_chat,n_msgs: A sequence number per conversation
  n_chat->>n_msgs: UPDATE conversations SET last_seq = last_seq + 1
  n_msgs-->>n_chat: last_seq = 1043
  Note over n_bob,n_gw_b: Delivered and read
  n_bob->>n_gw_b: WS frame (client -#62; server)
  Note over n_chat,n_sessions: Bob's phone is asleep
  n_chat->>n_sessions: SMEMBERS conn:u_bob
  n_sessions-->>n_chat: (empty array)
  Note over n_notify,n_push: Push notification, then catch-up
  n_notify->>n_push: POST /v1/send HTTP/1.1
  n_push-->>n_notify: HTTP/1.1 200 OK
  Note over n_chat,n_sessions: Group chat: store once, fan out to who is online
  n_chat->>n_sessions: SINTER members:g_team online
  n_sessions-->>n_chat: 1) #34;u_bob#34;
  Note over n_gw_b,n_sessions: Online and last seen
  n_gw_b->>n_sessions: SET presence:u_bob online EX 60
  n_sessions-->>n_gw_b: OK

D2

direction: right

n_alice: "Alice's app" {
  shape: rectangle
}
n_bob: "Bob's app" {
  shape: rectangle
}
n_chat: "Chat service" {
  shape: rectangle
}
n_msgs: "Message store" {
  shape: cylinder
}
n_gw_a: "Gateway A" {
  shape: rectangle
}
n_gw_b: "Gateway B" {
  shape: rectangle
}
n_sessions: "Session registry" {
  shape: hexagon
}
n_notify: "Push worker" {
  shape: rectangle
  style.multiple: true
}
n_push: "Push service" {
  shape: cloud
}

n_chat -> n_msgs
n_alice -> n_gw_a
n_gw_a -> n_chat
n_gw_b -> n_sessions
n_chat -> n_sessions
n_chat -> n_gw_b
n_chat -> n_gw_a {
  style.stroke-dash: 3
}
n_gw_a -> n_alice {
  style.stroke-dash: 3
}
n_chat -> n_notify
n_notify -> n_push
n_push -> n_bob
n_bob -> n_gw_b
n_gw_b -> n_bob

PlantUML: request sequence

@startuml
participant "Bob's app" as n_bob
participant "Chat service" as n_chat
participant "Gateway B" as n_gw_b
collections "Session registry" as n_sessions
database "Message store" as n_msgs
participant "Push worker" as n_notify
boundary "Push service" as n_push
note over n_bob, n_chat : Asking every two seconds: anything new?
n_bob -> n_chat : GET /v1/messages?since=1795400100 HTTP/1.1
n_chat --> n_bob : HTTP/1.1 200 OK
note over n_bob, n_gw_b : One open connection per device
n_bob -> n_gw_b : GET /ws HTTP/1.1
n_gw_b --> n_bob : HTTP/1.1 101 Switching Protocols
note over n_chat, n_sessions : Finding Bob's gateway
n_chat -> n_sessions : SMEMBERS conn:u_bob
n_sessions --> n_chat : 1) "gw-b/c_8812"
note over n_chat, n_msgs : Store by conversation, not by user
n_chat -> n_msgs : INSERT INTO messages
n_msgs --> n_chat : INSERT 0 1 -- shard 12 of 64
note over n_gw_b, n_bob : Two quick messages arrive swapped
n_gw_b -> n_bob : u_bob history, sorted by sent_at:
note over n_chat, n_msgs : A sequence number per conversation
n_chat -> n_msgs : UPDATE conversations SET last_seq = last_seq + 1
n_msgs --> n_chat : last_seq = 1043
note over n_bob, n_gw_b : Delivered and read
n_bob -> n_gw_b : WS frame (client -> server)
note over n_chat, n_sessions : Bob's phone is asleep
n_chat -> n_sessions : SMEMBERS conn:u_bob
n_sessions --> n_chat : (empty array)
note over n_notify, n_push : Push notification, then catch-up
n_notify -> n_push : POST /v1/send HTTP/1.1
n_push --> n_notify : HTTP/1.1 200 OK
note over n_chat, n_sessions : Group chat: store once, fan out to who is online
n_chat -> n_sessions : SINTER members:g_team online
n_sessions --> n_chat : 1) "u_bob"
note over n_gw_b, n_sessions : Online and last seen
n_gw_b -> n_sessions : SET presence:u_bob online EX 60
n_sessions --> n_gw_b : OK
@enduml

PlantUML: components

@startuml
left to right direction
agent "Alice's app" as n_alice
agent "Bob's app" as n_bob
component "Chat service" as n_chat
database "Message store" as n_msgs
component "Gateway A" as n_gw_a
component "Gateway B" as n_gw_b
stack "Session registry" as n_sessions
collections "Push worker" as n_notify
cloud "Push service" as n_push
n_chat --> n_msgs
n_alice --> n_gw_a
n_gw_a --> n_chat
n_gw_b --> n_sessions
n_chat --> n_sessions
n_chat --> n_gw_b
n_chat ..> n_gw_a
n_gw_a ..> n_alice
n_chat --> n_notify
n_notify --> n_push
n_push --> n_bob
n_bob --> n_gw_b
n_gw_b --> n_bob
@enduml

Recap

A chat is a store plus a set of open pipes.

1) WebSocket gateways hold connections; a registry tells where each user is. 2) Store first, partitioned by conversation. 3) Order by a per-conversation sequence, never by clocks; clients fill gaps by seq. 4) Receipts are one number per member. 5) Offline users get a push and catch up from the store. 6) Groups fan out per gateway, presence lives on a TTL.

Numbers: at the start and at the end

MetricStartEnd
Delivery p992 100 ms180 ms
Out-of-order messages0/day0/day
Unnoticed for 1 h+0/day15/day

Steps

  1. Problem Asking every two seconds: anything new?

    The first version: Bob's app asks the server every 2 seconds for new messages.

  2. Fix One open connection per device

    Each app opens a WebSocket to a gateway and keeps it open.

  3. Case Finding Bob's gateway

    Alice's message arrives at gateway A, which passes it to the chat service.

  4. Case Store by conversation, not by user

    Every message is written before it is delivered.

  5. Problem Two quick messages arrive swapped

    Alice sends "dinner tonight?" and right after it "at 7pm?".

  6. Fix A sequence number per conversation

    The shard that owns the conversation hands out the next number atomically: 1042, 1043, 1044.

  7. Case Delivered and read

    When Bob's device gets the message it acknowledges it: one tick for Alice.

  8. Problem Bob's phone is asleep

    The phone puts the app in the background and the operating system closes its socket.

  9. Fix Push notification, then catch-up

    When no connection is found, the chat service puts an event on a queue.

  10. Case Group chat: store once, fan out to who is online

    A team chat with 300 members.

  11. Case Online and last seen

    Each gateway sends a heartbeat for its users every 25 s, and the presence key lives for 60 s.

  12. Recap Recap

    A chat is a store plus a set of open pipes.