mufassall

Cheat sheet · Communication

Real-time updates

13 min · Intermediate · mufassall.dtwin.uz/en/l/realtime/

← Open the lesson

The problem

A chat must show a new message the moment it is sent. Polling, long polling, Server-Sent Events and WebSockets, and what it takes to run many open connections.

What you learn

  • What polling, long polling, SSE and WebSockets cost the server
  • Why open connections need their own gateway layer
  • How pub/sub delivers a message to a user on another gateway
  • How a client reconnects without missing messages
Final scene
20%45%30%40%30%15%

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_alice(["Alice"])
  n_bob(["Bob"])
  n_db[("Database")]
  n_lb{"Load balancer"}
  n_gw1["Gateway 1"]
  n_gw2["Gateway 2"]
  n_chat["Chat service"]
  n_bus{{"Pub/sub"}}
  n_alice --> n_lb
  n_bob -.->|"retry"| n_lb
  n_lb --> n_gw1
  n_lb --> n_gw2
  n_gw1 --> n_chat
  n_chat --> n_db
  n_gw1 -.-> n_alice
  n_gw2 -.-> n_bob
  n_chat --> n_bus
  n_bus -.-> n_gw1
  n_bus -.-> n_gw2
  n_gw2 --> n_chat

Mermaid: request sequence

sequenceDiagram
  participant n_bob as Bob
  participant n_api as Chat server
  participant n_alice as Alice
  participant n_lb as Load balancer
  participant n_gw1 as Gateway 1
  participant n_chat as Chat service
  participant n_bus as Pub/sub
  participant n_gw2 as Gateway 2
  Note over n_bob,n_api: Asking every two seconds
  n_bob->>n_api: GET /rooms/r_17/updates?since=m_730 HTTP/1.1
  n_api-->>n_bob: HTTP/1.1 200 OK
  Note over n_bob,n_api: Long polling: the server holds the request
  n_bob->>n_api: GET /rooms/r_17/updates?since=m_730#38;wait=25 HTTP/1.1
  n_api-->>n_bob: HTTP/1.1 200 OK
  Note over n_api,n_bob: Server-Sent Events: one stream down
  n_api->>n_bob: GET /rooms/r_17/stream HTTP/1.1
  n_bob-->>n_api: HTTP/1.1 200 OK
  Note over n_alice,n_api: WebSocket: one connection, both ways
  n_alice->>n_api: GET /ws HTTP/1.1
  n_api-->>n_alice: HTTP/1.1 101 Switching Protocols
  Note over n_bob,n_api: One server, 65 000 sockets
  n_bob-xn_api: chat-1: accept() failed: EMFILE (too many open files)
  Note over n_bob,n_lb: A gateway layer holds the connections
  n_bob->>n_lb: GET /ws?room=r_17 HTTP/1.1
  n_lb-->>n_bob: HTTP/1.1 101 Switching Protocols
  Note over n_gw1,n_alice: The message stays on the wrong gateway
  n_gw1->>n_alice: gw1: deliver m_740 to room r_17
  Note over n_chat,n_bus: Pub/sub between gateways
  n_chat->>n_bus: PUBLISH room:r_17
  n_bus-->>n_chat: (integer) 2
  Note over n_gw2,n_bob: The phone goes into a tunnel
  n_gw2-xn_bob: gw2: socket u_77 closed, code 1006
  Note over n_bob,n_lb: Reconnect and catch up from the last ID
  n_bob->>n_lb: WS frame (after reconnect)
  n_lb-->>n_bob: WS frame

D2

direction: right

n_alice: "Alice" {
  shape: rectangle
}
n_bob: "Bob" {
  shape: rectangle
}
n_db: "Database" {
  shape: cylinder
}
n_lb: "Load balancer" {
  shape: diamond
}
n_gw1: "Gateway 1" {
  shape: rectangle
}
n_gw2: "Gateway 2" {
  shape: rectangle
}
n_chat: "Chat service" {
  shape: rectangle
}
n_bus: "Pub/sub" {
  shape: hexagon
}

n_alice -> n_lb
n_bob -> n_lb: "retry" {
  style.stroke-dash: 3
}
n_lb -> n_gw1
n_lb -> n_gw2
n_gw1 -> n_chat
n_chat -> n_db
n_gw1 -> n_alice {
  style.stroke-dash: 3
}
n_gw2 -> n_bob {
  style.stroke-dash: 3
}
n_chat -> n_bus
n_bus -> n_gw1 {
  style.stroke-dash: 3
}
n_bus -> n_gw2 {
  style.stroke-dash: 3
}
n_gw2 -> n_chat

PlantUML: request sequence

@startuml
participant "Bob" as n_bob
participant "Chat server" as n_api
participant "Alice" as n_alice
participant "Load balancer" as n_lb
participant "Gateway 1" as n_gw1
participant "Chat service" as n_chat
collections "Pub/sub" as n_bus
participant "Gateway 2" as n_gw2
note over n_bob, n_api : Asking every two seconds
n_bob -> n_api : GET /rooms/r_17/updates?since=m_730 HTTP/1.1
n_api --> n_bob : HTTP/1.1 200 OK
note over n_bob, n_api : Long polling: the server holds the request
n_bob -> n_api : GET /rooms/r_17/updates?since=m_730&wait=25 HTTP/1.1
n_api --> n_bob : HTTP/1.1 200 OK
note over n_api, n_bob : Server-Sent Events: one stream down
n_api -> n_bob : GET /rooms/r_17/stream HTTP/1.1
n_bob --> n_api : HTTP/1.1 200 OK
note over n_alice, n_api : WebSocket: one connection, both ways
n_alice -> n_api : GET /ws HTTP/1.1
n_api --> n_alice : HTTP/1.1 101 Switching Protocols
note over n_bob, n_api : One server, 65 000 sockets
n_bob ->x n_api : [lost] chat-1: accept() failed: EMFILE (too many open files)
note over n_bob, n_lb : A gateway layer holds the connections
n_bob -> n_lb : GET /ws?room=r_17 HTTP/1.1
n_lb --> n_bob : HTTP/1.1 101 Switching Protocols
note over n_gw1, n_alice : The message stays on the wrong gateway
n_gw1 -> n_alice : gw1: deliver m_740 to room r_17
note over n_chat, n_bus : Pub/sub between gateways
n_chat -> n_bus : PUBLISH room:r_17
n_bus --> n_chat : (integer) 2
note over n_gw2, n_bob : The phone goes into a tunnel
n_gw2 ->x n_bob : [lost] gw2: socket u_77 closed, code 1006
note over n_bob, n_lb : Reconnect and catch up from the last ID
n_bob -> n_lb : [retry] WS frame (after reconnect)
n_lb --> n_bob : WS frame
@enduml

PlantUML: components

@startuml
left to right direction
agent "Alice" as n_alice
agent "Bob" as n_bob
database "Database" as n_db
node "Load balancer" as n_lb
component "Gateway 1" as n_gw1
component "Gateway 2" as n_gw2
component "Chat service" as n_chat
stack "Pub/sub" as n_bus
n_alice --> n_lb
n_bob ..> n_lb : retry
n_lb --> n_gw1
n_lb --> n_gw2
n_gw1 --> n_chat
n_chat --> n_db
n_gw1 ..> n_alice
n_gw2 ..> n_bob
n_chat --> n_bus
n_bus ..> n_gw1
n_bus ..> n_gw2
n_gw2 --> n_chat
@enduml

Recap

Pick the cheapest channel that fits: polling for rare updates, SSE for server-to-client streams, WebSockets for two-way chat.

1) Long-lived connections go to a separate gateway layer. 2) Gateways learn about messages through pub/sub. 3) Live delivery may miss; the database is the truth. 4) Clients reconnect with backoff and jitter and resume from their last ID.

Numbers: at the start and at the end

MetricStartEnd
Requests50 000/s300/s
Delivery delay1 000 ms45 ms
Missed0/min0/min

Steps

  1. Problem Asking every two seconds

    Each open chat asks the server “anything new since m_730?” every two seconds.

  2. Fix Long polling: the server holds the request

    The client sends the same request, but the server does not answer at once.

  3. Fix Server-Sent Events: one stream down

    The client opens one HTTP request and the server never finishes the answer.

  4. Fix WebSocket: one connection, both ways

    The client starts with an HTTP request and asks to upgrade it.

  5. Problem One server, 65 000 sockets

    Every user keeps a socket open all day.

  6. Fix A gateway layer holds the connections

    Connections move to a layer of gateways.

  7. Problem The message stays on the wrong gateway

    Alice sends a message.

  8. Fix Pub/sub between gateways

    After saving a message, the chat service publishes it to the room's channel.

  9. Problem The phone goes into a tunnel

    Bob's connection breaks without a goodbye.

  10. Fix Reconnect and catch up from the last ID

    Bob's app reconnects with growing pauses and random jitter, so thousands of phones do not reconnect in the same second.

  11. Recap Recap

    Pick the cheapest channel that fits: polling for rare updates, SSE for server-to-client streams, WebSockets for two-way chat.

Terms

Long polling
The client sends a request and the server holds it until something new appears or a timeout passes. Cheaper than asking every two seconds, but each answer needs a new request.
WebSocket
One long-lived connection between client and server where both sides can send at any time. Many open connections are held by a separate gateway layer.
Server-Sent Events (SSE)
The server keeps one HTTP response open and pushes events down it one after another. It is one-way (server to client), but it is plain HTTP and the browser reconnects by itself after a drop.