Cheat sheet · Communication
Real-time updates
The problem
A chat must show a new message the moment it is sent. Polling, long polling, Server-Sent Events and WebSockets, and what it takes to run many open connections.
What you learn
- What polling, long polling, SSE and WebSockets cost the server
- Why open connections need their own gateway layer
- How pub/sub delivers a message to a user on another gateway
- How a client reconnects without missing messages
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_alice(["Alice"])
n_bob(["Bob"])
n_db[("Database")]
n_lb{"Load balancer"}
n_gw1["Gateway 1"]
n_gw2["Gateway 2"]
n_chat["Chat service"]
n_bus{{"Pub/sub"}}
n_alice --> n_lb
n_bob -.->|"retry"| n_lb
n_lb --> n_gw1
n_lb --> n_gw2
n_gw1 --> n_chat
n_chat --> n_db
n_gw1 -.-> n_alice
n_gw2 -.-> n_bob
n_chat --> n_bus
n_bus -.-> n_gw1
n_bus -.-> n_gw2
n_gw2 --> n_chatMermaid: request sequence
sequenceDiagram
participant n_bob as Bob
participant n_api as Chat server
participant n_alice as Alice
participant n_lb as Load balancer
participant n_gw1 as Gateway 1
participant n_chat as Chat service
participant n_bus as Pub/sub
participant n_gw2 as Gateway 2
Note over n_bob,n_api: Asking every two seconds
n_bob->>n_api: GET /rooms/r_17/updates?since=m_730 HTTP/1.1
n_api-->>n_bob: HTTP/1.1 200 OK
Note over n_bob,n_api: Long polling: the server holds the request
n_bob->>n_api: GET /rooms/r_17/updates?since=m_730#38;wait=25 HTTP/1.1
n_api-->>n_bob: HTTP/1.1 200 OK
Note over n_api,n_bob: Server-Sent Events: one stream down
n_api->>n_bob: GET /rooms/r_17/stream HTTP/1.1
n_bob-->>n_api: HTTP/1.1 200 OK
Note over n_alice,n_api: WebSocket: one connection, both ways
n_alice->>n_api: GET /ws HTTP/1.1
n_api-->>n_alice: HTTP/1.1 101 Switching Protocols
Note over n_bob,n_api: One server, 65 000 sockets
n_bob-xn_api: chat-1: accept() failed: EMFILE (too many open files)
Note over n_bob,n_lb: A gateway layer holds the connections
n_bob->>n_lb: GET /ws?room=r_17 HTTP/1.1
n_lb-->>n_bob: HTTP/1.1 101 Switching Protocols
Note over n_gw1,n_alice: The message stays on the wrong gateway
n_gw1->>n_alice: gw1: deliver m_740 to room r_17
Note over n_chat,n_bus: Pub/sub between gateways
n_chat->>n_bus: PUBLISH room:r_17
n_bus-->>n_chat: (integer) 2
Note over n_gw2,n_bob: The phone goes into a tunnel
n_gw2-xn_bob: gw2: socket u_77 closed, code 1006
Note over n_bob,n_lb: Reconnect and catch up from the last ID
n_bob->>n_lb: WS frame (after reconnect)
n_lb-->>n_bob: WS frameD2
direction: right
n_alice: "Alice" {
shape: rectangle
}
n_bob: "Bob" {
shape: rectangle
}
n_db: "Database" {
shape: cylinder
}
n_lb: "Load balancer" {
shape: diamond
}
n_gw1: "Gateway 1" {
shape: rectangle
}
n_gw2: "Gateway 2" {
shape: rectangle
}
n_chat: "Chat service" {
shape: rectangle
}
n_bus: "Pub/sub" {
shape: hexagon
}
n_alice -> n_lb
n_bob -> n_lb: "retry" {
style.stroke-dash: 3
}
n_lb -> n_gw1
n_lb -> n_gw2
n_gw1 -> n_chat
n_chat -> n_db
n_gw1 -> n_alice {
style.stroke-dash: 3
}
n_gw2 -> n_bob {
style.stroke-dash: 3
}
n_chat -> n_bus
n_bus -> n_gw1 {
style.stroke-dash: 3
}
n_bus -> n_gw2 {
style.stroke-dash: 3
}
n_gw2 -> n_chatPlantUML: request sequence
@startuml
participant "Bob" as n_bob
participant "Chat server" as n_api
participant "Alice" as n_alice
participant "Load balancer" as n_lb
participant "Gateway 1" as n_gw1
participant "Chat service" as n_chat
collections "Pub/sub" as n_bus
participant "Gateway 2" as n_gw2
note over n_bob, n_api : Asking every two seconds
n_bob -> n_api : GET /rooms/r_17/updates?since=m_730 HTTP/1.1
n_api --> n_bob : HTTP/1.1 200 OK
note over n_bob, n_api : Long polling: the server holds the request
n_bob -> n_api : GET /rooms/r_17/updates?since=m_730&wait=25 HTTP/1.1
n_api --> n_bob : HTTP/1.1 200 OK
note over n_api, n_bob : Server-Sent Events: one stream down
n_api -> n_bob : GET /rooms/r_17/stream HTTP/1.1
n_bob --> n_api : HTTP/1.1 200 OK
note over n_alice, n_api : WebSocket: one connection, both ways
n_alice -> n_api : GET /ws HTTP/1.1
n_api --> n_alice : HTTP/1.1 101 Switching Protocols
note over n_bob, n_api : One server, 65 000 sockets
n_bob ->x n_api : [lost] chat-1: accept() failed: EMFILE (too many open files)
note over n_bob, n_lb : A gateway layer holds the connections
n_bob -> n_lb : GET /ws?room=r_17 HTTP/1.1
n_lb --> n_bob : HTTP/1.1 101 Switching Protocols
note over n_gw1, n_alice : The message stays on the wrong gateway
n_gw1 -> n_alice : gw1: deliver m_740 to room r_17
note over n_chat, n_bus : Pub/sub between gateways
n_chat -> n_bus : PUBLISH room:r_17
n_bus --> n_chat : (integer) 2
note over n_gw2, n_bob : The phone goes into a tunnel
n_gw2 ->x n_bob : [lost] gw2: socket u_77 closed, code 1006
note over n_bob, n_lb : Reconnect and catch up from the last ID
n_bob -> n_lb : [retry] WS frame (after reconnect)
n_lb --> n_bob : WS frame
@endumlPlantUML: components
@startuml
left to right direction
agent "Alice" as n_alice
agent "Bob" as n_bob
database "Database" as n_db
node "Load balancer" as n_lb
component "Gateway 1" as n_gw1
component "Gateway 2" as n_gw2
component "Chat service" as n_chat
stack "Pub/sub" as n_bus
n_alice --> n_lb
n_bob ..> n_lb : retry
n_lb --> n_gw1
n_lb --> n_gw2
n_gw1 --> n_chat
n_chat --> n_db
n_gw1 ..> n_alice
n_gw2 ..> n_bob
n_chat --> n_bus
n_bus ..> n_gw1
n_bus ..> n_gw2
n_gw2 --> n_chat
@endumlRecap
Pick the cheapest channel that fits: polling for rare updates, SSE for server-to-client streams, WebSockets for two-way chat.
1) Long-lived connections go to a separate gateway layer. 2) Gateways learn about messages through pub/sub. 3) Live delivery may miss; the database is the truth. 4) Clients reconnect with backoff and jitter and resume from their last ID.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Requests | 50 000/s | 300/s |
| Delivery delay | 1 000 ms | 45 ms |
| Missed | 0/min | 0/min |
Steps
- Problem Asking every two seconds
Each open chat asks the server “anything new since
m_730?” every two seconds. - Fix Long polling: the server holds the request
The client sends the same request, but the server does not answer at once.
- Fix Server-Sent Events: one stream down
The client opens one HTTP request and the server never finishes the answer.
- Fix WebSocket: one connection, both ways
The client starts with an HTTP request and asks to upgrade it.
- Problem One server, 65 000 sockets
Every user keeps a socket open all day.
- Fix A gateway layer holds the connections
Connections move to a layer of gateways.
- Problem The message stays on the wrong gateway
Alice sends a message.
- Fix Pub/sub between gateways
After saving a message, the chat service publishes it to the room's channel.
- Problem The phone goes into a tunnel
Bob's connection breaks without a goodbye.
- Fix Reconnect and catch up from the last ID
Bob's app reconnects with growing pauses and random jitter, so thousands of phones do not reconnect in the same second.
- Recap Recap
Pick the cheapest channel that fits: polling for rare updates, SSE for server-to-client streams, WebSockets for two-way chat.
Terms
- Long polling
- The client sends a request and the server holds it until something new appears or a timeout passes. Cheaper than asking every two seconds, but each answer needs a new request.
- WebSocket
- One long-lived connection between client and server where both sides can send at any time. Many open connections are held by a separate gateway layer.
- Server-Sent Events (SSE)
- The server keeps one HTTP response open and pushes events down it one after another. It is one-way (server to client), but it is plain HTTP and the browser reconnects by itself after a drop.