Cheat sheet · Scaling
Stateless servers
The problem
A shop API keeps logged-in users in its own memory. It works on one server and breaks the moment you add a second one or restart the first. Where the state should live instead.
What you learn
- Why a session in process memory ties a user to one server
- Why sticky sessions only hide the problem
- Moving sessions to a shared store such as Redis
- Signed tokens: no lookup, but no instant logout either
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_client(["Browser"])
n_app1["App server 1"]
n_lb{"Load balancer"}
n_app2["App server 2"]
n_redis{{"Session store"}}
n_client --> n_lb
n_lb --> n_app1
n_lb --> n_app2
n_app1 --> n_redis
n_app2 --> n_redisMermaid: request sequence
sequenceDiagram
participant n_client as Browser
participant n_app1 as App server 1
participant n_lb as Load balancer
participant n_app2 as App server 2
participant n_redis as Session store
Note over n_client,n_app1: Login on a single server
n_client->>n_app1: POST /login HTTP/1.1
n_app1-->>n_client: HTTP/1.1 200 OK
Note over n_client,n_app1: One server is not enough
n_client->>n_app1: GET /cart HTTP/1.1
n_app1-->>n_client: HTTP/1.1 503 Service Unavailable
Note over n_lb,n_app2: A load balancer and a second server
n_lb->>n_app2: upstream app {
Note over n_lb,n_app2: Server 2 has never seen this user
n_lb->>n_app2: GET /cart HTTP/1.1
n_app2-->>n_lb: HTTP/1.1 401 Unauthorized
Note over n_lb,n_app1: Sticky sessions: always the same server
n_lb->>n_app1: upstream app {
Note over n_lb,n_app1: A deploy restarts server 1
n_lb-xn_app1: app1 #124; SIGTERM: deploy v2.4
Note over n_app2,n_redis: Sessions move to a shared store
n_app2->>n_redis: GET sess:h_51e0
n_redis-->>n_app2: (bulk string, 38 bytes)
Note over n_lb,n_app2: The same deploy, nobody notices
n_lb->>n_app2: GET /cart HTTP/1.1
n_app2-->>n_lb: HTTP/1.1 200 OK
Note over n_client,n_lb: Another way: the state travels with the request
n_client->>n_lb: POST /login HTTP/1.1
n_lb-->>n_client: HTTP/1.1 200 OK
Note over n_app1,n_redis: The price: a token cannot be taken back
n_app1->>n_redis: SET deny:jti_7f2 1 EX 900
n_redis-->>n_app1: OKD2
direction: right
n_client: "Browser" {
shape: rectangle
}
n_app1: "App server 1" {
shape: rectangle
}
n_lb: "Load balancer" {
shape: diamond
}
n_app2: "App server 2" {
shape: rectangle
}
n_redis: "Session store" {
shape: hexagon
}
n_client -> n_lb
n_lb -> n_app1
n_lb -> n_app2
n_app1 -> n_redis
n_app2 -> n_redisPlantUML: request sequence
@startuml
participant "Browser" as n_client
participant "App server 1" as n_app1
participant "Load balancer" as n_lb
participant "App server 2" as n_app2
collections "Session store" as n_redis
note over n_client, n_app1 : Login on a single server
n_client -> n_app1 : POST /login HTTP/1.1
n_app1 --> n_client : HTTP/1.1 200 OK
note over n_client, n_app1 : One server is not enough
n_client -> n_app1 : GET /cart HTTP/1.1
n_app1 --> n_client : HTTP/1.1 503 Service Unavailable
note over n_lb, n_app2 : A load balancer and a second server
n_lb -> n_app2 : upstream app {
note over n_lb, n_app2 : Server 2 has never seen this user
n_lb -> n_app2 : GET /cart HTTP/1.1
n_app2 --> n_lb : HTTP/1.1 401 Unauthorized
note over n_lb, n_app1 : Sticky sessions: always the same server
n_lb -> n_app1 : upstream app {
note over n_lb, n_app1 : A deploy restarts server 1
n_lb ->x n_app1 : [lost] app1 | SIGTERM: deploy v2.4
note over n_app2, n_redis : Sessions move to a shared store
n_app2 -> n_redis : GET sess:h_51e0
n_redis --> n_app2 : (bulk string, 38 bytes)
note over n_lb, n_app2 : The same deploy, nobody notices
n_lb -> n_app2 : GET /cart HTTP/1.1
n_app2 --> n_lb : HTTP/1.1 200 OK
note over n_client, n_lb : Another way: the state travels with the request
n_client -> n_lb : POST /login HTTP/1.1
n_lb --> n_client : HTTP/1.1 200 OK
note over n_app1, n_redis : The price: a token cannot be taken back
n_app1 -> n_redis : SET deny:jti_7f2 1 EX 900
n_redis --> n_app1 : OK
@endumlPlantUML: components
@startuml
left to right direction
agent "Browser" as n_client
component "App server 1" as n_app1
node "Load balancer" as n_lb
component "App server 2" as n_app2
stack "Session store" as n_redis
n_client --> n_lb
n_lb --> n_app1
n_lb --> n_app2
n_app1 --> n_redis
n_app2 --> n_redis
@endumlRecap
Four rules for stateless servers.
1) Nothing between requests lives in process memory: any server can die or be added at any time. 2) Sticky sessions hide the problem, they do not solve it. 3) Shared state goes to a shared store (Redis, a database). 4) Signed tokens remove the lookup but make logout harder: keep them short-lived.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Servers in service | 1 | 2 |
| Unexpected logouts | 0/min | 0/min |
| Reply p95 | 80 ms | 72 ms |
Steps
- Case Login on a single server
The user logs in.
- Problem One server is not enough
Traffic tripled.
- Fix A load balancer and a second server
A load balancer now receives every request and hands them out in turn: one to server 1, the next to server 2.
- Problem Server 2 has never seen this user
The user logged in on server 1.
- Fix Sticky sessions: always the same server
The load balancer now hashes the session cookie and always sends a user to the same server.
- Problem A deploy restarts server 1
A new version is rolled out and server 1 restarts.
- Fix Sessions move to a shared store
The servers no longer keep sessions in their own memory.
- Case The same deploy, nobody notices
Server 1 restarts for the next release.
- Fix Another way: the state travels with the request
Instead of a session ID, login returns a signed token (JWT) that already contains the user ID and an expiry time.
- Case The price: a token cannot be taken back
The user presses “log out” on a stolen laptop.
- Recap Recap
Four rules for stateless servers.
Terms
- Stateless server
- A server that keeps nothing in process memory between requests. Sessions and files live in a shared store, so any server can die or be added at any time.
- Sticky session
- The balancer always sends a user to the same server. It hides the problem rather than solving it: a restart loses the session and load becomes uneven.