mufassall

Cheat sheet · Scaling

Stateless servers

10 min · Beginner · mufassall.dtwin.uz/en/l/stateless/

← Open the lesson

The problem

A shop API keeps logged-in users in its own memory. It works on one server and breaks the moment you add a second one or restart the first. Where the state should live instead.

What you learn

  • Why a session in process memory ties a user to one server
  • Why sticky sessions only hide the problem
  • Moving sessions to a shared store such as Redis
  • Signed tokens: no lookup, but no instant logout either
Final scene
42%30%42%8%

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_client(["Browser"])
  n_app1["App server 1"]
  n_lb{"Load balancer"}
  n_app2["App server 2"]
  n_redis{{"Session store"}}
  n_client --> n_lb
  n_lb --> n_app1
  n_lb --> n_app2
  n_app1 --> n_redis
  n_app2 --> n_redis

Mermaid: request sequence

sequenceDiagram
  participant n_client as Browser
  participant n_app1 as App server 1
  participant n_lb as Load balancer
  participant n_app2 as App server 2
  participant n_redis as Session store
  Note over n_client,n_app1: Login on a single server
  n_client->>n_app1: POST /login HTTP/1.1
  n_app1-->>n_client: HTTP/1.1 200 OK
  Note over n_client,n_app1: One server is not enough
  n_client->>n_app1: GET /cart HTTP/1.1
  n_app1-->>n_client: HTTP/1.1 503 Service Unavailable
  Note over n_lb,n_app2: A load balancer and a second server
  n_lb->>n_app2: upstream app {
  Note over n_lb,n_app2: Server 2 has never seen this user
  n_lb->>n_app2: GET /cart HTTP/1.1
  n_app2-->>n_lb: HTTP/1.1 401 Unauthorized
  Note over n_lb,n_app1: Sticky sessions: always the same server
  n_lb->>n_app1: upstream app {
  Note over n_lb,n_app1: A deploy restarts server 1
  n_lb-xn_app1: app1 #124; SIGTERM: deploy v2.4
  Note over n_app2,n_redis: Sessions move to a shared store
  n_app2->>n_redis: GET sess:h_51e0
  n_redis-->>n_app2: (bulk string, 38 bytes)
  Note over n_lb,n_app2: The same deploy, nobody notices
  n_lb->>n_app2: GET /cart HTTP/1.1
  n_app2-->>n_lb: HTTP/1.1 200 OK
  Note over n_client,n_lb: Another way: the state travels with the request
  n_client->>n_lb: POST /login HTTP/1.1
  n_lb-->>n_client: HTTP/1.1 200 OK
  Note over n_app1,n_redis: The price: a token cannot be taken back
  n_app1->>n_redis: SET deny:jti_7f2 1 EX 900
  n_redis-->>n_app1: OK

D2

direction: right

n_client: "Browser" {
  shape: rectangle
}
n_app1: "App server 1" {
  shape: rectangle
}
n_lb: "Load balancer" {
  shape: diamond
}
n_app2: "App server 2" {
  shape: rectangle
}
n_redis: "Session store" {
  shape: hexagon
}

n_client -> n_lb
n_lb -> n_app1
n_lb -> n_app2
n_app1 -> n_redis
n_app2 -> n_redis

PlantUML: request sequence

@startuml
participant "Browser" as n_client
participant "App server 1" as n_app1
participant "Load balancer" as n_lb
participant "App server 2" as n_app2
collections "Session store" as n_redis
note over n_client, n_app1 : Login on a single server
n_client -> n_app1 : POST /login HTTP/1.1
n_app1 --> n_client : HTTP/1.1 200 OK
note over n_client, n_app1 : One server is not enough
n_client -> n_app1 : GET /cart HTTP/1.1
n_app1 --> n_client : HTTP/1.1 503 Service Unavailable
note over n_lb, n_app2 : A load balancer and a second server
n_lb -> n_app2 : upstream app {
note over n_lb, n_app2 : Server 2 has never seen this user
n_lb -> n_app2 : GET /cart HTTP/1.1
n_app2 --> n_lb : HTTP/1.1 401 Unauthorized
note over n_lb, n_app1 : Sticky sessions: always the same server
n_lb -> n_app1 : upstream app {
note over n_lb, n_app1 : A deploy restarts server 1
n_lb ->x n_app1 : [lost] app1 | SIGTERM: deploy v2.4
note over n_app2, n_redis : Sessions move to a shared store
n_app2 -> n_redis : GET sess:h_51e0
n_redis --> n_app2 : (bulk string, 38 bytes)
note over n_lb, n_app2 : The same deploy, nobody notices
n_lb -> n_app2 : GET /cart HTTP/1.1
n_app2 --> n_lb : HTTP/1.1 200 OK
note over n_client, n_lb : Another way: the state travels with the request
n_client -> n_lb : POST /login HTTP/1.1
n_lb --> n_client : HTTP/1.1 200 OK
note over n_app1, n_redis : The price: a token cannot be taken back
n_app1 -> n_redis : SET deny:jti_7f2 1 EX 900
n_redis --> n_app1 : OK
@enduml

PlantUML: components

@startuml
left to right direction
agent "Browser" as n_client
component "App server 1" as n_app1
node "Load balancer" as n_lb
component "App server 2" as n_app2
stack "Session store" as n_redis
n_client --> n_lb
n_lb --> n_app1
n_lb --> n_app2
n_app1 --> n_redis
n_app2 --> n_redis
@enduml

Recap

Four rules for stateless servers.

1) Nothing between requests lives in process memory: any server can die or be added at any time. 2) Sticky sessions hide the problem, they do not solve it. 3) Shared state goes to a shared store (Redis, a database). 4) Signed tokens remove the lookup but make logout harder: keep them short-lived.

Numbers: at the start and at the end

MetricStartEnd
Servers in service12
Unexpected logouts0/min0/min
Reply p9580 ms72 ms

Steps

  1. Case Login on a single server

    The user logs in.

  2. Problem One server is not enough

    Traffic tripled.

  3. Fix A load balancer and a second server

    A load balancer now receives every request and hands them out in turn: one to server 1, the next to server 2.

  4. Problem Server 2 has never seen this user

    The user logged in on server 1.

  5. Fix Sticky sessions: always the same server

    The load balancer now hashes the session cookie and always sends a user to the same server.

  6. Problem A deploy restarts server 1

    A new version is rolled out and server 1 restarts.

  7. Fix Sessions move to a shared store

    The servers no longer keep sessions in their own memory.

  8. Case The same deploy, nobody notices

    Server 1 restarts for the next release.

  9. Fix Another way: the state travels with the request

    Instead of a session ID, login returns a signed token (JWT) that already contains the user ID and an expiry time.

  10. Case The price: a token cannot be taken back

    The user presses “log out” on a stolen laptop.

  11. Recap Recap

    Four rules for stateless servers.

Terms

Stateless server
A server that keeps nothing in process memory between requests. Sessions and files live in a shared store, so any server can die or be added at any time.
Sticky session
The balancer always sends a user to the same server. It hides the problem rather than solving it: a restart loses the session and load becomes uneven.