Cheat sheet · Scaling
CDN and edge caching
The problem
A shop's server sits on one continent, its buyers on another. Images and scripts move to edge caches near the users: how a request reaches the edge, when it goes back to the origin, how to replace a file and what must never be cached.
What you learn
- How an edge cache sits between users and the origin, and what a miss and a hit cost
Cache-Controldecides what may be cached and for how long- Purge vs versioned URLs for replacing a file
- An origin shield, and why private responses must never be cached
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_client(["Buyer, region A"])
n_origin["Origin server"]
n_bucket[("File storage")]
n_edge[/"Edge cache A"\]
n_client2(["Buyer, region B"])
n_edge2[/"Edge cache B"\]
n_shield[/"Origin shield"\]
n_origin --> n_bucket
n_client --> n_edge
n_client2 --> n_edge2
n_edge --> n_shield
n_edge2 --> n_shield
n_shield --> n_originMermaid: request sequence
sequenceDiagram
participant n_client as Buyer, region A
participant n_origin as Origin server
participant n_edge as Edge cache A
actor n_ops as Engineer
participant n_edge2 as Edge cache B
participant n_shield as Origin shield
participant n_client2 as Buyer, region B
Note over n_client,n_origin: Every image crosses half the planet
n_client->>n_origin: GET /img/sofa-1042.jpg HTTP/1.1
n_origin-->>n_client: HTTP/1.1 200 OK
Note over n_client,n_edge: A cache near the buyers
n_client->>n_edge: dig static.example.com
n_edge-->>n_client: static.example.com. 300 IN CNAME edge.cdn.example.net.
Note over n_edge,n_origin: A miss: the edge asks the origin once
n_edge->>n_origin: GET /img/sofa-1042.jpg HTTP/1.1
n_origin-->>n_edge: HTTP/1.1 200 OK
Note over n_client,n_edge: A hit: the origin is not involved
n_client->>n_edge: GET /img/sofa-1042.jpg HTTP/1.1
n_edge-->>n_client: HTTP/1.1 200 OK
Note over n_client,n_edge: The photo changed, the edge still has the old one
n_client->>n_edge: GET /img/sofa-1042.jpg HTTP/1.1
n_edge-->>n_client: HTTP/1.1 200 OK
Note over n_ops,n_edge: Purge: tell every edge to drop the file
n_ops->>n_edge: POST /v1/purge HTTP/1.1
n_edge-->>n_ops: HTTP/1.1 202 Accepted
Note over n_client,n_edge: Versioned URLs: a new file gets a new name
n_client->>n_edge: GET /img/sofa-1042.3f9a2c.jpg HTTP/1.1
n_edge-->>n_client: HTTP/1.1 200 OK
Note over n_edge2,n_origin: Each edge asks the origin on its own
n_edge2->>n_origin: GET /app.9c1d.js edge=edge-a MISS
Note over n_shield,n_origin: An origin shield in between
n_shield->>n_origin: GET /app.9c1d.js HTTP/1.1
n_origin-->>n_shield: HTTP/1.1 200 OK
Note over n_client2,n_edge2: A personal response was cached
n_client2->>n_edge2: GET /api/cart HTTP/1.1
n_edge2-->>n_client2: HTTP/1.1 200 OK
Note over n_client2,n_edge2: Personal responses: private, no-store
n_client2->>n_edge2: GET /api/cart HTTP/1.1
n_edge2-->>n_client2: HTTP/1.1 200 OKD2
direction: right
n_client: "Buyer, region A" {
shape: rectangle
}
n_origin: "Origin server" {
shape: rectangle
}
n_bucket: "File storage" {
shape: stored_data
}
n_edge: "Edge cache A" {
shape: oval
}
n_client2: "Buyer, region B" {
shape: rectangle
}
n_edge2: "Edge cache B" {
shape: oval
}
n_shield: "Origin shield" {
shape: oval
}
n_origin -> n_bucket
n_client -> n_edge
n_client2 -> n_edge2
n_edge -> n_shield
n_edge2 -> n_shield
n_shield -> n_originPlantUML: request sequence
@startuml
participant "Buyer, region A" as n_client
participant "Origin server" as n_origin
participant "Edge cache A" as n_edge
actor "Engineer" as n_ops
participant "Edge cache B" as n_edge2
participant "Origin shield" as n_shield
participant "Buyer, region B" as n_client2
note over n_client, n_origin : Every image crosses half the planet
n_client -> n_origin : GET /img/sofa-1042.jpg HTTP/1.1
n_origin --> n_client : HTTP/1.1 200 OK
note over n_client, n_edge : A cache near the buyers
n_client -> n_edge : dig static.example.com
n_edge --> n_client : static.example.com. 300 IN CNAME edge.cdn.example.net.
note over n_edge, n_origin : A miss: the edge asks the origin once
n_edge -> n_origin : GET /img/sofa-1042.jpg HTTP/1.1
n_origin --> n_edge : HTTP/1.1 200 OK
note over n_client, n_edge : A hit: the origin is not involved
n_client -> n_edge : GET /img/sofa-1042.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_client, n_edge : The photo changed, the edge still has the old one
n_client -> n_edge : GET /img/sofa-1042.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_ops, n_edge : Purge: tell every edge to drop the file
n_ops -> n_edge : POST /v1/purge HTTP/1.1
n_edge --> n_ops : HTTP/1.1 202 Accepted
note over n_client, n_edge : Versioned URLs: a new file gets a new name
n_client -> n_edge : GET /img/sofa-1042.3f9a2c.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_edge2, n_origin : Each edge asks the origin on its own
n_edge2 -> n_origin : GET /app.9c1d.js edge=edge-a MISS
note over n_shield, n_origin : An origin shield in between
n_shield -> n_origin : GET /app.9c1d.js HTTP/1.1
n_origin --> n_shield : HTTP/1.1 200 OK
note over n_client2, n_edge2 : A personal response was cached
n_client2 -> n_edge2 : GET /api/cart HTTP/1.1
n_edge2 --> n_client2 : HTTP/1.1 200 OK
note over n_client2, n_edge2 : Personal responses: private, no-store
n_client2 -> n_edge2 : GET /api/cart HTTP/1.1
n_edge2 --> n_client2 : HTTP/1.1 200 OK
@endumlPlantUML: components
@startuml
left to right direction
agent "Buyer, region A" as n_client
component "Origin server" as n_origin
storage "File storage" as n_bucket
frame "Edge cache A" as n_edge
agent "Buyer, region B" as n_client2
frame "Edge cache B" as n_edge2
frame "Origin shield" as n_shield
n_origin --> n_bucket
n_client --> n_edge
n_client2 --> n_edge2
n_edge --> n_shield
n_edge2 --> n_shield
n_shield --> n_origin
@endumlRecap
Five rules for a CDN.
1) Static files are served from an edge near the user; the origin only fills misses. 2) The origin sets Cache-Control explicitly on every response. 3) Replace files with versioned URLs; keep purge for emergencies. 4) An origin shield turns many edge misses into one. 5) Personal responses are private, no-store, always.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Page load p95 | 1 900 ms | 120 ms |
| Edge hit ratio | 0% | 93% |
| Requests to origin | 1 200/s | 120/s |
| Wrong content served | 0% | 0% |
Steps
- Problem Every image crosses half the planet
The shop's server is in one data centre.
- Fix A cache near the buyers
A CDN runs cache servers in many cities.
- Case A miss: the edge asks the origin once
The edge does not have the photo yet.
- Case A hit: the origin is not involved
Every following buyer in this region gets the photo straight from the edge.
- Problem The photo changed, the edge still has the old one
The shop uploads a better photo under the same file name.
- Fix Purge: tell every edge to drop the file
The engineer calls the CDN's purge API.
- Fix Versioned URLs: a new file gets a new name
The build adds a short hash of the content to every file name.
- Problem Each edge asks the origin on its own
Buyers in a second region get their own edge.
- Fix An origin shield in between
The edges now ask one middle cache, the origin shield, instead of the origin.
- Problem A personal response was cached
The cart endpoint was answered with
publicby mistake. - Fix Personal responses: private, no-store
Personal API responses now carry
private, no-store. - Recap Recap
Five rules for a CDN.
Terms
- CDN and edge
- A network of caches on edge servers near users that serves static files. The origin only fills misses, and Cache-Control decides what is cached and for how long.
- Origin shield
- A middle cache layer between the edges and the origin. Instead of every edge asking on its own, many edge misses turn into one request to the origin.