mufassall

Cheat sheet · Scaling

CDN and edge caching

12 min · Beginner · mufassall.dtwin.uz/en/l/cdn/

← Open the lesson

The problem

A shop's server sits on one continent, its buyers on another. Images and scripts move to edge caches near the users: how a request reaches the edge, when it goes back to the origin, how to replace a file and what must never be cached.

What you learn

  • How an edge cache sits between users and the origin, and what a miss and a hit cost
  • Cache-Control decides what may be cached and for how long
  • Purge vs versioned URLs for replacing a file
  • An origin shield, and why private responses must never be cached
Final scene
45%22%20%30%

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_client(["Buyer, region A"])
  n_origin["Origin server"]
  n_bucket[("File storage")]
  n_edge[/"Edge cache A"\]
  n_client2(["Buyer, region B"])
  n_edge2[/"Edge cache B"\]
  n_shield[/"Origin shield"\]
  n_origin --> n_bucket
  n_client --> n_edge
  n_client2 --> n_edge2
  n_edge --> n_shield
  n_edge2 --> n_shield
  n_shield --> n_origin

Mermaid: request sequence

sequenceDiagram
  participant n_client as Buyer, region A
  participant n_origin as Origin server
  participant n_edge as Edge cache A
  actor n_ops as Engineer
  participant n_edge2 as Edge cache B
  participant n_shield as Origin shield
  participant n_client2 as Buyer, region B
  Note over n_client,n_origin: Every image crosses half the planet
  n_client->>n_origin: GET /img/sofa-1042.jpg HTTP/1.1
  n_origin-->>n_client: HTTP/1.1 200 OK
  Note over n_client,n_edge: A cache near the buyers
  n_client->>n_edge: dig static.example.com
  n_edge-->>n_client: static.example.com. 300 IN CNAME edge.cdn.example.net.
  Note over n_edge,n_origin: A miss: the edge asks the origin once
  n_edge->>n_origin: GET /img/sofa-1042.jpg HTTP/1.1
  n_origin-->>n_edge: HTTP/1.1 200 OK
  Note over n_client,n_edge: A hit: the origin is not involved
  n_client->>n_edge: GET /img/sofa-1042.jpg HTTP/1.1
  n_edge-->>n_client: HTTP/1.1 200 OK
  Note over n_client,n_edge: The photo changed, the edge still has the old one
  n_client->>n_edge: GET /img/sofa-1042.jpg HTTP/1.1
  n_edge-->>n_client: HTTP/1.1 200 OK
  Note over n_ops,n_edge: Purge: tell every edge to drop the file
  n_ops->>n_edge: POST /v1/purge HTTP/1.1
  n_edge-->>n_ops: HTTP/1.1 202 Accepted
  Note over n_client,n_edge: Versioned URLs: a new file gets a new name
  n_client->>n_edge: GET /img/sofa-1042.3f9a2c.jpg HTTP/1.1
  n_edge-->>n_client: HTTP/1.1 200 OK
  Note over n_edge2,n_origin: Each edge asks the origin on its own
  n_edge2->>n_origin: GET /app.9c1d.js edge=edge-a MISS
  Note over n_shield,n_origin: An origin shield in between
  n_shield->>n_origin: GET /app.9c1d.js HTTP/1.1
  n_origin-->>n_shield: HTTP/1.1 200 OK
  Note over n_client2,n_edge2: A personal response was cached
  n_client2->>n_edge2: GET /api/cart HTTP/1.1
  n_edge2-->>n_client2: HTTP/1.1 200 OK
  Note over n_client2,n_edge2: Personal responses: private, no-store
  n_client2->>n_edge2: GET /api/cart HTTP/1.1
  n_edge2-->>n_client2: HTTP/1.1 200 OK

D2

direction: right

n_client: "Buyer, region A" {
  shape: rectangle
}
n_origin: "Origin server" {
  shape: rectangle
}
n_bucket: "File storage" {
  shape: stored_data
}
n_edge: "Edge cache A" {
  shape: oval
}
n_client2: "Buyer, region B" {
  shape: rectangle
}
n_edge2: "Edge cache B" {
  shape: oval
}
n_shield: "Origin shield" {
  shape: oval
}

n_origin -> n_bucket
n_client -> n_edge
n_client2 -> n_edge2
n_edge -> n_shield
n_edge2 -> n_shield
n_shield -> n_origin

PlantUML: request sequence

@startuml
participant "Buyer, region A" as n_client
participant "Origin server" as n_origin
participant "Edge cache A" as n_edge
actor "Engineer" as n_ops
participant "Edge cache B" as n_edge2
participant "Origin shield" as n_shield
participant "Buyer, region B" as n_client2
note over n_client, n_origin : Every image crosses half the planet
n_client -> n_origin : GET /img/sofa-1042.jpg HTTP/1.1
n_origin --> n_client : HTTP/1.1 200 OK
note over n_client, n_edge : A cache near the buyers
n_client -> n_edge : dig static.example.com
n_edge --> n_client : static.example.com. 300 IN CNAME edge.cdn.example.net.
note over n_edge, n_origin : A miss: the edge asks the origin once
n_edge -> n_origin : GET /img/sofa-1042.jpg HTTP/1.1
n_origin --> n_edge : HTTP/1.1 200 OK
note over n_client, n_edge : A hit: the origin is not involved
n_client -> n_edge : GET /img/sofa-1042.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_client, n_edge : The photo changed, the edge still has the old one
n_client -> n_edge : GET /img/sofa-1042.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_ops, n_edge : Purge: tell every edge to drop the file
n_ops -> n_edge : POST /v1/purge HTTP/1.1
n_edge --> n_ops : HTTP/1.1 202 Accepted
note over n_client, n_edge : Versioned URLs: a new file gets a new name
n_client -> n_edge : GET /img/sofa-1042.3f9a2c.jpg HTTP/1.1
n_edge --> n_client : HTTP/1.1 200 OK
note over n_edge2, n_origin : Each edge asks the origin on its own
n_edge2 -> n_origin : GET /app.9c1d.js edge=edge-a MISS
note over n_shield, n_origin : An origin shield in between
n_shield -> n_origin : GET /app.9c1d.js HTTP/1.1
n_origin --> n_shield : HTTP/1.1 200 OK
note over n_client2, n_edge2 : A personal response was cached
n_client2 -> n_edge2 : GET /api/cart HTTP/1.1
n_edge2 --> n_client2 : HTTP/1.1 200 OK
note over n_client2, n_edge2 : Personal responses: private, no-store
n_client2 -> n_edge2 : GET /api/cart HTTP/1.1
n_edge2 --> n_client2 : HTTP/1.1 200 OK
@enduml

PlantUML: components

@startuml
left to right direction
agent "Buyer, region A" as n_client
component "Origin server" as n_origin
storage "File storage" as n_bucket
frame "Edge cache A" as n_edge
agent "Buyer, region B" as n_client2
frame "Edge cache B" as n_edge2
frame "Origin shield" as n_shield
n_origin --> n_bucket
n_client --> n_edge
n_client2 --> n_edge2
n_edge --> n_shield
n_edge2 --> n_shield
n_shield --> n_origin
@enduml

Recap

Five rules for a CDN.

1) Static files are served from an edge near the user; the origin only fills misses. 2) The origin sets Cache-Control explicitly on every response. 3) Replace files with versioned URLs; keep purge for emergencies. 4) An origin shield turns many edge misses into one. 5) Personal responses are private, no-store, always.

Numbers: at the start and at the end

MetricStartEnd
Page load p951 900 ms120 ms
Edge hit ratio0%93%
Requests to origin1 200/s120/s
Wrong content served0%0%

Steps

  1. Problem Every image crosses half the planet

    The shop's server is in one data centre.

  2. Fix A cache near the buyers

    A CDN runs cache servers in many cities.

  3. Case A miss: the edge asks the origin once

    The edge does not have the photo yet.

  4. Case A hit: the origin is not involved

    Every following buyer in this region gets the photo straight from the edge.

  5. Problem The photo changed, the edge still has the old one

    The shop uploads a better photo under the same file name.

  6. Fix Purge: tell every edge to drop the file

    The engineer calls the CDN's purge API.

  7. Fix Versioned URLs: a new file gets a new name

    The build adds a short hash of the content to every file name.

  8. Problem Each edge asks the origin on its own

    Buyers in a second region get their own edge.

  9. Fix An origin shield in between

    The edges now ask one middle cache, the origin shield, instead of the origin.

  10. Problem A personal response was cached

    The cart endpoint was answered with public by mistake.

  11. Fix Personal responses: private, no-store

    Personal API responses now carry private, no-store.

  12. Recap Recap

    Five rules for a CDN.

Terms

CDN and edge
A network of caches on edge servers near users that serves static files. The origin only fills misses, and Cache-Control decides what is cached and for how long.
Origin shield
A middle cache layer between the edges and the origin. Instead of every edge asking on its own, many edge misses turn into one request to the origin.