Cheat sheet · Architecture
Event sourcing
The problem
An online shop keeps one row per order and updates it in place. Disputes cannot be answered and two edits at once silently overwrite each other. Event sourcing stores every change as an immutable event in a stream with a version, rebuilds state and read models from the events, and then has to deal with long streams, old event formats, read lag and the right to be forgotten.
What you learn
- Why a row updated in place loses history, and how an append-only stream with versions keeps it
- Optimistic concurrency with
expected_versionand a 409 that tells the writer what changed - Projections with checkpoints, rebuilds, snapshots for long streams and upcasting of old events
- CQRS read lag in the UI, and GDPR erasure of immutable events with crypto-shredding
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_app(["Shop app"])
n_support(("Support agent"))
n_api["Order service"]
n_store[("Event store")]
n_projector("Projector")
n_views[("Read model")]
n_vault>"Key vault"]
n_app --> n_api
n_support --> n_api
n_api --> n_store
n_projector --> n_store
n_projector --> n_views
n_api --> n_views
n_api --> n_vaultMermaid: request sequence
sequenceDiagram
participant n_api as Order service
participant n_orders as Orders table
participant n_store as Event store
actor n_support as Support agent
participant n_projector as Projector
participant n_views as Read model
participant n_app as Shop app
participant n_vault as Key vault
Note over n_api,n_orders: An order is a row that keeps changing
n_api->>n_orders: UPDATE orders
n_orders-->>n_api: UPDATE 1
Note over n_api,n_orders: Support overwrites the customer, nobody can tell
n_api->>n_orders: 10:02:11 app SET address = '12 Elm St'
Note over n_api,n_store: Append events, never update them
n_api->>n_store: INSERT INTO events (stream_id, version, type, data)
n_store-->>n_api: INSERT 0 1
Note over n_support,n_api: Expected version: the stale write is refused
n_support->>n_api: POST /v1/streams/order-ord_77/events HTTP/1.1
n_api-->>n_support: HTTP/1.1 409 Conflict
Note over n_projector,n_views: Read models are built from the events
n_projector->>n_views: projector order_list checkpoint 1204880
Note over n_api,n_store: A stream with 120 000 events
n_api->>n_store: SELECT type, data FROM events
n_store-->>n_api: 120037 rows 2.6 s
Note over n_api,n_store: Snapshots: start from a saved state
n_api->>n_store: snapshot account-b_12
n_store-->>n_api: then: events WHERE version #62; 120000
Note over n_projector,n_store: New code meets an event from 2024
n_projector->>n_store: projector order_list v2 started
Note over n_projector,n_store: Upcasting: old events are converted on read
n_projector->>n_store: stored in 2024 (never rewritten)
n_store-->>n_projector: after upcast v1 -#62; v2 (in memory)
Note over n_app,n_api: The user saves and sees the old value
n_app->>n_api: GET /v1/orders/ord_77?min_version=7 HTTP/1.1
n_api-->>n_app: HTTP/1.1 200 OK
Note over n_api,n_vault: Forget a customer without editing events
n_api->>n_vault: DELETE /v1/keys/subject/c_501 HTTP/1.1
n_vault-->>n_api: HTTP/1.1 200 OKD2
direction: right
n_app: "Shop app" {
shape: rectangle
}
n_support: "Support agent" {
shape: person
}
n_api: "Order service" {
shape: rectangle
}
n_store: "Event store" {
shape: cylinder
}
n_projector: "Projector" {
shape: rectangle
style.multiple: true
}
n_views: "Read model" {
shape: cylinder
}
n_vault: "Key vault" {
shape: cloud
}
n_app -> n_api
n_support -> n_api
n_api -> n_store
n_projector -> n_store
n_projector -> n_views
n_api -> n_views
n_api -> n_vaultPlantUML: request sequence
@startuml
participant "Order service" as n_api
database "Orders table" as n_orders
database "Event store" as n_store
actor "Support agent" as n_support
participant "Projector" as n_projector
database "Read model" as n_views
participant "Shop app" as n_app
boundary "Key vault" as n_vault
note over n_api, n_orders : An order is a row that keeps changing
n_api -> n_orders : UPDATE orders
n_orders --> n_api : UPDATE 1
note over n_api, n_orders : Support overwrites the customer, nobody can tell
n_api -> n_orders : [duplicate] 10:02:11 app SET address = '12 Elm St'
note over n_api, n_store : Append events, never update them
n_api -> n_store : INSERT INTO events (stream_id, version, type, data)
n_store --> n_api : INSERT 0 1
note over n_support, n_api : Expected version: the stale write is refused
n_support -> n_api : POST /v1/streams/order-ord_77/events HTTP/1.1
n_api --> n_support : HTTP/1.1 409 Conflict
note over n_projector, n_views : Read models are built from the events
n_projector -> n_views : projector order_list checkpoint 1204880
note over n_api, n_store : A stream with 120 000 events
n_api -> n_store : SELECT type, data FROM events
n_store --> n_api : 120037 rows 2.6 s
note over n_api, n_store : Snapshots: start from a saved state
n_api -> n_store : snapshot account-b_12
n_store --> n_api : then: events WHERE version > 120000
note over n_projector, n_store : New code meets an event from 2024
n_projector -> n_store : projector order_list v2 started
note over n_projector, n_store : Upcasting: old events are converted on read
n_projector -> n_store : stored in 2024 (never rewritten)
n_store --> n_projector : after upcast v1 -> v2 (in memory)
note over n_app, n_api : The user saves and sees the old value
n_app -> n_api : GET /v1/orders/ord_77?min_version=7 HTTP/1.1
n_api --> n_app : HTTP/1.1 200 OK
note over n_api, n_vault : Forget a customer without editing events
n_api -> n_vault : DELETE /v1/keys/subject/c_501 HTTP/1.1
n_vault --> n_api : HTTP/1.1 200 OK
@endumlPlantUML: components
@startuml
left to right direction
agent "Shop app" as n_app
actor "Support agent" as n_support
component "Order service" as n_api
database "Event store" as n_store
collections "Projector" as n_projector
database "Read model" as n_views
cloud "Key vault" as n_vault
n_app --> n_api
n_support --> n_api
n_api --> n_store
n_projector --> n_store
n_projector --> n_views
n_api --> n_views
n_api --> n_vault
@endumlRecap
Six rules for event sourcing.
1) Store changes as immutable past-tense events in streams with a version; state is a fold. 2) Append with expected_version; a 409 tells the writer what it missed. 3) Read models are disposable projections with a checkpoint in the same transaction; rebuild them by replay. 4) Snapshot only long streams, and treat snapshots as a cache. 5) Events are forever: version them and upcast old ones on read. 6) Plan for read lag in the UI, and encrypt personal data per subject so erasure is a key deletion.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Disputes with no answer | 0/day | 0/day |
| Overwritten edits | 0/day | 0/day |
| Read model lag | 0 ms | 340 ms |
| Load one stream | 3 ms | 30 ms |
Steps
- Case An order is a row that keeps changing
The order service keeps one row per order.
- Problem Support overwrites the customer, nobody can tell
At 10:02 the customer changes the delivery address in the app.
- Fix Append events, never update them
Every change becomes an event:
OrderPlaced,ItemAdded,AddressChanged,OrderShipped. - Fix Expected version: the stale write is refused
Every append says which version the writer last saw.
- Fix Read models are built from the events
Folding streams is fine for one order, not for "all open orders of this city".
- Problem A stream with 120 000 events
Wholesale buyers have an account stream: every order line, payment and credit change since 2021.
- Fix Snapshots: start from a saved state
Every 500 events the service saves a snapshot: the folded state of the stream at a version.
- Problem New code meets an event from 2024
The team splits the address into street, city and zip.
- Fix Upcasting: old events are converted on read
Every event carries a schema version.
- Fix The user saves and sees the old value
The customer changes the address and the app opens the order page.
- Fix Forget a customer without editing events
A customer asks to be erased.
- Recap Recap
Six rules for event sourcing.
Terms
- Event sourcing
- Storing every change as an immutable event in a versioned stream and computing current state by replaying those events.
- CQRS
- Separating the write model from read models that are built for queries and usually lag slightly behind.
- Crypto-shredding
- Erasing data that cannot be deleted, such as events or backups, by destroying the per-subject key it was encrypted with.