Cheat sheet · Shipping
Testing
The problem
A small shop ships every change straight to customers, and customers find the bugs. We add tests layer by layer, see what each layer catches and misses, and make CI the gate before deploy.
What you learn
- The test pyramid: many fast unit tests, fewer integration tests, a handful of end-to-end tests
- Why a mocked database hides real bugs, and how a real database in a container finds them
- Contract tests: catching a broken API between two teams before deploy
- What makes a test flaky, and how CI turns tests into a deploy gate
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_dev(("Developer"))
n_shopper(("Customer"))
n_orders["Orders service"]
n_pricing["Pricing service"]
n_db[("Database")]
n_ci("CI runner")
n_testdb[("Test database")]
n_browser(["Test browser"])
n_shopper --> n_orders
n_orders --> n_pricing
n_orders --> n_db
n_dev --> n_ci
n_ci --> n_testdb
n_ci --> n_pricing
n_ci --> n_browser
n_browser --> n_orders
n_ci --> n_ordersMermaid: request sequence
sequenceDiagram
actor n_shopper as Customer
participant n_orders as Orders service
actor n_dev as Developer
participant n_ci as CI runner
participant n_db as Database
participant n_testdb as Test database
participant n_pricing as Pricing service
participant n_browser as Test browser
Note over n_shopper,n_orders: Customers are the test suite
n_shopper->>n_orders: POST /orders HTTP/1.1
n_orders-->>n_shopper: HTTP/1.1 201 Created
Note over n_dev,n_ci: Unit tests: one function, milliseconds
n_dev->>n_ci: $ test unit price
Note over n_orders,n_db: Green tests, broken database call
n_orders->>n_db: INSERT INTO orders (order_no, user_id, total)
n_db-->>n_orders: ERROR: there is no unique or exclusion constraint
Note over n_ci,n_testdb: Integration tests with a real database in a container
n_ci->>n_testdb: $ test integration
Note over n_orders,n_pricing: Another team renamed a field
n_orders->>n_pricing: GET /prices/sku_77 HTTP/1.1
n_pricing-->>n_orders: HTTP/1.1 200 OK
Note over n_ci,n_pricing: Contract tests between services
n_ci->>n_pricing: $ contract verify --provider pricing
Note over n_ci,n_browser: A few end-to-end tests drive a real browser
n_ci->>n_browser: $ test e2e checkout
Note over n_ci,n_browser: A test that fails one run in twelve
n_ci->>n_browser: $ test e2e checkout
Note over n_browser,n_orders: Own data per test, wait for a condition
n_browser->>n_orders: POST /test/seed HTTP/1.1
n_orders-->>n_browser: HTTP/1.1 201 Created
Note over n_ci,n_orders: CI as the gate before deploy
n_ci->>n_orders: pipeline orders@a91c2eD2
direction: right
n_dev: "Developer" {
shape: person
}
n_shopper: "Customer" {
shape: person
}
n_orders: "Orders service" {
shape: rectangle
}
n_pricing: "Pricing service" {
shape: rectangle
}
n_db: "Database" {
shape: cylinder
}
n_ci: "CI runner" {
shape: rectangle
style.multiple: true
}
n_testdb: "Test database" {
shape: cylinder
}
n_browser: "Test browser" {
shape: rectangle
}
n_shopper -> n_orders
n_orders -> n_pricing
n_orders -> n_db
n_dev -> n_ci
n_ci -> n_testdb
n_ci -> n_pricing
n_ci -> n_browser
n_browser -> n_orders
n_ci -> n_ordersPlantUML: request sequence
@startuml
actor "Customer" as n_shopper
participant "Orders service" as n_orders
actor "Developer" as n_dev
participant "CI runner" as n_ci
database "Database" as n_db
database "Test database" as n_testdb
participant "Pricing service" as n_pricing
participant "Test browser" as n_browser
note over n_shopper, n_orders : Customers are the test suite
n_shopper -> n_orders : POST /orders HTTP/1.1
n_orders --> n_shopper : HTTP/1.1 201 Created
note over n_dev, n_ci : Unit tests: one function, milliseconds
n_dev -> n_ci : $ test unit price
note over n_orders, n_db : Green tests, broken database call
n_orders -> n_db : INSERT INTO orders (order_no, user_id, total)
n_db --> n_orders : ERROR: there is no unique or exclusion constraint
note over n_ci, n_testdb : Integration tests with a real database in a container
n_ci -> n_testdb : $ test integration
note over n_orders, n_pricing : Another team renamed a field
n_orders -> n_pricing : GET /prices/sku_77 HTTP/1.1
n_pricing --> n_orders : HTTP/1.1 200 OK
note over n_ci, n_pricing : Contract tests between services
n_ci -> n_pricing : $ contract verify --provider pricing
note over n_ci, n_browser : A few end-to-end tests drive a real browser
n_ci -> n_browser : $ test e2e checkout
note over n_ci, n_browser : A test that fails one run in twelve
n_ci -> n_browser : [retry] $ test e2e checkout
note over n_browser, n_orders : Own data per test, wait for a condition
n_browser -> n_orders : POST /test/seed HTTP/1.1
n_orders --> n_browser : HTTP/1.1 201 Created
note over n_ci, n_orders : CI as the gate before deploy
n_ci -> n_orders : pipeline orders@a91c2e
@endumlPlantUML: components
@startuml
left to right direction
actor "Developer" as n_dev
actor "Customer" as n_shopper
component "Orders service" as n_orders
component "Pricing service" as n_pricing
database "Database" as n_db
collections "CI runner" as n_ci
database "Test database" as n_testdb
agent "Test browser" as n_browser
n_shopper --> n_orders
n_orders --> n_pricing
n_orders --> n_db
n_dev --> n_ci
n_ci --> n_testdb
n_ci --> n_pricing
n_ci --> n_browser
n_browser --> n_orders
n_ci --> n_orders
@endumlRecap
Each test layer catches what the one below cannot.
1) Many unit tests for logic, in milliseconds. 2) Integration tests with a real database in a container, not a mock. 3) Contract tests between services, checked in the provider's CI. 4) A few end-to-end tests for the main paths. 5) Fix flaky tests with isolated data and condition waits. 6) CI runs it all, fast layers first, and only green builds deploy.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Bugs that reached customers | 9 | 1 |
| CI run time | 0 s | 165 s |
| Flaky runs | 0% | 0.5% |
Steps
- Problem Customers are the test suite
A developer changes the discount code and deploys it straight from the laptop.
- Fix Unit tests: one function, milliseconds
The price math now has unit tests: plain inputs in, expected numbers out.
- Problem Green tests, broken database call
The orders code saves with
ON CONFLICT, so a repeated order is ignored. - Fix Integration tests with a real database in a container
CI now starts a real database in a throwaway container, applies the same migrations as production and runs the repository code against it.
- Problem Another team renamed a field
The pricing team renamed
pricetounit_amount. - Fix Contract tests between services
The orders team writes down what it needs from pricing: this request, a
pricefield that is an integer. - Fix A few end-to-end tests drive a real browser
CI starts a browser and goes through checkout like a person: open the cart, press Pay, wait for the confirmation.
- Problem A test that fails one run in twelve
Checkout sometimes fails and passes on retry.
- Fix Own data per test, wait for a condition
Each test now creates its own user and empty cart and throws them away after.
- Fix CI as the gate before deploy
Every push runs the layers in order: unit first, then integration, contract and end-to-end.
- Recap Recap
Each test layer catches what the one below cannot.