mufassall

Cheat sheet · Shipping

Testing

12 min · Beginner · mufassall.dtwin.uz/en/l/testing/

← Open the lesson

The problem

A small shop ships every change straight to customers, and customers find the bugs. We add tests layer by layer, see what each layer catches and misses, and make CI the gate before deploy.

What you learn

  • The test pyramid: many fast unit tests, fewer integration tests, a handful of end-to-end tests
  • Why a mocked database hides real bugs, and how a real database in a container finds them
  • Contract tests: catching a broken API between two teams before deploy
  • What makes a test flaky, and how CI turns tests into a deploy gate
Final scene

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_dev(("Developer"))
  n_shopper(("Customer"))
  n_orders["Orders service"]
  n_pricing["Pricing service"]
  n_db[("Database")]
  n_ci("CI runner")
  n_testdb[("Test database")]
  n_browser(["Test browser"])
  n_shopper --> n_orders
  n_orders --> n_pricing
  n_orders --> n_db
  n_dev --> n_ci
  n_ci --> n_testdb
  n_ci --> n_pricing
  n_ci --> n_browser
  n_browser --> n_orders
  n_ci --> n_orders

Mermaid: request sequence

sequenceDiagram
  actor n_shopper as Customer
  participant n_orders as Orders service
  actor n_dev as Developer
  participant n_ci as CI runner
  participant n_db as Database
  participant n_testdb as Test database
  participant n_pricing as Pricing service
  participant n_browser as Test browser
  Note over n_shopper,n_orders: Customers are the test suite
  n_shopper->>n_orders: POST /orders HTTP/1.1
  n_orders-->>n_shopper: HTTP/1.1 201 Created
  Note over n_dev,n_ci: Unit tests: one function, milliseconds
  n_dev->>n_ci: $ test unit price
  Note over n_orders,n_db: Green tests, broken database call
  n_orders->>n_db: INSERT INTO orders (order_no, user_id, total)
  n_db-->>n_orders: ERROR: there is no unique or exclusion constraint
  Note over n_ci,n_testdb: Integration tests with a real database in a container
  n_ci->>n_testdb: $ test integration
  Note over n_orders,n_pricing: Another team renamed a field
  n_orders->>n_pricing: GET /prices/sku_77 HTTP/1.1
  n_pricing-->>n_orders: HTTP/1.1 200 OK
  Note over n_ci,n_pricing: Contract tests between services
  n_ci->>n_pricing: $ contract verify --provider pricing
  Note over n_ci,n_browser: A few end-to-end tests drive a real browser
  n_ci->>n_browser: $ test e2e checkout
  Note over n_ci,n_browser: A test that fails one run in twelve
  n_ci->>n_browser: $ test e2e checkout
  Note over n_browser,n_orders: Own data per test, wait for a condition
  n_browser->>n_orders: POST /test/seed HTTP/1.1
  n_orders-->>n_browser: HTTP/1.1 201 Created
  Note over n_ci,n_orders: CI as the gate before deploy
  n_ci->>n_orders: pipeline orders@a91c2e

D2

direction: right

n_dev: "Developer" {
  shape: person
}
n_shopper: "Customer" {
  shape: person
}
n_orders: "Orders service" {
  shape: rectangle
}
n_pricing: "Pricing service" {
  shape: rectangle
}
n_db: "Database" {
  shape: cylinder
}
n_ci: "CI runner" {
  shape: rectangle
  style.multiple: true
}
n_testdb: "Test database" {
  shape: cylinder
}
n_browser: "Test browser" {
  shape: rectangle
}

n_shopper -> n_orders
n_orders -> n_pricing
n_orders -> n_db
n_dev -> n_ci
n_ci -> n_testdb
n_ci -> n_pricing
n_ci -> n_browser
n_browser -> n_orders
n_ci -> n_orders

PlantUML: request sequence

@startuml
actor "Customer" as n_shopper
participant "Orders service" as n_orders
actor "Developer" as n_dev
participant "CI runner" as n_ci
database "Database" as n_db
database "Test database" as n_testdb
participant "Pricing service" as n_pricing
participant "Test browser" as n_browser
note over n_shopper, n_orders : Customers are the test suite
n_shopper -> n_orders : POST /orders HTTP/1.1
n_orders --> n_shopper : HTTP/1.1 201 Created
note over n_dev, n_ci : Unit tests: one function, milliseconds
n_dev -> n_ci : $ test unit price
note over n_orders, n_db : Green tests, broken database call
n_orders -> n_db : INSERT INTO orders (order_no, user_id, total)
n_db --> n_orders : ERROR: there is no unique or exclusion constraint
note over n_ci, n_testdb : Integration tests with a real database in a container
n_ci -> n_testdb : $ test integration
note over n_orders, n_pricing : Another team renamed a field
n_orders -> n_pricing : GET /prices/sku_77 HTTP/1.1
n_pricing --> n_orders : HTTP/1.1 200 OK
note over n_ci, n_pricing : Contract tests between services
n_ci -> n_pricing : $ contract verify --provider pricing
note over n_ci, n_browser : A few end-to-end tests drive a real browser
n_ci -> n_browser : $ test e2e checkout
note over n_ci, n_browser : A test that fails one run in twelve
n_ci -> n_browser : [retry] $ test e2e checkout
note over n_browser, n_orders : Own data per test, wait for a condition
n_browser -> n_orders : POST /test/seed HTTP/1.1
n_orders --> n_browser : HTTP/1.1 201 Created
note over n_ci, n_orders : CI as the gate before deploy
n_ci -> n_orders : pipeline orders@a91c2e
@enduml

PlantUML: components

@startuml
left to right direction
actor "Developer" as n_dev
actor "Customer" as n_shopper
component "Orders service" as n_orders
component "Pricing service" as n_pricing
database "Database" as n_db
collections "CI runner" as n_ci
database "Test database" as n_testdb
agent "Test browser" as n_browser
n_shopper --> n_orders
n_orders --> n_pricing
n_orders --> n_db
n_dev --> n_ci
n_ci --> n_testdb
n_ci --> n_pricing
n_ci --> n_browser
n_browser --> n_orders
n_ci --> n_orders
@enduml

Recap

Each test layer catches what the one below cannot.

1) Many unit tests for logic, in milliseconds. 2) Integration tests with a real database in a container, not a mock. 3) Contract tests between services, checked in the provider's CI. 4) A few end-to-end tests for the main paths. 5) Fix flaky tests with isolated data and condition waits. 6) CI runs it all, fast layers first, and only green builds deploy.

Numbers: at the start and at the end

MetricStartEnd
Bugs that reached customers91
CI run time0 s165 s
Flaky runs0%0.5%

Steps

  1. Problem Customers are the test suite

    A developer changes the discount code and deploys it straight from the laptop.

  2. Fix Unit tests: one function, milliseconds

    The price math now has unit tests: plain inputs in, expected numbers out.

  3. Problem Green tests, broken database call

    The orders code saves with ON CONFLICT, so a repeated order is ignored.

  4. Fix Integration tests with a real database in a container

    CI now starts a real database in a throwaway container, applies the same migrations as production and runs the repository code against it.

  5. Problem Another team renamed a field

    The pricing team renamed price to unit_amount.

  6. Fix Contract tests between services

    The orders team writes down what it needs from pricing: this request, a price field that is an integer.

  7. Fix A few end-to-end tests drive a real browser

    CI starts a browser and goes through checkout like a person: open the cart, press Pay, wait for the confirmation.

  8. Problem A test that fails one run in twelve

    Checkout sometimes fails and passes on retry.

  9. Fix Own data per test, wait for a condition

    Each test now creates its own user and empty cart and throws them away after.

  10. Fix CI as the gate before deploy

    Every push runs the layers in order: unit first, then integration, contract and end-to-end.

  11. Recap Recap

    Each test layer catches what the one below cannot.