mufassall

Cheat sheet · Business processes

Document lifecycle

14 min · Intermediate · mufassall.dtwin.uz/en/l/document-lifecycle/

← Open the lesson

The problem

An electronic invoice between two companies goes from draft to accepted, rejected or cancelled, and every legally important step must also be confirmed by an external registry. How to keep that state machine correct when two people click at once and the registry answers late, twice or never.

What you learn

  • Document states and a transition table instead of scattered ifs
  • Optimistic concurrency: a version column, 0 rows updated and a 409
  • Pending states until the registry confirms through a callback
  • Idempotent callbacks, a status sync job, history and an outbox
Final scene

Diagram as code

The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.

Mermaid: flowchart

flowchart LR
  n_seller(["Seller app"])
  n_buyer(["Buyer app"])
  n_api["Document API"]
  n_db[("Document DB")]
  n_registry>"Registry"]
  n_worker("Status sync")
  n_events[["Events"]]
  n_seller --> n_api
  n_buyer --> n_api
  n_api --> n_db
  n_api --> n_registry
  n_registry --x|"lost"| n_api
  n_worker --> n_db
  n_worker --> n_registry
  n_db --> n_events
  n_events --> n_buyer

Mermaid: request sequence

sequenceDiagram
  participant n_buyer as Buyer app
  participant n_api as Document API
  participant n_db as Document DB
  participant n_seller as Seller app
  participant n_registry as Registry
  participant n_worker as Status sync
  Note over n_buyer,n_api: An invoice is a state machine
  n_buyer->>n_api: GET /documents/inv_5120 HTTP/1.1
  n_api-->>n_buyer: HTTP/1.1 200 OK
  Note over n_api,n_db: Two users act at the same time
  n_api->>n_db: two clicks on inv_5120, 5 ms apart
  Note over n_api,n_db: Update only the version you read
  n_api->>n_db: UPDATE documents
  n_db-->>n_api: UPDATE 0
  Note over n_seller,n_api: The loser gets a 409, not a silent overwrite
  n_seller->>n_api: POST /documents/inv_5120/cancel HTTP/1.1
  n_api-->>n_seller: HTTP/1.1 409 Conflict
  Note over n_buyer,n_api: Allowed moves come from one transition table
  n_buyer->>n_api: POST /documents/inv_5120/reject HTTP/1.1
  n_api-->>n_buyer: HTTP/1.1 422 Unprocessable Content
  Note over n_api,n_registry: The registry has the last word, and it answers later
  n_api->>n_registry: POST /v1/documents/inv_5120/accept HTTP/1.1
  n_registry-->>n_api: HTTP/1.1 202 Accepted
  Note over n_registry,n_api: Wait in a pending state until the registry confirms
  n_registry->>n_api: POST /registry/callbacks HTTP/1.1
  n_api-->>n_registry: HTTP/1.1 200 OK
  Note over n_registry,n_api: Callbacks come twice, or never
  n_registry-xn_api: callback log, registry -#62; api
  Note over n_api,n_db: Handle each event id once
  n_api->>n_db: INSERT INTO registry_events (event_id, received_at)
  n_db-->>n_api: INSERT 0 0
  Note over n_worker,n_registry: A sync job asks about stuck documents
  n_worker->>n_registry: GET /v1/documents/inv_5133/status HTTP/1.1
  n_registry-->>n_worker: HTTP/1.1 200 OK
  Note over n_api,n_db: Every transition writes history and an outbox row
  n_api->>n_db: BEGIN#59;

D2

direction: right

n_seller: "Seller app" {
  shape: rectangle
}
n_buyer: "Buyer app" {
  shape: rectangle
}
n_api: "Document API" {
  shape: rectangle
}
n_db: "Document DB" {
  shape: cylinder
}
n_registry: "Registry" {
  shape: cloud
}
n_worker: "Status sync" {
  shape: rectangle
  style.multiple: true
}
n_events: "Events" {
  shape: queue
}

n_seller -> n_api
n_buyer -> n_api
n_api -> n_db
n_api -> n_registry
n_registry -> n_api: "lost" {
  style.stroke-dash: 3
  target-arrowhead.shape: cross
}
n_worker -> n_db
n_worker -> n_registry
n_db -> n_events
n_events -> n_buyer

PlantUML: request sequence

@startuml
participant "Buyer app" as n_buyer
participant "Document API" as n_api
database "Document DB" as n_db
participant "Seller app" as n_seller
boundary "Registry" as n_registry
participant "Status sync" as n_worker
note over n_buyer, n_api : An invoice is a state machine
n_buyer -> n_api : GET /documents/inv_5120 HTTP/1.1
n_api --> n_buyer : HTTP/1.1 200 OK
note over n_api, n_db : Two users act at the same time
n_api -> n_db : [duplicate] two clicks on inv_5120, 5 ms apart
note over n_api, n_db : Update only the version you read
n_api -> n_db : UPDATE documents
n_db --> n_api : UPDATE 0
note over n_seller, n_api : The loser gets a 409, not a silent overwrite
n_seller -> n_api : POST /documents/inv_5120/cancel HTTP/1.1
n_api --> n_seller : HTTP/1.1 409 Conflict
note over n_buyer, n_api : Allowed moves come from one transition table
n_buyer -> n_api : POST /documents/inv_5120/reject HTTP/1.1
n_api --> n_buyer : HTTP/1.1 422 Unprocessable Content
note over n_api, n_registry : The registry has the last word, and it answers later
n_api -> n_registry : POST /v1/documents/inv_5120/accept HTTP/1.1
n_registry --> n_api : HTTP/1.1 202 Accepted
note over n_registry, n_api : Wait in a pending state until the registry confirms
n_registry -> n_api : POST /registry/callbacks HTTP/1.1
n_api --> n_registry : HTTP/1.1 200 OK
note over n_registry, n_api : Callbacks come twice, or never
n_registry ->x n_api : [lost] callback log, registry -> api
note over n_api, n_db : Handle each event id once
n_api -> n_db : INSERT INTO registry_events (event_id, received_at)
n_db --> n_api : INSERT 0 0
note over n_worker, n_registry : A sync job asks about stuck documents
n_worker -> n_registry : GET /v1/documents/inv_5133/status HTTP/1.1
n_registry --> n_worker : HTTP/1.1 200 OK
note over n_api, n_db : Every transition writes history and an outbox row
n_api -> n_db : BEGIN;
@enduml

PlantUML: components

@startuml
left to right direction
agent "Seller app" as n_seller
agent "Buyer app" as n_buyer
component "Document API" as n_api
database "Document DB" as n_db
cloud "Registry" as n_registry
collections "Status sync" as n_worker
queue "Events" as n_events
n_seller --> n_api
n_buyer --> n_api
n_api --> n_db
n_api --> n_registry
n_registry -[#c62828,dashed]-> n_api : lost
n_worker --> n_db
n_worker --> n_registry
n_db --> n_events
n_events --> n_buyer
@enduml

Recap

Five rules for documents synchronised with a registry.

1) States and moves live in one transition table. 2) Every update checks version; 0 rows means 409. 3) Until the registry confirms, the document is pending, never final. 4) Callbacks are deduplicated by event_id, and a sync job catches the lost ones. 5) Each transition writes history and an outbox row in the same transaction.

Numbers: at the start and at the end

MetricStartEnd
Lost updates0/day0/day
Differs from registry00
Stuck in pending00
Duplicate effects0/day0/day

Steps

  1. Case An invoice is a state machine

    The seller creates invoice inv_5120 as a draft, signs it (signed) and sends it (sent).

  2. Problem Two users act at the same time

    The buyer clicks accept, and 5 ms later the seller clicks cancel.

  3. Fix Update only the version you read

    The update now names the version it was based on.

  4. Case The loser gets a 409, not a silent overwrite

    The client sends the version it showed to the user in If-Match.

  5. Fix Allowed moves come from one transition table

    A second buyer employee has a stale tab and clicks reject on the already accepted invoice, with a fresh version.

  6. Problem The registry has the last word, and it answers later

    An accepted invoice counts only once the registry has registered it.

  7. Fix Wait in a pending state until the registry confirms

    The table gets new rows: delivered + accept -> accept_pending, then accept_pending + registered -> accepted and accept_pending + rejected -> delivered.

  8. Problem Callbacks come twice, or never

    The API took 12 seconds on the first callback, the registry gave up after 10 and sent it again.

  9. Fix Handle each event id once

    The handler first records event_id in a table with a unique key.

  10. Fix A sync job asks about stuck documents

    Every 5 minutes a job selects documents in a pending state for longer than 15 minutes and asks the registry for their status.

  11. Fix Every transition writes history and an outbox row

    In one transaction the API updates the document, adds a history row (from, to, who, why, when) and an outbox row.

  12. Recap Recap

    Five rules for documents synchronised with a registry.

Terms

State machine
An object changes only through (state, action) pairs allowed by a transition table. Any other transition is rejected.
Pending state
An in-between state a document holds until an external system confirms. The final result is not shown before that.