Cheat sheet · Business processes
Subscription billing
The problem
A SaaS product sells monthly plans and charges cards every night. Two runs of the job charge the same customer twice, cards get declined, customers upgrade mid-month, months have different lengths and the provider reports results later by webhook. Each problem has a concrete fix that keeps money correct and customers on board.
What you learn
- Subscriptions, plans, billing periods and the invoice row behind every charge
- Overlapping billing runs: idempotency keys per invoice and leases on each row
- Dunning after a decline, and proration when a customer changes plan mid-period
- Month ends and time zones, provider webhooks, and cancelling at period end
Diagram as code
The lesson's final design as text: paste it into Jira, GitLab, GitHub, Notion or your docs, where Mermaid renders by itself. There are D2 and PlantUML (Confluence, IntelliJ) versions too.
Mermaid: flowchart
flowchart LR
n_customer(["Customer"])
n_api["Billing API"]
n_invoices[("Invoices DB")]
n_job("Billing job")
n_provider>"Payment provider"]
n_job2("Billing job #35;2")
n_mailer>"Email service"]
n_customer --> n_api
n_api --> n_invoices
n_job --> n_invoices
n_job -->|"call"| n_provider
n_job2 --> n_invoices
n_job --> n_mailer
n_provider -.->|"async"| n_apiMermaid: request sequence
sequenceDiagram
participant n_api as Billing API
participant n_invoices as Invoices DB
participant n_job as Billing job
participant n_job2 as Billing job #35;2
participant n_provider as Payment provider
Note over n_api,n_invoices: A subscription is a plan plus a billing period
n_api->>n_invoices: invoices: new row
Note over n_job,n_invoices: Every night a job renews what is due
n_job->>n_invoices: SELECT s.id, s.plan, s.current_period_end
n_invoices-->>n_job: 18 412 rows
Note over n_job2,n_provider: Two copies of the job charge the same customers
n_job2->>n_provider: 02:00 cron billing-run started on node-a
Note over n_job,n_provider: A lease per invoice and an idempotency key per charge
n_job->>n_provider: POST /v1/charges HTTP/1.1
n_provider-->>n_job: HTTP/1.1 200 OK
Note over n_job,n_provider: The card is declined
n_job->>n_provider: POST /v1/charges HTTP/1.1
n_provider-->>n_job: HTTP/1.1 402 Payment Required
Note over n_job,n_invoices: Dunning: retries over days, a grace period, then suspend
n_job->>n_invoices: dunning: row for in_7188
Note over n_api,n_invoices: Upgrade mid-period: credit the unused part, charge the new one
n_api->>n_invoices: upgrade sub_42 basic -#62; pro at 2026-11-16T00:00Z
Note over n_job,n_invoices: Month ends and time zones
n_job->>n_invoices: sub_77 anchor 2026-01-31T00:00 +05:00, anchor_day 31
Note over n_provider,n_api: The provider confirms payments later by webhook
n_provider->>n_api: POST /billing/webhooks HTTP/1.1
n_api-->>n_provider: HTTP/1.1 200 OK
Note over n_job,n_invoices: Cancel at period end, or now with a refund
n_job->>n_invoices: UPDATE subscriptions SET status = 'canceled'
n_invoices-->>n_job: UPDATE 312D2
direction: right
n_customer: "Customer" {
shape: rectangle
}
n_api: "Billing API" {
shape: rectangle
}
n_invoices: "Invoices DB" {
shape: cylinder
}
n_job: "Billing job" {
shape: rectangle
style.multiple: true
}
n_provider: "Payment provider" {
shape: cloud
}
n_job2: "Billing job #2" {
shape: rectangle
style.multiple: true
}
n_mailer: "Email service" {
shape: cloud
}
n_customer -> n_api
n_api -> n_invoices
n_job -> n_invoices
n_job -> n_provider: "call"
n_job2 -> n_invoices
n_job -> n_mailer
n_provider -> n_api: "async" {
style.stroke-dash: 3
}PlantUML: request sequence
@startuml
participant "Billing API" as n_api
database "Invoices DB" as n_invoices
participant "Billing job" as n_job
participant "Billing job #2" as n_job2
boundary "Payment provider" as n_provider
note over n_api, n_invoices : A subscription is a plan plus a billing period
n_api -> n_invoices : invoices: new row
note over n_job, n_invoices : Every night a job renews what is due
n_job -> n_invoices : SELECT s.id, s.plan, s.current_period_end
n_invoices --> n_job : 18 412 rows
note over n_job2, n_provider : Two copies of the job charge the same customers
n_job2 -> n_provider : [duplicate] 02:00 cron billing-run started on node-a
note over n_job, n_provider : A lease per invoice and an idempotency key per charge
n_job -> n_provider : POST /v1/charges HTTP/1.1
n_provider --> n_job : HTTP/1.1 200 OK
note over n_job, n_provider : The card is declined
n_job -> n_provider : POST /v1/charges HTTP/1.1
n_provider --> n_job : HTTP/1.1 402 Payment Required
note over n_job, n_invoices : Dunning: retries over days, a grace period, then suspend
n_job -> n_invoices : dunning: row for in_7188
note over n_api, n_invoices : Upgrade mid-period: credit the unused part, charge the new one
n_api -> n_invoices : upgrade sub_42 basic -> pro at 2026-11-16T00:00Z
note over n_job, n_invoices : Month ends and time zones
n_job -> n_invoices : sub_77 anchor 2026-01-31T00:00 +05:00, anchor_day 31
note over n_provider, n_api : The provider confirms payments later by webhook
n_provider ->> n_api : POST /billing/webhooks HTTP/1.1
n_api -->> n_provider : HTTP/1.1 200 OK
note over n_job, n_invoices : Cancel at period end, or now with a refund
n_job -> n_invoices : UPDATE subscriptions SET status = 'canceled'
n_invoices --> n_job : UPDATE 312
@endumlPlantUML: components
@startuml
left to right direction
agent "Customer" as n_customer
component "Billing API" as n_api
database "Invoices DB" as n_invoices
collections "Billing job" as n_job
cloud "Payment provider" as n_provider
collections "Billing job #2" as n_job2
cloud "Email service" as n_mailer
n_customer --> n_api
n_api --> n_invoices
n_job --> n_invoices
n_job --> n_provider : call
n_job2 --> n_invoices
n_job --> n_mailer
n_provider ..> n_api : async
@endumlRecap
Six rules for recurring billing.
1) Every charge belongs to one invoice for one half-open period. 2) Assume the billing job runs twice: lease each invoice and give each charge an idempotency key from the invoice and attempt. 3) After a soft decline, run dunning: retries over days, emails, a grace period, then suspend. 4) Prorate plan changes as invoice lines with one rounding rule. 5) Keep the anchor day and the customer's time zone; store moments in UTC. 6) Let signed, deduplicated webhooks finish payments, check stuck invoices with the provider, and cancel at period end unless the customer asks for now.
Numbers: at the start and at the end
| Metric | Start | End |
|---|---|---|
| Double charges | 0/day | 0/day |
| Unpaid invoices | 0 $ | 4 700 $ |
| Recovered after decline | 0% | 71% |
| Involuntary churn | 0% | 1% |
Steps
- Case A subscription is a plan plus a billing period
A customer picks the Basic plan, $20.00 a month.
- Case Every night a job renews what is due
At 02:00 UTC a billing job selects every active subscription whose period has ended.
- Problem Two copies of the job charge the same customers
A deploy started a new instance of the billing service while the old one was still running.
- Fix A lease per invoice and an idempotency key per charge
A worker claims a batch of due invoices with
FOR UPDATE SKIP LOCKEDand writeslease_until = now() + 10 min. - Problem The card is declined
Every night about 4% of renewals fail.
- Fix Dunning: retries over days, a grace period, then suspend
Each failed invoice gets a dunning record: retry on days 1, 3, 5 and 7, at mid-morning when salaries have landed rather than at 02:00.
- Case Upgrade mid-period: credit the unused part, charge the new one
On November 16 the customer upgrades from Basic ($20) to Pro ($50).
- Case Month ends and time zones
A customer in UTC+5 subscribed at midnight their time on January 31.
- Fix The provider confirms payments later by webhook
Some charges do not finish in the call: the bank asks for extra verification or processes the payment later.
- Fix Cancel at period end, or now with a refund
Most customers who cancel have already paid for this month, so they keep access until the period ends.
- Recap Recap
Six rules for recurring billing.