mufassall

Konspekt · Aloqa

Xabar navbatlari

14 daq · O'rta · mufassall.dtwin.uz/uz/l/queues/

← Darsni ochish

Muammo

Do'kon buyurtma xatini soniyasiga 100 ta xat qabul qiladigan provayder orqali yuboradi, aksiya esa soniyasiga 600 ta buyurtma olib keladi. Navbat to'lqinni qanday yutadi va unda nimalarni sozlash shart: lease, ack, backoff, dead-letter navbat va backpressure.

Nimani o'rganasiz

  • Producer va worker'lar orasidagi navbat to'lqinni qanday tekislaydi
  • Visibility timeout va ack: worker o'lsa yoki sekin bo'lsa nima bo'ladi
  • Eksponensial backoff va jitter bilan qayta urinish, zaharli xabarlar uchun dead-letter navbat
  • Iste'molchilarni backlog bo'yicha kengaytirish va producer'ni backpressure bilan sekinlatish
Yakuniy sahna
20%30%50%40%40%

Diagramma kod sifatida

Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.

Mermaid: sxema

flowchart LR
  n_buyers(["Xaridorlar"])
  n_api["Do'kon API"]
  n_mail>"Pochta provayderi"]
  n_queue[["Navbat"]]
  n_w1("Worker 1")
  n_w2("Worker 2")
  n_dlq[["Dead-letter navbat"]]
  n_campaign("Rassilka job'i")
  n_buyers -->|"chaqiruv"| n_api
  n_api -->|"asinxron"| n_queue
  n_queue -->|"asinxron"| n_w1
  n_w1 -->|"chaqiruv"| n_mail
  n_queue -->|"asinxron"| n_w2
  n_w2 -->|"chaqiruv"| n_mail
  n_queue --x|"asinxron"| n_dlq
  n_campaign -->|"asinxron"| n_queue

Mermaid: so'rovlar ketma-ketligi

sequenceDiagram
  participant n_api as Do'kon API
  participant n_mail as Pochta provayderi
  participant n_queue as Navbat
  participant n_w2 as Worker 2
  participant n_w1 as Worker 1
  participant n_dlq as Dead-letter navbat
  participant n_campaign as Rassilka job'i
  Note over n_api,n_mail: Xat buyurtma so'rovi ichida yuboriladi
  n_api->>n_mail: POST /v1/send HTTP/1.1
  n_mail-->>n_api: HTTP/1.1 202 Accepted
  Note over n_api,n_mail: Aksiya: soniyasiga 600 buyurtma
  n_api->>n_mail: POST /v1/send HTTP/1.1
  n_mail-->>n_api: HTTP/1.1 429 Too Many Requests
  Note over n_api,n_queue: Buyurtma va xat orasida navbat
  n_api->>n_queue: SEND emails
  n_queue-->>n_api: OK queued, depth 42000
  Note over n_queue,n_w2: Ko'proq worker, provayder shiftigacha
  n_queue->>n_w2: autoscaler: queue emails
  Note over n_queue,n_w1: Worker xabarni ushlab o'ldi
  n_queue->>n_w1: RECEIVE emails
  n_w1-->>n_queue: DELETE m_8812
  Note over n_queue,n_w2: Sekin ish lease'dan uzoq davom etdi
  n_queue->>n_w2: 10:02:00 w1 got m_9120 (receipt: 212 pages)
  Note over n_queue,n_w1: Ishlayotganda lease'ni uzaytir
  n_queue->>n_w1: EXTEND emails m_9120 visibility_timeout=60s
  n_w1-->>n_queue: OK until 10:03:20
  Note over n_w1,n_mail: Provayder tushdi, worker'lar unga yopiriladi
  n_w1->>n_mail: POST /v1/send HTTP/1.1
  n_mail-->>n_w1: HTTP/1.1 503 Service Unavailable
  Note over n_queue,n_w1: Jitter bilan eksponensial backoff
  n_queue->>n_w1: NACK emails m_8812 delay=86s
  Note over n_queue,n_dlq: Zaharli xabar dead-letter navbatga
  n_queue->>n_dlq: SET emails redrive_policy
  n_dlq-->>n_queue: m_7731 moved to emails-dlq after 5 receives
  Note over n_campaign,n_queue: Backpressure: producer sekinlashadi
  n_campaign->>n_queue: campaign: 2000000 newsletter emails to publish

D2

direction: right

n_buyers: "Xaridorlar" {
  shape: rectangle
}
n_api: "Do'kon API" {
  shape: rectangle
}
n_mail: "Pochta provayderi" {
  shape: cloud
}
n_queue: "Navbat" {
  shape: queue
}
n_w1: "Worker 1" {
  shape: rectangle
  style.multiple: true
}
n_w2: "Worker 2" {
  shape: rectangle
  style.multiple: true
}
n_dlq: "Dead-letter navbat" {
  shape: queue
}
n_campaign: "Rassilka job'i" {
  shape: rectangle
  style.multiple: true
}

n_buyers -> n_api: "chaqiruv"
n_api -> n_queue: "asinxron"
n_queue -> n_w1: "asinxron"
n_w1 -> n_mail: "chaqiruv"
n_queue -> n_w2: "asinxron"
n_w2 -> n_mail: "chaqiruv"
n_queue -> n_dlq: "asinxron" {
  target-arrowhead.shape: cross
}
n_campaign -> n_queue: "asinxron"

PlantUML: so'rovlar ketma-ketligi

@startuml
participant "Do'kon API" as n_api
boundary "Pochta provayderi" as n_mail
queue "Navbat" as n_queue
participant "Worker 2" as n_w2
participant "Worker 1" as n_w1
queue "Dead-letter navbat" as n_dlq
participant "Rassilka job'i" as n_campaign
note over n_api, n_mail : Xat buyurtma so'rovi ichida yuboriladi
n_api -> n_mail : POST /v1/send HTTP/1.1
n_mail --> n_api : HTTP/1.1 202 Accepted
note over n_api, n_mail : Aksiya: soniyasiga 600 buyurtma
n_api -> n_mail : POST /v1/send HTTP/1.1
n_mail --> n_api : HTTP/1.1 429 Too Many Requests
note over n_api, n_queue : Buyurtma va xat orasida navbat
n_api ->> n_queue : SEND emails
n_queue -->> n_api : OK queued, depth 42000
note over n_queue, n_w2 : Ko'proq worker, provayder shiftigacha
n_queue ->> n_w2 : autoscaler: queue emails
note over n_queue, n_w1 : Worker xabarni ushlab o'ldi
n_queue ->> n_w1 : RECEIVE emails
n_w1 -->> n_queue : DELETE m_8812
note over n_queue, n_w2 : Sekin ish lease'dan uzoq davom etdi
n_queue ->> n_w2 : [dublikat] 10:02:00 w1 got m_9120 (receipt: 212 pages)
note over n_queue, n_w1 : Ishlayotganda lease'ni uzaytir
n_queue ->> n_w1 : EXTEND emails m_9120 visibility_timeout=60s
n_w1 -->> n_queue : OK until 10:03:20
note over n_w1, n_mail : Provayder tushdi, worker'lar unga yopiriladi
n_w1 -> n_mail : [qayta urinish] POST /v1/send HTTP/1.1
n_mail --> n_w1 : HTTP/1.1 503 Service Unavailable
note over n_queue, n_w1 : Jitter bilan eksponensial backoff
n_queue ->> n_w1 : NACK emails m_8812 delay=86s
note over n_queue, n_dlq : Zaharli xabar dead-letter navbatga
n_queue ->> n_dlq : SET emails redrive_policy
n_dlq -->> n_queue : m_7731 moved to emails-dlq after 5 receives
note over n_campaign, n_queue : Backpressure: producer sekinlashadi
n_campaign ->> n_queue : campaign: 2000000 newsletter emails to publish
@enduml

PlantUML: komponentlar

@startuml
left to right direction
agent "Xaridorlar" as n_buyers
component "Do'kon API" as n_api
cloud "Pochta provayderi" as n_mail
queue "Navbat" as n_queue
collections "Worker 1" as n_w1
collections "Worker 2" as n_w2
queue "Dead-letter navbat" as n_dlq
collections "Rassilka job'i" as n_campaign
n_buyers --> n_api : chaqiruv
n_api --> n_queue : asinxron
n_queue --> n_w1 : asinxron
n_w1 --> n_mail : chaqiruv
n_queue --> n_w2 : asinxron
n_w2 --> n_mail : chaqiruv
n_queue -[#c62828]-> n_dlq : asinxron
n_campaign --> n_queue : asinxron
@enduml

Xulosa

Navbat uchun olti qoida.

1) Sekin va cheklangan ish so'rov yo'lidan chiqib, navbat orqali boradi. 2) Worker'larni backlog va eng eski xabar yoshi bo'yicha, quyi limitgacha kengaytiring. 3) Ack ishdan keyin; uzun ishlarda lease'ni uzaytiring. 4) Eksponensial backoff va jitter bilan qayta urining. 5) Zaharli xabarlar kuzatiladigan dead-letter navbatga. 6) Navbatni cheklang va producer'larga sekinlashishga imkon bering.

Ko'rsatkichlar: boshida va oxirida

Ko'rsatkichBoshidaOxirida
Buyurtma p95950 ms35 ms
Backlog020 000
Xat kechikishi1 s60 s
Yo'qolgan xat0/soat0/soat

Qadamlar

  1. Holat Xat buyurtma so'rovi ichida yuboriladi

    Buyurtma to'langach, API pochta provayderini chaqiradi va xaridorga javob berishdan oldin uning javobini kutadi.

  2. Muammo Aksiya: soniyasiga 600 buyurtma

    Soat 10:00 da aksiya boshlanadi va buyurtmalar o'n besh barobar oshadi.

  3. Yechim Buyurtma va xat orasida navbat

    API endi navbatga kichik vazifa qo'yadi va xaridorga darhol javob beradi.

  4. Yechim Ko'proq worker, provayder shiftigacha

    Autoscaler backlog va eng eski xabar yoshini kuzatadi va ikkinchi worker'ni ishga tushiradi.

  5. Holat Worker xabarni ushlab o'ldi

    Worker 2 m_8812 xabarini oladi va yuborishga ulgurmay o'ladi.

  6. Muammo Sekin ish lease'dan uzoq davom etdi

    Ulgurji buyurtmaning cheki 212 sahifa, PDF 45 soniya yasaladi.

  7. Yechim Ishlayotganda lease'ni uzaytir

    Ish davom etayotganda worker har 20 soniyada lease'ni uzaytiradi.

  8. Muammo Provayder tushdi, worker'lar unga yopiriladi

    Pochta provayderi yigirma daqiqaga ishdan chiqadi.

  9. Yechim Jitter bilan eksponensial backoff

    Xatodan keyin worker xabarni har safar ikki barobar oshadigan kechikish bilan qaytaradi: 10 s, 20 s, 40 s, 80 s, shifti 15 daqiqa.

  10. Yechim Zaharli xabar dead-letter navbatga

    Bitta xabar o'chirib yuborilgan xat shabloniga ishora qiladi.

  11. Yechim Backpressure: producer sekinlashadi

    Marketing rassilka boshlaydi: 2 million xat birdaniga o'sha navbatga.

  12. Xulosa Xulosa

    Navbat uchun olti qoida.

Atamalar

Navbat (queue)
Producer va worker'lar orasidagi bufer: sekin yoki cheklangan ish so'rov yo'lidan chiqib navbatga tushadi va worker'lar uni o'z tezligida bajaradi. Navbat to'lqinni yutadi, ack ish tugagach beriladi.
Dead-letter navbat
Bir necha urinishdan keyin ham ishlanmaydigan zaharli xabar ko'chiriladigan alohida navbat. U asosiy navbatni to'smaydi va kuzatib turiladi.
Visibility timeout (lease)
Navbat xabarni worker'ga berganda uni o'chirmaydi, shu vaqtga boshqalardan yashiradi. Worker vaqt ichida ack bermasa, xabar yana ko'rinadi va boshqa worker'ga ketadi.