mufassall

Konspekt · Aloqa

Pub/sub va hodisalar jurnali

13 daq · O'rta · mufassall.dtwin.uz/uz/l/pubsub/

← Darsni ochish

Muammo

Bitta hodisa, uni kutayotgan ko'p jamoa. Topic ularni bir-biridan qanday ajratadi va offset, consumer group, kalit va sxema nima beradi.

Nimani o'rganasiz

  • Nega bitta sekin obunachi checkout'ni hech qachon buzmasligi kerak
  • Offset: o'chib qolgan consumer qanday yetib oladi va qayta o'qiydi
  • Consumer group va partition'lar: bitta obunachini kengaytirish
  • Kalit bo'yicha tartib va hodisani o'quvchilarni buzmasdan o'zgartirish
Yakuniy sahna
55%50%25%

Diagramma kod sifatida

Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.

Mermaid: sxema

flowchart LR
  n_shop(["Do'kon ilovasi"])
  n_orders["Buyurtmalar"]
  n_email("Email")
  n_stock("Ombor")
  n_analytics("Analitika")
  n_topic[["Topic"]]
  n_stock_2("Ombor 2")
  n_shop --> n_orders
  n_orders --> n_topic
  n_topic --> n_email
  n_topic --> n_stock
  n_topic --> n_analytics
  n_topic --> n_stock_2

Mermaid: so'rovlar ketma-ketligi

sequenceDiagram
  participant n_orders as Buyurtmalar
  participant n_email as Email
  participant n_analytics as Analitika
  participant n_topic as Topic
  participant n_stock_2 as Ombor 2
  Note over n_orders,n_email: Buyurtmalar servisi hammani o'zi chaqiradi
  n_orders->>n_email: POST /send-receipt HTTP/1.1
  n_email-->>n_orders: HTTP/1.1 200 OK
  Note over n_orders,n_analytics: Analitika o'chdi, checkout ham o'chdi
  n_orders-xn_analytics: POST /track HTTP/1.1
  Note over n_orders,n_topic: Bitta hodisani topic'ga yozish
  n_orders->>n_topic: PRODUCE topic=order-events
  n_topic-->>n_orders: OK partition=2 offset=88412
  Note over n_topic,n_analytics: O'lik obunachi faqat orqada qoladi
  n_topic->>n_analytics: describe consumer group: analytics
  n_analytics-->>n_topic: PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
  Note over n_topic,n_analytics: Saqlangan offset'dan yetib olish
  n_topic->>n_analytics: FETCH topic=order-events partition=2 offset=76112
  n_analytics-->>n_topic: RECORDS 500 (offsets 76112..76611)
  Note over n_topic,n_stock_2: Consumer group partition'larni bo'lishadi
  n_topic->>n_stock_2: JOIN group=stock member=stock-2
  n_stock_2-->>n_topic: REBALANCE group=stock generation=7
  Note over n_orders,n_topic: Bekor qilish buyurtmadan oldin keldi
  n_orders->>n_topic: PRODUCE topic=order-events
  n_topic-->>n_orders: OK partition=5 offset=31907
  Note over n_orders,n_topic: Bir xil kalit, bir xil partition
  n_orders->>n_topic: PRODUCE topic=order-events
  n_topic-->>n_orders: OK partition=2 offset=88950
  Note over n_topic,n_analytics: Nomi o'zgargan maydon o'quvchini to'xtatadi
  n_topic->>n_analytics: RECORD partition=2 offset=90233
  n_analytics-->>n_topic: consumer stopped: field total is missing
  Note over n_orders,n_topic: Hodisaga versiya, o'zgarish faqat qo'shish orqali
  n_orders->>n_topic: PRODUCE topic=order-events
  n_topic-->>n_orders: OK partition=0 offset=41022

D2

direction: right

n_shop: "Do'kon ilovasi" {
  shape: rectangle
}
n_orders: "Buyurtmalar" {
  shape: rectangle
}
n_email: "Email" {
  shape: rectangle
  style.multiple: true
}
n_stock: "Ombor" {
  shape: rectangle
  style.multiple: true
}
n_analytics: "Analitika" {
  shape: rectangle
  style.multiple: true
}
n_topic: "Topic" {
  shape: queue
}
n_stock_2: "Ombor 2" {
  shape: rectangle
  style.multiple: true
}

n_shop -> n_orders
n_orders -> n_topic
n_topic -> n_email
n_topic -> n_stock
n_topic -> n_analytics
n_topic -> n_stock_2

PlantUML: so'rovlar ketma-ketligi

@startuml
participant "Buyurtmalar" as n_orders
participant "Email" as n_email
participant "Analitika" as n_analytics
queue "Topic" as n_topic
participant "Ombor 2" as n_stock_2
note over n_orders, n_email : Buyurtmalar servisi hammani o'zi chaqiradi
n_orders -> n_email : POST /send-receipt HTTP/1.1
n_email --> n_orders : HTTP/1.1 200 OK
note over n_orders, n_analytics : Analitika o'chdi, checkout ham o'chdi
n_orders ->x n_analytics : [yo'qoldi] POST /track HTTP/1.1
note over n_orders, n_topic : Bitta hodisani topic'ga yozish
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=2 offset=88412
note over n_topic, n_analytics : O'lik obunachi faqat orqada qoladi
n_topic -> n_analytics : describe consumer group: analytics
n_analytics --> n_topic : PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
note over n_topic, n_analytics : Saqlangan offset'dan yetib olish
n_topic -> n_analytics : FETCH topic=order-events partition=2 offset=76112
n_analytics --> n_topic : RECORDS 500 (offsets 76112..76611)
note over n_topic, n_stock_2 : Consumer group partition'larni bo'lishadi
n_topic -> n_stock_2 : JOIN group=stock member=stock-2
n_stock_2 --> n_topic : REBALANCE group=stock generation=7
note over n_orders, n_topic : Bekor qilish buyurtmadan oldin keldi
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=5 offset=31907
note over n_orders, n_topic : Bir xil kalit, bir xil partition
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=2 offset=88950
note over n_topic, n_analytics : Nomi o'zgargan maydon o'quvchini to'xtatadi
n_topic -> n_analytics : RECORD partition=2 offset=90233
n_analytics --> n_topic : consumer stopped: field total is missing
note over n_orders, n_topic : Hodisaga versiya, o'zgarish faqat qo'shish orqali
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=0 offset=41022
@enduml

PlantUML: komponentlar

@startuml
left to right direction
agent "Do'kon ilovasi" as n_shop
component "Buyurtmalar" as n_orders
collections "Email" as n_email
collections "Ombor" as n_stock
collections "Analitika" as n_analytics
queue "Topic" as n_topic
collections "Ombor 2" as n_stock_2
n_shop --> n_orders
n_orders --> n_topic
n_topic --> n_email
n_topic --> n_stock
n_topic --> n_analytics
n_topic --> n_stock_2
@enduml

Xulosa

Producer faktni bir marta yozadi; har obunachi guruhi uni o'z offset'i orqali o'z tezligida o'qiydi.

1) Sekin obunachini xato emas, lag ko'rsatadi. 2) Offset ish tugagach commit qilinadi, kerak bo'lsa qayta o'qiladi. 3) Obunachi guruh orqali kengaytiriladi, partition sonigacha. 4) Tartib faqat kalit ichida bor. 5) Hodisa — shartnoma: versiyalang va faqat qo'shish orqali o'zgartiring.

Ko'rsatkichlar: boshida va oxirida

Ko'rsatkichBoshidaOxirida
Checkout p95900 ms40 ms
Yiqilgan buyurtma0%0%
Lag (hodisa)00
Tartibsiz0/daq0/daq

Qadamlar

  1. Muammo Buyurtmalar servisi hammani o'zi chaqiradi

    Buyurtma berilganda buyurtmalar servisi email, ombor va analitikani birma-bir chaqiradi va faqat shundan keyin do'konga javob beradi.

  2. Muammo Analitika o'chdi, checkout ham o'chdi

    Analitika servisi qotib qoldi.

  3. Yechim Bitta hodisani topic'ga yozish

    Buyurtmalar endi bitta hodisa, order.placed, ni topic'ga yozadi va do'konga javob beradi.

  4. Holat O'lik obunachi faqat orqada qoladi

    Analitika yana o'chdi.

  5. Yechim Saqlangan offset'dan yetib olish

    Analitika qaytib keladi va oxirgi saqlangan offset'dan yozuvlarni so'raydi.

  6. Muammo Bitta obunachi ulgurmayapti

    Aksiya buyurtmalarni uch barobar oshirdi.

  7. Yechim Consumer group partition'larni bo'lishadi

    Topic'da 6 ta partition bor.

  8. Muammo Bekor qilish buyurtmadan oldin keldi

    Mijoz buyurtma beradi va bir soniyadan keyin bekor qiladi.

  9. Yechim Bir xil kalit, bir xil partition

    Producer endi yozuvning kalitini buyurtma ID'si qiladi.

  10. Muammo Nomi o'zgargan maydon o'quvchini to'xtatadi

    Buyurtmalar jamoasi total ni amount deb qayta nomlab, deploy qiladi.

  11. Yechim Hodisaga versiya, o'zgarish faqat qo'shish orqali

    Hodisalar versiyali sxema oladi va yozilayotganda tekshiriladi.

  12. Xulosa Xulosa

    Producer faktni bir marta yozadi; har obunachi guruhi uni o'z offset'i orqali o'z tezligida o'qiydi.

Atamalar

Pub/sub
Producer hodisani topic'ga bir marta yozadi, har obunachi guruhi uni o'z offset'i bilan o'z tezligida o'qiydi. Sekin yoki o'lik obunachi faqat orqada qoladi, producer'ni to'xtatmaydi.
Consumer group
Bitta obunachining bir nechta nusxasi topic partition'larini bo'lishib oladi, har partition'ni faqat bitta a'zo o'qiydi. Kengaytirish partition sonigacha ishlaydi, tartib esa faqat kalit ichida saqlanadi.