Konspekt · Aloqa
Pub/sub va hodisalar jurnali
Muammo
Bitta hodisa, uni kutayotgan ko'p jamoa. Topic ularni bir-biridan qanday ajratadi va offset, consumer group, kalit va sxema nima beradi.
Nimani o'rganasiz
- Nega bitta sekin obunachi checkout'ni hech qachon buzmasligi kerak
- Offset: o'chib qolgan consumer qanday yetib oladi va qayta o'qiydi
- Consumer group va partition'lar: bitta obunachini kengaytirish
- Kalit bo'yicha tartib va hodisani o'quvchilarni buzmasdan o'zgartirish
Diagramma kod sifatida
Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.
Mermaid: sxema
flowchart LR
n_shop(["Do'kon ilovasi"])
n_orders["Buyurtmalar"]
n_email("Email")
n_stock("Ombor")
n_analytics("Analitika")
n_topic[["Topic"]]
n_stock_2("Ombor 2")
n_shop --> n_orders
n_orders --> n_topic
n_topic --> n_email
n_topic --> n_stock
n_topic --> n_analytics
n_topic --> n_stock_2Mermaid: so'rovlar ketma-ketligi
sequenceDiagram
participant n_orders as Buyurtmalar
participant n_email as Email
participant n_analytics as Analitika
participant n_topic as Topic
participant n_stock_2 as Ombor 2
Note over n_orders,n_email: Buyurtmalar servisi hammani o'zi chaqiradi
n_orders->>n_email: POST /send-receipt HTTP/1.1
n_email-->>n_orders: HTTP/1.1 200 OK
Note over n_orders,n_analytics: Analitika o'chdi, checkout ham o'chdi
n_orders-xn_analytics: POST /track HTTP/1.1
Note over n_orders,n_topic: Bitta hodisani topic'ga yozish
n_orders->>n_topic: PRODUCE topic=order-events
n_topic-->>n_orders: OK partition=2 offset=88412
Note over n_topic,n_analytics: O'lik obunachi faqat orqada qoladi
n_topic->>n_analytics: describe consumer group: analytics
n_analytics-->>n_topic: PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
Note over n_topic,n_analytics: Saqlangan offset'dan yetib olish
n_topic->>n_analytics: FETCH topic=order-events partition=2 offset=76112
n_analytics-->>n_topic: RECORDS 500 (offsets 76112..76611)
Note over n_topic,n_stock_2: Consumer group partition'larni bo'lishadi
n_topic->>n_stock_2: JOIN group=stock member=stock-2
n_stock_2-->>n_topic: REBALANCE group=stock generation=7
Note over n_orders,n_topic: Bekor qilish buyurtmadan oldin keldi
n_orders->>n_topic: PRODUCE topic=order-events
n_topic-->>n_orders: OK partition=5 offset=31907
Note over n_orders,n_topic: Bir xil kalit, bir xil partition
n_orders->>n_topic: PRODUCE topic=order-events
n_topic-->>n_orders: OK partition=2 offset=88950
Note over n_topic,n_analytics: Nomi o'zgargan maydon o'quvchini to'xtatadi
n_topic->>n_analytics: RECORD partition=2 offset=90233
n_analytics-->>n_topic: consumer stopped: field total is missing
Note over n_orders,n_topic: Hodisaga versiya, o'zgarish faqat qo'shish orqali
n_orders->>n_topic: PRODUCE topic=order-events
n_topic-->>n_orders: OK partition=0 offset=41022D2
direction: right
n_shop: "Do'kon ilovasi" {
shape: rectangle
}
n_orders: "Buyurtmalar" {
shape: rectangle
}
n_email: "Email" {
shape: rectangle
style.multiple: true
}
n_stock: "Ombor" {
shape: rectangle
style.multiple: true
}
n_analytics: "Analitika" {
shape: rectangle
style.multiple: true
}
n_topic: "Topic" {
shape: queue
}
n_stock_2: "Ombor 2" {
shape: rectangle
style.multiple: true
}
n_shop -> n_orders
n_orders -> n_topic
n_topic -> n_email
n_topic -> n_stock
n_topic -> n_analytics
n_topic -> n_stock_2PlantUML: so'rovlar ketma-ketligi
@startuml
participant "Buyurtmalar" as n_orders
participant "Email" as n_email
participant "Analitika" as n_analytics
queue "Topic" as n_topic
participant "Ombor 2" as n_stock_2
note over n_orders, n_email : Buyurtmalar servisi hammani o'zi chaqiradi
n_orders -> n_email : POST /send-receipt HTTP/1.1
n_email --> n_orders : HTTP/1.1 200 OK
note over n_orders, n_analytics : Analitika o'chdi, checkout ham o'chdi
n_orders ->x n_analytics : [yo'qoldi] POST /track HTTP/1.1
note over n_orders, n_topic : Bitta hodisani topic'ga yozish
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=2 offset=88412
note over n_topic, n_analytics : O'lik obunachi faqat orqada qoladi
n_topic -> n_analytics : describe consumer group: analytics
n_analytics --> n_topic : PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
note over n_topic, n_analytics : Saqlangan offset'dan yetib olish
n_topic -> n_analytics : FETCH topic=order-events partition=2 offset=76112
n_analytics --> n_topic : RECORDS 500 (offsets 76112..76611)
note over n_topic, n_stock_2 : Consumer group partition'larni bo'lishadi
n_topic -> n_stock_2 : JOIN group=stock member=stock-2
n_stock_2 --> n_topic : REBALANCE group=stock generation=7
note over n_orders, n_topic : Bekor qilish buyurtmadan oldin keldi
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=5 offset=31907
note over n_orders, n_topic : Bir xil kalit, bir xil partition
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=2 offset=88950
note over n_topic, n_analytics : Nomi o'zgargan maydon o'quvchini to'xtatadi
n_topic -> n_analytics : RECORD partition=2 offset=90233
n_analytics --> n_topic : consumer stopped: field total is missing
note over n_orders, n_topic : Hodisaga versiya, o'zgarish faqat qo'shish orqali
n_orders -> n_topic : PRODUCE topic=order-events
n_topic --> n_orders : OK partition=0 offset=41022
@endumlPlantUML: komponentlar
@startuml
left to right direction
agent "Do'kon ilovasi" as n_shop
component "Buyurtmalar" as n_orders
collections "Email" as n_email
collections "Ombor" as n_stock
collections "Analitika" as n_analytics
queue "Topic" as n_topic
collections "Ombor 2" as n_stock_2
n_shop --> n_orders
n_orders --> n_topic
n_topic --> n_email
n_topic --> n_stock
n_topic --> n_analytics
n_topic --> n_stock_2
@endumlXulosa
Producer faktni bir marta yozadi; har obunachi guruhi uni o'z offset'i orqali o'z tezligida o'qiydi.
1) Sekin obunachini xato emas, lag ko'rsatadi. 2) Offset ish tugagach commit qilinadi, kerak bo'lsa qayta o'qiladi. 3) Obunachi guruh orqali kengaytiriladi, partition sonigacha. 4) Tartib faqat kalit ichida bor. 5) Hodisa — shartnoma: versiyalang va faqat qo'shish orqali o'zgartiring.
Ko'rsatkichlar: boshida va oxirida
| Ko'rsatkich | Boshida | Oxirida |
|---|---|---|
| Checkout p95 | 900 ms | 40 ms |
| Yiqilgan buyurtma | 0% | 0% |
| Lag (hodisa) | 0 | 0 |
| Tartibsiz | 0/daq | 0/daq |
Qadamlar
- Muammo Buyurtmalar servisi hammani o'zi chaqiradi
Buyurtma berilganda buyurtmalar servisi email, ombor va analitikani birma-bir chaqiradi va faqat shundan keyin do'konga javob beradi.
- Muammo Analitika o'chdi, checkout ham o'chdi
Analitika servisi qotib qoldi.
- Yechim Bitta hodisani topic'ga yozish
Buyurtmalar endi bitta hodisa,
order.placed, ni topic'ga yozadi va do'konga javob beradi. - Holat O'lik obunachi faqat orqada qoladi
Analitika yana o'chdi.
- Yechim Saqlangan offset'dan yetib olish
Analitika qaytib keladi va oxirgi saqlangan offset'dan yozuvlarni so'raydi.
- Muammo Bitta obunachi ulgurmayapti
Aksiya buyurtmalarni uch barobar oshirdi.
- Yechim Consumer group partition'larni bo'lishadi
Topic'da 6 ta partition bor.
- Muammo Bekor qilish buyurtmadan oldin keldi
Mijoz buyurtma beradi va bir soniyadan keyin bekor qiladi.
- Yechim Bir xil kalit, bir xil partition
Producer endi yozuvning kalitini buyurtma ID'si qiladi.
- Muammo Nomi o'zgargan maydon o'quvchini to'xtatadi
Buyurtmalar jamoasi
totalniamountdeb qayta nomlab, deploy qiladi. - Yechim Hodisaga versiya, o'zgarish faqat qo'shish orqali
Hodisalar versiyali sxema oladi va yozilayotganda tekshiriladi.
- Xulosa Xulosa
Producer faktni bir marta yozadi; har obunachi guruhi uni o'z offset'i orqali o'z tezligida o'qiydi.
Atamalar
- Pub/sub
- Producer hodisani topic'ga bir marta yozadi, har obunachi guruhi uni o'z offset'i bilan o'z tezligida o'qiydi. Sekin yoki o'lik obunachi faqat orqada qoladi, producer'ni to'xtatmaydi.
- Consumer group
- Bitta obunachining bir nechta nusxasi topic partition'larini bo'lishib oladi, har partition'ni faqat bitta a'zo o'qiydi. Kengaytirish partition sonigacha ishlaydi, tartib esa faqat kalit ichida saqlanadi.