Konspekt · Ishonchlilik
Rate limiting
Muammo
Bitta shovqinli klient API'ni hamma uchun sekinlashtirishi mumkin. Har klientni adolatli cheklash: oynalar, token bucket, 429 javobi va hamma serverlar bo'lishadigan hisoblagich.
Nimani o'rganasiz
- Fixed window, sliding window va token bucket, har biri qayerda sinadi
- Yaxshi
429javobi klientga nima aytadi - Kimni cheklash kerak: IP, API kalit yoki foydalanuvchi
- Nega hisoblagich umumiy bo'lishi kerak va u yo'qolsa nima qilish kerak
Diagramma kod sifatida
Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.
Mermaid: sxema
flowchart LR
n_partner(["Hamkor ilovasi"])
n_bot(["Skript"])
n_api["API"]
n_db[("Baza")]
n_office(["Ofis (300 kishi)"])
n_lb{"Load balancer"}
n_api2["API 2"]
n_api3["API 3"]
n_redis{{"Redis"}}
n_api -->|"chaqiruv"| n_db
n_partner -->|"chaqiruv"| n_lb
n_office -->|"chaqiruv"| n_lb
n_bot -->|"chaqiruv"| n_lb
n_lb -->|"chaqiruv"| n_api
n_lb -->|"chaqiruv"| n_api2
n_lb -->|"chaqiruv"| n_api3
n_api2 -->|"chaqiruv"| n_db
n_api3 -->|"chaqiruv"| n_db
n_api --x|"yo'qoldi"| n_redis
n_api2 --x|"yo'qoldi"| n_redis
n_api3 --x|"yo'qoldi"| n_redis
classDef down fill:#fde2e2,stroke:#c62828,color:#4a0d0d,stroke-dasharray:4 3
class n_redis downMermaid: so'rovlar ketma-ketligi
sequenceDiagram
participant n_bot as Skript
participant n_api as API
participant n_partner as Hamkor ilovasi
participant n_office as Ofis (300 kishi)
participant n_lb as Load balancer
participant n_redis as Redis
Note over n_bot,n_api: Bitta skript butun API'ni egallaydi
n_bot->>n_api: GET /v1/products?page=8812 HTTP/1.1
n_api-->>n_bot: HTTP/1.1 200 OK
Note over n_bot,n_api: So'rovlarni soniya bo'yicha sanash
n_bot->>n_api: GET /v1/products?page=8812 HTTP/1.1
n_api-->>n_bot: HTTP/1.1 429 Too Many Requests
Note over n_bot,n_api: Oyna chetida limit ikki barobar
n_bot->>n_api: limit 10 per second, fixed window, ip=198.51.100.7
Note over n_bot,n_api: Sliding window orqaga qaraydi
n_bot->>n_api: sliding window, ip=198.51.100.7, t=12:00:01.250
Note over n_partner,n_api: Halol to'lqin (burst) rad etiladi
n_partner->>n_api: partner app opens: 25 requests in 300 ms
Note over n_partner,n_api: Token bucket: to'lqinga ruxsat, o'rtacha qat'iy
n_partner->>n_api: token bucket ip=203.0.113.24: capacity=30 refill=10/s
Note over n_office,n_api: Bitta IP ortida 300 kishi
n_office->>n_api: GET /v1/orders HTTP/1.1
n_api-->>n_office: HTTP/1.1 429 Too Many Requests
Note over n_office,n_api: Manzil bo'yicha emas, kalit bo'yicha cheklash
n_office->>n_api: GET /v1/orders HTTP/1.1
n_api-->>n_office: HTTP/1.1 200 OK
Note over n_bot,n_lb: Uch server, uchta alohida hisoblagich
n_bot->>n_lb: local counters, key_id=k_88, limit 10/s
Note over n_api,n_redis: Redis'da bitta hisoblagich
n_api->>n_redis: EVALSHA 9f2c7a1e4b8d3f60a5c2e7b19d4f8a6c3e1b2d07 1 rl:k_88 30 10
n_redis-->>n_api: 1) (integer) 0
Note over n_api,n_redis: Limiter'ning o'zi ishdan chiqsa
n_api-xn_redis: rate limiter: redis timeout after 20 msD2
direction: right
n_partner: "Hamkor ilovasi" {
shape: rectangle
}
n_bot: "Skript" {
shape: rectangle
}
n_api: "API" {
shape: rectangle
}
n_db: "Baza" {
shape: cylinder
}
n_office: "Ofis (300 kishi)" {
shape: rectangle
}
n_lb: "Load balancer" {
shape: diamond
}
n_api2: "API 2" {
shape: rectangle
}
n_api3: "API 3" {
shape: rectangle
}
n_redis: "Redis" {
shape: hexagon
style.fill: "#fde2e2"
style.stroke: "#c62828"
style.stroke-dash: 3
}
n_api -> n_db: "chaqiruv"
n_partner -> n_lb: "chaqiruv"
n_office -> n_lb: "chaqiruv"
n_bot -> n_lb: "chaqiruv"
n_lb -> n_api: "chaqiruv"
n_lb -> n_api2: "chaqiruv"
n_lb -> n_api3: "chaqiruv"
n_api2 -> n_db: "chaqiruv"
n_api3 -> n_db: "chaqiruv"
n_api -> n_redis: "yo'qoldi" {
target-arrowhead.shape: cross
}
n_api2 -> n_redis: "yo'qoldi" {
target-arrowhead.shape: cross
}
n_api3 -> n_redis: "yo'qoldi" {
target-arrowhead.shape: cross
}PlantUML: so'rovlar ketma-ketligi
@startuml
participant "Skript" as n_bot
participant "API" as n_api
participant "Hamkor ilovasi" as n_partner
participant "Ofis (300 kishi)" as n_office
participant "Load balancer" as n_lb
collections "Redis" as n_redis
note over n_bot, n_api : Bitta skript butun API'ni egallaydi
n_bot -> n_api : GET /v1/products?page=8812 HTTP/1.1
n_api --> n_bot : HTTP/1.1 200 OK
note over n_bot, n_api : So'rovlarni soniya bo'yicha sanash
n_bot -> n_api : GET /v1/products?page=8812 HTTP/1.1
n_api --> n_bot : HTTP/1.1 429 Too Many Requests
note over n_bot, n_api : Oyna chetida limit ikki barobar
n_bot -> n_api : limit 10 per second, fixed window, ip=198.51.100.7
note over n_bot, n_api : Sliding window orqaga qaraydi
n_bot -> n_api : sliding window, ip=198.51.100.7, t=12:00:01.250
note over n_partner, n_api : Halol to'lqin (burst) rad etiladi
n_partner -> n_api : partner app opens: 25 requests in 300 ms
note over n_partner, n_api : Token bucket: to'lqinga ruxsat, o'rtacha qat'iy
n_partner -> n_api : token bucket ip=203.0.113.24: capacity=30 refill=10/s
note over n_office, n_api : Bitta IP ortida 300 kishi
n_office -> n_api : GET /v1/orders HTTP/1.1
n_api --> n_office : HTTP/1.1 429 Too Many Requests
note over n_office, n_api : Manzil bo'yicha emas, kalit bo'yicha cheklash
n_office -> n_api : GET /v1/orders HTTP/1.1
n_api --> n_office : HTTP/1.1 200 OK
note over n_bot, n_lb : Uch server, uchta alohida hisoblagich
n_bot -> n_lb : local counters, key_id=k_88, limit 10/s
note over n_api, n_redis : Redis'da bitta hisoblagich
n_api -> n_redis : EVALSHA 9f2c7a1e4b8d3f60a5c2e7b19d4f8a6c3e1b2d07 1 rl:k_88 30 10
n_redis --> n_api : 1) (integer) 0
note over n_api, n_redis : Limiter'ning o'zi ishdan chiqsa
n_api ->x n_redis : [yo'qoldi] rate limiter: redis timeout after 20 ms
@endumlPlantUML: komponentlar
@startuml
left to right direction
agent "Hamkor ilovasi" as n_partner
agent "Skript" as n_bot
component "API" as n_api
database "Baza" as n_db
agent "Ofis (300 kishi)" as n_office
node "Load balancer" as n_lb
component "API 2" as n_api2
component "API 3" as n_api3
stack "Redis" as n_redis #fde2e2;line:c62828;line.dashed
n_api --> n_db : chaqiruv
n_partner --> n_lb : chaqiruv
n_office --> n_lb : chaqiruv
n_bot --> n_lb : chaqiruv
n_lb --> n_api : chaqiruv
n_lb --> n_api2 : chaqiruv
n_lb --> n_api3 : chaqiruv
n_api2 --> n_db : chaqiruv
n_api3 --> n_db : chaqiruv
n_api -[#c62828,dashed]-> n_redis : yo'qoldi
n_api2 -[#c62828,dashed]-> n_redis : yo'qoldi
n_api3 -[#c62828,dashed]-> n_redis : yo'qoldi
@endumlXulosa
Rate limit qolgan hammani bitta klientdan himoya qiladi, «yo'q» deyish esa arzon bo'lishi shart.
1) Fixed window chetlarda teshik beradi; sliding window va token bucket bermaydi. 2) Token bucket halol to'lqinga ruxsat beradi va o'rtachani ushlaydi. 3) 429 ni Retry-After bilan qaytaring. 4) Kalit yoki foydalanuvchi bo'yicha cheklang, IP faqat anonim chaqiruvlarga. 5) Server ko'p bo'lsa, hisoblagichni bitta umumiy joyda saqlang va usiz qanday ishlashni oldindan hal qiling.
Ko'rsatkichlar: boshida va oxirida
| Ko'rsatkich | Boshida | Oxirida |
|---|---|---|
| Hamkor p95 | 2 400 ms | 87 ms |
| Oddiy klientlar xatosi | 9% | 0% |
| Limitdan oshish | 90 x | 1,2 x |
Qadamlar
- Muammo Bitta skript butun API'ni egallaydi
Kimdir butun katalogni imkon qadar tez varaqlaydigan skript ishga tushirdi: soniyasiga 900 so'rov.
- Yechim So'rovlarni soniya bo'yicha sanash
API endi har IP uchun joriy soniyaga hisoblagich yuritadi.
- Muammo Oyna chetida limit ikki barobar
Skript ritmni o'rganadi: bir soniyaning eng oxirida 10 ta, keyingisining eng boshida yana 10 ta so'rov.
- Yechim Sliding window orqaga qaraydi
Sliding window kalendar soniyasini emas, istalgan paytdagi oxirgi bir soniyani sanaydi.
- Muammo Halol to'lqin (burst) rad etiladi
Hamkor ilovasi ochilganda bosh ekranni birdaniga 25 ta so'rov bilan yuklaydi, keyin bir daqiqa jim turadi.
- Yechim Token bucket: to'lqinga ruxsat, o'rtacha qat'iy
Har klientga 30 tagacha token sig'adigan chelak (bucket) beriladi, u soniyasiga 10 tadan to'ladi.
- Muammo Bitta IP ortida 300 kishi
Mijozning 300 kishilik ofisi internetga bitta manzil orqali chiqadi.
- Yechim Manzil bo'yicha emas, kalit bo'yicha cheklash
Autentifikatsiyali chaqiruvlar endi har API kalit (yoki har foydalanuvchi) bo'yicha cheklanadi; IP limiti faqat login kabi anonim endpoint'larda qoladi va ancha yumshoq.
- Muammo Uch server, uchta alohida hisoblagich
Trafik o'sdi va API load balancer ortida uchta serverda ishlaydi.
- Yechim Redis'da bitta hisoblagich
Chelak Redis'ga ko'chadi va uni hamma API serverlar bo'lishadi.
- Holat Limiter'ning o'zi ishdan chiqsa
Redis ishlamayapti.
- Xulosa Xulosa
Rate limit qolgan hammani bitta klientdan himoya qiladi, «yo'q» deyish esa arzon bo'lishi shart.
Atamalar
- Rate limiting
- Bitta klient vaqt birligida qancha so'rov yuborishini cheklash, qolganlarni undan himoya qilish uchun. Limitdan oshganda Retry-After bilan 429 qaytariladi, hisoblagich esa barcha serverlar uchun umumiy bo'ladi.
- Token bucket
- Har klientda bir xil tezlikda to'ladigan, sig'imi cheklangan token chelagi bor, har so'rov bitta token oladi. Halol to'lqinga ruxsat beradi, o'rtacha tezlikni esa qat'iy ushlaydi.