mufassall

Konspekt · Ishonchlilik

Rate limiting

13 daq · O'rta · mufassall.dtwin.uz/uz/l/rate-limiting/

← Darsni ochish

Muammo

Bitta shovqinli klient API'ni hamma uchun sekinlashtirishi mumkin. Har klientni adolatli cheklash: oynalar, token bucket, 429 javobi va hamma serverlar bo'lishadigan hisoblagich.

Nimani o'rganasiz

  • Fixed window, sliding window va token bucket, har biri qayerda sinadi
  • Yaxshi 429 javobi klientga nima aytadi
  • Kimni cheklash kerak: IP, API kalit yoki foydalanuvchi
  • Nega hisoblagich umumiy bo'lishi kerak va u yo'qolsa nima qilish kerak
Yakuniy sahna
30%20%30%30%30%

Diagramma kod sifatida

Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.

Mermaid: sxema

flowchart LR
  n_partner(["Hamkor ilovasi"])
  n_bot(["Skript"])
  n_api["API"]
  n_db[("Baza")]
  n_office(["Ofis (300 kishi)"])
  n_lb{"Load balancer"}
  n_api2["API 2"]
  n_api3["API 3"]
  n_redis{{"Redis"}}
  n_api -->|"chaqiruv"| n_db
  n_partner -->|"chaqiruv"| n_lb
  n_office -->|"chaqiruv"| n_lb
  n_bot -->|"chaqiruv"| n_lb
  n_lb -->|"chaqiruv"| n_api
  n_lb -->|"chaqiruv"| n_api2
  n_lb -->|"chaqiruv"| n_api3
  n_api2 -->|"chaqiruv"| n_db
  n_api3 -->|"chaqiruv"| n_db
  n_api --x|"yo'qoldi"| n_redis
  n_api2 --x|"yo'qoldi"| n_redis
  n_api3 --x|"yo'qoldi"| n_redis
  classDef down fill:#fde2e2,stroke:#c62828,color:#4a0d0d,stroke-dasharray:4 3
  class n_redis down

Mermaid: so'rovlar ketma-ketligi

sequenceDiagram
  participant n_bot as Skript
  participant n_api as API
  participant n_partner as Hamkor ilovasi
  participant n_office as Ofis (300 kishi)
  participant n_lb as Load balancer
  participant n_redis as Redis
  Note over n_bot,n_api: Bitta skript butun API'ni egallaydi
  n_bot->>n_api: GET /v1/products?page=8812 HTTP/1.1
  n_api-->>n_bot: HTTP/1.1 200 OK
  Note over n_bot,n_api: So'rovlarni soniya bo'yicha sanash
  n_bot->>n_api: GET /v1/products?page=8812 HTTP/1.1
  n_api-->>n_bot: HTTP/1.1 429 Too Many Requests
  Note over n_bot,n_api: Oyna chetida limit ikki barobar
  n_bot->>n_api: limit 10 per second, fixed window, ip=198.51.100.7
  Note over n_bot,n_api: Sliding window orqaga qaraydi
  n_bot->>n_api: sliding window, ip=198.51.100.7, t=12:00:01.250
  Note over n_partner,n_api: Halol to'lqin (burst) rad etiladi
  n_partner->>n_api: partner app opens: 25 requests in 300 ms
  Note over n_partner,n_api: Token bucket: to'lqinga ruxsat, o'rtacha qat'iy
  n_partner->>n_api: token bucket ip=203.0.113.24: capacity=30 refill=10/s
  Note over n_office,n_api: Bitta IP ortida 300 kishi
  n_office->>n_api: GET /v1/orders HTTP/1.1
  n_api-->>n_office: HTTP/1.1 429 Too Many Requests
  Note over n_office,n_api: Manzil bo'yicha emas, kalit bo'yicha cheklash
  n_office->>n_api: GET /v1/orders HTTP/1.1
  n_api-->>n_office: HTTP/1.1 200 OK
  Note over n_bot,n_lb: Uch server, uchta alohida hisoblagich
  n_bot->>n_lb: local counters, key_id=k_88, limit 10/s
  Note over n_api,n_redis: Redis'da bitta hisoblagich
  n_api->>n_redis: EVALSHA 9f2c7a1e4b8d3f60a5c2e7b19d4f8a6c3e1b2d07 1 rl:k_88 30 10
  n_redis-->>n_api: 1) (integer) 0
  Note over n_api,n_redis: Limiter'ning o'zi ishdan chiqsa
  n_api-xn_redis: rate limiter: redis timeout after 20 ms

D2

direction: right

n_partner: "Hamkor ilovasi" {
  shape: rectangle
}
n_bot: "Skript" {
  shape: rectangle
}
n_api: "API" {
  shape: rectangle
}
n_db: "Baza" {
  shape: cylinder
}
n_office: "Ofis (300 kishi)" {
  shape: rectangle
}
n_lb: "Load balancer" {
  shape: diamond
}
n_api2: "API 2" {
  shape: rectangle
}
n_api3: "API 3" {
  shape: rectangle
}
n_redis: "Redis" {
  shape: hexagon
  style.fill: "#fde2e2"
  style.stroke: "#c62828"
  style.stroke-dash: 3
}

n_api -> n_db: "chaqiruv"
n_partner -> n_lb: "chaqiruv"
n_office -> n_lb: "chaqiruv"
n_bot -> n_lb: "chaqiruv"
n_lb -> n_api: "chaqiruv"
n_lb -> n_api2: "chaqiruv"
n_lb -> n_api3: "chaqiruv"
n_api2 -> n_db: "chaqiruv"
n_api3 -> n_db: "chaqiruv"
n_api -> n_redis: "yo'qoldi" {
  target-arrowhead.shape: cross
}
n_api2 -> n_redis: "yo'qoldi" {
  target-arrowhead.shape: cross
}
n_api3 -> n_redis: "yo'qoldi" {
  target-arrowhead.shape: cross
}

PlantUML: so'rovlar ketma-ketligi

@startuml
participant "Skript" as n_bot
participant "API" as n_api
participant "Hamkor ilovasi" as n_partner
participant "Ofis (300 kishi)" as n_office
participant "Load balancer" as n_lb
collections "Redis" as n_redis
note over n_bot, n_api : Bitta skript butun API'ni egallaydi
n_bot -> n_api : GET /v1/products?page=8812 HTTP/1.1
n_api --> n_bot : HTTP/1.1 200 OK
note over n_bot, n_api : So'rovlarni soniya bo'yicha sanash
n_bot -> n_api : GET /v1/products?page=8812 HTTP/1.1
n_api --> n_bot : HTTP/1.1 429 Too Many Requests
note over n_bot, n_api : Oyna chetida limit ikki barobar
n_bot -> n_api : limit 10 per second, fixed window, ip=198.51.100.7
note over n_bot, n_api : Sliding window orqaga qaraydi
n_bot -> n_api : sliding window, ip=198.51.100.7, t=12:00:01.250
note over n_partner, n_api : Halol to'lqin (burst) rad etiladi
n_partner -> n_api : partner app opens: 25 requests in 300 ms
note over n_partner, n_api : Token bucket: to'lqinga ruxsat, o'rtacha qat'iy
n_partner -> n_api : token bucket ip=203.0.113.24: capacity=30 refill=10/s
note over n_office, n_api : Bitta IP ortida 300 kishi
n_office -> n_api : GET /v1/orders HTTP/1.1
n_api --> n_office : HTTP/1.1 429 Too Many Requests
note over n_office, n_api : Manzil bo'yicha emas, kalit bo'yicha cheklash
n_office -> n_api : GET /v1/orders HTTP/1.1
n_api --> n_office : HTTP/1.1 200 OK
note over n_bot, n_lb : Uch server, uchta alohida hisoblagich
n_bot -> n_lb : local counters, key_id=k_88, limit 10/s
note over n_api, n_redis : Redis'da bitta hisoblagich
n_api -> n_redis : EVALSHA 9f2c7a1e4b8d3f60a5c2e7b19d4f8a6c3e1b2d07 1 rl:k_88 30 10
n_redis --> n_api : 1) (integer) 0
note over n_api, n_redis : Limiter'ning o'zi ishdan chiqsa
n_api ->x n_redis : [yo'qoldi] rate limiter: redis timeout after 20 ms
@enduml

PlantUML: komponentlar

@startuml
left to right direction
agent "Hamkor ilovasi" as n_partner
agent "Skript" as n_bot
component "API" as n_api
database "Baza" as n_db
agent "Ofis (300 kishi)" as n_office
node "Load balancer" as n_lb
component "API 2" as n_api2
component "API 3" as n_api3
stack "Redis" as n_redis #fde2e2;line:c62828;line.dashed
n_api --> n_db : chaqiruv
n_partner --> n_lb : chaqiruv
n_office --> n_lb : chaqiruv
n_bot --> n_lb : chaqiruv
n_lb --> n_api : chaqiruv
n_lb --> n_api2 : chaqiruv
n_lb --> n_api3 : chaqiruv
n_api2 --> n_db : chaqiruv
n_api3 --> n_db : chaqiruv
n_api -[#c62828,dashed]-> n_redis : yo'qoldi
n_api2 -[#c62828,dashed]-> n_redis : yo'qoldi
n_api3 -[#c62828,dashed]-> n_redis : yo'qoldi
@enduml

Xulosa

Rate limit qolgan hammani bitta klientdan himoya qiladi, «yo'q» deyish esa arzon bo'lishi shart.

1) Fixed window chetlarda teshik beradi; sliding window va token bucket bermaydi. 2) Token bucket halol to'lqinga ruxsat beradi va o'rtachani ushlaydi. 3) 429 ni Retry-After bilan qaytaring. 4) Kalit yoki foydalanuvchi bo'yicha cheklang, IP faqat anonim chaqiruvlarga. 5) Server ko'p bo'lsa, hisoblagichni bitta umumiy joyda saqlang va usiz qanday ishlashni oldindan hal qiling.

Ko'rsatkichlar: boshida va oxirida

Ko'rsatkichBoshidaOxirida
Hamkor p952 400 ms87 ms
Oddiy klientlar xatosi9%0%
Limitdan oshish90 x1,2 x

Qadamlar

  1. Muammo Bitta skript butun API'ni egallaydi

    Kimdir butun katalogni imkon qadar tez varaqlaydigan skript ishga tushirdi: soniyasiga 900 so'rov.

  2. Yechim So'rovlarni soniya bo'yicha sanash

    API endi har IP uchun joriy soniyaga hisoblagich yuritadi.

  3. Muammo Oyna chetida limit ikki barobar

    Skript ritmni o'rganadi: bir soniyaning eng oxirida 10 ta, keyingisining eng boshida yana 10 ta so'rov.

  4. Yechim Sliding window orqaga qaraydi

    Sliding window kalendar soniyasini emas, istalgan paytdagi oxirgi bir soniyani sanaydi.

  5. Muammo Halol to'lqin (burst) rad etiladi

    Hamkor ilovasi ochilganda bosh ekranni birdaniga 25 ta so'rov bilan yuklaydi, keyin bir daqiqa jim turadi.

  6. Yechim Token bucket: to'lqinga ruxsat, o'rtacha qat'iy

    Har klientga 30 tagacha token sig'adigan chelak (bucket) beriladi, u soniyasiga 10 tadan to'ladi.

  7. Muammo Bitta IP ortida 300 kishi

    Mijozning 300 kishilik ofisi internetga bitta manzil orqali chiqadi.

  8. Yechim Manzil bo'yicha emas, kalit bo'yicha cheklash

    Autentifikatsiyali chaqiruvlar endi har API kalit (yoki har foydalanuvchi) bo'yicha cheklanadi; IP limiti faqat login kabi anonim endpoint'larda qoladi va ancha yumshoq.

  9. Muammo Uch server, uchta alohida hisoblagich

    Trafik o'sdi va API load balancer ortida uchta serverda ishlaydi.

  10. Yechim Redis'da bitta hisoblagich

    Chelak Redis'ga ko'chadi va uni hamma API serverlar bo'lishadi.

  11. Holat Limiter'ning o'zi ishdan chiqsa

    Redis ishlamayapti.

  12. Xulosa Xulosa

    Rate limit qolgan hammani bitta klientdan himoya qiladi, «yo'q» deyish esa arzon bo'lishi shart.

Atamalar

Rate limiting
Bitta klient vaqt birligida qancha so'rov yuborishini cheklash, qolganlarni undan himoya qilish uchun. Limitdan oshganda Retry-After bilan 429 qaytariladi, hisoblagich esa barcha serverlar uchun umumiy bo'ladi.
Token bucket
Har klientda bir xil tezlikda to'ladigan, sig'imi cheklangan token chelagi bor, har so'rov bitta token oladi. Halol to'lqinga ruxsat beradi, o'rtacha tezlikni esa qat'iy ushlaydi.