mufassall

Konspekt · Butun tizimlar

URL qisqartirgich loyihasi

13 daq · O'rta · mufassall.dtwin.uz/uz/l/design-url-shortener/

← Darsni ochish

Muammo

Uzun havola kiradi, qisqa kod chiqadi, har bosish esa redirect. Butun servisni qadamma-qadam quramiz: hisob-kitob, qisqa kod, redirect, kesh, replika, bosishlar statistikasi, alias va muddat.

Nimani o'rganasiz

  • O'qish ko'p bo'lgan servis hajmini bir necha raqamdan qanday hisoblash
  • Nega hisoblagich + base62 URL hash'idan yaxshiroq
  • 301 yoki 302: har birida nimadan voz kechasiz
  • Kesh, replika va navbat bazadan yukni qayerda oladi
Yakuniy sahna
45%35%15%30%20%40%20%

Diagramma kod sifatida

Darsdagi yakuniy sxemani matn ko'rinishida oling va Jira, GitLab, GitHub, Notion yoki hujjatga qo'ying: Mermaid u yerlarda o'zi chiziladi. D2 va PlantUML (Confluence, IntelliJ) uchun alohida variantlar bor.

Mermaid: sxema

flowchart LR
  n_user(["Brauzer"])
  n_api["Havola API"]
  n_db[("Havolalar bazasi (primary)")]
  n_cache{{"Havola keshi"}}
  n_replica[("O'qish replikasi")]
  n_queue[["Bosishlar navbati"]]
  n_worker("Bosishlar worker'i")
  n_stats[("Statistika bazasi")]
  n_user --> n_api
  n_api --> n_db
  n_user --> n_api
  n_api -.-> n_user
  n_api --> n_cache
  n_api --> n_replica
  n_db --> n_replica
  n_api --> n_queue
  n_queue --> n_worker
  n_worker --> n_stats
  n_worker --> n_db

Mermaid: so'rovlar ketma-ketligi

sequenceDiagram
  participant n_user as Brauzer
  participant n_api as Havola API
  participant n_db as Havolalar bazasi (primary)
  participant n_cache as Havola keshi
  participant n_replica as O'qish replikasi
  participant n_queue as Bosishlar navbati
  Note over n_user,n_api: Nima quramiz va qancha katta
  n_user->>n_api: POST /api/links HTTP/1.1
  n_api-->>n_user: HTTP/1.1 201 Created
  Note over n_api,n_db: URL hash'i to'qnashadi
  n_api->>n_db: INSERT INTO links (code, long_url)
  n_db-->>n_api: ERROR: duplicate key value
  Note over n_api,n_db: Hisoblagich + base62: to'qnashuv bo'lishi mumkin emas
  n_api->>n_db: UPDATE id_blocks SET next_id = next_id + 1000
  n_db-->>n_api: next_id = 125001000
  Note over n_user,n_api: Redirect: 301 emas, 302
  n_user->>n_api: GET /8sujL HTTP/1.1
  n_api-->>n_user: HTTP/1.1 302 Found
  Note over n_api,n_db: Har bosish bazadan o'qish
  n_api->>n_db: SELECT long_url, expires_at FROM links
  n_db-->>n_api: (1 row)
  Note over n_api,n_cache: Baza oldida kesh
  n_api->>n_cache: GET link:8sujL
  n_cache-->>n_api: #34;https://shop.example.com/sale/42#34;
  Note over n_db,n_replica: Kesh miss'lari read replikaga
  n_db->>n_replica: SELECT now() - pg_last_xact_replay_timestamp()
  n_replica-->>n_db: lag = 00:00:00.180
  Note over n_api,n_db: Bosishlarni havolalar jadvalida sanash
  n_api->>n_db: UPDATE links SET clicks = clicks + 1
  n_db-->>n_api: UPDATE 1
  Note over n_api,n_queue: Bosishlar navbatga, worker sanaydi
  n_api->>n_queue: PRODUCE clicks key=8sujL partition=7
  n_queue-->>n_api: ack offset=88341201
  Note over n_api,n_db: O'z alias'i: bitta nomlar maydoni, ikki manba
  n_api->>n_db: INSERT INTO links (code, long_url, expires_at)
  n_db-->>n_api: (0 rows)
  Note over n_api,n_cache: Muddati tugaydigan havolalar
  n_api->>n_cache: SET link:spring-sale https://shop.example.com/s EX 3600
  n_cache-->>n_api: OK

D2

direction: right

n_user: "Brauzer" {
  shape: rectangle
}
n_api: "Havola API" {
  shape: rectangle
}
n_db: "Havolalar bazasi (primary)" {
  shape: cylinder
}
n_cache: "Havola keshi" {
  shape: hexagon
}
n_replica: "O'qish replikasi" {
  shape: cylinder
}
n_queue: "Bosishlar navbati" {
  shape: queue
}
n_worker: "Bosishlar worker'i" {
  shape: rectangle
  style.multiple: true
}
n_stats: "Statistika bazasi" {
  shape: cylinder
}

n_user -> n_api
n_api -> n_db
n_user -> n_api
n_api -> n_user {
  style.stroke-dash: 3
}
n_api -> n_cache
n_api -> n_replica
n_db -> n_replica
n_api -> n_queue
n_queue -> n_worker
n_worker -> n_stats
n_worker -> n_db

PlantUML: so'rovlar ketma-ketligi

@startuml
participant "Brauzer" as n_user
participant "Havola API" as n_api
database "Havolalar bazasi (primary)" as n_db
collections "Havola keshi" as n_cache
database "O'qish replikasi" as n_replica
queue "Bosishlar navbati" as n_queue
note over n_user, n_api : Nima quramiz va qancha katta
n_user -> n_api : POST /api/links HTTP/1.1
n_api --> n_user : HTTP/1.1 201 Created
note over n_api, n_db : URL hash'i to'qnashadi
n_api -> n_db : INSERT INTO links (code, long_url)
n_db --> n_api : ERROR: duplicate key value
note over n_api, n_db : Hisoblagich + base62: to'qnashuv bo'lishi mumkin emas
n_api -> n_db : UPDATE id_blocks SET next_id = next_id + 1000
n_db --> n_api : next_id = 125001000
note over n_user, n_api : Redirect: 301 emas, 302
n_user -> n_api : GET /8sujL HTTP/1.1
n_api --> n_user : HTTP/1.1 302 Found
note over n_api, n_db : Har bosish bazadan o'qish
n_api -> n_db : SELECT long_url, expires_at FROM links
n_db --> n_api : (1 row)
note over n_api, n_cache : Baza oldida kesh
n_api -> n_cache : GET link:8sujL
n_cache --> n_api : "https://shop.example.com/sale/42"
note over n_db, n_replica : Kesh miss'lari read replikaga
n_db -> n_replica : SELECT now() - pg_last_xact_replay_timestamp()
n_replica --> n_db : lag = 00:00:00.180
note over n_api, n_db : Bosishlarni havolalar jadvalida sanash
n_api -> n_db : UPDATE links SET clicks = clicks + 1
n_db --> n_api : UPDATE 1
note over n_api, n_queue : Bosishlar navbatga, worker sanaydi
n_api -> n_queue : PRODUCE clicks key=8sujL partition=7
n_queue --> n_api : ack offset=88341201
note over n_api, n_db : O'z alias'i: bitta nomlar maydoni, ikki manba
n_api -> n_db : INSERT INTO links (code, long_url, expires_at)
n_db --> n_api : (0 rows)
note over n_api, n_cache : Muddati tugaydigan havolalar
n_api -> n_cache : SET link:spring-sale https://shop.example.com/s EX 3600
n_cache --> n_api : OK
@enduml

PlantUML: komponentlar

@startuml
left to right direction
agent "Brauzer" as n_user
component "Havola API" as n_api
database "Havolalar bazasi (primary)" as n_db
stack "Havola keshi" as n_cache
database "O'qish replikasi" as n_replica
queue "Bosishlar navbati" as n_queue
collections "Bosishlar worker'i" as n_worker
database "Statistika bazasi" as n_stats
n_user --> n_api
n_api --> n_db
n_user --> n_api
n_api ..> n_user
n_api --> n_cache
n_api --> n_replica
n_db --> n_replica
n_api --> n_queue
n_queue --> n_worker
n_worker --> n_stats
n_worker --> n_db
@enduml

Xulosa

Redirect yo'li bu mahsulotning o'zi: uni qisqa tuting va primary'ni undan chetda saqlang.

1) Avval hisoblang: 40 yozish/s, 4 000 o'qish/s, 3 TB. 2) Bloklab olingan hisoblagich + base62 takrorlarsiz noyob kod beradi. 3) 302 har bosishni ko'rinadigan qiladi. 4) Issiq havolalarga kesh, miss'larga replika, bosishlarni sanashga navbat. 5) Alias jadvalni bo'lishadi, lekin kod shaklini emas; muddat kesh TTL'ini cheklaydi.

Ko'rsatkichlar: boshida va oxirida

Ko'rsatkichBoshidaOxirida
Redirect p9930 ms12 ms
Primary'dan o'qish300/s20/s
Kod to'qnashuvi0/kun0/kun

Qadamlar

  1. Holat Nima quramiz va qancha katta

    Ikki chaqiruv: POST /api/links uzun URL'ni qisqa kodga aylantiradi, GET /<code> esa unga yo'naltiradi.

  2. Muammo URL hash'i to'qnashadi

    Birinchi g'oya: uzun URL'dan MD5 olamiz, base62 ga o'giramiz va dastlabki 7 belgini qoldiramiz.

  3. Yechim Hisoblagich + base62: to'qnashuv bo'lishi mumkin emas

    Har havolaga noyob raqam beramiz va uni base62 da yozamiz: 125000417 → 8sujL.

  4. Holat Redirect: 301 emas, 302

    Bosish GET /8sujL bo'lib keladi.

  5. Muammo Har bosish bazadan o'qish

    Trafik bahodagi 4 000 redirect/s ga yetdi.

  6. Yechim Baza oldida kesh

    API avval keshdan link:<code> ni so'raydi.

  7. Yechim Kesh miss'lari read replikaga

    Qolgan miss'lar primary'dan o'zgarishlarni oqim bilan oladigan read replikaga o'tadi.

  8. Muammo Bosishlarni havolalar jadvalida sanash

    Marketing bosishlar sonini so'raydi.

  9. Yechim Bosishlar navbatga, worker sanaydi

    Endi redirect faqat kichik hodisa (event) chiqaradi va javob qaytaradi.

  10. Holat O'z alias'i: bitta nomlar maydoni, ikki manba

    Foydalanuvchilar /spring-sale kabi o'qiladigan havola istaydi.

  11. Yechim Muddati tugaydigan havolalar

    Havolada expires_at bo'lishi mumkin.

  12. Xulosa Xulosa

    Redirect yo'li bu mahsulotning o'zi: uni qisqa tuting va primary'ni undan chetda saqlang.

Atamalar

Base62 kod
Har havolaga noyob raqam berib, uni harf va raqamlardan iborat qisqa kodga aylantirish. Har xil raqam doim har xil kod beradi, shuning uchun hash kabi to'qnashuv bo'lmaydi.