Glossary
The key terms of the course, each in a sentence or two, with the lesson that explains it.
A
- API gateway
One front door in front of all services: it checks the token, applies rate limits, adds a request id and routes by path. It hides the inside from clients.
In the lesson: Microservices →- At-least-once delivery
A message is delivered at least once, and sometimes twice. So consumers drop duplicates by event_id or write idempotently.
In the lesson: Sagas and the outbox →
B
- B-tree
A structure that keeps data in sorted pages and updates them in place. Reads and range scans are fast, but every small write rewrites a whole page at a random spot.
In the lesson: Storage engines: B-tree and LSM →- Backpressure
Slowing the producer down when consumers cannot keep up. The queue is bounded, and the producer waits instead of the queue growing without limit.
In the lesson: Message queues →- Base62 code
Give every link a unique number and encode it as a short string of letters and digits. Different numbers always give different codes, so there are no collisions as with hashing.
In the lesson: Design a URL shortener →- Batch vs stream
A batch job processes data in bulk on a schedule, such as nightly: simple and rerunnable, but late. A stream counts events in windows as they arrive: fresh, but harder.
In the lesson: Batch vs stream →- Blue-green deploy
Two full sets of servers: the new version goes to the idle set and the load balancer moves all traffic in one step. Rollback takes under a second; the cost is twice the servers.
In the lesson: CI/CD: from commit to production →
C
- Cache invalidation
Removing the old cached copy when the data changes. The key is deleted after the commit, so the next read fetches the fresh value.
In the lesson: Caching →- Cache stampede
When a hot key expires, thousands of requests miss at once and hit the database with the same query. Request coalescing and jittered TTLs prevent it.
In the lesson: Caching →- Cache-aside
The most common cache pattern: the app looks in the cache first, on a miss reads the database and puts the answer in the cache. Later requests hit and are served from memory.
In the lesson: Caching →- Canary deploy
The new version first gets a small share of traffic, such as 5%, and its error rate is compared with the old one. A bad release reaches few users, and a rule rolls it back by itself.
In the lesson: CI/CD: from commit to production →- CAP
When the network splits, a system must choose: stay consistent (CP) and refuse writes on the cut-off side, or stay available (AP) and accept writes on both sides. During the cut you cannot have both.
In the lesson: CAP: when the link breaks →- CDN and edge
A network of caches on edge servers near users that serves static files. The origin only fills misses, and Cache-Control decides what is cached and for how long.
In the lesson: CDN and edge caching →- CI/CD
CI builds and tests every commit once and produces one immutable artifact; only green builds deploy. Deploys become small, automatic and easy to undo.
In the lesson: CI/CD: from commit to production →- Circuit breaker
When the failure rate passes a threshold, the breaker opens and for a while does not call the dead service at all, returning a fallback. Then, half-open, one probe call decides whether it closes or opens again.
In the lesson: Timeouts and retries →- Consensus (Raft)
Several nodes agree on a leader and a shared log by majority vote. A change is committed once a majority stored it, and terms and votes allow at most one leader per term.
In the lesson: Consensus and leader election →- Consistent hashing
Nodes and keys sit on one ring, and a key belongs to the next node clockwise. When a node joins or leaves only about 1/N of the keys move; virtual nodes even out the load.
In the lesson: Consistent hashing →- Consumer group
Several instances of one subscriber split the topic's partitions, and each partition is read by exactly one member. It scales up to the partition count, and order holds only per key.
In the lesson: Pub/sub and event logs →- Cursor pagination
The next page is requested with a cursor pointing at the last item seen, not with OFFSET. New items no longer cause repeats or gaps.
In the lesson: API styles →
D
- Dead-letter queue
A separate queue that receives a poison message after it fails several attempts. It stops blocking the main queue and is watched by people.
In the lesson: Message queues →- Dirty read
Reading another transaction's uncommitted draft. If that transaction rolls back, the reader used data that never existed.
In the lesson: Transactions →
E
- Encryption at rest
Data is encrypted before it is written, so disks and backups hold only ciphertext. If the key sits next to the data, this protects nothing.
In the lesson: Encryption and secrets →- Envelope encryption (KMS)
Each record is encrypted with its own data key, and the data key is wrapped by a master key that never leaves the KMS. Only the wrapped key is stored, and rotation re-wraps keys instead of re-encrypting data.
In the lesson: Encryption and secrets →- Eventual consistency
If writes stop, all copies end up the same, but not at every instant. With asynchronous replication some reads are briefly stale.
In the lesson: CAP: when the link breaks →- Exponential backoff
The wait before each retry doubles. Instant retries no longer grow into a storm that buries a struggling service.
In the lesson: Timeouts and retries →
F
- Failover
When the primary database dies, a standby replica is promoted to primary and the app is pointed at it. Automatic failover does this without a human, in seconds.
In the lesson: Redundancy and failover →- Fan-out (on write / on read)
Fan-out on write pushes a new post's id into every follower's feed cache at posting time; fan-out on read builds the feed when it is opened. A hybrid works best: push for most, pull for huge accounts.
In the lesson: Design a news feed →- Feature flag
Code ships to production switched off and a flag turns it on for staff, then for a share of users. If it misbehaves, the flag goes off in a second: deploy is not release.
In the lesson: CI/CD: from commit to production →- Fencing token
An increasing number handed out with every new lock. The protected resource remembers the highest number it has seen and rejects writes from a worker that wakes up with an old lock.
In the lesson: Consensus and leader election →
G
- GraphQL
The client writes which fields it needs in the query and gets them in one call. The N+1 moves to the server, so resolvers are batched and query cost is limited.
In the lesson: API styles →- gRPC
A fast RPC protocol with a strict schema and a binary format. It fits internal calls between your own services.
In the lesson: API styles →
H
- Health check
The balancer probes each server regularly and takes one that stops answering out of the pool within seconds. It comes back once it recovers.
In the lesson: Load balancing →- Horizontal scaling
Adding more servers instead of one bigger one. It needs a load balancer in front and stateless app servers.
In the lesson: Scaling up vs scaling out →- Hot key
A single key with far more load than the rest, such as a huge channel or a product on sale. It overheats one shard or cache node, so it is split into buckets.
In the lesson: Sharding →
I
- Idempotency
Running an operation once or several times gives the same result. Messages can be delivered again, so every handler must be idempotent.
In the lesson: Reliable callbacks →- Idempotency key
A unique key the client sends with every create request. The server stores it and answers a repeat with the same response instead of creating a second order.
In the lesson: API styles →- Index
A separate sorted structure, usually a B-tree, that lets the database find rows without scanning the whole table. In a composite index the column order matters, and every index slows down every write.
In the lesson: Database indexes →- Isolation level
Sets how much concurrent transactions see of each other's changes: READ COMMITTED, REPEATABLE READ, SERIALIZABLE. A higher level closes more anomalies but causes more retries.
In the lesson: Transactions →
J
- Jitter
A random part added to a wait so that many clients do not retry at the same instant. It is also added to cache TTLs so keys do not expire together.
In the lesson: Timeouts and retries →- JWT
A signed token that any service can verify without a database lookup. It cannot be revoked instantly, so the access token is kept short-lived and paired with a refresh token.
In the lesson: Authentication →
L
- Latency
How long one request takes from start to finish. Under load, time spent waiting in line is added to the work itself.
In the lesson: Latency and throughput →- Load balancer
A component that spreads incoming traffic across several servers. L4 balances connections, L7 balances requests; it picks by round robin, least connections or weights.
In the lesson: Load balancing →- Load shedding
When a server is full, extra requests get an instant 503 instead of waiting. A fast "try again" for a few beats a slow failure for everyone.
In the lesson: Latency and throughput →- Long polling
The client sends a request and the server holds it until something new appears or a timeout passes. Cheaper than asking every two seconds, but each answer needs a new request.
In the lesson: Real-time updates →- LSM-tree
A store that appends each write to the WAL, sorts it in an in-memory memtable and later flushes it to disk as an SSTable file. Writes are very fast; the price is read amplification and compaction.
In the lesson: Storage engines: B-tree and LSM →
M
- Message queue
A buffer between a producer and workers: slow or limited work leaves the request path and goes into the queue, and workers process it at their own pace. The queue absorbs spikes; ack comes after the work.
In the lesson: Message queues →- Microservice
A small service with its own deploys and its own data; others reach it only through its API or events. The cost is network calls, partial failure and no shared transaction, so you start with a clean monolith.
In the lesson: Microservices →
N
- Napkin math
Sizing a system with a chain of simple multiplications: users, requests per second, bytes per day, then disks, bandwidth and servers. Write each assumption down and plan for the peak, not the average.
In the lesson: Napkin math →- Network partition
A broken link between parts of a system so they can no longer see each other. Both sides keep running but cannot agree.
In the lesson: CAP: when the link breaks →- NoSQL (document, wide-column, key-value)
Non-relational stores: a document store returns a whole object in one read, a wide-column store takes huge write rates, a key-value store looks up by key only. Each is shaped for a known access pattern, while a relational database stays open to new questions.
In the lesson: SQL and NoSQL →
O
- OAuth2 / OIDC
The user logs in at an identity provider and comes back with a one-time code, which the API exchanges server-to-server for an id token. The app never sees the password.
In the lesson: Authentication →- Observability (logs, metrics, traces)
Logs say what happened, metrics say how much and how often, traces say where the time went. Use RED metrics for services, USE for resources, and alert on p95, not on averages.
In the lesson: Observability →- Optimistic locking (version)
Instead of locking the row up front, the write checks that a version column has not changed. If someone else wrote first, the update changes nothing and the operation retries.
In the lesson: Transactions →- Origin shield
A middle cache layer between the edges and the origin. Instead of every edge asking on its own, many edge misses turn into one request to the origin.
In the lesson: CDN and edge caching →- Outbox
A service writes the data and the event into an outbox table in one local transaction, and a separate relay publishes the event to the broker. The event can be late, but it can no longer be lost.
In the lesson: Sagas and the outbox →
P
- Password hashing (salted)
Store a slow hash (argon2id or bcrypt) with a random salt per user, never the password itself. Login gets slower by an unnoticeable amount, while cracking a leaked table becomes millions of times harder.
In the lesson: Authentication →- Percentile (p50, p95, p99)
p95 means 95% of requests are faster than this value, so one in twenty is slower. The average hides the slow tail, so you watch p95 and p99.
In the lesson: Latency and throughput →- Presigned URL
A short-lived link signed by the API that lets the client upload a file straight to object storage. Bytes skip the API, which only checks who the user is and how big the file may be.
In the lesson: File uploads →- Pub/sub
A producer writes an event to a topic once, and each subscriber group reads it at its own pace through its own offset. A slow or dead subscriber only falls behind; it does not stop the producer.
In the lesson: Pub/sub and event logs →
Q
- Quorum (R + W > N)
A write counts once W of N replicas confirm it, and a read asks R replicas. If R + W > N, every read overlaps the latest write on at least one replica, and only the majority side can write.
In the lesson: CAP: when the link breaks →
R
- Rate limiting
Capping how many requests one client may send per unit of time, to protect everyone else. Over the limit the answer is 429 with Retry-After, and the counter is shared by all servers.
In the lesson: Rate limiting →- RBAC and ABAC
RBAC grants permissions by role, ABAC by rules about attributes of the user, the resource and the context. Both live in one policy in one place, denying by default.
In the lesson: Authorization →- Read-your-writes
A guarantee that users always see their own writes. After a write, that user's reads go to the leader until the replica has caught up.
In the lesson: Database replication →- Replication (leader and followers)
The leader takes every write and streams its log to followers, which replay it and keep a copy. Replicas give read capacity and a spare for failover.
In the lesson: Database replication →- Replication lag
How far an asynchronous replica is behind the leader. It grows under load, so a read from a replica can return stale data.
In the lesson: Database replication →- Request coalescing
On a miss, only one request takes a short lock, reads the database and refills the cache; the rest wait a few milliseconds and read the fresh key. Thousands of identical queries become one.
In the lesson: Caching →- REST
One URL per resource and standard HTTP methods. Simple and cache-friendly, but it can lead to over-fetching, under-fetching and N+1 calls.
In the lesson: API styles →
S
- Saga
Work across several services runs as a chain of local transactions, each with a compensating action. If a step fails, earlier steps are undone by their compensations; an orchestrator or events (choreography) drive the chain.
In the lesson: Sagas and the outbox →- Service discovery
Instances write their name and address to a registry on start and are removed when their heartbeat stops. Callers get addresses from the registry, not from config: themselves (client-side), via a load balancer (server-side) or via DNS.
In the lesson: Service discovery →- Shard key
The value that decides which shard holds a row. Pick the column your most common query filters on, and hash keys that grow over time or one shard runs hot.
In the lesson: Sharding →- Sharding
Splitting a table that no longer fits on one server across several servers (shards) by rows. The shard key decides where each row lives.
In the lesson: Sharding →- Single point of failure (SPOF)
A part whose loss stops the whole system. The fix is a spare that takes over without a human.
In the lesson: Redundancy and failover →- Snowflake ID (time + machine + sequence)
A 64-bit id made of milliseconds since a chosen epoch, a machine number and a counter within that millisecond. Each server makes ids alone, and waits or refuses if its clock moves back.
In the lesson: Unique IDs →- Split brain
Two nodes both believe they are the primary and both accept writes. The data drifts apart, so the old primary must be fenced off.
In the lesson: Redundancy and failover →- Stateless server
A server that keeps nothing in process memory between requests. Sessions and files live in a shared store, so any server can die or be added at any time.
In the lesson: Stateless servers →- Sticky session
The balancer always sends a user to the same server. It hides the problem rather than solving it: a restart loses the session and load becomes uneven.
In the lesson: Stateless servers →
T
- Throughput
How many requests a system finishes per second. It is linked to latency through the number of requests in flight.
In the lesson: Latency and throughput →- Timeout
How long to wait for an answer before giving up. Without one, a single slow dependency can tie up every thread and freeze everything; set it from real latency.
In the lesson: Timeouts and retries →- TLS
The protocol that encrypts a connection: in the handshake both sides agree on a shared key and the server proves its identity with a certificate. Certificates must renew themselves or they expire at midnight.
In the lesson: Encryption and secrets →- Token bucket
Each client has a bucket of limited size refilled at a steady rate, and each request takes one token. Honest bursts fit, while the average rate stays fixed.
In the lesson: Rate limiting →- Trace and span
Each service records a span for a request: what it did, when it started and how long it took. Spans share a trace id and are assembled into one tree, so the slow hop stands out.
In the lesson: Observability →- Transaction
Several related writes inside BEGIN ... COMMIT: either all are saved or none are. Even if the server dies halfway, no half-done change remains.
In the lesson: Transactions →- TTL (time to live)
How long a cached key lives before it expires by itself. A safety net so stale data cannot stay forever if invalidation is missed.
In the lesson: Caching →- Two-phase commit (2PC)
A coordinator first asks every participant to prepare and vote, and sends commit only if all vote yes. Participants hold locks the whole time and get stuck if the coordinator dies, so teams avoid it.
In the lesson: Sagas and the outbox →
U
- UUIDv7
A 128-bit id whose first 48 bits are the time in milliseconds and the rest is random. New ids land at the end of the index, so it does not tear a B-tree apart like random UUIDv4.
In the lesson: Unique IDs →
V
- Vertical scaling
Replacing a server with a bigger, stronger one. It is simple, but one day there is no bigger box, and it is still a single point of failure.
In the lesson: Scaling up vs scaling out →
W
- WAL (write-ahead log)
Every change is first appended to a log file that only grows at the end, then applied to the main structure. If the server crashes, data is recovered from this log.
In the lesson: Storage engines: B-tree and LSM →- Watermark
A marker that decides how long a stream processor waits for late events: the newest event time seen minus the allowed lateness. A window is final only once the watermark passes its end; later events go to a side output.
In the lesson: Batch vs stream →- WebSocket
One long-lived connection between client and server where both sides can send at any time. Many open connections are held by a separate gateway layer.
In the lesson: Real-time updates →- Write-through and write-back
Write-through writes the cache and the database in the same request: always fresh, but every write pays twice. Write-back writes only the cache and a worker flushes later: very fast, but unflushed data is lost if the cache dies.
In the lesson: Caching →