Availability and the nines

For this lesson, we'll turn nines into minutes, do the two pieces of math every design round needs (boxes in a chain, boxes side by side), and look at the assumption that quietly breaks that math. Then we'll pick a target on purpose and keep it when a zone disappears. Interviewers like this topic because "highly available" is the most hand-waved phrase in system design, and a candidate who says "99.95% for writes, here's the chain, here's what degrades" stands out in under a minute.

01Picture it first: nines are a downtime budget

A percentage feels abstract. Minutes don't. Here's the table I keep in my head, computed with a 365-day year (525,600 minutes), a 30-day month (43,200 minutes) and a 24-hour day.

The rule behind every row is one multiplication: downtime = (1 - availability) x period. For 99.99% over a year that's 0.0001 x 525,600 minutes = 52.56 minutes. The yearly column matches the availability table in Google's SRE book, which is the one to quote if anyone asks where your numbers come from.

Two things to notice. Each extra nine is ten times less downtime, and usually far more than ten times the effort. And at four nines the daily budget is 8.6 seconds: no human can respond that fast, so detection and repair have to be automated.

There are two ways to count:

• Time-based: minutes up divided by total minutes. Easy to explain, but crude for a service that's half broken.
• Request-based: successful requests divided by valid requests. Google's SRE book prefers this for global services, because they're almost never fully up or fully down. Its example: a service handling 2.5 million requests a day with a 99.99% daily target can fail up to 250 of them.

I default to request-based in interviews, because it handles partial outages naturally.

02Under the hood: serial and parallel math

Almost every availability question reduces to two rules.

Serial (hard dependencies) multiply. If a request needs the load balancer AND the app AND the database, all three must be up, so you multiply their availabilities. AWS's reliability pillar uses this exact example: a 99.99% service with hard dependencies on two other 99.99% services can reach about 99.97%.

Parallel (redundant copies) multiply the failures. If either of two replicas will do, the tier is down only when both are down at once. So you multiply the unavailabilities and subtract from one: availability = 1 - (1 - a)^n. Two independent 99.9% replicas: 1 - (0.001 x 0.001) = 99.9999%.

Here's the worked version, all on a 365-day year:

Notice three things.

• The chain is always below its weakest link. 99.84% is worse than every single box in it.
• Redundancy pays where the failures are. Doubling the app tier removed most of its contribution, and the database now owns most of the remaining downtime. Fix the next weakest link, not the one you already fixed.
• Small numbers compound. Ten hard dependencies at 99.99% each come to about 99.90%. That's how a microservice graph of individually "four nines" services ends up at three.

03Where the math lies: independence

That last row says 99.99%, and I'd still not promise it. The parallel formula assumes the replicas fail independently and that switching between them is instant. Neither is free.

Things that make replicas fail together:

• Shared infrastructure. Same rack, same power feed, same zone, same region. That's why cloud zones are built not to share power and are physically separated.
• Shared change. A bad config push or deploy goes to every replica within minutes. Redundant boxes running the same broken config are one box.
• Shared dependency. Both app servers call one auth service, one secrets store, one DNS provider. Your redundancy ends at their availability.
• The same bug. A poison request or a leap-second edge case crashes every replica that sees it.

And switching isn't instant. A database standby that takes two minutes to promote costs you two minutes every time the primary dies. At 99.99% (52.56 minutes a year) you can afford about 26 of those; at 99.999% (5.26 minutes), two. We cover realistic switch times in Failover.

04Pick a target on purpose

The tempting target is 100%. It's the wrong one, for two reasons from Google's SRE book. Past a point users can't tell the difference, because their own phone and network fail more often than your service. And each extra nine can cost far more than the last, in hardware, slower releases and engineers working on risk instead of features.

The book also shows how to price a nine. For a service earning $1M, moving from 99.9% to 99.99% recovers 0.09% of revenue, about $900. If that extra nine costs more than $900 to build and run, it isn't worth it. That's the shape of argument I want to hear.

So I set targets per user journey, not per system:

These are a starting proposal, not industry constants. AWS's reliability pillar has a similar table that pairs 99.95% with online commerce and 99.99% with video delivery. Once you've set a target, the gap between it and 100% is your error budget; how teams spend and alert on it lives in Reliability.

05Static stability: stay up without doing anything

The usual plan for losing a zone is reactive: "auto scaling will launch replacements in the other zones." Amazon's Builders' Library calls this out as the weaker pattern, because recovery now depends on the control plane (launching instances, fetching config, registering with discovery) at the exact moment things are already broken.

The alternative is static stability: run enough spare capacity, all the time, that losing a zone needs no action at all. Amazon describes running stateless fleets across three zones overprovisioned by 50%, so each zone normally runs at about 66% of the load it was tested for. When one zone drops, the load balancer's health checks steer traffic to the other two, which absorb it without launching anything.

The arithmetic is worth knowing: with three zones you need 1.5x your peak capacity; with two zones each must carry the whole load, so you need 2x. That's a strong argument for three zones over two.

Static stability pairs with graceful degradation: when a non-critical dependency fails, serve something smaller instead of an error. Recommendations go blank, the feed shows cached posts, search falls back to popular items. The checkout path stays up. That's how a request-based availability number stays high during a partial outage. See Fault tolerance and the fallback pattern for the techniques.

06How it shows up in design problems

Availability shows up as a non-functional requirement in almost every round. Name a number per path and show where it comes from.

• Design a payment system: writes favour correctness over uptime, so a payment can return "pending" and finish later rather than risk a double charge. This is where CAP and idempotent retries meet the nines.
• Design a notification system: the accept path must be very available, so you put a durable queue in front and let delivery lag behind during an outage.
• Design a key-value store: replicas across zones and quorum reads and writes; the availability question becomes "how many replicas can die before reads or writes stop?"
• Design a URL shortener: redirects are read-heavy and cacheable, so the read path can target four nines even if creating links targets three.

For region-level planning, backups and recovery objectives, go to Disaster recovery. For how health checks actually notice a dead node, see Heartbeats and the health checks pattern.

07How to say it in the interview

When I ask "how available does this need to be?", I'm listening for three things: a number per path instead of one number for everything, a dependency chain with the weak link named, and a story for what degrades instead of failing. Bonus points for mentioning correlated failure without being prompted.