Reliability & Scale · Lesson 13

Capacity estimation (designing for N users)

Back-of-the-envelope calculation is the secret handshake of system design interviews — interviewers aren't testing your arithmetic, they're watching whether you can impose structure on uncertainty and arrive at an order-of-magnitude answer that actually guides architecture decisions.

Why back-of-the-envelope matters

Imagine a chef asked to cook for 10,000 people instead of 10. They don't need to calculate grams to the decimal place — they need to know whether the kitchen can fit the pots, whether the delivery truck can carry the ingredients, and whether the gas line can handle 50 burners running at once. Getting the order of magnitude right is everything; being off by a factor of 1.3 is irrelevant.

Capacity estimation works the same way. The goal is not a precise number — it's a number that's in the right power-of-ten bucket so that you know you need two app servers versus two hundred, and whether your storage bill is $50/month or $50,000/month. Powers of ten are your unit of measurement. An estimate within 3× of reality is an excellent estimate.

In interviews, the estimator who states their assumptions aloud and walks through a structured chain of reasoning signals that they can handle ambiguity without freezing — a critical skill for any engineer designing systems that don't exist yet.

The derivation chain: five steps from users to resources

Every capacity estimate follows the same logical spine. Each step takes the output of the previous one and produces a new quantity. Run through these in order and you can derive any resource requirement from scratch.

1. Step 1 — Total registered users → DAU (daily active users). Not every registered account is active every day. A typical consumer app sees 10–30% of registered users as daily actives. For a 10M-registered-user platform, that's 1–3M DAU. Choose a single number and state it. Example: 10M registered × 10% DAU rate = 1M DAU.
2. Step 2 — DAU → requests per user per day. Think about what a single user does during a typical session. Behaviors vary sharply by app type: a social feed user might make 30 read requests (scrolling, refreshing) and 2 write requests (posting, liking) per day. A URL-shortener user might make 5 redirect lookups and 0.1 URL creations per day (creating a URL every 10 days on average). Separate reads from writes — they scale differently.
3. Step 3 — Average QPS.
   avg_QPS = (DAU × requests_per_user_per_day) ÷ 86,400
   86,400 is the number of seconds in a day. This converts a daily total to a per-second rate. Round aggressively — this is your baseline.
4. Step 4 — Peak QPS.
   peak_QPS = avg_QPS × peak_multiplier
   Traffic is never evenly spread. Social apps see a 3× evening prime-time spike. Flash-sale platforms see 10× during sale windows. E-commerce during Black Friday: 5–8×. Use 3× as a default for social, 2× for internal tools, 10× only if you have evidence of flash-load events.
5. Step 5 — Resources from peak QPS. Now derive each resource category independently:
   • Servers: ceil(peak_QPS / per_server_capacity) — then add 20–50% headroom.
   • Storage: DAU × uploads_per_user_per_day × avg_object_size × retention_days
   • Bandwidth (egress): peak_QPS × avg_response_payload_size
   • Cache size: estimate hot data set; if too large for RAM, cache references/metadata, not raw objects.

The derivation chain — visualised

Numbers worth knowing — a reference table

Estimation requires ballpark figures for what each component can handle. These are rough orders of magnitude based on typical cloud hardware and widely cited benchmarks — actual numbers vary significantly with workload type, tuning, and hardware generation.

Fully worked example: photo-feed app at 1,000,000 DAU

Let's walk through every arithmetic step for a simplified photo-feed service (think Instagram Lite). State your assumptions first, then compute each quantity in turn.

Stated assumptions

• 1,000,000 DAU
• Each user: 20 feed reads/day, 1 photo upload every 5 days (= 0.2 uploads/day), 10 likes/day, 5 comments/day
• Average compressed photo size: 500 KB
• Feed response payload (one page of thumbnails): 2 KB
• Cache hit rate on reads: 80% (cache handles 80% of read QPS)
• Data retention: 5 years
• Peak multiplier: 3× (evening prime time for a social app)

🧮 Step-by-step math walkthrough

Let's break down the arithmetic step-by-step to understand where every number came from, including conversions and unit calculations:

Summary table

Resource derivation — visualised

Under the hood: why the math works

The derivation chain is not arbitrary — each step has a mechanical justification rooted in queueing theory and traffic engineering. Understanding the why lets you adapt the formula when your system doesn't fit the defaults.

86,400: the normalization constant

A day has 86,400 seconds (60 × 60 × 24). Dividing a daily total by 86,400 converts it to a per-second rate, which is what QPS (queries per second) means. The insight is that users distribute their activity over a 24-hour window, and the time window for averaging is exactly that: one day. This works because user behavior at the level of a whole platform tends to average out across time zones, even though any individual user is bursty.

The peak multiplier: accounting for traffic shape

Real traffic follows a diurnal pattern — it peaks in the evening local time of your dominant user base and troughs overnight. If you plot hourly traffic over a week, you get a wave with peaks roughly 3–4× the overnight minimum and 2–3× the daily average. The Pareto principle applies here in time: roughly 20% of the day (about 5 hours of prime time) can carry 80% of the total daily request volume.

For flash-sale events (Black Friday, concert ticket drops), the peak-to-average ratio can reach 10–50×. Designing only for average load in these cases means your system will fail the moment it matters most.

Read:write ratio as architectural signal

Once you have separate read QPS and write QPS, their ratio tells you where to invest in scaling:

• High read:write (100:1 or higher): most requests are reads. Scale with caching (Redis, CDN), read replicas, and denormalized read models. The write path is rarely the bottleneck.
• Moderate read:write (10:1 to 100:1): typical for most consumer apps. Cache aggressively, add a few read replicas, and your single primary should hold.
• Low read:write (below 5:1): write-heavy — uncommon, but signals you need sharding, async write queues, or a write-optimized store (Cassandra, DynamoDB with write partitioning). Question your assumptions first: many "low ratio" estimates are miscounts.

Server count: ceiling, not floor

The formula ceil(peak_QPS / per_server_capacity) gives you the bare minimum to survive peak load at 100% CPU utilization — which is not survivable in practice, because a single slow request or GC pause will cascade. Add 20–50% headroom: if the math says 2.5 servers → round to 3 → add 30% headroom → 4. Deploy in pairs across availability zones for redundancy.

Storage compounding

Storage is not a snapshot — it accumulates over time. Objects uploaded in Year 1 are still stored in Year 5 unless you have an explicit deletion or tiering policy. Always compute Year-1 storage, then multiply by retention years. Add 20% overhead for metadata (database rows, indexes, thumbnails) and 2–3× for replication copies (3× for geo-redundant durability).

Sanity-check table: does this estimate pass the smell test?

After running the numbers, ask yourself whether each result falls in a plausible range. The table below captures the most common failure modes.

How to debug & inspect it

Capacity estimation is verified in production by comparing estimates against live resource metrics. Use tools like Prometheus, Grafana, and system monitors (htop, iostat, netstat) to monitor resources under load.

When not to use this

Back-of-the-envelope sizing is for order-of-magnitude architecture, not for finance-grade capacity contracts.

• Do not treat BOTE as a provisioning ticket for production autoscaling. Load-test your real p99 and connection pools before buying hardware from an interview-style estimate.
• Do not use 3× peak for flash-sale or launch events that need 10–50×; under-peak is the classic Black Friday failure mode.
• Do not average away multi-modal traffic (batch import hour + steady API). Split write-heavy and read-heavy paths and size each.
• Do not over-index on storage TB when egress and connection concurrency will fail first — check bandwidth and Little’s Law concurrency too.
• Cost of cargo-cult estimation: fake precision (1,234.56 QPS), silent unit bugs (req/min vs req/s), and architectures justified by numbers that were never re-checked offline.

Sources & further reading

• Google SRE Book — Site Reliability Engineering (O'Reilly / Google)
• AWS Well-Architected Framework — Performance Efficiency Pillar (Amazon Web Services)
• Latency Numbers Every Programmer Should Know — Jonas Bonér
• Numbers Everyone Should Know — Jeff Dean (Google), LADIS 2009