AmouAI Hub/Courses/Software Engineering for AI Engineers/Chapter 15
The Right Architecture for This Phase
There is no correct architecture, only an architecture correct for a phase. The design that gets a product to its first ten customers is the wrong one at ten thousand — and, less obviously, the design built for ten thousand is much worse at ten, because it spends the one resource an early product cannot replace, which is the speed of finding out you were wrong.
By the end of this chapter you can
- Identify which phase a system is actually in, from evidence rather than ambition
- Say what each phase optimises for, and what it deliberately gives up
- Distinguish the signals that mean “move” from the ones that only look like them
- Cost a change now against the same change later
- Leave a seam in place before you need it, cheaply
- Refuse both premature scaling and deferred migration, for the same reason
1Three phases, three correct answers
Each phase has a different scarce resource, and the architecture that is right is the one that spends the abundant resource to protect the scarce one.
| Phase | Scarce | Optimise for | Deliberately give up |
|---|---|---|---|
| Prototype 0–100 users, question unanswered | Time to find out you were wrong | Being thrown away cheaply | Durability, tests, operability, most correctness |
| First production Real users, real money, small team | Attention — the same people build and operate | Being understood at 3 am | Elasticity, per-component scaling, sophistication |
| Scale One dimension is genuinely binding | Whatever the binding constraint is | That one dimension, specifically | Simplicity, and some of the ability to change everything at once |
Two failures follow from getting the phase wrong, and they are mirror images.
Building for scale at phase one spends the scarce resource on the abundant problem. A team with forty users and a sharded multi-region deployment has bought elasticity it does not need with the weeks it needed to find out whether anyone wants the product. This is the more fashionable mistake and the more expensive one, because the cost is invisible: nothing breaks, the system just fails to learn anything fast enough.
Staying at phase one too long is the opposite and it is not better. A prototype serving real customers has no tests, no runbook and one person who understands it, and every month it stays that way the migration gets larger — until the change that was a refactor becomes a rewrite, which is a project nobody will approve.
What both have in common is a refusal to name the phase. The useful discipline is to say out loud which phase you are in, what that means you are giving up, and what signal would say you have moved — which makes the next architecture conversation a check against a written expectation rather than a fresh argument.
2Prototype: optimise for being thrown away
A prototype exists to answer a question. Every property that does not serve that question is a cost, including several that are virtues everywhere else.
The right prototype is one that can be deleted without regret, which means the goal is not to write bad code — it is to write little code, on top of things you did not build, in a shape that makes deletion obvious. Hard-coded values, no configuration, one file if one file works, and the fewest possible moving parts.
| Skip | Keep | Why this one |
|---|---|---|
| Tests for the happy path | A way to reset to a known state | You will re-run this fifty times; starting from scratch each time is the actual cost |
| Error handling on paths that have not come up | Errors that print loudly | A prototype that fails silently wastes the one thing it was meant to save |
| Configuration and environments | Version control | Free, and it is what lets you try the other idea without losing this one |
| Abstraction over the model provider | A note of what you tried and what it cost | The notes are the deliverable; the code is the by-product |
| Migrations, deploy pipelines, monitoring | An honest record of what you faked | Whoever builds the real thing needs to know which parts were never solved |
The one non-negotiable is the last row, and it is the difference between a prototype and a trap. A prototype that hard-codes a customer id, ignores multi-tenancy, and processes one invoice at a time is answering its question perfectly — and if that is not written down, the demo it produces looks like a working product to everyone who was not in the room.
The failure mode to design against is promotion by deadline. A prototype that looks finished, deploys cleanly and has a nice interface is one urgent customer away from being production, and the decision will never be made explicitly. The defences are cheap and structural: keep it visibly unfinished, state at the start that the code will be deleted, and give the real implementation its own budget before the demo rather than after.
3First production: optimise for being understood at 3am
The distinguishing feature of this phase is that the people who build the system are the people who get woken up by it. Every design choice should be read in that light.
This is where most software lives, and most of it lives here forever — which is worth saying out loud, because architecture writing is disproportionately about phase three and most teams will never need it. The scarce resource is attention: five engineers cannot hold an elaborate system in their heads while also shipping features, and anything that fragments their attention is expensive in a way that does not appear on any dashboard.
What that implies, concretely:
- One deployable, until something forces otherwise. Chapter 13 made this argument in full; the phase framing adds that the forcing function is organisational as often as it is technical.
- Boring, well-understood components. A conventional stack means the answer to an incident is a search rather than an investigation. Chapter 14’s innovation tokens are a phase-two budget.
- Enough operability to be woken up usefully. Structured logs with a request id, a health check that means something, a runbook for the three things that have gone wrong before, and one dashboard a tired person can read. Not a platform — a page.
- Deliberate module boundaries, enforced by the build. This is the cheapest thing on the list and the one that determines how expensive phase three is.
- Recovery over prevention. A fast, boring rollback is worth more than an elaborate scheme for not needing one, because it works for the failures you did not anticipate.
Notice what is absent: autoscaling, per-component scaling, multi-region, a service mesh, an event bus. Each is a real tool for a real problem, and none of those problems is the one a five-person team with 8,000 customers actually has.
4Scale: optimise for the one dimension that is binding
“We need to scale” is not a statement until somebody names the dimension. There are at least six, they have nothing in common, and the architectures that serve them conflict.
| Dimension | Sounds like | The move | What it costs |
|---|---|---|---|
| Traffic | “We can’t handle the load” | More instances, caching, read replicas | Money, and usually little else |
| Data volume | “Queries have got slow” | Indexes, partitioning, archival, retention | Chapter 8 and Chapter 11 — mostly cheap |
| Team size | “We keep blocking each other” | Module boundaries first, then services | Chapter 13’s whole bill |
| Feature surface | “Nobody understands the codebase” | Boundaries, deletion, documentation | Discipline. No architecture fixes this |
| Geography | “It’s slow in Australia” | CDN, then read replicas, then multi-region | Enormous at the far end. Try the first two properly |
| Reliability | “We need more nines” | Redundancy, graceful degradation, error budgets | Each nine costs roughly ten times the last |
The discipline is to identify the one that is binding and address that one. A team that is blocked on team size and responds with autoscaling has bought nothing; a team that is blocked on data volume and responds by splitting into services has made the problem worse, because the slow query is now behind a network hop.
It is also worth noticing which rows are cheap. Traffic and data volume — the two that sound most like “scale” — are usually the least architectural: another instance, an index, a retention policy. The expensive rows are team size and feature surface, which are organisational, and geography and reliability, which are physical. Most teams that say they need to scale are describing one of the cheap rows and reaching for a remedy from the expensive ones.
5The signals that say it is time — and the ones that do not
A phase change should be triggered by something recurring and measurable, not by a feeling that the system has got big.
| Real signal | Looks like a signal, is not |
|---|---|
| Two teams have blocked each other on releases three times this quarter | “We have two teams now” |
| The p95 of the main endpoint has doubled and the cause is a resource we cannot add more of | “Traffic is growing” |
| A single component’s failures caused three of the last five incidents | “It’s a single point of failure” |
| The deploy pipeline takes 50 minutes and people batch changes to avoid it | “The codebase is large” |
| Onboarding an engineer to a feature takes six weeks, measured | “The code is hard to understand” |
| One workload’s resource profile starves another, observed in production | “These should really be separate services” |
The right-hand column is not wrong — each of those statements may be true. They are just not decidable: there is no threshold at which “traffic is growing” becomes an action, which is why the conversation recurs every quarter and resolves differently depending on who is in the room. The left-hand column has a number in it, and a number can be checked next month.
The practical move is to write the signal down when you make the phase decision, not when somebody proposes changing it. “We are staying with one deployable. We will revisit when two teams have blocked each other on releases three times in a quarter” converts every future version of this argument into a check against a written expectation — and it means the person who proposed the split has a way to be right later, which is what makes a refusal survivable.
6Build: the seam you leave in place before you need it
The cheapest architectural work is done at the moment the code is written and is nearly impossible afterwards. It is also invisible, which is why it does not get done.
A seam left in place is not a service, an abstraction layer or a plugin system. It is a small set of properties that cost almost nothing now and determine how expensive the next phase is:
| Leave this | Costs now | Saves later |
|---|---|---|
| A module boundary the build enforces | An afternoon of configuration | Extraction becomes a week rather than a quarter |
| No cross-module database access — go through the module’s interface | Some joins become two queries | The data can be separated at all, which is the thing that makes a split real |
| A request id threaded through everything | One parameter, one middleware | Distributed tracing is a config change instead of a project |
| An interface over anything you might replace | Slightly less expressive than the raw client | A one-way door becomes a two-way one (Chapter 14) |
| Idempotency keys on write endpoints | One column, one check | Retries become safe, which every later architecture assumes |
| A stated retention policy per table | A conversation | Data volume never becomes the binding dimension by accident |
Every row has the same shape: a small, boring cost now against a large, structural saving later. That is also why none of them happens by default — the cost is paid by the person writing the code today and the saving goes to whoever is here in two years, and nothing in a sprint review rewards that trade. Naming these as seams, in the same document where you name the phase, is what makes them survive a deadline.
One caution to keep this honest: a seam is not a plugin system. The failure mode of this advice is a team that builds an abstraction for every conceivable future variation and ends up with a framework nobody asked for, which is premature scaling wearing a different hat. The test is cost: if leaving the seam takes more than a day and makes the current code harder to read, it is not a seam, it is a speculative feature.
✓Checkpoint
▶Playground
The same product, at 8,000 customers with five engineers. Same three dimensions as the phase-one board above, and the weights have moved — which is the entire lesson.
✓Exercise set
Twelve problems: classifiers and cost models that turn phase judgements into something checkable, and three architecture decisions where the same product gets different right answers at different sizes. Your work is saved in this browser.
Chapter 16 — Decisions on the Record, and the Spec the Agent Reads
An undocumented decision is re-litigated forever — and your codebase now has a second reader who will helpfully undo anything nobody wrote down. Next chapter closes Part 3.