Choosing a Store, Including the Vector One

1Six shapes, thirty products

Product names change every eighteen months. The shapes have not changed in twenty years, and it is the shapes that determine whether your access pattern is cheap or expensive.

Every storage product is a set of decisions about how data is laid out on disk, what is indexed by default, what a single operation is allowed to touch, and what happens when a machine dies. Those decisions cluster, and the clusters are the shapes. Learn the six and every new product you meet is a variation on one of them, usually with a better developer experience and a worse operational story than the marketing suggests.

There is a specific failure this framing prevents. Asked “what database should we use?”, teams reach for the product they read about most recently and then bend their access patterns to fit it. The order that works is the reverse: write down the two or three queries that will run millions of times, identify which shape serves them naturally, and only then argue about products. Chapter 6 was the first half of this; this chapter is the second.

2What each shape is actually for

Six rows. The column that matters most is the last one, because it is the one people skip.

Two of these rows deserve expansion, because they are the ones teams get wrong in opposite directions.

Relational is under-rated by ambition and over-rated by inertia. The common mistake is leaving it too early, on the strength of a benchmark or a conference talk, for a workload a single well-indexed Postgres instance would have carried for another five years. The opposite mistake is real too — modelling append-only telemetry at a million events a minute in a general-purpose relational engine is genuinely the wrong shape, and no index rescues it.

Vector is over-rated by novelty and under-rated as a feature of something you already run. The question is almost never “vector database, yes or no”. It is “do I need a dedicated vector service, or does a vector index in the database I already operate cover this?” For a few million vectors with metadata filtering and transactional writes, an extension in your existing database is usually both simpler and better, because the embeddings stay next to the rows they describe and a single transaction can update both. Dedicated services earn their place at scales and query volumes most products never reach, and at the cost of a second consistency problem.

3The operability tax nobody costs

The spreadsheet in the proposal has hosting costs in it. The costs that actually decide the question are not in the spreadsheet.

Adding a store to a system adds, permanently: a backup that must be tested, a restore procedure someone must have practised, an upgrade path, a monitoring and alerting setup, an on-call runbook, a capacity model, a security review, a set of credentials to rotate, and at least one person who understands its failure modes well enough to be useful at 3 am. None of that appears on the pricing page, and all of it recurs every year.

A rough figure worth carrying into design reviews: a new production datastore costs a small team somewhere between a quarter and a full engineer-year in the first year, and a meaningful fraction of that every year after. You do not need this number to be precise. You need it to be present, because the alternative it is competing against — an index, a materialised view, or a cache on the store you already run — costs close to zero on that axis and is usually left out of the comparison entirely.

There is a second-order cost too, and it is the one that bites in year two. Every additional store is a place where your data can be, which means every future question — where does this field live, what is the source of truth, which copy is stale — has more possible answers. Teams describe this as “the system got complicated”, but it is specifically this: the number of places a fact can live went up, and nobody wrote down which one wins.

4Embeddings, and where they belong

An embedding is derived data. Treat it as anything else and you will eventually be unable to answer the only question that matters about it: what produced this, and can I make it again?

An embedding is a function of three things: some source text, a model, and the parameters you fed it (chunking, truncation, any preprocessing). Change any one of them and you get a different vector for the same source. That makes an embedding exactly like a cache entry, a thumbnail, or a materialised view: it is worth storing, and it must never be the only copy of anything.

The failure this prevents is specific and common. A team embeds a corpus, stores the vectors, and does not store which model version produced them, or the chunk boundaries, or the text of the chunk itself. Eighteen months later the model is deprecated. There is now no way to reproduce the index, no way to migrate it, and no way to verify that any particular vector still corresponds to the document it claims to. The corpus is still there; the index over it has become unmaintainable.

The content hash is the highest-value column on that list and the one most often missing. With it, a re-embedding job is a join: find every chunk whose source hash no longer matches the stored one, and recompute only those. Without it, your only options are to recompute everything — which for a large corpus is a real bill and a long job — or to guess from timestamps, which quietly misses edits that did not update a timestamp you controlled.

One more design rule, which follows from “derived data”: retrieval finds candidates; it does not supply facts. A vector search tells you which five invoices are probably relevant. The amounts in your answer come from a SELECT against those invoice ids, not from text embedded in a chunk. Systems that quote numbers straight out of retrieved chunks produce confident, well-cited, wrong figures, and they do it in exactly the situation where a wrong figure is worst: when the citation makes it look verified.

5Polyglot persistence: correct, or a symptom

Using several stores is sometimes exactly right. It is also the most common shape of a system nobody can reason about. The difference is whether each store owns something.

Polyglot persistence is correct when each store is the source of truth for a distinct set of facts and the boundaries are written down. Invoices live in Postgres. Uploaded PDFs live in object storage, with the metadata row in Postgres holding the key. Session tokens live in Redis and are allowed to be lost. Three stores, three clear owners, no fact stored in two places.

It is a symptom when the same fact lives in several stores and something has to keep them agreeing. Then you have not chosen a storage architecture; you have taken on a synchronisation problem, and synchronisation problems fail in the ugliest way available — silently, partially, and only under load.

The healthiest version of the rule: a second store is allowed to hold derived data, and is not allowed to hold the only copy of anything. A search index, a cache, a vector index and a pre-aggregated analytics table all pass that test — each can be dropped and rebuilt from the primary store, which means an outage in any of them degrades the product rather than losing data. The moment a secondary store holds the only copy of a fact, you have promoted it to a primary store and it needs the full backup, restore and correctness treatment of one.

This rule also gives you a clean answer to the question that usually stalls these discussions, which is who gets to add a store. Anyone may add a store that holds only derived data, because the blast radius of being wrong is a rebuild. Promoting a store to hold the only copy of a fact is an architectural decision that belongs in a written record with a named owner — which is exactly what Chapter 16 is about.

6Build: Ledger’s storage map

Six kinds of data in one product, and the map is more interesting than any single choice in it.

Ledger stores invoices and customers, uploaded PDF attachments, session tokens, webhook delivery records, an audit trail, and — for the new AI feature that drafts payment-chase emails — embeddings over invoice history. Here is the map, with the reasoning that matters:

Two things about this map are worth arguing over, and both are deliberate.

The first is that five of the six live in the same Postgres instance. That is not laziness — it is the operability tax applied consistently. Each additional store would have to beat “one more table in the database we already back up, monitor and know how to restore”, and for a product at this scale none of them does. The one exception, object storage, wins clearly because large binaries in a relational database make every backup, restore and page-cache decision worse for no benefit.

The second is the last row. Embeddings are the fashionable place to add a store, and here they are a table with a vector index. The two reasons are the ones from section four: the embedding stays beside the invoice it describes so it cannot silently drift, and the filter that every query needs — WHERE org_id = ?, because one organisation must never retrieve another’s invoices — is an ordinary indexed predicate rather than a metadata filter in a second system with its own semantics. That second point is a security property, not a performance one, which is why it outranks the benchmark.

It is worth saying what would change the answer, because a map with no stated exit condition is dogma rather than design. Three things would move the embeddings out: a corpus large enough that index build time becomes an operational event rather than a job, a query volume high enough that similarity search competes with transactional traffic for the same CPU, or a retrieval quality requirement that needs an index type the extension does not offer. None of those is a guess about the future — each is a number you can watch, which is what makes them useful.

✓Checkpoint

▶Playground

The selector again, locked to the shape of the AI feature: embeddings, nearest-neighbour reads. Change the other two answers and watch how much the consistency and scale requirements move the ranking — and where the recommendation stops being obvious.

✓Exercise set

Twelve problems: seven build the retrieval machinery by hand so that none of it stays magic, two are architecture decisions graded against stated constraints, and three ask you to choose and defend. Your work is saved in this browser.