Eventual consistency
A Socratic walk-through of eventual consistency — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why is a system that is briefly wrong everywhere sometimes better than one that is always right in one place?
There is a phrase that sounds like an excuse for sloppy engineering: the system is eventually consistent. Eventually? A bank ledger that is eventually right is a bank ledger you would not use. And yet some of the most heavily engineered systems in the world — shopping carts, timelines, DNS, the address books that hold the internet together — deliberately choose to be briefly wrong in many places rather than reliably right in one.
So the question worth sitting with is not "is eventual consistency acceptable?" It is: what exactly do you get in exchange for that window of wrongness, and why is it sometimes the better trade?
Reasoning it through
REASONING #Start with the single-copy version, since that is the thing eventual consistency is being compared against. One database, one authoritative value. Every reader sees the same answer because there is only one answer to see. Beautifully simple. Now ask two awkward questions of it.
First: how far away is that copy from the reader? Light in fibre travels roughly 200 kilometres per millisecond, so a Sydney reader querying a copy in Virginia pays about 160 milliseconds of round trip before the database has done any work. That is physics, not engineering. Second, and worse: when the network between reader and copy breaks, the reader gets nothing. Not a stale answer — no answer.
So put a second copy near the reader. Now you have introduced the only real problem in this whole subject: two copies can disagree. And here you must choose. You can make the copies agree before anyone is allowed to read — coordinate on every write, wait for a quorum to confirm, block if you cannot reach it. Or you can let each copy answer immediately from what it currently knows, and reconcile afterwards.
Notice that this is a genuine fork, not a matter of effort. This is the substance behind the CAP theorem, which Eric Brewer proposed in 2000 and Gilbert and Lynch proved in 2002: when a network partition occurs, a system must give up either consistency or availability. Its real content is narrower than the slogan suggests — it constrains behaviour during a partition, not in normal operation — but the fork it names is real. If the two copies cannot talk and you insist they never disagree, then at least one of them must refuse to answer.
Now the interesting part. What does "eventually" actually mean mechanically? Not "we hope it sorts itself out." A convergent system needs three concrete things. It needs to propagate every update to every replica — through gossip, a replication log, or a background repair pass often called anti-entropy. It needs a deterministic rule for settling disagreements, so two replicas handed the same set of conflicting updates reach the same answer independently: last-writer-wins on a timestamp is the crude version; version vectors, or conflict-free replicated data types that make merges mathematically commutative and associative, are the careful versions. And it needs the update stream to pause occasionally, because convergence is defined as "if no new updates arrive, all replicas eventually agree."
Does that last clause bother you? It should slightly. It means convergence is an equilibrium the system is always heading towards and, under continuous writes, never quite standing in. In practice the window is measured in milliseconds to seconds within a datacentre, and that measurement is the thing to demand from a vendor — ask for the observed replication lag, not the adjective.
One more refinement, because it is where the concept becomes usable rather than scary. "Eventually consistent" is a floor, not a ceiling. Real systems bolt targeted guarantees on top: read-your-writes, so you always see your own change even if others do not yet; monotonic reads, so you never see time run backwards; and per-key quorums, where reading and writing majorities that overlap gives you a strong read on demand. So the choice is rarely all-or-nothing — it is usually per-operation. The cart may be eventually consistent while the checkout is not.
The analogy
THE ANALOGY #Think of a rumour spreading through a large office. If everyone had to gather in one room before any fact could be stated, nobody could be told anything while somebody was out at lunch — but nobody would ever hold a wrong version either. Instead each person tells the people near them, and for a few minutes different corridors hold different versions of the story. Given a pause in new gossip, everyone converges on the same account, and anyone can be answered instantly the whole time.
office rumours degrade as they pass and have no rule for settling contradictions, whereas replicas propagate the exact bytes and apply a deterministic merge rule — convergence in a real system is a designed property with a proof behind it, not a hope that the story survives retelling.
Clarifying the model
THE MODEL #Three things worth pinning down.
Eventual consistency does not mean unbounded wrongness, and it does not mean data loss. Every accepted write is durable somewhere; what is temporarily uncertain is which of several concurrent writes wins and how quickly a given reader learns the answer. A system that drops writes is not eventually consistent, it is broken.
It also does not follow that strong consistency is "safer" in every dimension. A strongly consistent system that must coordinate on every write has a wider blast radius: a partition or a slow quorum member converts directly into refused requests. Availability is a safety property too, if you are the customer who cannot check out.
And the trade is not really consistency versus speed — it is where you pay the coordination cost. Coordinate before the write, and the user waits. Coordinate after, and the application must be written to tolerate a stale read and a possible merge. The complexity does not vanish; it moves from the database into the application's semantics. That is the honest price, and it is the reason eventual consistency suits a shopping cart (where a merge rule is obvious — keep both items) far better than a seat reservation (where "keep both" is a double booking).
A picture of it
THE PICTURE #How to readFollow the numbered arrows down the page. Step 2 is the whole bargain — Dublin acknowledges without waiting for Sydney. Steps 3 and 4 are the cost: the Sydney reader is answered immediately, and briefly wrongly. Step 5 and the note are the repair, and by step 7 the replicas agree. Trace which arrows cross the ocean: none of them does so before a user is answered.
What became clearer
WHAT CLEARED #Eventual consistency is not a weaker promise made by people who could not manage a stronger one. It is a decision about when to pay for agreement. Paying up front buys a single truth at the cost of latency and of refusing service when the network splits; paying afterwards buys instant local answers and survival through partitions, at the cost of a measurable window in which replicas disagree and an application that must know how to merge. The window is bounded, the merge rule is designed, and the choice is usually made per operation rather than per system.
Where to go next
ONWARD #- How conflict-free replicated data types achieve merges that need no coordination at all, and what kinds of data they cannot express.
- What "consistency" means in ACID versus in CAP — two different properties that share a word and cause endless confusion.
Key terms
TERMS #| Term | What it means |
|---|---|
| Anti-entropy | a background process that compares replicas and repairs differences, so convergence does not depend on every real-time message arriving. |
| CAP theorem | the result that during a network partition a system must sacrifice either consistency or availability; it says nothing about the partition-free case. |
| Read-your-writes | a targeted guarantee that a client always sees at least its own most recent write, layered on top of eventual consistency. |
| CRDT | a data type whose merge operation is commutative, associative and idempotent, so replicas converge regardless of the order updates arrive in. |
Every term the collection defines is gathered in the glossary.