The test pyramid

Why do the tests that most resemble real use catch the fewest problems worth catching?

Here is a proposition that seems unarguable: the closer a test comes to what a real user does, the more trustworthy it is. A test driving a real browser against a real server is, by construction, testing the thing you ship. A test calling one function with a made-up argument is testing something no user will ever do directly.

So why does almost every experienced team end up with thousands of the second kind and a couple of dozen of the first — not as a compromise they regret, but as the arrangement they would choose again? Something in that reasoning must be incomplete, and it is worth finding the missing piece rather than accepting the pyramid as folklore.

Start by separating two things the word "trustworthy" is doing at once. A test can be faithful — what it exercises resembles production. And a test can be decisive — when it goes red, you know what to do. Notice that these are not the same property, and ask whether they might trade against each other.

Consider a failing end-to-end test. It says: a user could not complete checkout. What broke? It might be the payment adapter, a database migration, a race between two services, a certificate that expired an hour ago, or the test's own assumption that a button would appear within two seconds. The test was faithful precisely because it covered all of that — and that same breadth is what makes it undecisive. It has told you that something in a large region is wrong, which is genuinely useful, but it has not located anything.

Now a failing unit test. It says: this function returns the wrong value for an empty list. That is nearly the whole diagnosis. It is decisive because it is narrow, and it is unfaithful for the same reason.

So the shape of the trade is: fidelity buys coverage of interactions, and pays for it in diagnostic resolution. Does that alone explain the pyramid? Not quite — there is a second cost, and it compounds.

Think about what has to be true for a broad test to be reliably green. Every component it touches must behave, and must also be available and fast enough at the moment the test runs. Chain enough of those together and the probability that the whole thing passes when the code is correct starts to slip. That slippage is flakiness, and it is corrosive rather than merely annoying: a suite that fails sometimes for no reason trains people to re-run it rather than read it. Once that habit forms, the broad tests stop catching anything at all — not because they cannot detect bugs, but because their verdicts are no longer believed.

A third pressure is plainer: time. A unit test runs in a millisecond; an end-to-end test runs in tens of seconds and often needs an environment stood up first. If the whole suite is to run before every change — the only way tests prevent defects rather than merely documenting them — the total runtime is a hard budget, and slow tests spend it fast.

Put the three together and the pyramid stops being a rule and starts being an inevitability. Many cheap, fast, decisive tests at the base; fewer broad, slow, faithful ones at the top; and at the very top a handful of end-to-end journeys that exist to answer one question the lower tests genuinely cannot — is this system wired together at all?

One honesty note. The familiar proportions — something like seventy per cent unit, twenty per cent service, ten per cent end-to-end — come from Mike Cohn's Succeeding with Agile (2009), where the shape was the point and the numbers were illustrative. They were never a measured optimum, and they are widely disputed. Kent C. Dodds' "testing trophy" argues the middle layer should be the fattest for typical web applications, and the argument is a reasonable one. The shape is a claim about costs; the exact ratios are not settled.

Three refinements.

First, this is not an argument that end-to-end tests are bad. They answer a question nothing else answers — does the assembled system work when the pieces are the real pieces? — and a suite without one can be entirely green while the application fails to start. The claim is only that a few suffice, because the answer is nearly the same for all of them: the tenth checkout journey tells you little the first did not.

Second, the inverted pyramid — the "ice cream cone", heavy at the top — is not usually chosen; it accumulates, because a broad test is the easiest thing to write when you do not want to change the code. That is worth noticing: the pyramid is partly a claim about design. Code that is hard to unit-test is usually code whose parts cannot be separated.

Third, "unit" is a vaguer word than it sounds, and arguing about it wastes time. What matters is not whether a test exercises exactly one class but whether it is fast, deterministic, and points somewhere specific when it fails. Google's internal scheme drops the vocabulary entirely and classifies tests as small, medium, or large by what resources they may touch — which measures the thing that actually drives the trade.

• How contract testing tries to buy integration confidence without paying the end-to-end runtime.
• Whether mutation testing is a better measure of a suite's worth than line coverage.

Every term the collection defines is gathered in the glossary.