Ubiquity of the bell curve

Why do measurements of completely unrelated things pile up into the same bell-shaped curve?

Measure the heights of ten thousand adults, the errors of ten thousand repeated weighings, the totals of ten thousand rolls of five dice. Three subjects with nothing in common, and three histograms with the same silhouette: a single hump, symmetric, thinning fast on both sides.

Coincidence is not a good explanation for a coincidence that keeps happening. So what do the three actually share? Not their subject matter. Something about how each number was produced.

Look at what is common: in each case the recorded number is a total. A person's height is the accumulation of many genetic and developmental contributions. A weighing error is the accumulation of many small disturbances. A dice total is literally a sum. None of these is a single cause acting once.

So the question narrows usefully. What does adding independent things do to shape?

Do the smallest case by hand. One fair die is flat — each face 1 in 6, no hump anywhere. Add a second die and count: of 36 equally likely pairs, exactly one gives 2, but six give 7. Already a peak, purely from counting, because there are more routes to the middle than to the edges. Add a third: of 216 triples, 27 give 10 and 27 give 11, against a single triple for 3. In two additions a perfectly flat distribution has grown a hump you did not put there.

Notice what produced it. Not any property of dice — only the fact that extreme totals require every contribution to agree, while middling totals can be reached in many ways. Agreement of independent things is rare; that rarity is the thinning tail.

Now be careful, because this is where the explanation usually overreaches. "More ways to reach the middle" genuinely explains why a sum concentrates and why it becomes single-humped. It does not explain why the limiting shape is that particular curve rather than some other hump. The honest reason is different in kind: among distributions with finite variance, the normal is the unique shape that reproduces itself when you add two independent copies and rescale. Adding is a smoothing operation, and the normal is its fixed point — everything else is dragged toward it. Deriving that is a genuine piece of analysis, and I am not going to dress a hand-wave up as it.

What we can be precise about is the statement. Take independent, identically distributed measurements with a finite mean and a finite variance. Subtract the mean, divide the sum by the square root of the count, and the distribution of that quantity approaches the standard bell curve as the count grows. Every clause in that sentence is doing work.

The theorem is far narrower than its reputation, and the conditions are the interesting part.

Finite variance. Without it, nothing converges. The Cauchy distribution — the ratio of two independent normal variables, and a real physical shape, describing where a randomly aimed beam strikes a line — has no finite variance. The average of a thousand independent Cauchy values has exactly the same distribution as a single one. Not approximately, not slowly: no improvement whatever. This is the observation that would refute the claim, and it is available to anyone who simulates it: plot the histogram of averages of n draws, and if it stays as wide at n = 1000 as at n = 1, the finite-variance hypothesis has failed. My neighbouring walk-through of unstable averages follows exactly that failure.

Independence, and no dominant term. If the contributions are strongly correlated, or if one of them is comparable in size to the sum of all the rest, the limit does not hold. The identical-distribution requirement is the one that can be dropped most cheaply — Lindeberg's condition replaces it with a requirement that no individual term dominates.

Addition. The theorem is about sums. Quantities built by multiplying many independent factors — incomes, particle sizes, the spread of an epidemic — have normal logarithms, and so are lognormal: right-skewed, and not bell-shaped at all. City populations and wealth follow power laws, further still from a bell. That so much of the world is not normally distributed is not a failure of the theorem; it is the theorem being read correctly.

Two further limits worth holding. First, the theorem concerns the average or sum, not the individual measurement. Heights are roughly bell-shaped because a height is itself an accumulation, not because the theorem applies to any measured quantity. Second, convergence happens in the middle first. The Berry-Esseen bound gives an error shrinking like one over the square root of the count — fine for the central region, poor for the far tails at any practical sample size. Estimating a one-in-ten-thousand event from a normal approximation fitted to a few hundred observations is precisely the move that goes wrong in practice.

One boundary with a neighbour: the walk-through of error cancellation in averages asks how big the error of a mean is, and answers one over the square root of n. This one asks a different question about the same sum — what shape the deviation has. Size and shape are separate results, and the second is what lets you turn a standard error into a confidence interval.

• Why the Cauchy distribution's average never settles, and what stable distributions replace the normal limit.
• How the lognormal arises from multiplication, and why so many biological and economic quantities take it.
• The Berry-Esseen bound: how fast the approximation actually converges, and why the tails lag.

Every term the collection defines is gathered in the glossary.