Unstable averages
A Socratic walk-through of unstable averages — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why can the average income in a room change completely when one more person walks in?
Ten people are in a room, each earning 30,000 a year. The average is 30,000. An eleventh walks in earning 3,000,000. Now the total is 3,300,000 across eleven people, so the average is 300,000 — ten times what it was a second ago. Every person in the original room is now described as "below average" by a factor of ten.
The arithmetic is trivial. The discomfort is not: nothing changed except one entrant, and a summary statistic swung tenfold. Ask what the average was supposed to be doing, and whether it can be relied on to do it.
Reasoning it through
REASONING #Try the same trick with height. Ten people of 1.7 metres, and an eleventh walks in. The average barely moves, because no human is 100 times as tall as another. Income has no such ceiling. So the instability is not a property of averaging — it is a property of the quantity averaged.
Give that property a shape. Suppose the chance of exceeding a value x falls off like x to the power minus alpha — a power law, the standard model for a heavy tail. Two integrals settle everything, and both are small enough to check.
The mean is the integral of x times the density. With that tail the density behaves like x to the power minus alpha minus one, so the integrand behaves like x to the power minus alpha, whose integral to infinity converges only if alpha exceeds 1. The second moment carries another factor of x, so it converges only if alpha exceeds 2.
Read what those two thresholds mean.
If alpha exceeds 2, the variance is finite, and everything comfortable is true: the sample mean settles, and its error shrinks like one over the square root of the count.
If alpha lies between 1 and 2, a mean exists as a property of the distribution — but the variance is infinite. The sample average still converges to it, by the law of large numbers, but not at the square-root rate and not smoothly. It approaches in fits: long quiet stretches, then a jump when a new record arrives.
If alpha is 1 or below, there is no mean to converge to. The running average simply wanders; more data does not improve it, because there is nothing for it to improve toward.
Now the sharpest way to see it. For a light-tailed quantity, the largest observation becomes a vanishing fraction of the total as the sample grows — no single person's height moves the sum. For alpha below 1 the ratio of largest term to sum does not vanish: the biggest single observation stays a substantial share of the whole, however many you collect. The average is then largely a report on its own maximum, dressed up as a summary of everyone.
The analogy
THE ANALOGY #Think of estimating the typical weight of an object on a conveyor belt by weighing the whole belt and dividing by the count. If the objects are apples, this works, and works better the longer you watch. If the belt occasionally carries an anvil, the running estimate sits quietly at apple-weight for a long stretch, then leaps when an anvil passes, then sags back as more apples arrive — and it never settles, because a heavier anvil is always still to come.
anvils and apples are two visibly different populations, so you could sort them out by eye, whereas a genuine heavy tail is a single population with no natural break — there is no size at which items stop being "normal" and start being "anvils", which is exactly why you cannot fix the problem by excluding the ones that look wrong.
Clarifying the model
THE MODEL #Several things get run together here, and separating them is most of the value.
Heavy tails are not outliers. An outlier suggests contamination — a typing error, a wrongly included population — and the remedy is removal. A heavy-tailed draw is a legitimate member of the distribution, arriving exactly as often as the model says. Deleting it does not clean the data; it silently substitutes a different distribution, and yields an estimate that is stable and wrong.
The distribution's mean and your estimate of it are different objects. For alpha between 1 and 2 the mean genuinely exists and is a real number. What fails is your ability to learn it from a sample of realistic size. Saying "the average is meaningless" overshoots; saying "this sample average carries much less information than its precision suggests" is the accurate complaint.
This failure is not the same as bias. The neighbouring walk-through of error cancellation in averages shows averaging defeated by correlation and systematic bias, which no sample size cures. The failure here is different in kind: the draws may be perfectly independent and the estimator perfectly unbiased, and averaging still fails, because the variance it relies on does not exist. That same hypothesis is what the central limit theorem needs, which is why the bell-curve machinery gives no useful confidence interval here either.
What would refute the claim. Plot the running average as observations accumulate, on a logarithmic count axis. Light-tailed data traces a band that narrows steadily and stops moving. Heavy-tailed data shows step jumps that do not diminish as the count grows, each coinciding with a new record. If a quantity's running average settles and stays settled across several decades of sample size, it is not heavy-tailed in the sense that matters, whatever a few large values suggest.
What to do instead. The median is stable under exactly this stress — the median of that room stays 30,000 whichever eleventh person enters. But it answers a different question, and for anything that adds up, such as a total insurance liability, the mean is the quantity you actually need; the honest response is a wider interval, not a substituted statistic. Estimates of the tail exponent for wealth commonly land near 1.5, a figure I state from recall — squarely in the band where a mean exists and a sample struggles to find it.
A picture of it
THE PICTURE #How to readFollow the running sample average, not the data. It starts in Settling, quietly narrowing. Which way it leaves depends on the tail: with finite variance it reaches Converged and stops, which is the only exit from the diagram. With a heavy tail it is knocked into Jumped by a single record observation, and the return edge is the slow dilution as ordinary draws accumulate — the cycle between those two states is the fits-and-starts convergence of the middle regime. Wandering is the case with no mean at all, and note that it has no path to Converged: its only edge leads back into another jump.
What became clearer
WHAT CLEARED #An average is a promise that individual observations do not matter much, and that promise is underwritten by finite variance. Where the tail is heavy the promise is void: the sum is dominated by its largest few terms, the average becomes a report on the maximum, and more data buys far less than the square-root rule advertises — or, past a threshold, nothing at all. The room with the millionaire is not an anomaly to be excluded. It is the distribution behaving as designed, and the average behaving as it must.
Where to go next
ONWARD #- How tail exponents are estimated from data, and why the estimates are themselves unstable.
- Why insurance and catastrophe risk are priced with extreme-value methods rather than sample means.
- The generalised central limit theorem, and the stable distributions that replace the bell curve when variance is infinite.
Key terms
TERMS #| Term | What it means |
|---|---|
| Heavy tail | a distribution whose probability of large values falls off slowly, as a power rather than exponentially. |
| Tail exponent (alpha) | the power governing that decay; the mean exists only above 1, the variance only above 2. |
| Law of large numbers | the sample mean converges to the distribution's mean, provided that mean exists. |
| Trimmed mean | an average computed after discarding the most extreme observations; stable, but an estimate of a different quantity. |
Every term the collection defines is gathered in the glossary.