Dropping incomplete records

Why can discarding the rows with blanks distort an answer more than filling the blanks in would?

A survey comes back with blanks. The instinct is almost moral: we will not make anything up, so we keep only the complete rows and analyse those. It feels like the conservative choice.

Yet the alternative sounds worse. Filling a blank means writing down a number nobody supplied. How can inventing values be safer than deleting rows we know nothing about? The answer turns on a question we have not asked: why is that cell empty?

Notice first that deletion is not neutral. Keeping only complete rows analyses a subsample whose membership was decided by whatever caused the blanks. So dropping is not "no assumption"; it is the assumption that the surviving rows resemble the whole. That is a claim about a mechanism, and the mechanism has three possible shapes.

Suppose a lab machine failed on random days. Whether a cell is blank has nothing to do with what would have been in it, nor with anything else in the row. Missing completely at random, MCAR. Here the complete cases really are a random subsample, so their mean is unbiased — you have lost precision and nothing else.

Now suppose older patients are more likely to skip a question, but among people of the same age, whether you answer has nothing to do with your answer. The blank depends on something you observed — age — not on the hidden value. Missing at random, MAR, a badly named term that means "random once you condition on what you have". Complete-case analysis now generally goes wrong, because the surviving rows are skewed young. Yet here is a nuance worth having: if you are fitting a regression of income on age, and missingness depends only on age, complete-case regression is still unbiased — you are already conditioning on the thing that drove the deletion. Deletion damages the marginal mean while leaving that conditional relationship intact.

Third, suppose the highest earners decline to state their income. The blank depends on the very value that is missing. Missing not at random, MNAR, and nothing in the observed data can repair it. Put numbers on it: take 100 respondents with a true mean income of 50, of whom the top 20 — averaging 90 — refuse. The complete-case mean is (100×50 − 20×90)/80 = 3200/80 = 40. A fifth of the answer has vanished, and no diagnostic run on the 80 remaining rows will show it, because the evidence left with the people.

So why can filling in beat deleting? Two reasons, and they are different. The first is arithmetic. Blanks scatter across columns, so completeness is a conjunction: with ten variables each 5% missing independently, the share of fully complete rows is 0.95¹⁰ ≈ 0.599. You have deleted 40% of your data to repair 5% of each column — and every deleted row still held nine good values. The second is bias: under MAR, a method that uses the observed part of an incomplete row — multiple imputation, or full-information maximum likelihood — recovers the relationship deletion destroys, because it conditions on exactly the observed variables that drove the missingness.

That last clause is the correction the question's framing needs. "Filling the blanks in" is not automatically better than dropping; done naively it is worse. Substituting each column's mean for its blanks shrinks that column's variance, drags correlations toward zero, and then reports a confidence interval computed as though the invented values were measurements. Single imputation lies about precision. Multiple imputation exists precisely to fix that: impute several times from a model that includes random draws, analyse each completed dataset, and combine by Rubin's rules so that the between-imputation variance is added back. The gain over deletion comes from that variance bookkeeping, not from the filling in.

Now the honest limits. MCAR is testable — compare observed variables between complete and incomplete rows, and systematic differences refute it. But MAR versus MNAR is not testable from the data you hold, and this is a hard boundary, not a gap in current technique: the two mechanisms make identical predictions about everything you can see and differ only about the values you cannot. The practical response is sensitivity analysis — redo the estimate assuming the missing values sit some amount above what the model predicts, and find the shift at which the conclusion flips. If it flips at an implausible shift, the finding survives; if it flips at a small one, the finding was never robust.

This also places the idea against its noisier neighbour. Survivorship bias — reading only the records that survived a filter tied to the outcome — is a special case: whole rows missing, and missing MNAR. The taxonomy here is the general frame naming when deletion is survivable and when it is fatal. Practice is genuinely contested, too. Multiple imputation is the default in medical statistics, but statisticians working with administrative and survey data argue that imputing from a misspecified model can be worse than a well-chosen complete-case analysis, particularly when the missingness sits in a predictor rather than the outcome.

• How inverse-probability weighting achieves the same correction as imputation by reweighting the complete cases instead.
• Why attrition in a randomised trial threatens the randomisation itself, and what an intention-to-treat analysis does about it.

Every term the collection defines is gathered in the glossary.