Confounded factorial effects

Why does testing eight factors in eight runs hide some of the answers inside each other?

An engineer suspects eight things matter and has budget for eight runs. Changing one factor at a time would buy her almost nothing. Vary them all at once in a clever pattern, the promise goes, and eight runs will size all eight effects.

Something in that promise should feel too generous. Eight runs produce eight numbers. So the question worth asking before the trial rather than after is: what exactly comes out, and what did the clever pattern cost?

Start by counting what the runs can pay for. Eight runs give eight measured responses. Every estimate is some weighted combination of those responses, and there are only eight linearly independent combinations to be had — eight numbers cannot be stretched into nine. One of them is inevitably the grand mean. Seven quantities remain, and no arrangement of plus and minus signs changes that.

That already answers the question as posed, more bluntly than expected: eight factors do not fit in eight runs at all. Seven do, exactly, with nothing spare. So take the honest version — seven factors, eight runs — and ask what those seven contrasts actually contain.

Build it. Three factors A, B and C in eight runs is the full factorial: every combination present, and seven contrast columns available — A, B, C, AB, AC, BC, ABC. Each column is a pattern of signs across the eight runs, and they are mutually orthogonal. Now we want four more factors, and there is nowhere to put them except on columns that already have jobs. Write D on the AB column, E on AC, F on BC, G on ABC.

The trap is now visible, and it is arithmetic rather than statistical. The column you contrast to estimate D is the AB column. Not similar to it, not correlated with it — identical, sign for sign. So the number that emerges is not the effect of D; it is the effect of D plus the AB interaction, and no quantity of data collected under this design can prise them apart, because the data holds one column where the model has two terms.

Work the smallest case to see it rather than take it. Two factors in four runs, with C written on the AB column, gives the runs (A,B,C) = (−,−,+), (+,−,−), (−,+,−), (+,+,+). The contrast estimating A is (−y₁ + y₂ − y₃ + y₄)/2 — read the signs off A's column. Now build the BC column by multiplying B's signs (−,−,+,+) by C's (+,−,−,+): you get (−,+,−,+), which is A's column exactly. What you have labelled "A" is the estimate of A + BC.

There is compact bookkeeping for this. Since C was defined as AB, the product ABC is the all-plus identity column: write I = ABC. Multiply through by A, noting any column times itself is all-plus, and A = BC falls out. The generator you chose is the alias structure.

Carry that into the seven-factor design and each main effect drags three two-factor interactions with it. A is aliased with BD, CE and FG; D with AB, CG and EF. And that is forced: there are 7 × 6 / 2 = 21 two-factor interactions and seven columns, so at three per column all 21 are accounted for, with no spare column for any of them to hide in.

The answers were not hidden inside each other by accident, then. They were put there, by the choice of which column each factor was written on.

Three refinements hold the pieces together.

First, "confounded" here does not mean what it means in observational research. There, a confounder is a lurking variable you failed to control. Here nothing lurks: the aliasing is known before a single run, computable from the generators, and printed in the design's alias table. It is a deliberate purchase — runs saved, in exchange for terms that arrive fused.

Second, resolution grades that purchase, and it is nothing more mysterious than the length of the shortest defining word. Our design carries I = ABD among its relations, a word of length three, so it is resolution III: main effects aliased with two-factor interactions. Push the shortest word to length four and main effects come clear of two-factor interactions, though those still alias each other; length five and both are clear. Resolution is bought with runs and with nothing else.

Third, the assumption that makes a resolution III design usable at all: effect sparsity and hierarchy — few factors matter much, and interactions are typically smaller than the main effects they involve. These are empirical regularities of many industrial systems, not theorems. A system with strong interactions, chemistry near a phase boundary say, violates them and will mislead you through a design executed perfectly.

So how would you learn that you had been misled? Fold over: run the same eight points again with every sign reversed. Across the sixteen runs, main effects change sign under the reversal and two-factor interactions do not, so averaging the two blocks separates them. If your estimate of A barely shifts, the interactions it was carrying were negligible and the first reading stood. If it moves a long way, the first reading was mostly interaction. That is the refuting observation, and it costs eight more runs — which is exactly the price of the resolution you declined to buy at the start.

• Why sequential experimentation — a small screen, then a fold-over, then a response-surface design — beats one large design of the same total size.

Every term the collection defines is gathered in the glossary.