Sixty-four runs cut to sixteen, and what I gave up to do it

Choosing a reduced design, and the information deliberately sacrificed.

Cutting sixty-four runs down to sixteen means giving up the ability to separate certain effects cleanly from one another, a deliberate trade in which some interactions are allowed to become tangled together in exchange for running a quarter of the experiments a full factorial study would need.

Two circuits on one breaker

A full study of six factors, each tried at two settings, needs sixty-four runs to visit every combination, and from those runs every factor's effect and every interaction can be calculated as a separate number. A fractional design runs a quarter of those combinations, chosen by a fixed pattern so that the factors are still varied together in a structured way. With fewer runs than things to distinguish, some pairs of effects end up changing in exactly the same pattern across every run performed, and the data cannot say which of the two caused the result. Those effects are confounded, and no analysis afterwards can separate them, because the runs never varied them independently.

A house where two circuits were, by an old wiring mistake, connected to the same breaker shows the trap. Tripping the breaker labelled kitchen also kills a hallway light, and flipping breakers alone can never reveal which of the two the breaker really protects, because they have always moved together. Only tracing the wiring or rewiring the circuits apart would separate them. A study that never checks its confounding pattern can likewise go on quoting a number for one effect that has always been a blend of that effect and another.

Sixty-four effects squeezed into sixteen columns

Six two-level factors produce sixty-four things that could be estimated (the overall average, six main effects and fifty-seven interactions), but sixteen runs can only yield sixteen separate numbers, so each number is a blend of four effects. A well chosen reduction arranges those blends so that each main effect shares its number only with interactions among three or more factors, which are rarely large in physical systems. The loss is therefore concentrated in the effects least likely to matter, which is why the trade is usually worth making.

A badly chosen reduction can instead tangle a main effect with an interaction that matters a great deal, producing a result that looks clean while being unable to say which of two different things was responsible. The confounding pattern can be read from the way the runs were chosen, before the first one is carried out. Reduced designs are normally taken from published tables, including the orthogonal arrays behind Taguchi methods, so that the pattern is a known property of the design from the start.

My key error with this

The mistake was thinking that reducing the number of runs was something I had to justify from first principles, which meant I spent a genuinely embarrassing amount of time trying to derive the correlation structure between my variables myself, as though a reduced design were only legitimate if I personally understood every step of the mathematics underneath it. It cost me weeks of reading that never produced anything usable, and it delayed the actual study while the machines sat idle. What eventually replaced that belief was the realisation that this problem had been solved carefully by people who spend their careers on it, that the designs were published, tabulated and validated long before I arrived at them, and that my job was to find the right established design for the kind of variables I was working with and apply it correctly rather than to rebuild statistics from the ground up. Knowing where the established answer lives, and being willing to use it, turned out to be a more valuable skill than deriving it would have been.