Why changing one thing at a time is a trap
Why the obvious method misses interactions and wastes runs.
Changing one variable at a time is a trap because it can only ever reveal how each factor behaves on its own, while a great many real systems respond to combinations of factors in ways that never show up when everything else is held fixed while a single thing is varied.
Sixty-four variations of a bracket design were more than there was any real hope of simulating one at a time before a deadline, and the way out turned out to be a piece of statistics dating back to the 1920s.
Two shower taps
Turning a shower's hot tap up while leaving the cold one alone shows, correctly, that more hot water makes the shower warmer. Turning the cold tap up on its own shows, just as correctly, that more cold water makes it cooler. Neither test, nor both done in sequence, reveals that the comfortable temperature sits at one particular setting of both taps together. Comfort is a joint property of the two taps, and a method that only ever moves one while the other sits fixed maps each tap's behaviour perfectly and misses the answer. Worse, from a badly chosen starting position it can report that neither tap matters much, if both happen to have only a weak effect there.
Engineering factors behave the same way whenever they interact, meaning the effect of changing one depends on where the other is set. The one-at-a-time method assumes every effect simply adds to the others, which is close enough to true for many systems, but it only ever explores a narrow cross of settings radiating out from one starting point. Any interaction living away from that cross stays invisible.
Seven runs out of sixty-four
Six factors, each tried at just two settings, make sixty-four combinations. The one-at-a-time approach reaches its apparent answer in seven runs, a baseline plus one change per factor, and those seven have visited about a tenth of the space. Add a seventh factor and the combinations double to 128 while the one-at-a-time study grows by a single run, so the share left unexplored keeps growing.
Running all sixty-four is rarely the fix either, since that number grows punishingly fast. The alternative, tracing back to agricultural field trials in the 1920s, is a designed experiment, which spreads a limited budget of runs across combinations of factors, so that interactions have a chance to show themselves without every combination being tested.
When one at a time is enough
Plenty of real systems are close to additive across the range that matters, particularly where the factors act through separate, physically unconnected mechanisms, and there a one-at-a-time study reaches a trustworthy answer for a fraction of the effort. The mistake lies in using the method without first asking whether the factors have any physical reason to interact. A study that has reasoned through why its factors should behave independently, and checked that reasoning against a few combined runs, earns far more trust than one that assumed independence because it was the easier experiment to run.