Automating a test before you have done it by hand
Building a machine for a procedure that was not yet understood.
Automating a test before running it by hand at least a few times tends to build a fast, precise machine for a procedure nobody yet fully understands, freezing whichever steps happened to be in the manual version, including the ones that were only ever habit, and losing the small judgement calls a hand would have made without even noticing it was making them.
Why automation freezes whatever the procedure happened to be
A machine does exactly what it is told, every time, without the small unconscious adjustments a person makes in the moment, and this is normally the entire point of automating something, consistency where a human would drift. The trouble is that automation can only freeze a procedure that already exists, and if the procedure was never actually worked out by hand first, what gets frozen is a guess dressed up as a fixed sequence of motor moves and timed pauses. A person running the same test manually a dozen times learns, almost without meaning to, that a certain step needs to happen a little slower on humid days, or that a reading taken too soon after a part is loaded reads differently than one taken a minute later once things have settled, and none of that learning happens if the very first version of the test is already a machine following a script nobody has had the chance to correct yet.
The bread-dough comparison
A baker who has kneaded bread by hand enough times learns to recognise a specific point, dough that stretches into a thin, translucent sheet without tearing, as the signal that kneading is actually finished, a judgement made by feel rather than by watching a clock. A stand mixer programmed with a fixed kneading time before anyone had learned to recognise that signal by hand would faithfully run for exactly that long every single time, regardless of whether that particular batch of flour, on that particular day's humidity, actually reached the same state a minute early or two minutes late. The machine is not at fault, it is doing precisely what it was built to do, but what it was built to do was never actually verified against the thing that matters, dough that is properly kneaded, only against a proxy for it, a fixed amount of time, chosen before anyone had learned by hand which amount of time was even the right one to fix.
Why doing it by hand first is what teaches which steps matter
Running a procedure manually is slow and inconsistent compared with a machine, which is exactly why it is worth doing anyway before automating: the slowness is what gives a person time to notice which steps genuinely change the outcome and which ones are just leftover habit carried in from an earlier, cruder version of the process. A hand-run test also surfaces failure modes a script never will, since a person naturally reacts to something looking or feeling wrong in a way a machine following fixed instructions cannot, stopping, checking, adjusting, in exactly the moments that later turn out to matter most. Only once those moments have been noticed enough times to describe clearly can they be written into a script with any confidence that the script is automating the actual procedure, rather than automating a plausible-looking approximation of it. This is also the reason a manual phase should not be rushed through with the automated version already half-built in the background, since the temptation to treat the first hand-run attempts as a formality rather than as the real source of information tends to produce exactly the same blind spots automating on day one would have, only slightly delayed.
The one number worth remembering
A test procedure written down and automated after only one or two manual runs typically still contains several steps that later turn out to be unnecessary, carried over simply because they were part of the very first version anyone happened to try, and untangling which of those steps were load-bearing from which were habit after the fact, once a machine has already been built around all of them, usually costs several times the effort it would have taken to work that out by hand before any automation began.
Where automating early still makes sense
None of this argues against automation generally, and for a procedure that is already well understood, repetitive, and free of any real judgement calls, building the machine first loses very little and saves a great deal of tedious manual repetition. The distinction worth holding onto is between automating a known procedure, which is almost always worth doing early, and automating a procedure that is still being discovered, which usually is not, since the second case risks locking in decisions made before anyone actually knew which decisions mattered. A short, deliberately manual phase before automation is cheap insurance against exactly that risk, and it rarely costs more than the first handful of the many runs the eventual machine will go on to perform. The judgement call worth making explicitly, rather than leaving to whoever happens to be under schedule pressure at the time, is whether the procedure in front of a team is genuinely finished changing or is still being figured out, since the two situations look almost identical from the outside and only the second one has any real risk attached to building the machine too soon.
What this changes in practice
The practical habit worth adopting is treating the first several runs of any new test as explicitly manual and exploratory, resisting the pull toward building the machine immediately even when doing so looks like the obvious time-saving move, since a machine built too early does not actually save time once the cost of discovering and correcting its baked-in mistakes is counted honestly. Only once a procedure has been run by hand enough times to know, with real confidence, which steps genuinely drive the result does automating it stop being a bet on an unverified guess and start being what it was always meant to be, a way of doing a known-good procedure faster and more consistently than a hand ever could.