What a list of engineering problems has in common with a revision timetable

Why a development matrix should rank open problems by how unsolved they are as well as by how critical they are to the system.

Both are ranked wrongly if they are ranked by importance alone, because time spent on a topic only pays back to the extent that the topic is still unresolved, and a subject already known cold returns almost nothing however many marks it carries. A development plan built around a list of critical systems makes the same mistake in a more expensive form, since the most critical parts of a machine are very often the parts the rest of the industry settled decades ago, and a plan that puts them at the top sends the scarcest engineering time toward work that a catalogue, a standard and an afternoon of reading would have finished anyway.

When I first drew up a matrix of the open problems on a machine, I ranked them the obvious way, by how badly the machine would fail if each one went wrong, and the ordering that fell out was almost useless as a guide to what to do on Monday morning. Sealing sat at the top and stayed there for weeks without ever being the thing that was actually holding the project up.

Why criticality answers the wrong question

A criticality ranking answers a genuine question, which is what happens to the whole system if this particular part does not work, and for anything touching safety or any failure that cannot be recovered in the field, that question has to be asked. The trouble is that it describes consequences rather than effort, and those two things come apart almost immediately in practice. Water ingress into a powered enclosure is about as critical as a failure gets, since it takes out the electronics and often the machine with them, yet the engineering content of preventing it amounts to choosing a rated enclosure, choosing a rated connector, sizing a groove for an O-ring from a table somebody else compiled, and confirming afterwards that the assembled result behaves the way the IP rating promised. It is critical and it is bought, and the hours it consumes bear almost no relationship to the damage it would do if it were skipped.

Set against that is a problem the field has genuinely not closed at the size, cost or power budget a given machine has to live within. Nobody can be sent to a supplier's catalogue for it, no table gives up the dimension, and the only honest estimate of how long it will take is a range wide enough to swallow the schedule. Such a problem might be a middling entry on any criticality list, since the machine would still limp along in some reduced form without it, and it will nonetheless decide when the machine is finished.

The exam-revision comparison

A student with a fortnight left and six topics to cover has exactly this problem in a smaller and more familiar form. The intuitive plan is to rank the topics by how many marks each is worth and work down the list, which feels responsible and is close to worthless, since the topic worth the most marks is frequently the one already understood well enough that another evening on it changes almost nothing. The plan that actually raises the final mark weights each topic by the marks available and by the share of those marks currently being dropped, so that a middling topic understood badly outranks a heavyweight topic understood well.

The arithmetic is worth doing once, because the result is more lopsided than intuition suggests. A topic carrying a fifth of the available marks but already scoring nine answers out of ten has at most two marks left in it however long it is studied, while a topic carrying a tenth of the marks and scoring nothing has ten marks sitting there untouched, five times the return from a topic worth half as much. The same multiplication governs a development matrix, where the value of spending a week somewhere is roughly the importance of the problem multiplied by the fraction of it still unsolved, and it explains why the ordering flips so readily between the two ways of sorting one list of problems.

Why a solved problem can be borrowed long before it is designed

The deeper reason a solved problem consumes so little of a schedule is that it can be faked convincingly while the rest of the machine is being tested. An existing answer, even an ugly one, can be bolted on in an afternoon: an off-the-shelf enclosure several sizes too large, a commercial gearbox far heavier than the final design will tolerate, a bought sensor standing in for the one that will eventually be integrated. None of that survives to production, and none of it needs to, because its whole purpose is to hold the solved part of the problem steady so that everything downstream of it can be exercised honestly. A team can therefore be testing the parts of the machine nobody understands yet while the sealing, the power supply and the gearbox are all represented by jerry-rigged stand-ins bought from three different suppliers.

Solved problems are also cheap to check, which matters more than it sounds. A settled problem arrives with a settled test: a defined procedure, a defined pass mark and often an outside house that will run it for a fee, so the result of the work is knowable in advance and the remaining uncertainty collapses into money and lead time, both of which can be put on a schedule with some confidence. The same is true of the expertise itself, since an established problem has established people, and somebody who has sealed a hundred enclosures can be brought in for a week and will compress months of fumbling into an afternoon of specific advice. The commercial version of this is renting a floor sander for a weekend rather than building one, and the reason nobody hesitates is that the tool already exists, is known to work and can be had immediately.

The test of whether a problem counts as solved is therefore whether a working answer can actually be obtained by this project, which is stricter than asking whether the answer exists somewhere. A technique demonstrated only at ten times the available budget, a part sold only in quantities far beyond what a small run needs, or a method that works reliably only on equipment nobody on the project can reach, all sit on the unsolved half of the axis in any practical sense, because getting to them costs the same original effort that inventing something would. Engineering knowledge accumulates as a stock of things that no longer have to be worked out from first principles, the argument Vincenti makes in What Engineers Know and How They Know It, and the useful question is which parts of that stock are genuinely within reach on this machine rather than which parts exist at all.

Why unsolved problems stay unsolved

Nothing sits open for long in a field with money in it unless there is a reason, and the reason is rarely that nobody has thought about it. A problem that remains open usually sits at the intersection of constraints that actively fight one another, where accuracy is available but not at that power budget, or the mechanism works but not at that size, or the whole thing is achievable and costs twenty times what the product can carry. Removing any one constraint would have solved it years ago, which is exactly why it is still open with all of them in place.

Such problems also come in layers, and the first layer is by far the cheapest. Making something work once on a bench, with a careful operator, good lighting and three attempts, is a different problem from making it work repeatably, which is different again from making it work repeatably at a price and a size the product can carry, which is different once more from making it work unattended in the field for a year with nobody watching. Each of those layers can cost as much as the one before it, and the visible demonstration is almost always the first and shallowest of them. A kitchen recipe behaves in exactly the same way, since a dish that comes out beautifully at home on a Sunday afternoon is nowhere near a dish that two hundred covers a night can be built on, and everything that makes the difference, consistency, speed, cost, the fact that somebody else has to cook it, lives in the layers underneath the version that already worked. Most serious underestimates of an unsolved problem come from having seen one of these demonstrations and quietly assuming it was the last layer rather than the first.

Why solving one is worth more than the machine it was solved for

The hours spent on a solved problem buy a result any competitor can also buy, on the same catalogue page, at roughly the same price. The hours spent on an unsolved one buy something nobody else currently has, and that asymmetry is the whole of a small company's defensibility argument. A competitor can match bought-in sealing within a week of deciding to, but a competitor cannot match a genuine solution to a problem the rest of the field is still stuck on without spending what it cost to get there in the first place. For an established company the same logic arrives as a lead measured in years rather than as survival, though it is the same mechanism underneath.

What makes this more than an argument about market position is that the knowledge outlives the product it was made for. A problem solved once has been solved for every subsequent machine, for adjacent applications nobody has looked at yet, and for the team itself, which now contains people who understand something few others do. Even a cancelled programme leaves that behind, and a programme cancelled after solving something genuinely hard can be a net gain, while a programme cancelled after a year of carefully specifying bought parts leaves almost nothing that could not be reassembled from purchase orders. This is also the sharpest argument against the ordering that criticality alone produces, since a plan that spends its first six months on critical bought-in items and its last six on the open questions has arranged for the transferable value to be the part that gets cut when the schedule slips.

None of that makes every open problem worth attacking. A problem can be unsolved simply because it never mattered enough for anyone to bother, and the corner of the matrix holding things that are neither critical nor solved is where a great deal of genuinely interesting engineering lives and where almost none of it belongs during a schedule that matters. The unsolved axis earns attention only where it crosses something the machine actually needs.

What this changes on Monday morning

Sorting the same list of open problems by how unsolved each one is, rather than by the damage each would cause, changes what gets scheduled first and who the work goes to. The solved-but-critical items become procurement and verification tasks with dates attached, safely handled by following an established method carefully, and deliberately faked with borrowed hardware in the meantime so they stop blocking anything. The unsolved items become the only genuine unknowns in the plan and therefore the only honest source of schedule risk, in the same way that a journey with one motorway leg and one unmapped track is decided almost entirely by the track. Whatever later turns out to have delayed the machine is almost always found on the unsolved half of the matrix, which is the practical argument for putting the earliest and best attention there, while there is still time for the estimate to be wrong and still time to be glad it was.