Our playtest robot bought machines for ten hours and never switched any of them on

The last few of these have been about checks that pass while looking at the wrong thing. This one is a different animal, and worse in a specific way: nothing was checking anything. A measuring instrument was quietly measuring something else, and it produced numbers that looked exactly like the numbers it was supposed to produce.

What it did

We have a probe that plays the game as a flawless player — buys the right thing at the right moment, never misses a collection, runs for hours of simulated time in seconds. Everything we know about pacing comes from it, and previous entries here have leaned on it heavily.

Buying a machine and starting it are two separate commands.

The probe sent the first one. It never sent the second.

So every machine it ever bought sat idle for the rest of the run. Ten hours of simulated time, a shop full of equipment, and not one part was ever printed by any of it.

Why nothing caught it

The runs completed. No errors, no crashes, no assertion failed anywhere — the probe deliberately has no assertions, because it is an instrument rather than a test, and I have defended that decision in this devlog before. I still think it is right. But it does mean the only signal an instrument gives you is its output, and the output looked fine.

It looked fine because the columns were plausible. A table headed with machine names, filled with non-zero numbers that rose as the run went on. It was counting purchases. Reading it as production required no misunderstanding at all — the label said one thing, the arithmetic underneath said another, and a plausible number is far more dangerous than a missing one. A blank column gets investigated. A column of reasonable-looking figures gets quoted.

And it got quoted. Here, among other places.

What survived, and this is the part worth keeping

When the correction landed, most of the shop-level figures moved. But one headline number did not: the estimate of how little a returning player finds waiting for them after a long absence.

It survived because it had never come from the probe. It was arithmetic done directly on the catalogue — print duration, buffer size, multiply out — a completely separate method that happened to answer the same question.

That is the whole lesson, and it is cheaper than it sounds:

> A number you can reach two different ways is a number you can trust. A number with one source is a number you are hoping about.

We did not do that deliberately. We got lucky, in that one figure had been derived twice by accident, and it was the only one still standing when the instrument turned out to be wrong.

Three more wrong numbers, three different species

Correcting the probe turned up others in the same sweep, and they are worth separating because the fixes are not the same.

A true number about a thing that no longer exists. One figure had a research track taking over thirteen hours. It was accurate — for a version of the game that was retired weeks ago, whose progression system no longer exists. Nobody mis-measured anything; the world moved and the number stayed. The current figure is closer to an hour and a half. Correctly measured, and still false, which is a category most people do not check for.

A units slip. One machine's price was out by a factor of a hundred, in the direction that turns a modest purchase into an absurd one. Cents where dollars were meant. It had been sitting in a table being read as a real figure.

A number that was simply better than reported. The engaged player was recorded as running at around two-thirds efficiency. Re-measured, they are at a hundred percent — the buffer they were thought to be overflowing peaks at roughly half its capacity. That correction went the pleasant direction, and it is worth saying out loud that the sweep found some of those too. A correction pass that only ever finds bad news is a pass that is confirming something.

The rule this one leaves behind

The instrument had no assertions and that was the correct design. It reports; a human decides. But an instrument still needs one thing a test does not:

> Something has to check that the instrument is exercising what it claims to exercise.

Not the values — the behaviour. In this case: did any machine ever transition out of idle? A single check for that would have failed on the very first run, ten hours of nothing would never have been recorded, and several months of quoted figures would never have needed a correction.

That check now exists. It is one line, and it is embarrassing how cheap it was.