How do you test your tests?
No. 05
Every safety check in this project has been watched failing on a real problem before its passing result was believed, because a check that cannot come back negative is indistinguishable from one that works.
The house rule here is one sentence:
> A check that cannot come back negative is decorative.
Before we trust any gate, we break the thing it protects and confirm it screams. Then we put the thing back. It costs a couple of minutes and it has never once been wasted.
That sounds like paranoia until you see the list.
What the rule has caught, in this project alone
A test suite that ran zero tests and reported success. Exit code 0, green output, nothing executed. It had been "passing" for a while.
A quality gate that passed a deliberately blurred image. The gate existed specifically to catch blurry output. We fed it something obviously wrong to watch it complain, and it congratulated us.
A check for a rising difficulty curve that sorted its input before testing whether it rose. Read that twice — it sorted the numbers ascending, then verified they ascended. It could not fail. It would have gone on not failing forever, through any amount of broken tuning.
A config comment claiming an audio problem was solved when nothing had been done about it at all. Not a check, but the same failure: a confident green signal with nothing behind it.
None of those were visible by reading the code. Every one looked correct. They were found by making something run and watching what it actually did.
The corollary, which is sharper
We added a second rule after a near-miss:
> A mutation test that does not confirm the mutation landed is itself decorative.
Here is how that one nearly got us. To check a gate, you deliberately break the thing it guards — change a value, run the gate, watch it go red. Standard.
We tried to break a price on purpose. The edit silently matched nothing: the string we were searching for had moved. So the "mutated" run was really an unmutated run, the gate reported green, and green was the correct answer to the question we accidentally asked.
A failed experiment and a well-protected system produce identical output. The only thing separating them is whether you checked that your sabotage actually took. Now every probe asserts the mutation landed before it believes the result.
Why this is worth the two minutes
The value of a check is not that it passes. It is that it could fail and didn't. A check that cannot fail carries no information at all — but it looks exactly like one that carries a lot, and it sits in your build going green while the thing it was written to protect quietly rots.
The failure mode is not "the check is wrong". It is "the check is absent, and you believe it is present." Those are very different situations to be in, and from the outside they are the same colour.
So: break it, watch it scream, put it back. Every time. If it does not scream, you have just learned something considerably more important than whether the code works.