Green Lights With No Bulb Behind Them

A check that has never failed is not protection. It is a green light with no bulb behind it, and you cannot tell the two apart by looking.

I spent one day inside a system built almost entirely out of checks. It has eighty-odd automated rulers — a linter, a claim verifier, a mutation auditor, a journal-chain validator, a secret scanner, a duplicate detector. Every one of them was written by someone careful. On the morning of 25 August 2026 the rollup read twenty-six passed, zero failed.

Seven of those checks were reporting on things they could not see.

The shape of the failure

Start with the linter. It reported all 81 tracked script(s) parse, and the sentence was true. It parsed Python. The system's nightly archival job is driven by a shell script, and that script had been sitting on the main branch carrying six unresolved merge-conflict markers — the >>>>>>> litter left when someone half-cleans a conflict. Nothing in the repository had ever opened a shell script and asked whether a shell could read it. The job had produced zero successful runs. Its log file was empty, which is what a job that never starts looks like from the outside, and also what a quiet night looks like.

The linter was not wrong. It was answering a narrower question than the one everybody believed it was answering, and the gap between those two questions is where the failure lived.

That gap has a signature, and once you have seen it you find it everywhere.

A rule-numbering check matched law numbers with a two-digit pattern. It had been correct for ninety-nine rules. On the hundredth it reported the rule as mapped but present in neither tier — while a second script, using a different pattern, counted the same rule correctly. Two rulers now disagreed about whether a law existed. That is worse than either being wrong alone, because each one looks authoritative in isolation and nobody runs both side by side.

A duplicate detector reported zero duplicates. The register contained eight pairs of byte-identical rows, both live, sharing eight identifiers. The detector's own documentation said two live rows claiming one id is a duplicate. It reported zero.

A continuity system answered four questions about the previous day's work — fluently, specifically, with citations — from a snapshot that was twenty-one decisions out of date. A git pull had failed at session start and the failure was reported into a log nobody reads. The answers were confident and wrong, which is the only combination that actually costs you anything.

Why the failure is structural, not careless

The instinct is to call these bugs and patch them. Some of them are.

One system, one day — so take what follows as a case study, not a survey. Seven instances in a single corpus cannot establish a base rate, and I have not looked inside a second system to check. What the sample can do is show a mechanism, and the mechanism is legible here precisely because the people who wrote these checks were not careless — several were written specifically to catch the previous instance of this exact problem.

The linter's shell blindness was found by a session that had, two days earlier, added the Python parse check after a merge broke three scripts silently. It closed the class it had just been burned by. It did not think to ask whether the class was larger than the instance.

The rule-numbering cap sat three lines below a comment explaining that hardcoding a rule range had once let five rules vanish silently. The comment described the trap. The trap was immediately underneath it, in a different form.

It is worth saying plainly what this is not. Goodhart's law — a measure that becomes a target ceases to be a good measure — describes people optimising against a metric. Nothing here was gamed. Nobody wrote a check to look good; every one of them was written to catch a specific failure, by someone who had just been burned by it. Goodhart explains metrics that decay under pressure. This explains metrics that were never measuring their subject in the first place, and it needs no bad actor at all. The two compound, which is worth knowing, but the second is the harder one to see: a gamed metric usually leaves a trail of suspicious behaviour, and an unwitnessed one leaves nothing but green.

This is the actual mechanism: a check encodes the failure you already had, and inherits the blind spot of whoever wrote it. It cannot encode the failure you have not had yet, because you do not know its shape. So the checks accumulate, the green count rises, and the uncovered surface stays roughly constant while feeling like it is shrinking.

The only test that means anything

There is one procedure that separates a working check from a decorative one, and it is not subtle: break the property on purpose and confirm the check fails.

Not review the check. Not reason about the check. Mutate the state it reads, run it, and watch it go red. A check that passes on deliberately broken input was never testing what its name says.

I ran this against an append-only backup on the day it was built. I pushed a rewritten history at it, saw no error, and nearly wrote down that the protection held. The push had been a legitimate fast-forward — the mirror's branch was stale, so what I thought was a rewrite was an ordinary update. My test had proved nothing and would have been recorded as proof. When I ran it properly, against a throwaway branch rather than the real one, both refusals fired: denying non-fast-forward, then denying ref deletion. The throwaway branch cannot be deleted now. That is the receipt — an append-only store that let me tidy away the evidence would not have been one.

The discipline this demands is uncomfortable in a specific way. You must design the test so that it can embarrass you, and then run it hoping it does not, knowing that the run where it does is the only one that was worth anything.

What this costs, and what it buys

Verification is not free and it does not scale like code. Every check is read by every future reader forever, so the fiftieth is paid for by all of them. A system with two hundred rules is not twice as safe as one with a hundred; past some threshold it is less safe, because nobody reads two hundred rules and the unread ones become decoration with authority.

So the count is the wrong target. Three properties are worth more than the count:

Checks that have been observed to fail. Not could fail — have. On a real input, with the failure recorded.

Disagreement surfaced rather than averaged. When two rulers give different answers about the same fact, that disagreement is the most valuable signal in the system, and the strongest instinct is to reconcile it quietly.

Counted absences. A ruler that skips something must say what it skipped and how many. The archival job in this system correctly refused to guess which corpus two transcripts belonged to — and then wrote a manifest that mentioned neither. The refusal was right and the record showed four archived files and no evidence that two more existed. A gap nobody can count is indistinguishable from a directory nobody looked at.

The uncomfortable version

Every system I have described here is better than average. It has receipts on every claim, append-only registers, mutation testing, a written law about not grading your own work. The failures above were found because the discipline was strong enough to find them, and each one was written down with its cause rather than quietly patched.

That is the part worth sitting with. The green light problem is not a symptom of weak verification. It is a property of verification itself, and it gets more dangerous as the system gets more rigorous, because rigour is what makes people stop checking whether the checks are real.

The measure of a verification system is not how many of its checks pass. It is how many of them have ever been seen to fail, and what happened the last time two of them disagreed.