The Third Test Harness
![[ The Third Test Harness ] — Preventing specification drift in autonomous, AI-augmented development workflows. Terminal prompt: system_verify --target live --mode external_truth → STATUS: DRIFT DETECTED](/assets/images/blog/third-test-harness/th-hero.png)
A supplier scored 72. The frontend showed a green ring and the label "Strong." The backend API returned "Moderate." Both systems were passing all their tests. Both were correct — according to their own definitions.
![[ The Third Test Harness ] — Preventing specification drift in autonomous, AI-augmented development workflows. Terminal prompt: system_verify --target live --mode external_truth → STATUS: DRIFT DETECTED](/assets/images/blog/third-test-harness/th-hero.png)
A supplier scored 72. The frontend showed a green ring and the label "Strong." The backend API returned "Moderate." Both systems were passing all their tests. Both were correct — according to their own definitions.
The AI industry has decided the future is a swarm. Dozens of agents, competing, debating, voting on outputs, selecting the best result through evolutionary pressure. It sounds elegant. It borrows the right metaphors from biology. And it is a spectacular misunderstanding of what makes AI useful in real engineering work.
A company spent three years planning an ERP migration. Their architecture catalog was clean, tidy, and fully documented. It was also seventeen connections short of the truth — before they even opened the deployment config.

In February 2024, a Canadian small claims tribunal ruled against Air Canada. Their chatbot had told a passenger he could book a full-fare ticket and claim a bereavement discount retroactively. He couldn't. When he tried, Air Canada's position was: the chatbot said that, not us. The tribunal disagreed. You deployed it, you own what it says.
The ruling was correct. But the more interesting problem was underneath: when the incident happened, nobody could reconstruct what context the chatbot had been given. Nobody could confirm whether a human had ever reviewed the policy the bot was consulting. Nobody could determine whether the specific bereavement policy text had been modified between deployment and the passenger's interaction. The audit trail recorded that the system was deployed. It did not record what the system knew.
That's the provenance problem. And every organization running AI agents at enterprise scale is about to hit a version of it.
I have been watching the AI agent space closely for the past year. The frameworks are impressive. The orchestration tools are clever. The models are increasingly capable. And yet, most agent deployments I see make the same quiet mistake: they treat the knowledge problem as solved.
It is not solved. It is barely addressed. And until it is, all the reasoning capability in the world will not make your agents reliably useful.
In February 2009, I wrote a blog post called "Clouded Vision." The central argument was straightforward: "developers have fundamentally misunderstood how cloud computing delivers its benefits." They saw cheaper prices but never stopped to consider where the savings came from. They expected to move existing applications, full of what I called "enterprise DNA" -- static configuration, vertical clusters, high administration costs -- onto cloud platforms with minimal change. Then they complained when it proved difficult. I wrote nineteen posts about cloud computing that year. Most of them circled the same frustration: the industry was adopting a new technology while completely misunderstanding the structural shift it required.
A practical guide to giving your AI coding assistant an institutional memory
You've tried Claude Code. Maybe you love it. Maybe you've noticed that on your 300K-line, 20-module Maven project it spends the first five minutes figuring out where anything is.
That's not a model limitation. That's a context problem. And it's solvable.
In February 2009, I wrote a post called "Clouded Vision" where I argued that "developers have fundamentally misunderstood how cloud computing delivers its benefits. They see the cheap prices but don't stop to consider where the cost saving comes from." The post described a specific architectural mistake: teams were taking their existing applications, full of what I called "enterprise DNA," and deploying them to cloud platforms with minimal change. Then they complained when it proved difficult and expensive.
There is a narrative forming in the industry that goes something like this: AI will replace junior developers, senior developers will become more valuable, and if you have enough experience you have nothing to worry about. I think this misreads what is actually happening. The shift is real, but it is not the one most people describe.
When we finished lib-pcb, the question we got most was: "How?"
Not "what model did you use?" Not "what IDE?" Those questions miss the point entirely. The model is the least interesting variable. What made 197,831 lines of Java, 7,461 tests, and 474 commits in 11 days possible was a methodology. Specifically: six practices that we have now codified under the name Skill-Driven Development.