Skip to content

Architecture

The Human in the Loop — at Design Time

Tim O'Reilly posted something this week about craftsmanship in the AI age. The question he was circling: how do you maintain quality standards when agents are doing the work?

The default answer in the industry is: keep the human in the loop. For every meaningful decision, have a human review before proceeding.

That model contains a fatal flaw.

From What to Why: When AI Reveals Questions You Didn't Ask

For most of my career, analysis meant asking a question and getting an answer. How many deployments last quarter? Which modules have the most open defects? What is the test coverage of the payment service? The tools were built for this. You formulated a query, you ran it, you got a number. The number was correct. And the quality of your insight was entirely bounded by the quality of your question.

I did not think of this as a limitation. It was just how analysis worked. You got better at it by learning to ask better questions. Thirty years of architecture experience is, in large part, thirty years of learning which questions to ask and in what order. The senior architect's advantage was not access to better data. It was knowing which query to run.

That model is breaking. Not because the tools got faster at answering questions, but because a new class of tooling -- AI-augmented, temporally aware, relationship-tracking -- does something structurally different. It does not just answer your question. It tells you what you should have asked instead.

Why Temporal Matters: The Time Capsule Graph

Most systems I have built over thirty years answer one question well: what is the current state? A database query returns the latest row. A service responds with the live configuration. A dashboard shows what is happening right now. Current state is the default, and it is sufficient most of the time.

Some systems go further. They add history. An audit table, an event log, a change data capture stream. Now you can answer: what was the state at time T? Useful for compliance, useful for debugging. But still a limited question, because history stored as a sequence of snapshots tells you what changed without telling you how those changes relate to each other.

The questions I keep running into -- the ones that matter most -- are different. How did we get here? In what order? What does that trajectory mean? Those questions require something most architectures are not built to answer.