Skip to content

Governance, Trust & Compliance

The LLM Cautionary Tales

In late 2024 and through 2025, we published a series of short horror stories about building with LLMs. Not fictional in the sense of being made up — fictional in the sense of being slightly dramatised versions of things that happen, or will happen, or already have happened to someone you know.

The format was deliberate. Security warnings written as dry checklists get skimmed. Security warnings written as campfire stories get remembered.

Here are all eight tales.

The Organisational Amnesia Problem

Here's a question most organisations can't answer: what was Pump 47's vibration reading before it failed?

Not a complex question. A specific measurement, at a specific point in time, for a specific piece of equipment. But in most asset management systems, if that reading was overwritten when the failure was logged, it's gone. The data archaeology required to reconstruct it — if it's possible at all — involves manual investigation across multiple systems, log files, and human memory.

The same pattern applies everywhere.