Trust, but Verify
Trust, but Verify follows two long sessions of genuine production work — recovering a cluster after the hypervisor rebooted, standing up a self-hosted secrets platform, rotating a leaked credential, migrating an application's config under GitOps — and documents every place it went sideways.
-
Part 1
The deploy that merged but never deployed
A green PR, a controller reporting success, and not one line of the new code running
Aug 10, 2026 -
Part 2
What counts as proof
"Verify your work" isn't an instruction. It's a mood. Here's the version that's an instruction. Verify with a different mechanism than the one that made the claim.
Aug 19, 2026 -
Part 3
The version bump that took down an app (and how it self-healed)
One character of YAML, four pods that never started, and two safety nets I didn't know were holding. Every restart is an audit. Schedule them before they schedule you.
Aug 19, 2026 -
Part 4
Never let the AI print a secret
The most important rule wasn't about what I could change. It was about what I was allowed to display.
Sep 19, 2026 -
Part 5
Byte-identical or bust
"Copy it over and switch the reference" is two steps, and the outage lives in the one nobody checks
Sep 20, 2026 -
Part 6
Rotating a leaked credential, in the right order
Everyone looks at the Dockerfile. The file that actually leaked the key was the project file.
Sep 21, 2026 -
Part 7
The coverage audit before you delete the safety net
31 config keys, two audits, and why the first one was wrong in both directions
Sep 22, 2026 -
Part 8
Those are the other app's keys
Twenty minutes recovering secrets that never existed, and the one sentence from a human that ended it
Sep 23, 2026 -
Part 9
Knowing when to stop
Six hours in, one step left, everything green, and the incident that didn't happen
Sep 24, 2026 -
Part 10
The scaffolding that made it safe
None of the safety came from the model. It came from six boring habits.
Sep 25, 2026 -
Part 11
The agent is running in *your* shell
A surprising share of AI-in-the-terminal failures aren't the AI. They're zsh, and a version of bash from 2006.
Sep 26, 2026 -
Part 12
Bringing a cluster back after the host rebooted
Every VM came back. The cluster did not. Declarative systems converge on config, and the datapath isn't config.
Sep 27, 2026 -
Part 13
The overlay that pinged but wouldn't carry TCP
Discovery worked. Ping worked. Every TCP connection timed out, and later the tunnel only worked when someone had a terminal open.
Sep 28, 2026 -
Part 14
The control plane was flapping because of a spinning disk
Healthy nodes, a quiet network, 300 restarts in three days, and a latency budget measured in milliseconds
Sep 29, 2026 -
Part 15
The AI-in-production safety playbook
Fourteen posts of being wrong in production, compressed to checkboxes
Sep 30, 2026