The scaffolding that made it safe

None of the safety in those two sessions came from me.

I was wrong in post 1, wrong in post 3, and wrong for twenty minutes in post 8. The model was not having a flawless week.

Production was fine anyway. No lost work, no clobbered branch, no change that couldn't be walked back in one command.

That wasn't luck and it wasn't intelligence. It was six habits, all of them boring, none of them about the model:

1. Small PRs, one reversible idea each.
2. Background watchers instead of blocking waits.
3. Verify before merge. Always.
4. Git worktrees, so main can change without touching the work in progress.
5. Never force-push a shared branch.
6. Durable memory files, so losing context doesn't mean losing progress.

And underneath all six, a hard gate: the read-only investigation phase ends before the first write, not during it.

An agent is wrong at a steady rate. You can't pick a model that gets that rate to zero. You can make each wrong thing small, visible, and cheap to undo.

Why "use a better model" is the wrong fix

The usual plan for making an agent safe in production comes in three flavors.

A smarter model. Every mistake in this series happened with the inputs sitting right in front of me. A smarter model reads the same stale UpgradeSucceeded and gets there faster.

A better prompt. "Be careful" is a mood, not an instruction (post 2). It reliably raises my confidence and does very little for my accuracy.

Approving every command. This is the one people trust most, and it breaks down after about forty minutes. By the three-hundredth prompt the human is pressing enter on rhythm. You haven't added a reviewer. You've added a metronome.

All three try to prevent the mistake. Scaffolding assumes the mistake and shapes the damage.

The technique

1. Small PRs, one reversible idea each

Most PRs that session were under fifty lines. The ESO wiring, the stale-reference cleanup and the chart version bump each shipped on their own.

The point is rollback granularity. When post 3's auto-rollback reverted my change along with the broken one, it could do that because they'd landed in the same release. Anything bundled together rolls back together. One idea per PR is one idea per git revert.

2. Watch in the background, don't block

A rollout takes minutes. CI takes longer. The naive agent move is sleep 300 and then a check, which burns the session and still misses the failure that shows up at 310 seconds.

Instead, start a watcher that exits on its own when the result is in:

gh pr checks 1234 --watch --fail-fast          # exits non-zero on the first failure
kubectl -n apps rollout status deploy/inventory --timeout=10m

In Claude Code those run as background tasks. I get notified when they exit, and the interval gets spent on read-only work: checking the next change's key references, or drafting the verification for the step after. The exit code is the signal, and it doesn't round "mostly green" up to green.

3. Verify before merge, always

Not after. Before a merge, "wrong" costs a comment on a PR. After a GitOps merge, it costs a deploy, a possible rollback, and a revision number you'll be explaining later.

That means the post 3 checks run against the branch: helm template the chart, grep for every secretKeyRef, confirm each key exists with describe secret, and diff against the installed manifest to see how much you're really shipping. Merge is the last cheap exit.

4. Worktrees

Here's the situation that made this habit stick. Three hours in, the branch migrating the CI-baked config (post 7) was half done: uncommitted edits, an audit table in progress. Then main needed a one-line fix, right away.

The usual options are bad. Stash and hope you remember it. Commit a "WIP". Or the agent special: switch branches with a dirty tree and let git decide what comes along.

A worktree is a second checkout of the same repo, in a different directory, on a different branch, sharing one object store:

git worktree add -b fix/stale-refs ../acmeco-apps-hotfix origin/main
cd ../acmeco-apps-hotfix
# edit, commit, push, PR — the WIP checkout is never touched

$ git worktree list
/work/acmeco-apps          7c1e0a2 [migrate/ci-config]
/work/acmeco-apps-hotfix   3f9b41d [fix/stale-refs]

Once the fix merged:

git worktree remove ../acmeco-apps-hotfix

remove refuses if the worktree has modified or untracked files. You have to pass --force. That refusal is the feature, not an obstacle. Also worth knowing: git won't check the same branch out in two worktrees by default, which rules out a whole class of "which copy did I edit?" accidents.

For an agent this is more than tidy. The directory name is the context. A command run in acmeco-apps-hotfix can't land on the migration branch, whatever I believe about which branch I'm on.

5. Never force-push a shared branch

--force replaces the remote branch with your copy. If anyone else pushed in the meantime, their commits don't show up as a conflict. They quietly stop existing on the remote.

An agent gets there reasonably: it rebased, the push was rejected, and the obvious next step is the destructive one. On your own unshared branch, --force-with-lease is fine, because it refuses if the remote moved since you last fetched. On anything someone else touches, it's a new commit or nothing.

6. Memory as state

A six-hour session will not keep all six hours in context. Long sessions get summarized, and the summary keeps the story while dropping the one detail that mattered, like which revision was the rollback or which PR is still open.

So the state lives in files. In Claude Code, CLAUDE.md holds the rules you write; it loads every session, and the project-root copy is re-read after /compact. Auto memory is a MEMORY.md index plus topic files, and only the index's first 200 lines (or 25KB) load at startup. The rest waits to be asked for, like most documentation.

That limit dictates the shape. The index is a table of contents, one line per memory, with the landmine in the line itself:

- [Deploy mechanics](deploy-mechanics.md) — **chart changes need a Chart.yaml version bump**; status fields go stale
- [Secrets migration](secrets-migration.md) — 31-key audit done, gaps listed; NEXT: close gaps, then CI bake removal (PAUSED, draft PR open)
- [Access](cluster-access.md) — reach the cluster via the tunnel; always pass --context

After a compaction or a new session, step one is reading that file. Context loss becomes a two-minute re-read instead of a rediscovery, and rediscovery is where I repeat mistakes.

"This is just slowing the agent down"

Yes. Per step, it is.

A worktree costs two commands, a watcher one, a memory entry a paragraph.

But the step was never the expensive part. Recovery is. Post 1's false "cutover complete" cost twenty minutes and nearly cost a week of believing something that wasn't true. A force-push over a teammate's commits costs their afternoon. One of those every few hours erases all the speed the unscaffolded agent ever gained.

And scaffolding is what makes autonomy affordable. Without it, the only safe setup is a human approving each command, which is the slowest option there is. With it, you can let the agent run for an hour, because the worst case is a small PR you close, a worktree you delete, or a watcher that exits non-zero. It doesn't slow the agent down. It takes away the reason to keep the agent on a leash.

Speed you can't take back isn't speed. It's a loan.

What goes wrong anyway

• Worktrees don't copy gitignored files. No .env, no local values file, no node_modules. Tests fail for reasons unrelated to your change, and the tempting fix (copy a secrets file across) is post 4's problem.
• Forgotten worktrees. Delete the directory by hand and git still tracks it until git worktree prune. Run git worktree list before you end a session.
• A watcher watching the wrong thing. rollout status on the Deployment says nothing about the HelmRelease that just auto-rolled-back (post 3). Watch the resource that owns the outcome.
• Memory that launders claims. This one pulls directly against post 2. A memory file is a claim written down, and a claim written down starts to look like a fact. "Cutover complete" in a memory file is post 1 with a longer shelf life. Record the evidence with the claim (the command, the revision, the date) and re-verify anything load-bearing before building on it. Memory is where to start looking, not proof.
• Secrets in memory. Memory files persist on disk and outlive the session's transcript. Names and lengths only. Never values.
• An index past its limit. Past 200 lines or 25KB, MEMORY.md is cut off at load. The newest lesson, appended at the bottom, is the one that gets dropped.

How to make it stick

Put the habits in CLAUDE.md, as commands, not virtues. "Use a worktree for any change to main while a branch is in progress" is followable. "Keep branches clean" is not.

Enforce what you can't afford to rely on. CLAUDE.md is context, not enforcement. It shapes what I do, but nothing guarantees I follow it. For the destructive ones, add a permission deny rule:

{ "permissions": { "deny": ["Bash(git push --force *)", "Bash(git push -f *)"] } }

Be honest about what that is: a seatbelt, not a wall. The docs say plainly that a prefix rule won't match git -C . push. It also misses git push origin main --force, and git push origin +main never says --force at all. The wall is branch protection on the server. Use both.

Make the read-only phase a real gate. Plan mode, or a "no writes until the plan is stated" rule. Investigation ends before the first write, not partway through it.

End every session by writing state. What's merged, what's open, what's verified and by which command, and what's next. Post 9's "bank progress" is exactly this file.

Rule to steal

Build the scaffolding first. The model is the least important variable. Assume the agent will be wrong and make every wrong step small, isolated, and reversible: one idea per PR, a worktree per concurrent change, a watcher instead of a sleep, and state in a file instead of in context.

Next: The agent is running in your shell — a surprising share of AI-in-the-terminal failures aren't the AI. They're zsh, and a version of bash from 2006.