# break-it-check — the mandatory anti-theater gate

**This gate is mandatory. No phase latches without it.** Execute mode's step 4
is `break-it-check`, invoked between "write the tests for this phase" and
"write `Landed:` and commit" — nothing skips it, and nothing writes `Landed:`
without it passing.

It discharges *"the test suite should fail loudly when you fix bugs."* For
each behavior named on the phase's `Catches:` line: inject that bug into a
**disposable copy** of the production code, confirm the new test goes red,
discard the copy.

The mechanism is adapted from `verification-before-completion/SKILL.md:84-88`:

```
Write → Run (pass) → Inject bug → Run (MUST FAIL) → Discard copy → Run (pass)
```

with two inversions from that skill: there is no fix to revert — the bug is
**injected** into working code — and the injection happens in a throwaway
`git worktree`, so nothing in the developer's tree is ever at risk (Inviolate
#2: production code is never permanently modified).

**When you tell the developer what you're doing, use plain words — never
"break-it-check," "the gate," "mutation," or "latch"** (see
`references/test-pushback.md § Talking to the developer`). For example: *"Before
I call these tests done, I'll make sure they actually work — I'll slip one
realistic bug into a throwaway copy of the code and confirm the tests catch it,
then throw the copy away. Your real code is never touched."* What each possible
result means for the developer is spelled out in *Explaining results*, below.

## Protocol

Six steps, in order. The order is load-bearing — steps 1 and 2 each protect
against a failure mode that only shows up if they run first.

1. **Sweep, then create one worktree for the phase.** First fix this run's
   identity: run `echo "$(date +%Y%m%d-%H%M%S)-$$"` **once**, at the first
   `break-it-check` of the session, and reuse the exact string it prints —
   shape `<YYYYMMDD-HHMMSS>-<pid>` — as `<run-id>` for every phase after it.
   **Run the command; never invent an id, and never copy one out of any
   example.** Two runs holding the same id sweep each other's live worktrees,
   which is the failure the id exists to prevent. Never regenerate it per
   phase either: a run that changes its own id stops recognizing its own
   strands.

   Then reclaim strands left by a crashed or aborted prior run:

   - `git worktree prune` — reclaims records whose directory is gone or
     unreachable; it never deletes checked-out files.
   - Enumerate `git worktree list --porcelain` and force-remove a worktree
     **iff one of its path components is exactly `paad-test-roadmap-<run-id>`**
     — split the path on `/` and compare components for **equality**. Never a
     substring or "contains" test: run-ids differ only by PID, so a sibling's
     `paad-test-roadmap-20260726-141133-481712` *contains* your
     `paad-test-roadmap-20260726-141133-4817`, and a contains-test
     force-removes that sibling's live worktree mid-mutation. Never match on a
     `${TMPDIR:-/tmp}/…` prefix you construct either: git records the
     symlink-resolved path (`/var/…` prints as `/private/var/…`) and `$TMPDIR`
     already ends in `/`, so a constructed prefix matches nothing and the sweep
     silently becomes a no-op.
   - **Do not add an age or mtime arm to catch what run-id scoping leaves
     behind.** The worktree is created once and reused for the whole phase, so
     its mtime does not move while a slow suite runs — any threshold you pick
     force-removes a live phase mid-mutation, which is that same failure
     rescheduled onto a timer. *Why a disposable worktree* covers what happens
     to those strands instead.

   Then create the phase's worktree. Name the directory `phase-<N>` — never the
   roadmap's phase title, which is prose (`Phase X of Y`) and would silently
   nest if it ever contained a `/`:

   ```
   git worktree add "${TMPDIR:-/tmp}/paad-test-roadmap-<run-id>/phase-<N>" HEAD
   ```

   That worktree is **reused across the baseline run and every mutation in the
   phase** — one per phase, not one per mutation. This is structural, not an
   optimization to drop under time pressure: a fresh checkout has none of the
   gitignored build artifacts (`node_modules/`, `target/`, venv) that the skill
   cannot language-agnostically reinstall, so the cost is paid once per phase
   rather than once per mutation. See *Why a disposable worktree* for the full cost.

2. **Copy the phase's new test files into the worktree, *then* mutate.**
   Never the other way around. In colocated-test ecosystems — Rust's
   `#[cfg(test)] mod tests` is the reproduced case — the test file *is* the
   production file. If the mutation were applied first and the test copied
   in second, the copy would silently overwrite the mutation: the test would
   stay green, and the gate would misread a mutation that was never actually
   applied as a theater test. Copy-then-mutate is the only order that can't
   erase its own injected bug.

3. **Baseline run.** Run the phase's own tests, unmutated, in the worktree.
   They **must pass**, and the result must be **stable across two runs**. See
   the *Failure table* below for what each way of failing this step means —
   do not proceed past a baseline that isn't clean green and stable.

4. **Per behavior on `Catches:`, inject one mutation via a blind mutator
   subagent.** The subagent receives:

   - the `Catches:` behavior text,
   - the production file,
   - the worktree path,
   - the operator list (below),

   and explicitly **not the test**. It returns one hunk. The main agent
   applies the **first** hunk returned, runs the phase's own tests, and
   checks for red. There is no re-rolling for a more convenient hunk — see
   *Constrained mutation* for why.

   Blinding the subagent to the test is the point: a subagent that can see
   the test could tune the mutation to whatever the test happens to check,
   which reintroduces the exact failure mode this gate exists to catch — the
   same author writing both the test and the thing meant to indict it.

5. **`git worktree remove --force`.** `--force` is required, not optional —
   the worktree is unclean by construction (the copied-in test is untracked,
   the injected mutation is a tracked-file modification), and a bare
   `git worktree remove` refuses an unclean worktree, telling you to pass
   `--force`. There is nothing to restore, nothing to stage, no fence to
   satisfy first: this is only the fast path. Step 1's sweep is the
   guarantee — see *Why a disposable worktree* below.

6. **Commit only after the check passes** — and after execute mode's step 5
   clean-run gate also passes — on the developer's real tree. This gate proves
   the phase's tests catch a bug; that one proves they run clean in the full
   suite on the branch. Both precede the commit. The operator name(s) used here
   are echoed to the transcript and recorded on the phase's `Landed:` line as
   provenance.

**"Run the test" means the phase's own tests, not the whole suite**, at every
step above. Mutating production code *should* break unrelated tests — that's
expected, not signal — so there's no reason to run the full suite here, and a
legacy suite's pre-existing failures stay out of frame.

## Constrained mutation

The injected mutation is not free-form. It is a single change drawn from a
named operator set, and the subagent must **declare which operator by
name** when it returns its hunk:

- flip a comparison (`<` ↔ `<=`, `==` ↔ `!=`)
- off-by-one a boundary
- negate a condition
- alter a constant
- drop a state transition

**Explicitly banned, regardless of operator name:**

- an early `return` / `raise` / `throw` at function entry
- removing a whole function body

These two are banned by shape, not by judgment call, because they are
sledgehammers: either one makes *any* test go red — including a theater test
like `assert result is not None` — which would let a genuinely worthless test
pass the gate for the wrong reason. "Keep the mutation small" is a
rationalizable instruction; "no early return at function entry" is a concrete
line an agent cannot talk itself around. Requiring the operator to be named
turns a dishonest mutation into an affirmative written claim instead of a
silent choice.

**If the first hunk leaves the test green, that is the theater row in the
table below — rewrite the test. Do not re-roll the mutation.** Re-rolling
would let the main agent shop for a hunk that happens to turn the test red,
which quietly restores the identity the blind-mutator subagent exists to
break: one author controlling both the test and the thing meant to indict
it. A test that survives one honest, named mutation attempt on the behavior
it claims to cover has already told you what it needs to tell you.

## Failure table

Five outcomes. Each is a different failure of a different precondition the
gate depends on, and each has one correct action — never "fix the code" and
never "latch anyway."

| Outcome | Meaning | Action |
|---|---|---|
| Baseline red (fails unmutated, run alone) — but green in a full-suite run | **Order dependence**, not a mischaracterization: the test relies on state another test sets up. The isolated gate cannot judge it. | **Surface it. The phase cannot use the isolated gate as-is.** Do not latch, do not "fix" the code. |
| Baseline red (fails unmutated) — and also red in a full-suite run | The test does not describe current behavior — **mischaracterization**. | **Rewrite the test.** Do not fix the code, do not latch. |
| Baseline result unstable across two runs | The test is **flaky** — its verdict is noise, so the gate cannot judge it. | **Surface it.** Do not latch. |
| No code path implements this `Catches:` behavior (no valid site to inject) | The phase was planned against code that has since moved or vanished. | **Phase invalidated — re-plan it.** Do not latch. |
| Mutation applied, test stayed green | **Theater** — the test does not catch what the phase claims. | **Rewrite the test.** Do not latch. Do not re-roll the mutation. |

**"Run the test" in every row above means the phase's own tests, not the
whole suite.** The order-dependence and flaky rows exist because the
isolated gate silently *depends on* the phase's tests being isolable and
deterministic — this table is where those two preconditions get checked,
and where a violation of either gets named and surfaced rather than papered
over. Naming the violation is the whole of this gate's engagement with
suite health; it detects the two failure modes, it does not repair them.

The "no code path" row is the design's staleness detector: a bug cannot be
injected into a code path that has moved or vanished, and no phase latches
without passing this gate — so a stale plan gets caught here, not silently
latched against a `Produces:` path that no longer means what it once did.

## Explaining results to the developer

The table's labels — "theater," "mischaracterization," "order dependence,"
"flaky," "invalidated" — are for the agent. Tell the developer what the result
means for their tests, in plain words (§ Talking to the developer):

| Outcome | What to say to the developer |
|---|---|
| Mutation applied, test stayed green (*theater*) | *"These tests still passed after I deliberately broke the code they're supposed to check, so they aren't actually testing that behavior. I'll rewrite them so they catch it."* |
| Baseline red, also red in full suite (*mischaracterization*) | *"These tests disagree with how the code behaves right now. Since the job here is to pin down current behavior, I'll fix the tests to match what the code actually does."* |
| Baseline red alone, green in full suite (*order dependence*) | *"These tests only pass when run together with others — run on their own they fail — so I can't confirm them in isolation. That's worth cleaning up before relying on them."* |
| Baseline unstable across two runs (*flaky*) | *"These tests give different results on repeat runs, so I can't trust the outcome yet. They need to be made reliable first."* |
| No code path implements `Catches:` (*invalidated*) | *"The code this set of tests was planned around has moved or been removed, so the plan for it is out of date and I'll redo it."* |

## Why a disposable worktree, not an in-tree mutation

An earlier draft mutated the developer's working tree directly and protected
it with an elaborate write-fence: a whole-tree `git status --porcelain`
snapshot compared before and after each mutation, plus a
`git add`-before-`git checkout` staging dance to survive `git checkout`'s
restore-from-index behavior. Roughly forty lines of protocol existed solely
to make an in-tree mutation survivable — and it still had a window: a crash,
context exhaustion, or a `--watch` runner mid-mutation could leave altered
production code on disk, caught only on the *next* run, after the developer
already had it in front of them.

The worktree deletes the problem instead of fencing it. The mutated copy is
disposable, so there is no invariant left to protect: no snapshot
comparison, no staging, no path-based fence — and the colocated-test hazard
that broke every such fence (in Rust/Zig/D the test file *is* under `src/`)
becomes irrelevant, because you're mutating a copy you're about to throw
away regardless.

**Why `$TMPDIR` and not somewhere under the repo.** Nothing this gate creates
may land in the working tree: a second full checkout under the repo root is
visible to test discovery, linters, the `find`/`grep` recon in this plugin's
own skills, and `git status` — and the skill cannot exempt it, because it does
not edit the developer's `.gitignore`. An earlier draft used
`.git/test-roadmap-worktrees/`, which satisfied that property but hard-fails
wherever `.git` is a *file* rather than a directory: a `git worktree` checkout,
a submodule, a `--separate-git-dir` repo. `$TMPDIR` keeps the
nothing-in-the-working-tree property, behaves identically in all three of those
repo shapes, and gives the sweep a fixed path component to match on.

The **run-id** in that path is what keeps one run's sweep off another's live
worktree. This is not hypothetical caution about a rare double-booking: a
sibling's sweep force-removing a live worktree out from under a running
mutation **has been reproduced**. Two agents on one repo is not exotic — a
developer can open a second session while one sits mid-phase, and both modes
of this skill are built to resume across sittings. Path prefix alone
identifies "worktrees this skill made," not "worktrees belonging to *my*
run." The `paad-test-roadmap-` half of the component is what keeps the
sweep off the developer's *own* hand-made worktrees; the run-id half is what
keeps it off a concurrent session. Both live in the path, so both are
answerable from `git worktree list` output alone.

The same kind of crash the write-fence could not survive can still strand
the **worktree itself** — the disposable checkout, plus its
`.git/worktrees/` record — on any exit that skips step 5: a failure-table
short-circuit, context exhaustion, a `--watch` runner, a hard kill. That is
hygiene, not a safety gap — the developer's tree is never touched either
way, so Inviolate #2 holds regardless of whether step 5 ever runs — but the
leak this time is closed the way the write-fence's window was **not**: on
the next entry, never on the current exit. Step 1's sweep (`git worktree
prune`, plus a force-remove filtered to test-roadmap's own path component
**and** this run's id) runs on the one path a crash cannot skip — the start
of the next phase — so a strand left by a short-circuited phase dies when
the session next enters the gate, instead of persisting for the rest of the
run.

A strand left by a session that died *outright* is out of the run-id sweep's
reach by design, and what reclaims it is `$TMPDIR` — eventually, with an
honest ceiling: `systemd-tmpfiles` defaults to ten days on Linux, many
container and CI images run no cleaner at all, and a project-local `$TMPDIR`
may never be swept. So "the OS reclaims it" can mean "not for ten days" or
"never." That is accepted rather than fixed because the residue is bounded and
non-destructive: disk held by a checkout, plus a phantom row in `git worktree
list` until the directory does go away and `git worktree prune` clears the
record. The developer's tree is untouched either way, which is the property
this design actually protects — and every mechanism that would reclaim it
sooner (an age arm, an unscoped prefix sweep) buys that back by risking a live
sibling's worktree. On-exit removal (step 5) is only the fast path;
the sweep is the guarantee, because no on-exit cleanup survives a crash this
design already declined to trust once.

The cost, stated honestly: a fresh worktree starts with no `node_modules/`,
no `target/`, no venv, no `_build/`. Some ecosystems need a full dependency
install before their tests can even run, and that can cost minutes. This
skill cannot know the install command stack-agnostically, so the mitigation
here is structural, not a lookup table — one worktree per phase, reused
across every mutation in that phase (protocol step 1) — never a built-in
per-language command table.

# Build mode — the five stages

Loaded when `paad/test-roadmap/test-roadmap.md` does not exist yet — the first run against a
repo, or any run after that file was deleted. Five stages, run in order, each
handing a concrete artifact to the next: Detect, Grade, Plan, Critique, Write.

## Stage 1 — Detect (main agent)

Identify the stack from manifests and config rather than assumption —
`package.json`, `pyproject.toml`/`setup.cfg`, `go.mod`, `Cargo.toml`, `Gemfile`,
`pom.xml`/`build.gradle`, `composer.json`, `cpanfile`/`Makefile.PL`, `*.csproj`,
`mix.exs`, and so on. This list is examples, not a closed table — the skill must
never hardcode a language. Where it needs a per-ecosystem fact, it looks for the
signal in the repo rather than consulting a built-in list.

Detect **four** things, all from repo signals:

1. **How tests are invoked, and what test files exist.** Read the manifest's test
   script/target and enumerate the test files it would run.
2. **Whether a coverage tool is available.** Used read-only, later, in Stage 3 for
   gap-finding — **never as a quality signal**. Where none is detectable, Stage 3
   falls back to agent judgment reading source, and says so in the plan rather
   than silently proceeding as if coverage output existed.
3. **Whether a test-data mechanism exists** — fixtures, factories, seed scripts,
   testcontainers, a `conftest.py`, a `factories/` or `fixtures/` directory. This
   feeds Stage 3: an integration or e2e phase that needs constructed data cannot
   be planned honestly against a repo with no way to construct it.
4. **How this repo separates test tiers, and the per-tier run + coverage
   command.** The three tiers — unit, integration, e2e — must end up
   *distinguishable* and *independently coverage-runnable* (a stated requirement,
   see *The three tiers must be independently runnable*, below). The mechanism
   that makes them so is **not universal** — it is a *selector* the ecosystem
   already expresses, and it varies: a test directory (`tests/unit/`,
   path-selected), a filename marker, a test tag/marker (`pytest -m`, Go
   `//go:build`), a build target (`cargo test --lib`/`--test`, a `.csproj` or
   Jest project), or a test label (CTest `-L`). Detect which the repo uses, and
   derive per tier the command that runs *just that tier* and the command that
   runs it *with coverage*. **Do not assume the selector is a directory.** These
   per-tier commands are recorded in `## Decisions` (Stage 5).

**Stage 1's output is a discovery, not a boundary.** It tells you what tests exist
and how to run them; it does not exhaustively partition the repo into "test" and
"production," and nothing downstream may treat it as a safety fence. There is no
path fence anywhere in this skill — bug injection happens in a disposable
worktree (`references/break-it-check.md`), so no path-based fence is needed to
keep mutation off production code.

## Stage 2 — Grade (fan-out subagents)

**Skipped entirely when no tests exist.** Greenfield goes straight to Stage 3 —
there is nothing to grade.

Where tests exist, dispatch subagents partitioned by test tier or directory.
Each subagent reads its partition against `references/test-theater.md` and
returns a compact **structured verdict** — never pasted test source, never
diffs. A legacy suite is thousands of lines the main agent must not absorb.

Each subagent returns:

- **Per weak test found:** file, line, pattern name (verbatim from
  `test-theater.md`), a one-sentence statement of why it fails to catch
  regressions, and a suggested replacement.
- **Per mock or fixture encountered:** a ledger classification (see *The
  test-double & fixture ledger*, below).

**Cardinality containment.** The number of instances is unbounded by design — a
legacy suite may yield hundreds of weak tests — and the design contains that
rather than merely asserting it away:

- The **full verdict list** is written to `paad/test-roadmap/test-suite-analysis.md`, never
  returned to main context.
- **Main context receives counts and the top-K by severity only.** A 400-item
  grading result must never land in the resume context budget. Bounding each
  verdict's size while leaving the list itself unbounded in main context is the
  same defect wearing a different hat — both the per-item size and the list
  length must be bounded before anything reaches the main agent.

## Stage 3 — Plan (main agent)

Identify behaviors worth testing that are not tested. Group them into phases
across the three tiers — unit, integration, e2e. Build the ledger. Draft phases
in the canonical format defined in *Stage 5 — Write*, below.

Two Stage 1 findings feed this stage:

- **Coverage, for gap-finding only.** Where a coverage tool was detected, run it
  read-only to locate behaviors with *no test at all*. Coverage answers "what is
  untested," which it does well; it never answers "is this test any good," which
  it cannot — a line can be covered by a test that asserts nothing. This
  distinction is inviolate: coverage percentage is never evidence a test is
  worth keeping, in this stage or any other. Where no coverage tool exists, this
  is agent judgment reading source, and the plan states that plainly rather than
  presenting judgment as if it were tool output.
- **Test-data mechanism.** If a phase needs constructed data and Stage 1 found
  no mechanism to construct it, the phase carries a **surfaced note** — *"requires
  a test-data mechanism this repo lacks"* — rather than being planned as if it
  were executable as written. Finding un-runnability at plan time is the point
  of having a plan stage; finding it at the gate is the expensive place to find
  it.

**Legacy phases pin current behavior.** Where a phase characterizes existing
code, every assertion it proposes must match what the code *currently does*.
Behavior that looks wrong is never turned into a different assertion or a code
fix — characterizing a legacy system and fixing its bugs are two hard problems,
and this stage does the first one only. Where a wrong-looking behavior clears the
inclusion gate in *The findings log* (below), it is recorded there — a concrete,
actionable entry the developer works from later — and the phase that pins it
carries a one-line pointer to that entry. Where it does not clear the gate, it is
dropped, not written down as a vague note: a findings log of hunches is noise the
developer learns to skip, the same cry-wolf failure the clean-run rule exists to
prevent.

## Stage 4 — Critique

Invoke `references/test-pushback.md` mode `critique-plan` against the draft
plan Stage 3 just produced. Findings are **fixed inline, in the draft, on the
spot** — no report is written; there is nothing to hand off, only phases to
rewrite or drop before Stage 5 writes them to disk.

The governing rule, restated because it is the gate the whole stage exists to
enforce:

> **Every phase must name the bug it would catch.** If a phase cannot answer
> *"what breakage makes these tests go red?"*, it is not a phase — it is
> coverage theater. Rewrite it or drop it.

See `test-pushback.md` for the full pass, including the supporting rules (exit
status and coverage are never evidence; every double and fixture is classified;
legacy phases characterize, they don't fix).

## Stage 5 — Write (main agent)

All generated files live under `paad/test-roadmap/` — create that directory if
it does not yet exist. Two artifacts always, plus a third when this run turned up
any qualifying finding:

- **`paad/test-roadmap/test-roadmap.md`** — the `## Decisions` section first, then the
  phases.
- **`paad/test-roadmap/test-suite-analysis.md`** — full grading verdicts and the ledger.
  This is the sink for anything unbounded (Stage 2's full list, the ledger in
  full); main context never carries it.
- **`paad/test-roadmap/test-roadmap-findings.md`** — the findings log (see *The findings
  log*, below), written only if Stage 3/4 recorded at least one entry. If they
  recorded none, this file is not created here — execute mode creates it the
  first time a phase surfaces a qualifying finding.

**These artifacts are read by the developer, so they follow `references/test-pushback.md
§ Talking to the developer`.** In particular the ledger's class names
(`boundary`, `scaffold`, `data`) and any pattern names cited from
`test-theater.md` are jargon to a newcomer — gloss each in plain words the first
time it appears in the analysis doc (e.g. *"`scaffold` — a stand-in that only
exists because the code is hard to test as written; it marks test debt to retire
later"*). A weak-test verdict states, in plain terms, what regression the test
would let through, not just its pattern label.

**`paad/test-roadmap/test-roadmap.md` is always the target.** Everything this
skill generates lives under `paad/test-roadmap/` — its own namespace, so it never
clutters the developer's `docs/` and sits where PAAD tooling expects it. That
also sidesteps colliding with a hand-written `docs/roadmap.md` a repo may keep for
other reasons. The roadmap file's path is the router's signal for which mode to
load on the next invocation, so it is **load-bearing and not configurable** — do
not rename it, alias it, move it back under `docs/`, or write phases anywhere
else.

**Then commit the artifacts.** `git add paad/test-roadmap/test-roadmap.md
paad/test-roadmap/test-suite-analysis.md` — and `paad/test-roadmap/test-roadmap-findings.md` if it was
written — and commit them before build mode ends. This is
not optional bookkeeping: the roadmap is the router's resume signal, and
requirement 2 is resumability across *fresh clones*. An uncommitted roadmap
does not survive a clone, so the next run finds no `paad/test-roadmap/test-roadmap.md` and
silently rebuilds from scratch — the exact churn resumability exists to
prevent. Execute mode already commits each `Landed:` line the same way (step
5); build mode committing its own output is the same rule at the front of the
lifecycle, not a new one.

**Then list every file this run wrote**, one line per path — the roadmap is the
whole deliverable of build mode and it is easy to miss among the conversation:

```
Files written or updated:
  new  paad/test-roadmap/test-roadmap.md  (14 phases)
```

**Then tell the developer the total, in plain words.** Build mode's approach
menus fire *before* any plan exists, so during them there is no phase total to
show — the first instant it is knowable is right here, once Stage 5 has written
every phase. State it at the handoff, and make the loop explicit — **this run
planned the work, it did not do any of it**, and one phase lands per run from
here: *"Planned 14 phases — no tests written yet. Run `/test-roadmap` again
to write Phase 1 of 14, and keep running it: one phase per run, 14 runs to
finish."* This is the developer's first sight of the end of the tunnel, and from
here on every phase the skill names carries its `Phase X of Y` per
`references/test-pushback.md § Talking to the developer`. A developer who stops
after this run has a plan and zero tests, so do not end build mode without
saying what the next invocation does.

### The canonical phase format

Every phase, in both build mode's draft and execute mode's updates, takes this
exact shape:

```markdown
## Phase 3: Billing retry & dunning integration tests

Tier:     integration
Catches:  a retry exhausting without transitioning the account to dunning;
          a partial refund double-crediting.
Produces: tests/integration/billing/
Branch:   billing-integration-tests
Landed:
```

Field semantics:

- **`Tier:`** — exactly one of `unit`, `integration`, `e2e`. Stage 3 already
  groups behaviors by tier, so this is honestly authorable at plan time (unlike
  the rejected `Covers:` field, which asked Stage 3 to enumerate something it
  never produces). It makes *which tier a phase belongs to* explicit in the
  roadmap itself — satisfying the distinguishability half of the tier
  requirement without depending on where the files physically land — and it
  selects which recorded per-tier run/coverage command (see `## Decisions`)
  applies to this phase.
- **`Catches:`** — the anti-theater gate from Stage 4, preserved in the artifact
  so a later reader can audit whether the phase was honest. It is also
  `break-it-check`'s behavior input: it names the *behavior* to mutate, not a
  path. A path is neither necessary (the behavior locates the code) nor
  sufficient (directory granularity cannot pick a line to mutate).
- **`Produces:`** — paths the phase creates, as **documentation only**. It is
  **not** a completion signal. A later reader uses it to find the phase's
  output; no control flow in this skill infers phase status from it.
- **`Branch:`** — a human-readable breadcrumb recording the working branch the
  phase's tests were committed on. **Not a machine signal, and not a per-phase
  branch the skill creates** — execute mode commits each phase onto the current
  working branch and the suite accumulates there (see `execute-test-roadmap.md
  § What execute mode writes`).
- **`Landed:`** — empty until `break-it-check` passes for this phase, then set
  to `YYYY-MM-DD <sha> (<operator>)`. **Human-clearable; the skill itself never
  clears it.** A cleared `Landed:` line is a human declaring the phase needs
  redoing — the skill has no mechanism that would ever produce that state on
  its own.

Each phase's definition-of-done includes a recommendation to run `agentic-review` on the accumulated working branch before it is merged upstream, if available. This is not bundled into execute mode: `agentic-review` requires a fresh session with no substantive history, so calling it from within the test-roadmap session would be inert.

### The test-double & fixture ledger

Every test double and fixture, existing or proposed, gets one classification:

| Class | Meaning |
|---|---|
| `boundary` | Permanent and correct. A real external edge: network, clock, randomness, a payment gateway, a third-party API, filesystem where relevant. |
| `scaffold` | Exists only because the surrounding code resists testing. Test debt. Names the refactor that retires it — that refactor is *deferred*, named now and actually done only when it is actually planned, not during this pass. |
| `data` | Constructed test data that is permanent and correct — a seeded account, a factory-built order, a fixture record. Neither an external edge (`boundary`) nor debt a refactor retires (`scaffold`). |

An unclassified double is a gap in the plan, not a detail to leave for later.
**When in doubt between `boundary` and `scaffold`, or when a double goes
unexamined, default to `scaffold`.** The harm is asymmetric: a `scaffold`
silently promoted to permanent hides test debt indefinitely, while a `boundary`
mistakenly recorded as `scaffold` only leaves a note a human later deletes.

When a unit resists testing so completely that even a `scaffold`-mock test can't
be written, the artifact is not a mock but a **skipped stub** that names the
obstacle and surfaces it through the framework's skip-with-reason mechanism — see
execute mode's *Units too hard to test*. It is still `scaffold`-class debt (the
refactor that makes the unit testable retires it); it is a last resort, and it
never counts as coverage.

### The findings log

While pinning current behavior, the skill will notice code that looks wrong — a
return that contradicts its own docstring, a validator that accepts what its name
says it rejects. It never fixes these (Inviolate #1: characterize now, fix
later), but it records the good ones so the developer finishes with a concrete
to-do list instead of a vague memory that "something looked off." That list is
`paad/test-roadmap/test-roadmap-findings.md`.

This is **not** the "findings" of *Approaches vs findings* below — those are
test-quality verdicts (a weak test, a mock's class) and go to the ledger. This
log is production-code bugs the developer will fix later, watching the
characterization tests break as they do.

**The inclusion gate — verified and actionable, or it is not logged.** A log of
hunches is noise the developer learns to skip, the same cry-wolf failure the
clean-run rule guards against. An entry is written **only if all three hold**;
miss any one and the observation is dropped, never downgraded to a vague note
(there is no second-tier "maybe" list, and nothing lands elsewhere):

1. **Demonstrable current behavior** — a specific input→output or code path, not
   a general unease. Where a characterization test pins it, cite that test: the
   test *is* the reproduction.
2. **A concrete contradiction, cited** — in-repo evidence the behavior violates:
   a docstring, a type signature, an adjacent validation, a stated invariant, a
   test name asserting otherwise. This is a **citation of two things in the repo
   that disagree**, never the agent's own ruling on what the code *should* do —
   which of the two is right stays the developer's call, and Inviolate #1 stays
   intact.
3. **A clear action** — a specific next step, e.g. *"reconcile the docstring at
   `email.py:40` with the return at `email.py:52`."*

**Entry format** — one block per finding:

Each field is its own bold-labeled paragraph, separated by blank lines — never
bare adjacent lines, which markdown reflows into a single paragraph:

```markdown
## F3 — `is_valid_email` accepts the empty string

**Where:** src/email.py:52

**Behavior:** `is_valid_email("")` returns True.

**Contradicts:** the function's own docstring (src/email.py:40): "returns True
only for a syntactically valid address."

**Action:** decide which is correct and reconcile them.

**Pinned by:** Phase 6 — its test locks in the current True. Fixing the code
turns that test red; that is the signal to update the test, not a regression.
```

`Pinned by:` is filled where a phase's test pins the behavior (always so for a
finding surfaced *while writing* that phase; for one surfaced at plan time, it
names the phase that will pin it). It is the line that makes the finding
actionable *and* ties it to the suite: the developer knows in advance which test
will go red, and that red is success.

The log is **append-only and committed with whatever produced it**, so it
survives a fresh clone like the roadmap does. The main agent never holds the
whole file in context — it appends entries, it does not re-read the accumulated
log each run.

### Approaches vs findings

Two different things reach the developer during a run, and they are not
handled the same way.

**Approaches** — genuine forks where the answer is taste or is expensive to
reverse. Roughly four to six across a whole run: test framework/runner, suite
strategy (unit-first / e2e-first / risk-first), phase ordering, whether to
rewrite the weak tests Stage 2 found. Every approach goes to a menu whose full
format is defined once in `references/test-pushback.md § Presenting approaches`
— **load that section before presenting the first menu**, because by the time a
run reaches this point it is deep in analysis and the format is no longer in
context. The compact rule, restated here so it is in front of you at the moment
you present rather than only in a file that may not be loaded:

> Put the menu's substance in a **written prose menu in your own reply**: one
> decision, every option with its own pro and con, a recommendation grounded in
> this repo, and the deep-dive (a skeptical adversarial pass over the options)
> as the last option. Then collect the answer with **one** structured-question
> call — the harness's picker — carrying **exactly one question**, the options
> as short labels only (the deep-dive included). That call is just the enter-key:
> it blocks until the developer answers, so *one decision at a time* and *wait
> for the answer* are enforced by the tool, not left to you remembering to stop.
> **Never put more than one question in that call, and never let the picker
> carry the menu's substance** — pros and cons, recommendation, and deep-dive
> stay in the prose above it. A picker carrying the whole menu is what batched
> decisions as tabs and dropped three of the four required parts in the field; a
> one-question selector does neither, and unlike a bare prose menu it cannot be
> followed by more work until the developer answers. Full format and wording:
> `§ Presenting approaches`.

This is the one place the reference's format is deliberately restated rather
than only pointed at: it is a *reflex at the point of use*, and a pointer to an
unloaded file is not a reflex.

**Not on the menu — recorded defaults.** Some things look like a choice but have
a dominant default and are cheap to reverse, so they are decided once and
recorded, not voted on. **Test organization** is one of these: it is a *detected
fact* about how the repo already separates tiers, not a taste choice, so it goes
to `## Decisions`, not to a menu — putting it on one would blow the 4–6-approach
ceiling for a decision that isn't the developer's to make.

The organization must leave the tiers distinguishable and independently
coverage-runnable (see below), but the *form* is whatever the detected ecosystem
expresses — a tier directory only where the ecosystem uses an external,
path-selected test tree, and a tag/marker/build-target/label elsewhere. An
earlier version of this note argued against a "group by tier" layout on the
grounds that two phases sharing a tier directory would manufacture a false-landed
collision. That reasoning is now spent: the completion model was rewritten so
`Landed:` is the sole signal (see `execute-test-roadmap.md § Completion model`) —
nothing infers completion from a directory's contents anymore, so two phases in
one tier directory each latch via their own `Landed:` line and collide on
nothing. Tier grouping is therefore safe; it just isn't an operator *menu* item,
because it's detected, not chosen.

**Findings** — verdicts derived from evidence: a weak-test classification, a
mock classification. These are not approaches; asking a human to vote on a
classification is asking them to vote on a fact. Findings go to the ledger,
consistent with Stage 4's own rule that findings are fixed inline, not
reported.

**`scaffold` mocks are batched, not surfaced one-by-one.** They are recorded to
the ledger en bloc, not raised as individual decisions during grading — on a
legacy codebase, the case that produces the most `scaffold` entries, one-by-one
surfacing would mean dozens of serial decisions in a single sitting. The
retiring refactor for each is named when that refactor is actually planned,
which is both less overwhelming and a better moment to name it than during a
grading fan-out.

**Silence defaults to `scaffold`** — restated here because it governs what gets
written to the ledger during this stage, same reasoning as above: the harm of
under-classifying is asymmetric, so inattention is made safe by construction
rather than relying on a human reading every entry carefully.

### The three tiers must be independently runnable

The suite must leave unit, integration, and e2e (1) **distinguishable** — a
developer new to the repo can tell which tier a test belongs to — and (2)
**independently coverage-runnable** — they can run coverage over *just* the unit
tier, or *just* integration, and so on.

The distinguishability half is carried by the `Tier:` field on every phase (and,
where the ecosystem's convention already separates tiers on disk, by the layout
too). The independent-coverage half is carried by **recording, per tier, the
command that runs it with coverage** — because per-tier coverage is *not* a
universal "point the coverage tool at a directory" operation:

- Coverage instruments the **source under test**, never the test files, so
  pointing a coverage tool at a directory of tests does not scope coverage to a
  tier. What scopes it is the tier's *selector* applied to the coverage run.
- The selector is a directory only in ecosystems that use external,
  path-selected test trees (Python, PHP, Ruby, Perl). Elsewhere it is a tag
  (Go `-tags`, `pytest -m`), a build target (`cargo test --lib`/`--test`, a
  `.csproj`, a Jest project), or a label (CTest `-L`).
- In colocated-test ecosystems the tier directory *cannot exist*: Go `_test.go`
  files must live in the package under test, and Rust unit tests are
  `#[cfg(test)]` modules compiler-bound inside `src/` — both reproduced by
  construction (see the design's *Verification notes*). Mandating a `unit/`
  directory there is impossible, not merely awkward.

So the skill records the *commands*, not a layout. This is the only formulation
that holds across every ecosystem — the invariant is "there is a command that
runs exactly one tier with coverage," and its selector is detected, not assumed.

### `## Decisions`

Every recorded default and every approach the developer picked is written to a
`## Decisions` section at the top of `paad/test-roadmap/test-roadmap.md`, before the first
phase. Execute mode reads this section on every resume and **never re-asks**
anything recorded there — without it, menus would fire again on every run and
the resumability requirement fails through the front door. Record at minimum:
the chosen test framework/runner, the suite strategy, phase ordering, the
weak-test rewrite decision (if grading ran), the test-organization default, and
**the per-tier run + coverage commands** — one line each for unit, integration,
and e2e — so both execute mode and the developer can run and cover each tier
independently. A worked shape:

```markdown
## Decisions

Tiers (run | coverage):
- unit:        `pytest tests/unit`            | `pytest --cov=pkg tests/unit`
- integration: `pytest -m integration`        | `pytest --cov=pkg -m integration`
- e2e:         `playwright test`              | (no coverage tool for this tier)
```

Where a tier has no coverage tool, say so on its line rather than omitting it —
an absent line reads as "forgotten," a stated "(none)" reads as "checked."
Keep each entry to the decision and its one-line reason — this section is a
record for a machine to skip past, not a design document.

# Execute mode — the completion protocol

Loaded when `paad/test-roadmap/test-roadmap.md` already exists — the path every run after
the first takes. This is the run that must be quiet about anything the
developer already settled: it reads `## Decisions` once, per phase asks
exactly the one question the completion model requires, and otherwise gets on
with writing tests.

## The execute-mode loop

Eight steps, in order:

1. **Read `## Decisions` from the roadmap. Never re-ask anything recorded
   there.** The chosen test framework/runner, suite strategy, phase ordering,
   the weak-test rewrite decision, the test-organization default, the per-tier
   run + coverage commands — all of it was already decided during build mode's
   Stage 3/Stage 5 and is written down for exactly this reason. Re-asking it
   here is not caution, it is the resumability requirement failing through the
   front door. Place the phase's tests according to its `Tier:` and the recorded
   organization, and use the recorded per-tier command when running or covering
   that tier.
2. **Select the candidate phase per the *Completion model* protocol, below.**
   Exactly one phase at a time — the protocol's questioning fires for the
   candidate phase only, never as a batch sweep across every phase in the
   roadmap.
3. **Write the tests for that phase.** This is the only code this step
   writes — see *What execute mode writes*, below. Reading the code this
   closely is where real bugs surface; log the ones that qualify to the
   findings file — see *Logging suspected bugs*, below.
4. **Run `references/break-it-check.md`.** This gate is mandatory. **No
   phase latches without it** — not a fast path for a phase that "obviously"
   passes, not a phase where the developer is confident the tests are
   right. Skipping this step is skipping the entire mechanism that
   distinguishes this skill's suite from `assert result is not None` a green
   exit code would have waved through.
5. **Run the developer's whole suite on their branch, and don't latch a red
   or noisy run.** Using the recorded `## Decisions` commands, run every test
   the way the developer would — once normally, once under coverage — and read
   both runs for failing tests, spurious output, and anomalies. See *Keep the
   test run clean*, below, for what to run, why the coverage pass earns its
   place, and how to dispose of what you find. Like `break-it-check`,
   this gate can only **block or surface** — it never edits production code and
   never latches a dirty run.
6. **Verify the phase touched no production code, then latch.** Before writing
   `Landed:`, confirm from git that this phase's pending diff only *adds* test
   code and *writes* under `paad/test-roadmap/` — no existing production code
   modified, deleted, moved, or added (see *The phase touched no production
   code*, below). Only then write `Landed: YYYY-MM-DD <sha> (<operator>)` and
   commit on the current working branch — written only after steps 4, 5, *and*
   this check all pass, so it is those gates' own output, not bookkeeping that
   could drift out of sync with them.
7. **Surface run instructions for the tests just landed.** See *Run
   instructions*, below.
8. **Hand off to the next run — or say the roadmap is finished.** One phase
   lands per run, so the run always ends by telling the developer whether to
   invoke `/test-roadmap` again and why. See *Ending the run*, below. This
   step is not optional and is not merged into step 7: a developer who is not
   told to run it again assumes one invocation was the whole skill, and stops
   with a roadmap and one phase of tests.

### Ending the run

**The single most common way this skill fails is a developer running it once
and stopping.** They get a roadmap, or a roadmap plus one phase, and never
learn there were thirteen more. Nothing in the phase's output implies "come
back" unless the run says so, so every run says so — explicitly, as its last
words.

**First, list every artifact this run wrote or updated.** The test files are
visible in the run instructions, but the roadmap and the findings log are not,
and a developer who never learns a bug was logged never reads it. One line per
path, each marked new or updated. Name the test files too when a phase landed
only a few; past a handful, give them as a count with a pointer to the diff, so
the roadmap and findings lines stay the first thing read:

```
Files written or updated:
  new      tests/integration/billing/test_retry.py
  updated  paad/test-roadmap/test-roadmap.md          (Phase 3 marked done)
  updated  paad/test-roadmap/test-roadmap-findings.md (F4 added)
```

Name the findings log only when this run actually added an entry to it, and say
what the entry was about in a few words, so a real bug does not sit unread.

Then recompute the counts from the roadmap
(`references/test-pushback.md § Talking to the developer`) and end the run one
of two ways:

- **Phases remain** — name the next one and say plainly that another
  invocation is what runs it:

  ```
  Run /test-roadmap again to do the next one — one phase per run, so it
  takes another 6 runs to finish the roadmap.

  Next is Phase 8 of 14 — 7 done, 6 to go after this: Logger level filtering
  & message formatting.
  ```

  Say *why* it is another run, not a continuation: each phase is written,
  proved against an injected bug, and committed on its own, and a fresh run
  keeps the context clear for the phase it is working on. A new session is
  fine — the roadmap file is the memory, and the run picks up from it.

- **No phases remain** (every phase in the roadmap has a populated `Landed:`)
  — do not ask for another run. Say the roadmap is finished: all N phases are
  done, the tests are committed on this branch, and point at
  `paad/test-roadmap/test-roadmap-findings.md` if it has entries, since that
  bug list is the developer's to act on and this skill never will. Running the
  skill again in this state is a no-op that finds nothing to do, so do not
  invite it.

### Run instructions

The developer running this skill may not know the codebase, the test runner, or
even the language. So when a phase's tests are ready, do not leave them to
reconstruct how to run anything. Name the phase as `Phase X of Y` throughout this
mode — the run-instructions header, the "next phase" prompt, and the completion
question of step 2 — per `references/test-pushback.md § Talking to the developer`,
so the developer always sees how many phases remain. Print a short block, drawn
verbatim from the `## Decisions` per-tier commands (never invented on the spot),
giving:

- **The new tests** — the command that runs *just this phase's* tests.
- **This tier with coverage** — the recorded coverage command for the phase's
  `Tier:`, so the developer can see what these tests cover. Where that tier has
  no coverage tool, say so explicitly rather than omitting the line.
- **The whole suite** — the command that runs every test, so they can confirm
  nothing else broke.

A worked shape:

```
Phase 3 of 14 (integration) tests are in and committed. To run them:

  just these tests:   pytest tests/integration/billing/
  with coverage:      pytest --cov=billing -m integration
  the full suite:     pytest
```

Keep it to the commands and one label each — this is an aide-mémoire for someone
unfamiliar with the repo, not documentation. The commands are the recorded ones;
if a command has drifted (the runner changed since build mode), that surfaces
here as a failed run the developer can see, not as silent wrong advice.

### What execute mode writes

Execute mode writes **test code only**. It never writes production code. Two
different things enforce that, on two different axes: `break-it-check`'s
*mutation* never touches the developer's tree because it runs in a throwaway
`git worktree` (Inviolate #2), and this mode's *authoring* — step 3, writing the
tests — is verified after the fact by the step 6 check (*The phase touched no
production code*, below) before anything latches. Neither is a path-based fence:
that was rejected (it fails open in colocated-test ecosystems), and the step 6
check is deliberately diff-aware instead.

**Each phase's tests commit onto the current working branch; the skill creates
no per-phase branch.** The suite accumulates in place — as phases land, the
test directory on the branch the developer is already on fills up, and the
`Landed:` SHA points at a commit that branch can actually reach. This is the
literal reading of `break-it-check`'s "commit on the developer's real tree"
(step 6). The skill deliberately does **not** spin each phase onto its own
branch and leave it unmerged: unmerged local branches do not survive a fresh
clone — the very failure the completion model rejects branch-merge detection
over — so putting the test *files* there would strand the suite off the
working branch and break resumability (requirement 2). Per-phase review is not
lost: each phase is its own commit, reviewable with `git show <Landed-sha>`,
and `agentic-review` runs against the accumulated working branch before it is
merged upstream.

### The phase touched no production code

*What execute mode writes* states the rule — test code only. This step **verifies
it from git** before a phase latches, because a rule the skill is trusted to
follow is worth confirming: a stray edit while writing tests, or a
refactor-to-make-testable that slipped past Inviolate #1, would otherwise commit
onto the branch unnoticed.

**The invariant:** a phase's commit may only *add* test code and *write* under
`paad/test-roadmap/`. It may never modify, delete, move, or add production code.

**The check is diff-aware, not path-aware.** A path fence — "no non-test *file*
changed" — is the write-fence this design already rejected, and it fails the same
way: in colocated-test ecosystems (Rust `#[cfg(test)] mod tests`, Zig, D) the unit
test lives *inside* the production file, so a path fence flags every honest test
or fails open on a real edit. Use the test organization Stage 1 already detected
(recorded in `## Decisions`):

- **Separate-file ecosystems** (Python, JS, Go's `_test.go`, Ruby, PHP, Perl):
  every changed path must be a test file or under `paad/test-roadmap/`.
- **Same-file colocation** (Rust and the like): a change to a production file
  must be **purely additive test code** — a new test module or test functions,
  with *no* pre-existing line altered. Any edit to an existing line is a violation.

Walk this phase's pending changes — `git diff --name-status` plus the diff itself,
staged and unstaged, *this phase's own commit, not the whole branch* — and check:

- A **delete or rename** (`D`/`R`) of any production path → violation.
- A **new non-test file** → violation (the skill adds tests, not source).
- A **modified production file** → violation, unless it is the colocated case
  above and the diff is purely additive test code.
- A **build/config/manifest change** (`package.json`, `Cargo.toml`, `.gitignore`,
  CI config) → **do not latch silently; surface it and ask.** A test sometimes
  needs a dev-dependency, or the coverage tool the clean-run gate already asked to
  install; latch it only on the developer's OK. This is the one change outside
  tests + docs that may be legitimate — everything else in the production tree is
  not.
- **Stray generated artifacts** (`.coverage`, `coverage/`, `target/`, editor
  cruft), caught by the same invariant → they must not be committed; drop them
  from the commit.

**On any violation, stop — do not latch.** Show the developer the exact change in
plain words — *"while writing these tests I ended up changing `src/billing.py`,
which shouldn't happen; the skill only adds tests. Here's what changed — it needs
undoing before I can call this phase done."* **Never silently revert it**: that
could destroy work the developer meant to keep or bury a real bug. The human
decides.

### Keep the test run clean

A phase is not done until the developer's suite runs clean. So before `Landed:`
(step 5 of the loop, after `break-it-check`), **run the whole suite on the
developer's branch the way they would** — the recorded `## Decisions` commands,
verbatim, never invented — **twice**:

1. **Normally** — the plain whole-suite command.
2. **Under coverage** — the recorded coverage command. Not for the percentage
   (coverage never certifies a test — Inviolate #3); this is an **anomaly probe**.
   Coverage instrumentation reorders imports and shifts timing, so a warning,
   failure, or hang can appear under coverage that a normal run hides. Run both
   and compare.

   Where the phase's tier has **no coverage tool** (`## Decisions` records it as
   `(none)`), ask the developer **once**, in plain words, whether to install one
   or go without — then record their answer in `## Decisions` so no later phase
   re-asks. That is the only place this pauses for input; it is not a per-phase
   question.

Read both runs for three things: **failing tests**, **spurious output** (a
`Wide character in print` warning, deprecation spam, stray STDOUT), and **any
other anomaly** — a hang, a result that changes between runs — that would make a
developer distrust the suite. Two reasons this matters, both about the developer:
noise buries the signal (twenty lines of framework chatter around one real
failure trains people to skim past it), and a suite that always warns teaches the
developer to ignore warnings — at which point the one warning that *was* a real
bug scrolls past unnoticed. The run must end **green and quiet**, or its remaining
noise must be a deliberate, recorded decision.

For output that appears, in order of preference:

1. **Treat meaningful output as behavior — capture and assert it.** If the code
   under test emits a deprecation warning, a log line, or a message that *matters*
   (it should fire, or it should *not* fire), that is behavior worth pinning:
   capture it and assert on it, the same as any return value. A warning you
   assert on is no longer noise — it's a test.
2. **Suppress noise you can't assert, at the test boundary.** Third-party
   chatter you can't control and don't need to pin gets silenced narrowly in the
   test setup — not by muting all output globally, but scoped so that a *new*,
   unexpected warning still stands out.
3. **Never let the tests you write add their own noise** — no leftover debug
   prints, no `warn`/`console.log` scaffolding shipped in the committed test.

**Every fix here is test-side. Production code is never edited (Inviolate #1) —
where the fix belongs decides what you do:**

- **A failure or noise rooted in the tests this mode wrote** — a leftover print, a
  test mishandling its own output, a bad assertion — is **yours to fix.** Fix it,
  re-run both ways, confirm clean.
- **A failure or noise rooted in production code** — the code under test emits the
  wide character, not your test — is **never patched here.** Stay on the test side:
  assert it if it's meaningful behavior, suppress it narrowly if it's
  uncontrollable, and where it is a real defect, record it in the findings log
  (`build-test-roadmap.md § The findings log`) if it clears the gate. The suite
  goes quiet without production behavior changing — and the developer gets the bug
  on their list instead of a silenced warning they never see.
- **A pre-existing failure in tests this mode did *not* write** is not this
  phase's job to fix — but it is **never silently latched over.** Surface it
  plainly and ask the developer how to proceed; do not fix unrelated tests, and do
  not mark a phase done on a suite you can see is red.

If a clean run genuinely isn't achievable for a phase (the code is noisy by
construction and the noise can't be captured or scoped), say so plainly rather
than shipping a suite that cries wolf. Whenever any of the above needs a judgment
you can't make alone, discuss it with the developer in plain words
(`references/test-pushback.md § Talking to the developer`) — the failing test or
the warning described for what it means, not by its label.

### Units too hard to test

Some units resist testing so completely that neither a real characterization
test nor a `scaffold`-mock test (see the ledger in `build-test-roadmap.md`) can
be written honestly. When that happens — and **only as a last resort, after both
a real test and a scaffold-mock have been judged infeasible, not merely
inconvenient** — write a **skipped stub test** that names the obstacle, rather
than silently leaving the unit out of the suite. An untested unit that is simply
absent looks like one nobody needed to test; a skipped stub that says *why* keeps
the gap visible.

**Surface it through the framework's own skip mechanism, with a reason** —
`pytest.skip(reason=…)`, `@Disabled("…")`, `t.Skip("…")`, whatever the detected
framework provides — so the message rides with the test and shows in the run
summary and CI, the same standard-tooling channel *Keep the test run clean* uses
for meaningful output. The reason states in plain language that the unit is hard
to test and what makes it so, e.g. *"skipped — builds its own DB handle in the
constructor; untestable until that dependency is injected."* Only where the
detected framework has no machine-surfaced skip reason does this fall back to a
single one-line STDERR statement — never a wall of prints.

A skipped stub is a **gap-marker, not coverage**:

- It **never goes through `break-it-check`** — nothing is asserted, so there is
  nothing to mutate.
- It **never counts toward a phase's `Landed:` line.** A phase latches on its
  real, bug-catching tests; a skip latches nothing.
- **A phase made only of skipped stubs catches no bug and is dropped as theater
  (Inviolate #5), not landed.** Skips ride *alongside* real coverage; they never
  constitute a phase.
- The obstacle is also recorded as `scaffold`-class debt in the ledger /
  `paad/test-roadmap/test-suite-analysis.md`, naming the refactor (usually a dependency-
  injection seam) that would make the unit testable and retire the skip.

When the phase's tests land, tell the developer in plain words how many units
were too hard to test and why, so the skips are a decision they can see rather
than a silent omission.

### Logging suspected bugs

Writing a phase's tests means reading the code closely, which is exactly when a
real bug surfaces — a return that contradicts its own docstring, a check that
lets through what it claims to reject. Do not fix it (Inviolate #1: pin current
behavior; the developer fixes later, watching these tests break). Instead, where
it clears the inclusion gate, record it in `paad/test-roadmap/test-roadmap-findings.md`.

**The gate and the entry format are defined once in `build-test-roadmap.md
§ The findings log` — use them verbatim.** In short: log an entry only if you can
state (1) the demonstrable current behavior, citing the characterization test
that pins it; (2) a concrete in-repo contradiction it violates — a citation, not
your own ruling on what is correct; and (3) a clear action. Miss any one and drop
the observation — never write it down as a vague note. Set the entry's `Pinned
by:` to this phase's test, and add a one-line pointer on the phase block in the
roadmap where the finding maps to it.

Create `paad/test-roadmap/test-roadmap-findings.md` if it does not yet exist (build mode
writes it only when its own stages found something); otherwise append. **Commit
it in the same commit as the phase's tests** (step 6 of the loop), so a finding
never lands without the test that pins it, and both survive a fresh clone.

When the phase lands, tell the developer in plain words how many findings you
logged and point them at the file — *"I logged 2 concrete bugs I hit while
writing these; they're in `paad/test-roadmap/test-roadmap-findings.md`, each with the test
that proves it"* — so the log is a decision they can see, not a file they
stumble on later.

## Why TDD is not bundled

`superpowers:test-driven-development` is deliberately **not** loaded by
execute mode, even though this mode is, superficially, "write a test, run it."

For a characterization phase — the case this skill exists for — the code
already works. The test step 3 writes therefore passes on its first run *by
construction*: there is no bug yet to make it fail, because none has been
injected. Verified at `test-driven-development/SKILL.md:126`:

> **Test passes?** You're testing existing behavior. Fix test.

Applied literally to a characterization test, that instruction is not merely
inapplicable — it is destructive. The test passing on its first run is
correct behavior here, not a signal that the test is wrong, and an agent
following that rule would rewrite a correct characterization test until it
fails for no reason, or invent a reason to make it fail. Either way it
corrupts the one thing this phase was supposed to produce: an honest pin on
current behavior.

The red phase a characterization test actually needs does not come from
authorship — writing the test a second, "more red," way — it comes from
**mutation**: inject a real bug and confirm the existing test catches it.
That is exactly what `break-it-check` (step 4, above) provides, and it is why
this mode reaches for that gate instead of for TDD's red-green-refactor loop.

## Completion model

### Problem being solved

A hand-written status label goes stale the moment work is merged outside the
agent session — another developer lands the branch, or the same developer
merges it between sessions. A fresh session with no memory of the previous
one then reads the stale label and asks the human whether already-finished
work is finished, which is exactly the churn requirement 2 (resumability)
exists to prevent.

An earlier draft solved this by inferring completion from a `Produces:` path:
path present on disk → treat the phase as landed; path absent → treat that as
proof the phase never started, and execute it silently. That is repo state
the skill does not control, and it fails in both directions:

- **Toward re-churn:** a directory renamed in a reorg, or a fresh clone of a
  repo whose layout changed since, makes landed work look absent, so the
  skill silently re-executes it.
- **Toward silent skip:** two phases sharing a directory — any "group by
  tier" test-organization layout can produce this — makes an unstarted
  phase's path look present, so the inference finds the *other* phase's
  output and reports the unstarted one as landed. That is the one direction
  the governing principle below forbids outright.

### Governing principle

**No signal auto-marks anything done, and no absence is ever proof of
not-done** (Inviolate #4). Signals decide only whether to ask, and supply the
evidence attached to the asking. Being wrong toward "ask with evidence" costs
one keystroke. Being wrong toward "auto-mark done" silently skips real work —
the one outcome the whole design is built to keep from happening quietly.

The original bug behind the `Produces:`-path draft was never that the agent
asked the human something. It was that it asked **empty-handed** — and, just as
bad for a developer new to the repo, in the skill's own jargon. The question
must carry its evidence *and* be in plain words (see `references/test-pushback.md
§ Talking to the developer`):

- Bad (empty-handed): *"Phase 3 shows Pending. Is it complete?"*
- Bad (jargon): *"Phase 3 declares `tests/integration/billing/`; its `Landed:`
  line is empty — did it land, or should I execute it?"*
- Good: *"I don't see tests yet for **billing retries** (I'd add them under
  `tests/integration/billing/`). Did someone already write these — maybe on a
  branch called `billing-integration-tests` — or should I write them now?"*

The phase's recorded details supply the evidence; the developer supplies the
answer. Because they may not know the repo's history, tell them where to look if
unsure — *"you can check with `git log tests/integration/billing/`, or see
whether those test files already exist"* — so the question is answerable without
prior knowledge of what has and hasn't been done here.

### Protocol

`Landed:` is the sole completion signal, and this protocol runs for the
**candidate phase only** — never as a sweep across the whole roadmap.

| Step | Where | Action |
|---|---|---|
| 1 | main agent | `Landed:` populated? → done. Stop. No git, no question. |
| 2 | main agent | `Landed:` empty → ask the developer, in plain words (§ Talking to the developer), whether these tests were already written or should be written now — using the phase's recorded details as evidence and pointing them at where to check. On *"write them,"* run the phase (the execute-mode loop, above). On *"already done,"* record it from what they report. |

Step 1 ends the churn permanently for a phase that has already latched: once
`Landed:` is populated, the control flow never re-examines that phase again —
**except** that `Landed:` is human-clearable (see below), which is a human
decision, not something step 1 itself does.

Step 2 asks rather than infers, and **absence of a "yes" means execute** — the
safe direction — never a silent skip. This is the case that fires only for
work that may have landed *outside* an agent session; work the skill lands
itself goes straight from `break-it-check` passing to a written `Landed:`
line, no question asked, because there the skill *is* the authority on
whether it happened.

**`Landed:` is human-clearable; the skill itself never clears it.** A human
who learns a phase's tests were condemned by a later review, or that the gate
was gamed, can clear the line by hand, and the phase re-enters the flow at
step 2 on the next run. No agent inference is ever the thing that un-lands
work — only a human, deliberately, gets that lever.

### Why not branch-merge detection

Considered and rejected as a signal. Verified behavior of
`git branch --merged main`:

| Merge style | Result |
|---|---|
| `--no-ff` merge commit | works |
| Squash merge (common default) | **fails** — branch tip is not an ancestor |
| Rebase merge | **fails** — commits rewritten, new SHAs |
| Branch deleted after merge | **fails** — nothing left to query |
| Fresh clone | **fails** — no local branches exist |

Four of five cases fail, and all four fail **toward "not done"** — the
skill would regenerate work already landed, which is churn, not safety, but
churn that the resumability requirement was written to prevent.

### Why not commit-log scanning, or `Produces:`-absence

Rejected for the same underlying reason: both infer "did this land" from repo
state the skill does not control. Matching a phase's title against commit
messages is string-matching human prose that was never written to be a
machine contract. `Produces:`-absence infers "not started" from a path
layout that renames, that two phases can collide on, and that — in a
colocated-test ecosystem — may never independently exist at all. Git log
answers *when* a thing landed and *which commit* did it well; it answers
*whether* badly. So the skill does not use it as a signal. `Landed:` is the
signal; a human who wants to know when a phase landed runs `git log`
themselves, with the surrounding context to read what it means — context a
blind scan does not have.

### Why not an anchor commit or a `Covers:` field

Considered: record a `generated-at: <sha>` and a per-phase `Covers:` field
naming the production paths it depends on, then diff `<sha>..HEAD` against
`Covers:` on resume to detect a phase planned against code that has since
moved.

Rejected on two grounds:

1. **It fails silent.** A change to a transitive dependency outside the
   listed paths produces an empty intersection, so the skill reports "no
   drift" for a phase that is actually invalidated. Closing that hole would
   require per-language transitive-closure analysis, which contradicts the
   stack-agnostic requirement this design is built to hold.
2. **`Covers:` is not honestly authorable.** Stage 3 produces behaviors and
   phase groupings; it never enumerates the production paths behind them. The
   field would have to be filled in by inference after the fact, failing
   silent in exactly the same direction the mechanism was meant to close.

`break-it-check`'s "no code path implements this `Catches:`" row already
detects the same condition — code a phase depended on has moved or
vanished — terminally and loudly, at gate time, at no authoring cost and with
no separate field to keep in sync.

# test-pushback — how the skill talks to the developer, plus critique mode

This file holds three independent, referenceable pieces. The mode files and
`break-it-check.md` point at them by name rather than inlining them, so each
stays defined once:

- **`§ Talking to the developer`** — the plain-language rule that governs
  *everything* the skill says to the developer.
- **`§ Presenting approaches`** — the menu format used whenever the skill
  puts a genuine fork in front of the developer.
- **mode `critique-plan`** — the adversarial pass Stage 4 runs against the
  skill's *own* draft plan before Stage 5 writes it.

The sections don't depend on each other and can be pointed at on their own.

---

## § Talking to the developer

Everything the skill says to the developer is in **plain language**. The person
reading may not know this codebase, the test runner, or the programming
language — and they know nothing about this skill's internals. So the design's
own working vocabulary stays *inside these files* and never reaches them
unglossed: `break-it-check`, "the gate," "latch," "mutation," "operator," "the
ledger," `boundary`/`scaffold`/`data`, "characterization," "theater,"
`Landed:`, "the phase block," "build mode"/"execute mode" are words for the
agent, not for the developer.

Translate, don't emit:

- **Name what you're doing in ordinary words, not by its codename.** Not
  "running break-it-check" — instead: *"I'll check these tests actually work by
  slipping a realistic bug into a throwaway copy of the code and confirming the
  tests catch it. Your real code is never touched."*
- **Report findings as what they mean for the tests, not by their label.** Not
  "this test is theater" — instead: *"this test still passed after I
  deliberately broke the code it's meant to check, so it isn't really testing
  that behavior."*
- **When a term does live in the written artifacts** (a tier name, a
  `boundary`/`scaffold` classification), gloss it in plain words the first time
  the developer sees it.

The test is simple: if a sentence would only make sense to someone who has read
this skill's design, rewrite it until it makes sense to someone who has not.

**Always locate a phase for the developer: "Phase X of Y."** Whenever you name a
phase to the developer, render `Phase <its written number> of <total phases in
paad/test-roadmap/test-roadmap.md>`, so they always know how much is left — the end of the
tunnel, not just the current step. On the forward-looking "next phase" prompt and
on any completion question, add progress: `— N done, M to go after this`.
Recompute all three from the roadmap on every run — never cache them:

- **Y** = count of phase blocks currently in `paad/test-roadmap/test-roadmap.md`. Because it
  is recomputed each run, it stays honest when a human adds or drops a phase: the
  total just updates next time, rather than going stale.
- **done** = count of phases whose `Landed:` line is filled in.
- **M** = Y − done − 1 (everything neither finished nor the phase in hand).

X (a phase's written number) and the done-count are **independent** — a phase can
be finished out of order, so do not assume everything numbered below X is done;
report both from the file. Use plain words: say "done," never "landed," "latched,"
or "of Y phases in the roadmap." A worked shape, numbers adding up (7 + this one +
6 = 14): *"Next is **Phase 8 of 14** — 7 done, 6 to go after this: Logger level
filtering & message formatting."*

---

## § Presenting approaches

Use this format only for **approaches** — genuine forks where the answer is
taste or is expensive to reverse (test framework/runner, suite strategy,
phase ordering, whether to rewrite weak tests grading found). Do not use it
for **findings** — verdicts derived from evidence, such as a weak-test
classification or a mock classification. A finding is a fact, not a choice;
asking a human to vote on a fact is the wrong tool. Findings are fixed
inline or recorded to the ledger, not put on a menu.

**Assume the developer does not know this repository.** The person answering
may have opened it for the first time this session — legacy and unfamiliar
code is the case the skill exists for. So every menu must be answerable with
no prior knowledge of the codebase: state each option in plain terms, and
make the recommendation strong enough to follow blind, justified by what
*this repo actually is* — its detected stack, its existing tests, its
structure — not by generic preference. A developer who knows nothing about
the code should be able to take the recommended option and be right.

Every menu, presented unbatched, contains:

- **Each option**, with **pros and cons** stated for it — in terms a
  newcomer to this codebase can weigh, not insider shorthand.
- A **recommendation**, with the **reason** grounded in what was detected in
  *this* repo — not a bare pick and not a generic default.
- A final **deep dive** option — internally, dispatch a subagent to run an
  adversarial pushback against the presented options; but *present it to the
  developer in plain words*, e.g. *"dig deeper: have a second, skeptical pass
  challenge these options and check whether there's a better one."* It
  challenges the menu itself and surfaces any better option it missed, before
  committing to any one on it. All menu text follows `§ Talking to the
  developer`.

**One decision at a time, unbatched.** A menu asks for exactly one decision.
Do not bundle a second question onto it ("...and while we're at it, do you
also want X?") — it splits the developer's attention and degrades the
answer to both. Resolve the decision in front of them, act on it, and only
then raise the next one. A follow-up that is fully determined by the answer
just given is not a second decision — it does not need its own menu, just do
it.

**Collect the answer with a single blocking question, not a prose menu you
merely intend to stop after.** After presenting the menu above, make exactly
**one** structured-question call — your harness's picker — carrying **one
question**, the options as short labels (the deep-dive included). It blocks
until the developer answers, and that is what actually enforces *wait for the
answer* and *one at a time*: a prose menu you only plan to stop after does not
stop the run — a blocking question does. Never carry more than one question in
that call, and never move the menu's substance (pros and cons, the
recommendation, the deep-dive rationale) into it — that stays in the prose above;
the picker is only the selector. A picker made to carry the whole menu is what
batched decisions as tabs and dropped three of the four required parts; a
one-question selector does neither.

If an external `pushback` skill is available in the environment, offer it
as an addition to the adversarial-review option above — never as a
replacement for it and never as a requirement to proceed. This skill is
self-contained by design (Inviolate #6): the menu format works with or
without that skill present, on any agent.

---

## mode `critique-plan`

An adversarial pass against the plan Stage 3 just drafted — the skill
critiquing its own work product before committing to it. Findings from this
pass are **fixed inline, in the draft, on the spot**. No separate report is
written; there is nothing here to hand off, only phases to rewrite or drop
before Stage 5 writes them to disk.

This is not the approach-menu format above — nothing in this mode asks the
developer to choose anything. It is the plan's author, one pass later,
reading the draft adversarially.

### The governing rule

> **Every phase must name the bug it would catch.** If a phase cannot
> answer *"what breakage makes these tests go red?"*, it is not a phase —
> it is coverage theater. Rewrite it or drop it.

This is a gate, not a suggestion. A phase that fails it does not proceed to
Stage 5 in its current form — it gets a `Catches:` line that names an
actual breakage, or it does not survive the draft. Walk every phase in the
draft and ask the question explicitly; do not let a phase through on the
strength of a plausible-sounding title. "Billing retry & dunning
integration tests" is not, on its own, an answer — *"a retry exhausting
without transitioning the account to dunning; a partial refund
double-crediting"* is (see *Phase format*'s worked example). If the answer
you write down is vague enough that it would survive being attached to any
phase in any codebase, it has not actually named a bug — push until it
names the specific breakage this phase's tests would turn red for.

### Supporting rules

**1. No success criterion may reduce to "the command exits 0."** A green
exit code and a covered line are never evidence a test is good — a test
that asserts nothing produces both, forever, including against code that is
obviously broken. This is verified tool behavior, not a hypothetical:

- `go test ./...` exits 0 against a package with no `_test.go` files.
- `jest --passWithNoTests` exits 0 with zero tests collected.
- A fully-skipped test file exits 0 in every runner examined.

Check every phase's stated success criterion against this. If a phase's
definition of done is satisfied by any of the three cases above — or by
anything with the same shape, a run that can complete having asserted
nothing — rewrite the criterion to name the specific assertion that would
have to hold, tied to the bug the phase names it would catch. Coverage
percentage is not evidence either, for the same reason: it is legitimate
only for finding code with no test at all, never for certifying that a
covered line is well-tested.

**2. Every test double and fixture is classified, and every `scaffold`
mock names the refactor that retires it.** Walk the ledger entries the
draft plan touches — new mocks and fixtures a phase would introduce, and
existing ones a phase's tests would exercise. Each must carry one of the
three classes: `boundary` (a genuine external edge — network, clock,
randomness, a third-party API), `scaffold` (exists only because the
surrounding code resists testing — test debt), or `data` (constructed test
data that is permanent and correct — a seeded account, a factory-built
order). An unclassified double is a gap in the draft, not a detail to leave
for later. A `scaffold` mock additionally names the refactor that will
retire it; that refactor is **deferred** — named now, actually done when
that refactor is actually planned, not during this pass. When in doubt
between `boundary` and `scaffold`, or when a double goes unexamined,
default to `scaffold`: the harm is asymmetric, since a `scaffold` silently
promoted to permanent hides test debt, while a `boundary` mistakenly
recorded as `scaffold` only leaves a note someone later deletes.

**3. Legacy phases characterize current behavior; they do not fix it.**
Where a phase's tests would pin behavior of existing code, check that
every assertion the phase proposes matches what the code *currently does*,
not what it *should* do. If the pass turns up behavior that looks wrong,
it is never turned into a different assertion or a code change.
Characterizing a legacy system and fixing its bugs are two different hard
problems; this pass exists to police the first, and to stop the second
from sneaking in disguised as a "more correct" assertion. A wrong-looking
behavior that clears the findings-log inclusion gate (`build-test-roadmap.md
§ The findings log`) is recorded there — verified, actionable, with the
phase that pins it — so the developer gets a real to-do list; one that
does not clear the gate is dropped, not written down as a vague note.

### After the pass

Every phase that survives can answer the governing rule's question in one
sentence, has every double and fixture it touches classified, and — where
it characterizes legacy code — pins current behavior only. Phases that
can't be fixed to clear this bar are dropped from the draft, not carried
forward with a note to "revisit."

If a `pushback` skill is available in the environment, running it in
addition to this pass is strictly additive and should be offered — it is
not required. This skill remains self-contained (Inviolate #6): the gate
above is complete on its own, on any agent, whether or not that skill is
present.

# Test theater — the weak-test catalog

Reference catalog, not a procedure. Stage 2 (*Grade*) subagents read a test
partition against this file and cite pattern names from it in their verdicts.
This file does not tell an agent how to run the grading pass — see the skill's
Stage 2 instructions for that — it only names what to look for and what
vocabulary to use when reporting it.

**Governing rule, stated explicitly because it is load-bearing for every
pattern below:** a passing command and a covered line are never evidence that
a test is good. Exit status proves the test executed without throwing.
Coverage proves a line ran while some assertion (anywhere in the test) held.
Neither proves the test would fail if the behavior it's supposedly pinning
changed. A test that never asserts on behavior can have 100% line coverage
and a green exit code forever, on every version of the code, including
versions that are obviously broken. Coverage is legitimate for one purpose
only — finding code with *no test at all* — and illegitimate for every other
purpose, including certifying that a test which does exist is worth keeping.

Every pattern entry below states, as a fixed field, **what breakage it lets
through** — the regression that could ship with this test green. A pattern
entry that cannot fill in that field is not a weak-test pattern; drop it
before it enters a verdict.

No pattern here is tied to a language, framework, or test runner. Examples
are illustrative pseudocode or named from more than one ecosystem; the
detector cues describe a *shape*, not a syntax, because the shape recurs
everywhere — a mock-echo in Python looks like a mock-echo in Go.

## Verdict shape

Per weak test found, a Stage 2 verdict is: `file`, `line`, **pattern name**
(one of the names below, verbatim), a one-sentence statement of why it fails
to catch regressions, and a suggested replacement. Pattern names are the
`##` headings below (e.g. `over-mocked test`, `happy-path-only test`) — cite
the heading text, not a paraphrase, so verdicts stay greppable across a
400-item list.

This catalog does not define severity or ranking; that's a Stage 2 concern
(top-K by severity, full list to `paad/test-roadmap/test-suite-analysis.md`). It only
defines what the items in that list are called and how to tell them apart.

---

## assertion-free test

**Detector cue:** the test runs code and then asserts only that *something
non-crashing happened* — a null/undefined check, a "no exception thrown," a
type check, an existence check — with no assertion on the *value* the
behavior was supposed to produce. `assert result is not None` is the
canonical case: it passes for a correct result, a wrong result, and often a
partially-constructed garbage result, as long as it isn't literally absent.

**What breakage it lets through:** any regression that changes the *content*
of the result without making it disappear. A billing calculation that starts
returning the wrong total, a parser that returns a malformed-but-non-null
tree, a lookup that returns the wrong record — all pass.

**What a real replacement asserts instead:** the specific expected value, or
a specific expected shape with field-level checks (`total == 4200`, not
`total is not None`), tied to a case the phase names as worth pinning.

---

## snapshot-only test with no meaningful invariant

**Detector cue:** the test captures whatever the code currently outputs
(a serialized blob, a rendered string, a full object dump) and asserts
future runs match that capture byte-for-byte, with no comment or companion
assertion identifying *which part* of the snapshot is the behavior under
test. Passes on regeneration. The developer's daily move when it goes red is
to regenerate it, because nothing in the test tells them what would make a
diff meaningful versus noise.

**What breakage it lets through:** none, technically — the first time it's
run against a regression it *will* go red. The breakage is procedural, not
detectable-in-code: the fix-it reflex for a failing snapshot is
"regenerate," and a snapshot with no named invariant gives a developer under
time pressure no way to tell a real regression from formatting churn before
regenerating over it. In practice this pattern converts a real regression
into a silently accepted new baseline.

**What a real replacement asserts instead:** either a narrower assertion on
the specific field/property that matters, or a snapshot scoped tightly
enough (a single computed value, not a whole rendered page) that any diff is
self-evidently the thing under test, plus a comment naming the invariant the
snapshot exists to protect.

---

## tautological assertion

**Detector cue:** the test configures a mock or stub to return a value, then
asserts that calling the mocked function returns that value. The assertion
checks the test double's own configuration, not the code under test. A
variant: asserting a spy was called with exactly the arguments the test just
constructed and passed in, with no check on what the code *did* with the
result of that call.

**What breakage it lets through:** every regression in the actual code path
under test, because the assertion never touches the real code — it touches
the double. The code under test could be deleted entirely, calling the mock
directly, and the assertion would still pass.

**What a real replacement asserts instead:** an assertion on what the
*production code* did as a consequence of the mock's return value — the
downstream state change, the value it computed from the mock's output, the
side effect it triggered — not the mock's return value reflected back at
itself.

---

## exit-status-or-coverage-as-evidence

**Detector cue:** a test file, script, or CI step whose only success
criterion is "the process exited zero" or "coverage stayed above N%," with
no assertions inside the run that check specific behavior — e.g. a
smoke-test script that just imports every module and exits, or a coverage
gate treated as the definition of "tested."

**What breakage it lets through:** any regression that doesn't throw and
doesn't touch the covered-vs-not-covered boundary — which is most
regressions. A function can be exercised (covered) by a test that asserts
nothing, return a wrong value, and both the exit code and the coverage
report stay green.

**What a real replacement asserts instead:** behavior-level assertions
inside the run itself. Coverage output is retained only as a *gap-finder* —
"this branch has zero coverage, so nothing pins it" — never cited as
evidence that a covered branch is adequately tested.

---

## over-mocked test

**Detector cue:** enough of the collaborators around the unit under test are
mocked or stubbed that the mocks, taken together, implement the behavior the
test claims to verify. A giveaway: reading the test requires reading the
mock setup to know what the "real" answer would be, because the mock *is*
the answer. Another: a mock stands in for a same-process collaborator
(another class in the same codebase) rather than a genuine external edge.

**What breakage it lets through:** any regression in the mocked
collaborator's real behavior, and any regression in how the unit under test
actually integrates with that collaborator (wrong method called, wrong
argument order, wrong error handling on a real failure the mock is never
configured to produce).

**What a real replacement asserts instead:** depends on what the mock is
standing in for — this is exactly the `boundary` / `scaffold` / `data`
distinction from the test-double & fixture ledger, and grading should
classify the mock accordingly, not just flag the test:

- If the double stands in for a genuine external edge (network, clock,
  randomness, a third-party API) — a `boundary` — keep the double, but move
  the assertion to what the unit under test *does* with the boundary's
  response, including its failure modes, not to the boundary's own
  configured return value.
- If the double exists only because the surrounding code resists testing
  any other way — a `scaffold` — the replacement isn't a better mock, it's
  the refactor that retires the mock. Name that refactor in the verdict;
  don't paper over test debt with a more elaborate double.
- If what's being asserted on is actually constructed test data (a seeded
  record, a factory-built object) rather than a stand-in for behavior, it
  isn't this pattern at all — classify it `data` and grade it as a real
  fixture, not a mock.

---

## happy-path-only test

**Detector cue:** every test in a phase's partition exercises the success
case — valid input, available dependency, expected response — and none
exercises the boundary the phase names as the reason the phase exists: the
error path, the timeout, the malformed input, the concurrent write, the
retry-and-give-up. The phase's own stated purpose (see *Phase format* and
Inviolate #5 — every phase names the bug it would catch) is checkable
against what's actually asserted; a happy-path-only suite for a phase titled
around retry/dunning behavior, timeout handling, or invalid-input rejection
is a mismatch between the phase's stated purpose and its tests.

**What breakage it lets through:** any regression in error handling,
retries, timeouts, validation, or edge-of-range input — which for
integration and e2e tiers is frequently the entire reason the phase was
planned. A payment-retry phase with only successful-charge tests catches
nothing about the retry logic it exists to pin.

**What a real replacement asserts instead:** at least one case per boundary
the phase names — the declined charge, the expired token, the dependency
that's down, the input one past the valid range — asserting the specific
handling behavior (retried once then surfaced, rejected with a specific
error, degraded to a specific fallback), not merely that an exception was
thrown.

---

## Using this catalog

A test can match more than one pattern at once (a tautological assertion
inside an otherwise happy-path-only phase is both, and both survive in the
verdict). Matching zero patterns is not a certificate that a test is good —
this catalog names known-weak shapes; it is not exhaustive, and a test
absent from every pattern here still has to actually assert on behavior the
phase cares about. Absence of theater is necessary, not sufficient.