# Anti-Patterns: Extended Notes

Detail offloaded from the SKILL.md Anti-Patterns section. The symptom/fix one-liners live inline; this file holds the mechanics, root-cause narratives, and fix ladders.

## Persistent test infrastructure state contamination

**Root cause:** persistent test infrastructure -- a long-running `docker compose up`, a shared local database, a volume left between iterations -- accumulates state across test runs. The current run's data sits on top of the previous run's data; assertions counting rows or jobs see the sum. The numbers look like a code bug ("the loop runs N times instead of once"), but they are clean integer multiples of the expected value, and the same test passes in CI on a fresh container.

**Fix ladder**, in order of preference:

1. **Ephemeral containers per test session** (`testcontainers`, `pytest-postgresql`, or `docker compose run --rm <service>` for one-shot runs) -- slowest to start, strongest isolation. Default for CI.
2. **Fixture-driven `TRUNCATE` / `DROP DATABASE`** in a session-scoped or per-test fixture -- fast, but requires careful coverage of every stateful table.
3. **Volume teardown between iterations** (`docker compose down -v` before each run) when running locally -- manual but reliable.

Never rely on tests "cleaning up after themselves." If a previous run errored mid-test, the cleanup didn't run, and the next run inherits the partial state.

## Synchronous adapters hide timing-dependent races

**Wire-latency mechanics:** a test fires two or more parallel requests through a mock/adapter that resolves synchronously (a promise that settles in the same microtask, an in-memory fake with zero latency) and asserts a coalescing/dedup/single-flight guard held. It passes -- but only because every call observed the shared in-flight state before any reset ran. Under real wire latency the staggered arrivals miss the window, and the guard spawns N operations instead of one.

Same-tick microtask concurrency is not a proxy for production burst behavior. For dedup/coalescing logic, inject controllable latency (fake timers, deferred resolution staggered across ticks) so a later arrival lands after the reset, and assert the guard holds for arrival-staggered bursts.

## Constructing the object-under-test below the layer that transforms it

**Extended rationale:** when a fix guards or transforms a field in an upstream layer (a parser, normalizer, `from_api_response` constructor, serializer) and the test builds the object directly via the leaf constructor -- `Model(field=x)`, `new T(...)`, the raw initializer -- the test injects the already-correct value. The upstream strip/transform never runs, the guard never fires, and the test is green while production is still broken. The test cannot fail for the exact bug it was written to catch.

Enter through the same entry point production uses. If a test must construct the leaf form directly for other reasons, it is not covering the transform; add a separate test that feeds the pre-transform input (the raw API payload, the unparsed dict) so the transform under test executes.

# False-Pass Oracle Traps

Assertion oracles that can report success without observing failure, moved from the SKILL.md Anti-Patterns section.

## Piping a command into `grep -q` to assert on its output

**Symptom:** the assertion reports "absent" for a string plainly present in a manual run. Under `set -o pipefail` the pipeline's status is the *writer's*: `grep -q` exits on first match and closes the pipe, so the producer dies of SIGPIPE and the pipeline fails **because the assertion matched**. Independently, a command that legitimately exits non-zero -- a refusal path, a status code that is part of the contract -- fails the pipeline regardless of the match. Either way the false negative reads as a behavioral finding and sends you into the production code.

**Fix:** capture, then match. `out=$(cmd 2>&1)` on one line, `grep -q 'needle' <<<"$out"` on the next; never put the command and the matcher in one pipeline. Two adjacent shapes to avoid in assertions: `grep -c` prints `0` *and* exits 1, so `grep -c p f || printf 0` emits `0\n0`; and `cmd && x || y` is not if-then-else -- `y` also runs when `x` fails.

## A comparison oracle that fails open

**Symptom:** the harness compares two producers with `diff -q <(producer_a | filter) <(producer_b | filter)`. `diff` observes the streams, not whether either producer succeeded -- two failed producers yield two empty streams and compare equal. Comparing only added lines has the same hole: two deletions of *different* content both produce an empty `^+` stream, and identical file and line counts do not mean identical content.

**Fix:** capture each producer to a temporary file and check its exit status before comparing. Compare both added and removed hunk bodies from a zero-context diff, not summaries or diffstats. Classify a producer failure, a binary or metadata-only patch, or a comparison I/O error as *undecidable*, never as *identical*.

## A green suite over a feature the test environment disables

**Symptom:** The code path that would fail is behind a config or environment flag that defaults off, and the test environment sets no override. Every test exercising the affected object passes, including tests written for it, and the failure appears on the first write in an environment where the flag is on. Grepping the repository reinforces the wrong conclusion, because the enabled value lives in deployment configuration (a task definition, a parameter store), not in the codebase -- the only value in the tree is the `false` default.

**Fix:** Before reading a pass as coverage, check whether the flag gating the consumer that would fail is on under test. Re-run one existing test with the flag forced on, alongside a test touching only unaffected objects as a control, so a failure is attributable to the flag rather than to the environment change. When the disabling guard keys on the test runner itself -- an environment variable the runner sets -- no in-suite test can observe the behavior at all. Verify that surface out of band with a standalone process using production configuration and a capturing transport, and treat every in-suite assertion about it as vacuous until one has been shown able to fail.

## An expected-output pattern that ends in a bare wildcard

**Symptom:** the expectation matches the run's real output, and then a trailing wildcard absorbs whatever the harness appends after an abnormal exit. A run that prints the expected line and immediately crashes passes.

**Fix:** anchor the expectation to the real last line. Plant the failure -- kill the process after the matched line -- and confirm that expectation fails. Prefer a greedy wildcard to a lazy one before an end anchor: a lazy quantifier can exhaust the engine's backtrack budget, and most engines report that exhaustion as an error the harness reads as "no match".

## A conformance harness that normalizes before comparing

**Symptom:** the spec suite compares through a normalizer that ignores insignificant formatting, so a defect living inside that class passes identically with and without the bug. "652/652 against the spec" is a claim about the normalized form, not about bytes.

**Fix:** assert against the spec's literal output wherever consumers care about bytes. When a byte-exact test disagrees with an upstream green suite, suspect the normalizer before suspecting the test.

## Gating a test lane on a grep of the runner's summary

**Symptom:** `runner | tee out; grep -q '^Tests failed *: [1-9]' out` goes green when the runner colorizes its output and the anchor never matches, and again when the runner crashes before printing a summary at all. `rc=$?` after the pipe is `tee`'s status, not the runner's.

**Fix:** read `${PIPESTATUS[0]}` and gate on it. Corollary: a historically green lane is not evidence the suite passed, so a tightening fix that turns a lane red is evidence the old gate was masking.

## A readiness predicate whose alternation admits a mid-stream match

**Symptom:** a background producer is waited on with a condition that ORs the real completion marker against something cheap -- `until [ -s "$OUT" ] && grep -qiE 'done marker|^\s*$' "$OUT"; do sleep 15; done`. The `^\s*$` branch matches the first blank line, which the producer emits before it answers anything, so the wait reports ready on poll #1. The half-written artifact then reads as a *wrong answer* rather than a visibly incomplete one: a tail lands in the middle of the producer's own echoed input, which files as a defect report about the tool when the truth was a read that landed a few kilobytes short of the end of the write. Nothing about the output's appearance separates the two.

**Fix:** match a marker that cannot appear mid-stream -- the final result block, never a whitespace pattern and never a token the stream can print more than once -- and confirm the producer has exited before reading anything it wrote. A tail is evidence about the file; only the process table is evidence about completion. Prefer the harness's own completion signal to a hand-rolled poll, since the hand-rolled loop is what gets reached for when the wait has to happen inside one turn, and that is exactly when the predicate goes sloppy.

## A GNU-only matcher inside a cross-OS assertion

**Symptom:** `grep -qP` as an `if` condition fails open on BSD grep -- there is no `-P`, exit 2 reads as "pattern absent", and the check passes. The same gap points the other way in a summary parser: `grep -oP` errors, the parsed count defaults to zero, and the "suite not effective" branch fires on a green suite.

**Fix:** parse with POSIX awk, and gate on exit status before interpreting any text.

## An oracle newer than the supported floor

**Symptom:** the API under test is version-gated, but the helper the test calls to compute the expected answer is not. The suite is green on the development version, and that green gets read as "supported across the range" -- on the oldest supported build the file fails to load, so the case never runs and never reports, or reports a load error nobody traces back to the oracle.

**Fix:** express the oracle in the oldest supported dialect, and run any new test once against a floor build before pushing.

## A smoke input that does not exercise a budget

**Symptom:** a trivial payload passes under a size, time, or token limit that production-shaped input exhausts, so the limit is never reached and the green run gets read as capacity.

**Fix:** probe with input of the real shape and size against the real limit, and parse the result rather than the exit status -- a truncation signal alongside empty content is the signature.

## Retiring a test suite on a similar test count

**Symptom:** a suite is rewritten in another runner and the migration is declared done because the counts match. Parameterized cases collapse many legacy assertions, and a translated expectation can faithfully repeat its source's mistake.

**Fix:** keep the old suite frozen as an independent oracle until four gates pass: every legacy assertion or named section maps to a collected replacement contract; both suites run against the same built artifacts and are compared on exit codes, raw bytes, file modes, and artifacts rather than summaries; focused mutations of fail-closed boundaries make *both* suites fail on the intended assertion and go green again after cleanup; and the replacement passes serially, concurrently, and in randomized order. Never change a product expectation while translating it -- record the discovered defect separately and land its regression with the product fix.

# Generated-Corpus Techniques

Cases enumerated by a generator rather than typed by hand, offloaded from the SKILL.md Writing Good Tests section. Reach for these where the behavior under test emerges from a whole table, or where the change is mechanical and the existing suite is the wrong instrument.

## A hand-picked corpus for emergent-precedence behavior

**Symptom:** the behavior emerges from a whole table -- phrase lists, route priorities, rule sets -- but the cases are typed by hand, so they test the author's model of the table rather than the table. Every case the author did not think of is a precedence interaction nobody has seen.

**Fix:** build the corpus as the cross-product of the axes the table enumerates, drive it through the real matcher against both the old and the new table, and diff the outcomes. Grouping the analysis on the hand-picked axis re-imposes the same blind spot the corpus was built to remove. Sample a real corpus wherever one exists, and keep the generated diff as the review artifact rather than a prose summary of it.

## Proving a mechanical refactor behavior-preserving with a generated transcript

**Symptom:** a mechanical refactor is declared safe because the suite is green. The suite covers what someone thought to test, and a mechanical refactor can move anything else -- including the surfaces nobody wrote a case for.

**Fix:** enumerate the public surface by reflection, call each entry with a per-type pool of edge values varying one parameter at a time, and print one deterministic line per call: return value, warning, exception. Run that against both revisions and diff the transcripts. Rebuild fixtures before every call, so a mutated fixture does not read as a behavior change. Require every surviving difference to map to an intended change, and treat an unexplained difference as the finding rather than as transcript noise.

# Isolation and Sandbox Traps

Checks whose result is decided by the harness, the sandbox, or the surrounding environment rather than by the code under test, moved from the SKILL.md Anti-Patterns section.

## Bounding a containerized command with an outside `timeout`

**Symptom:** `timeout N docker exec <container> <test command>` returns 124 and the run looks bounded. `timeout` signalled the client, not the process inside the container, which keeps running -- and an orphaned test holds its transaction or lock, so every later run against the same database hangs for reasons unrelated to the code under test. The symptom looks like the *new* tests hanging.

**Fix:** put the timeout inside the container (`docker exec <container> timeout N <cmd>`) or bound the runtime itself. After any aborted containerized run, sweep for orphans (`ps -eo pid,etime,args` inside the container) before trusting the next result -- an `etime` far larger than any run you know about is the tell. Better still, assert the function returns within the test rather than racing a wall clock against a suspected infinite loop.

## The harness sandboxes the subject in the primitive under test

**Symptom:** A probe about commit durability runs under a transaction-wrapping test trait (Laravel `RefreshDatabase`, Rails `use_transactional_tests`, a pytest rollback fixture). The `COMMIT` under test becomes a savepoint, the write never becomes durable, and "did the row survive the failure?" is unanswerable -- but the probe still returns a plausible, well-formed answer. Sometimes there is a loud tell (on Postgres, `SQLSTATE 25P02` refusing every later statement); when the injected failure does not poison the connection, or the assertion reads state captured before the failure, there is no tell at all, and a wrong commit-ordering result gets quoted downstream as measured evidence.

**Fix:** Before trusting a probe, ask what the harness wraps the subject in and whether that is the same mechanism the question is about. Drop the trait for that one probe and assert a control proving the removal happened -- print the transaction nesting depth and require 0. Without the control the probe is unfalsifiable: a nested run and a clean run produce output of the same shape. Same test for a filesystem probe run under a chroot of the path under test, or any sandbox built from the primitive being measured.

## Global before/after snapshots as an isolation check

**Symptom:** the isolation fixture hashes an entire shared directory, ledger, or registry before and after the run and requires it unchanged. The assertion cannot name a writer, so any legitimate concurrent process flips it and the test implicates itself. Retrying to green then hides a real fixture leak exactly as easily as it hides the unrelated write.

**Fix:** assert on a per-run canary instead -- a unique token or uniquely-named artifact only this run can produce. Assert it lands in the disposable location and that the shared one holds zero copies. Verify both directions: an unrelated mutation elsewhere in the shared tree must leave the fixture green. If a leaked canary must be cleaned up, remove that one derived path, never sweep the shared directory.

## Asserting how a stream was chunked

**Symptom:** a fixture logs one line per read, so the expectation pins where the kernel happened to split the stream. The identical bytes delivered in different segments fail it, and a re-run does not clear it, because segmentation is decided by the pipe and the scheduler rather than by the code. The tell is expected and actual holding the same bytes redistributed, with exactly one test failing.

**Fix:** assert on the reassembled payload. Where segmentation is itself the contract, drive it deliberately through an injected transport that emits chosen boundaries.

## A killed worker with no failure summary

**Symptom:** a parallel run ends on a killed process and broken-pipe writes and prints no summary at all. Two multi-gigabyte tests were scheduled concurrently on a lane with few workers and the box ran out of memory; nothing in the output names either test.

**Fix:** find the offenders by allocation size -- grep the suite for large literals -- and mark them mutually exclusive through the runner's conflict mechanism. Re-running does not fix a scheduling-dependent ceiling, it reshuffles it.

## A stubbed-dependency fixture is an allowlist

**Symptom:** a harness copies the subject into a sandbox and stubs everything it calls. It works only for the dependency set that existed when it was written: add a step without adding its fixture line and the run dies on a missing file, with a generic harness error that names the harness rather than the new step.

**Fix:** add the stub in the same commit as the step it supports, and run the harness before committing.

## Relocating an environment variable and calling it isolation

**Symptom:** the suite points `HOME`, `TMPDIR`, or a state-root variable at a temp directory and concludes the code under test is sandboxed. The relocation only redirects paths derived from that variable *at call time*; anything anchored to the executable's own location, a compile-time constant, or the current uid still resolves to the real one. The fixture is green either way, because reading the real file usually succeeds -- the tell is absent by construction.

**Fix:** enumerate the anchors (the running binary's neighbours, baked build-time paths, uid-derived paths, literal `/tmp/` prefixes) and pin each with a planted decoy plus a positive control. Run the suite a second time against *installed* binaries in a real install layout: a build-tree-only run cannot observe any defect whose trigger is the install layout, and no amount of fixture review closes that gap.

# oracle smells

### An aggregate assertion that names no offender

**Symptom:** `assert all(r.returncode == 0 for r in results)` renders as a bare `assert False`. The failure names neither the failing item nor its message, so diagnosing it costs an extra cycle re-running under a patched assertion.

**Fix:** collect the offenders and assert on the list -- `assert [r for r in results if r.returncode != 0] == []` -- so the output carries identity and error text. Same rule for regression pins: pin the specific offending names, not a count. A defect of this class re-enters as a different plausible-looking value, and a count notices nothing.

### Persistent test infrastructure state contamination

**Symptom:** Integration tests fail with row-count multipliers (expected 2 rows, got 8) yet pass on a fresh container -- persistent infrastructure kept state from prior runs. **Diagnostic shortcut:** a clean integer multiple (2x, 3x, 4x...) between expected and actual means state contamination, not a logic bug -- logic bugs rarely produce uniform multipliers across unrelated assertions.

**Fix:** Reset infrastructure state between runs -- ephemeral containers, fixture `TRUNCATE`, or volume teardown (ladder in the reference); never rely on tests "cleaning up after themselves."

Isolation and sandbox traps -- containerized-timeout leaks, a harness sandboxing the subject in the primitive under test, global before/after snapshots, relocated environment variables, assertions pinned to stream chunking, an out-of-memory kill that prints no failure summary, and stub fixtures that quietly become allowlists -- are in [isolation-and-sandbox-traps.md](./isolation-and-sandbox-traps.md).

### Vacuous forall over an empty collection

**Symptom:** A `forall`-style assertion (`every`, `all`, `.iter().all()`) passes vacuously -- the factory never attached children, and every such operator returns `true` over an empty collection.

**Fix:** Attach a realistic child set and confirm the predicate flips for at least one populated case.

**One spelling of a construct is not coverage of the construct.** Where a parser routes two accepted spellings of one thing through different callbacks -- a self-closing tag and its bare form -- a counter built on those callbacks can balance for one spelling and not the other, while the test pinning whichever spelling the author typed reads as "this class is handled". Enumerate the spellings the real producer emits.

### Constructing the object-under-test below the layer that transforms it

**Symptom:** The fix lives in an upstream transform (parser, normalizer, `from_api_response`), but the test builds the object via the leaf constructor with the already-correct value -- the transform never runs; green test, broken production.

**Fix:** Feed the test the raw pre-transform input (API payload, unparsed dict), never the leaf constructor, so the transform under test executes.

### Synchronous adapters hide timing-dependent races

**Symptom:** Parallel requests through a zero-latency mock settle in the same microtask, so a dedup/coalescing guard passes -- under real wire latency, staggered arrivals miss the window and spawn N operations.

**Fix:** Inject controllable latency (fake timers, staggered deferred resolution); assert the guard holds for arrival-staggered bursts, not just same-tick ones.

**Reproducing the race deterministically.** Spawning N processes, or sleeping between the two steps, hits the window intermittently, and a fixture that fails two runs in six reads as flake and gets retried away. Release N threads from a single barrier so every participant enters the window on the first round, against a freshly-cold resource each round. Where the race spans an external boundary, use a deterministic hook between the two operations rather than a timing sleep. Validate with a mutant: with the guard removed the fixture must fail every run, not most of them.

### Asserting only presence, never absence

**Symptom:** Payload/serializer tests assert expected fields exist but never that unexpected fields are absent -- a field leaking into a reused builder (CREATE vs UPDATE) passes every existing test.

**Fix:** Where a field set is a contract, pin absence as well as presence: `assert "proof_document_id" not in payload`.

### Looping cases inside one test method

**Symptom:** a table of cases iterated inside a single method. Every reset the framework provides -- transaction rollback, container rebinding, fake state -- is scoped to the method, so cases 2..N run against case 1's leftovers. A uniformly passing run is the shape that hides it.

**Fix:** one case per method, with the control in its own method. Where a loop is unavoidable, assert first on a field that must differ between iterations, and print one identifier the loop did not set -- a repeat of that value is the tell.

### Negative assertions left behind by a relocated observable

**Symptom:** a refactor moves the layer an observable lives at. The positive assertions go red and get fixed; the negatives (`assertNotSent`, "no row written") now hold unconditionally and pass forever, including on the day the guard breaks.

**Fix:** after any such move, sweep the inverted direction: enumerate the subjects the new layer handles, grep every negative assertion naming them, and re-prove each against a planted violation. Confirm the relocated layer is the only path to the observable before calling a hit vacuous.

### A precondition supplied by the fixture, with no production actor

**Symptom:** `setUp()` establishes something nothing in production does -- a seeder that never runs on a deployed environment, a factory default that steers away from the overloaded enum member, a hand-built relation.

**Fix:** for every fixture step, name the production actor that performs the same write, and what request #1 sees if nobody does. An idempotent backfill is not a production write path.

### A bound the regression still satisfies

**Symptom:** `assertLessThanOrEqual(N, queryCount)` with N at or above the unfixed count. It stays green when the fix is reverted, and it also passes at 0.

**Fix:** assert the exact optimized value, or seed two input sizes and assert invariance across them -- the only shape that separates O(1) from O(n). A count invariance proves constant statements, never constant work, so assert a resolved value alongside it.

### A mutation whose application was never asserted

**Symptom:** the mutation never landed (quoting, a wrong anchor, no interpreter inside the container) or landed and tested a different proposition (a body retyped from memory relocates sequenced work; branch structure derived from a filtered view). Either way the run reports PASS, and a broken mutation reads as a credible criticism of someone else's suite.

**Fix:** produce the mutated tree from the ref itself (`git show <ref>:<path>`), assert the token occurs exactly once before writing, and assert both sides after -- new-only token absent, old-only token present -- with fixed-string matching anchored by line content. Keep the assertion on the same side of any container boundary as the edit, and give a zero-valued assertion its own control. Make the restore oracle the artifact (a checksum against `git show`), not a pattern match. Read the file-granularity pass list of a known-bad control: a file green in both runs is a layer that cannot see this class. A passing mutation is a validity signal before it is a coverage finding, and a mutation proves a line is load-bearing, never why.

### Proving a new assertion fires, when the claim was that it was missing

**Symptom:** the coverage gap is argued by mutating the code and watching the new assertion go red. That proves the new assertion has power, not that the pre-fix suite would have missed the defect.

**Fix:** run the same mutation at the pre-fix commit. Only a pass there establishes the gap.

### A fallback test with no guard on the primary path

**Symptom:** the test exercises a retry or fallback only while the primary path still fails. Once upstream is fixed the fallback never runs, and the test keeps passing for a reason it was not written for.

**Fix:** pair it with a guard test asserting the unguarded call still raises the specific error the fallback exists to absorb.

### Two guards with the same observable outcome

**Symptom:** the input trips a header check and a body-parse check alike, so the test proves only that something rejected it -- delete the guard under test and the assertion still holds.

**Fix:** satisfy every guard except the one under test, and assert the specific exception type rather than the shared status code.

### A "string X must not appear in the output" test that contains X

**Symptom:** anything capturing source context -- tracebacks with surrounding lines, error trackers attaching frame locals -- copies the test file into its own output, so the probe reports its own literal and inverts the verdict.

**Fix:** load the needle from a data file, `grep -c` the probe's own source for it and require zero, and assert on structure where possible (the frame's variable map is empty) rather than on a substring.

### A skipped test is green

**Symptom:** a case that skips because its feature, extension, or capability is missing from the build reports as success in every summary the suite prints.

**Fix:** assert that the specific test reported PASS, not that the suite exited zero. Enable whatever the harness helper itself needs -- a helper can pull in unrelated capabilities that each skip for their own reason.

False-pass oracle traps -- the `grep -q` pipefail trap, comparison oracles that fail open, feature-flag-disabled coverage illusions, retiring a suite on count alone, expectations ending in a bare wildcard, conformance harnesses that normalize before comparing, lane gates built on a summary grep, GNU-only matchers in cross-OS assertions, oracles newer than the supported floor, smoke inputs that never reach a budget, and readiness predicates satisfied by a mid-stream match -- are in [false-pass-oracle-traps.md](./false-pass-oracle-traps.md).

# Rationalization Table

Load this reference when you catch yourself arguing against writing a test. Each rationalization below is the excuse; the reality column is why the excuse is wrong.

| Rationalization | Reality |
|----------------|---------|
| "This is too simple to need tests" | Simple code still breaks. Tests document expected behavior. |
| "I manually tested it" | Manual testing is ephemeral — it can't be re-run, it proves nothing to the next person |
| "Tests will slow me down" | Debugging without tests slows you down more. Tests catch bugs at write time instead of production. |
| "I'll add tests later" | Later never comes. The context you have now is gone later. |
| "The tests would just test the framework" | Then you're not testing your logic. Find the logic and test that. |
| "It's just a refactor, behavior didn't change" | Run the existing tests. If they pass, you're done. If none exist, this is exactly when to add them. |
| "100% coverage is overkill" | Nobody said 100%. But 0% is negligence. Test the important paths. |
| "Mocks are faster" | Mocks are faster to run and slower to maintain. They test assumptions, not behavior. |
| "I already wrote the implementation" | Sunk cost. Tests written after pass immediately and prove nothing about the original bug. |
| "The test is too hard to write" | Hard-to-test code signals a design problem. Simplify the interface, not the test. |
| "I need to understand the code first" | Write the test to express what you expect. The test IS your understanding, made executable. |
| "This is a prototype / throwaway" | Prototypes become production code. Every time. The test costs 5 minutes now vs. hours debugging later. |
| "The deadline is too tight for tests" | The deadline is too tight to debug without tests. Tests catch bugs at write time, not in production under deadline pressure. |

# regression proof

## Red-Green-Refactor (When It Applies)

Tests-first answer "what should this do?"; tests-after answer "what does this do?" -- tests written after implementation are biased toward verifying what was built, not what's required. For bug fixes, the failing test first proves the bug exists and the fix works; for new features, the order matters less than the quality.

### Bug fixes: prove-it pattern

1. Write a test that reproduces the bug
2. **Run it and watch it fail** -- confirm it fails for the right reason. A test that fails due to a typo or import error hasn't captured the bug. The failure message should describe the buggy behavior.
3. Apply the fix
4. **Run it and watch it pass** -- confirm the fix addresses the specific failure AND other tests still pass. A fix that breaks something else isn't a fix.
5. If the test passes immediately without a fix, the test is verifying existing behavior, not the bug. Go back to step 1.

**Absence and isolation assertions need a manufactured red phase.** A test that asserts something did *not* happen has no bug in hand to fail against, so it goes green on day one and stays green every day after, including the days the guard is broken. Supply the missing red step: plant exactly the violation the assertion forbids, using a value only this run could produce (a run-unique token, a uniquely-named artifact), and confirm *that specific* assertion fails -- not merely that some assertion fails. Then remove the plant and watch it pass. A fixture that cannot observe the behavior under test passes vacuously in both directions, and nothing else in the suite will notice.

**The manufactured red phase needs its own two guards.** A negative control that stubs enforcement, rebuilds, and watches the fixture fail is evidence only if the stub applied *and* the artifact was built from it: a scripted replace that matches nothing returns the input unchanged with no error, and a build queued behind another on the same lock can publish an artifact from pre-stub source. Assert the edit changed the file, print an applied-marker the control can read back, and run the control only once that specific build invocation's own exit status is in hand -- a timestamp proves staleness in one direction and nothing in the other. The tell is an implausible pass, not an error.

### New features: test alongside

Write tests alongside the implementation, not after. By the time the feature is done, tests exist and pass -- whether a test was written 5 minutes before or 5 minutes after the code matters less than whether it exists and is good.

**Minimum viability during green phase:** When making a test pass, write the simplest code that satisfies it -- not the abstraction that seems "right," not the feature that might be needed next. Refactor only after the test is green.

# test completion

## When Stuck

| Stuck on... | Do this |
|-------------|---------|
| Don't know how to test | Write the assertion first (desired outcome), then build the test around it |
| Test too complicated | Simplify the interface being tested |
| Must mock everything | Code is too coupled -- use dependency injection |
| Test setup too large | Extract helpers that reduce noise without hiding test intent (see DAMP). Still complex? Simplify the design |

## Rationalization Table

If about to skip, defer, or argue against writing a test for any reason, STOP and load [rationalization-table.md](./rationalization-table.md) first. Thirteen common excuses with their counter-truths. When arguing against writing a test, the argument is probably lost.

## Verify

Before considering tests complete:

- [ ] Every new public function/endpoint has at least one test
- [ ] Each test has a descriptive name stating expected behavior
- [ ] Tests use real objects where possible (mocks only at system boundaries)
- [ ] Edge cases covered (empty, null, boundary, error paths)
- [ ] Each acceptance criterion has a discriminating case a naive wrong implementation would fail
- [ ] Every absence or isolation assertion was proven able to fail -- the forbidden violation was planted with a run-unique value and that specific assertion failed
- [ ] Tests assert on outcomes, not implementation details
- [ ] Snapshot, golden, fixture, and generated-expectation changes were reviewed semantically rather than regenerated to obtain green
- [ ] Tests are independent -- no shared mutable state between tests. If tests pass individually but fail together, use bisection to find the polluter (run one-by-one in isolation until the offending test is found)
- [ ] Tests run fast enough to run frequently (< 30 seconds for unit suite)
- [ ] Bug fix tests reproduce the original bug
- [ ] Mutation check run: mentally mutate the code (wrong constant, flipped branch, dropped side effect, empty/default return) and confirm some test fails for each

## Integration

This skill covers generic test discipline. For framework-specific patterns, conventions, and tooling:

- **Laravel/PHP** → `ia-php-laravel` (PHPUnit, factories, feature/unit split, facade faking, data providers)
- **React/TypeScript** → `ia-react-frontend` (Vitest, RTL, component/hook patterns, Playwright E2E, mocking patterns)

When both are active, framework-specific guidance takes precedence for tooling and conventions.

# test design

## Core Principle

Tests prove behavior works. A test that can't fail is worthless. A test that tests mocks instead of real code is theater.

## Discover the Test Setup First

Before writing the first test, establish what this repository actually runs. Reaching for a default command is how a suite goes green locally and red in CI.

- **Runner and its config**: whichever manifest and test-config file the project's ecosystem uses. Framework-specific detail belongs to the language skills listed under Integration.
- **The checked-in wrapper over any global binary.** A globally installed binary routinely resolves to a different version than the project pins, so prefer the project-local invocation (`uv run pytest` over bare `pytest`, `vendor/bin/phpunit` over `phpunit`).
- **Focused vs. full invocation**: the edit loop needs to run one file or one test; completion needs the whole suite. Learn both forms.
- **Where tests live and how neighbouring test files are named** -- match the existing convention rather than importing one.
- **The command CI gates on** (`.github/workflows/*.yml`). When CI and the README disagree, CI is authoritative.

## Writing Good Tests

### One behavior per test

Each test should verify exactly one thing. If the test name needs "and" in it, split it into two tests.

```
Good:  "creates user with valid email"
Good:  "rejects user with duplicate email"
Bad:   "creates user and sends welcome email and updates counter"
```

### When trivial code earns a test

Getters, constructors, constants, and pass-through wrappers earn a test only if they validate, normalize, default, derive, enforce, or carry a side effect -- otherwise assert the first consumer-visible result that depends on them.

### Derive test cases from three sources

Build test coverage from three independent sources and verify every item maps to at least one test:

1. **User requirements** -- what was requested (spec, issue, conversation)
2. **Features implemented** -- what the code actually does (scan the diff)
3. **Claims in the response** -- what is about to be reported to the user as working

Anything in any source with no corresponding test is a coverage gap -- implemented-but-untested features, claimed-but-unverified behavior.

For each acceptance criterion, include at least one discriminating case that a naive wrong implementation would fail. Prefer the negative, boundary, or state-transition case that separates the intended contract from a hard-coded happy path. Do not add a meaningless negative-case quota when one strong case already distinguishes the behavior.

**Make the fixture adversarial on the axis under test.** Realistic sample data carries globally unique ids, distinct values, and non-overlapping keys -- which is exactly what lets a wrong implementation pass. If the contract is a composite key, build a fixture where every child id collides across parents; if it is ordering, give every item the same timestamp. The right fixture is the one a naive implementation cannot survive, not the one that looks most like production. The default fixture also tends to select the container's fast internal representation -- sequential keys, short strings, and small maps take a packed or inline layout, so the restructure or rehash path never executes. When a change touches a container's internals, build one case whose keys, size, or contents force the alternate representation, and confirm it fails without the fix.

**An assertion of absence is discriminating only once it has been made to fail.** A test asserting that nothing was written to the shared path, nothing leaked into the production channel, or the fallback was never taken passes identically whether or not the guard works. Plant the forbidden violation and watch that specific assertion fail before trusting it (mechanics under Red-Green-Refactor).

For each source, enumerate user journeys ("As a [role], I want to [action], so that [benefit]") and generate test cases from each, so tests cover user-visible behavior rather than implementation details.

### Differential-fuzz anything that must byte-match another implementation

Hand-written cases pick round values and miss the format's conditional branches. Generate a few thousand values per type shape from the reference implementation itself, compare the two outputs, and include nested and composite shapes -- a fast path for leaf values inherits every scalar bug it delegates to. Commit a representative slice as fixed cases, and re-run the full sweep on every change to either side.

Two longer generation techniques -- building a corpus as the cross-product of a table's axes instead of hand-picking cases, and proving a mechanical refactor behavior-preserving with a reflection-driven transcript -- are in [generated-corpus-techniques.md](./generated-corpus-techniques.md).

### DAMP over DRY in tests

Each test should be independently readable without chasing shared setup through helpers. Duplication in tests is acceptable -- even desirable -- when it makes intent obvious at a glance. Extract shared setup only when it reduces noise without hiding what the test does.

### Test pyramid

For API/web projects, aim for ~80% unit / ~15% integration / ~5% E2E; adjust for risk profile (data pipelines may need heavier integration, CLI tools minimal E2E).

- **Unit**: fast, isolated, one behavior per test, no database/network/filesystem -- the cheap, fast-feedback foundation.
- **Integration**: verify component boundaries against real dependencies (real test database, wired services, queue producer + consumer) -- catch the wiring bugs mocks hide.
- **E2E**: critical user paths through the real system only (signup, checkout, core workflow) -- every E2E test must justify its maintenance cost.

### Name tests by expected behavior

The test name should describe what happens, not what's being called.

```
Good:  "returns 404 when user does not exist"
Bad:   "test getUserById"
Good:  "sends notification after order is placed"
Bad:   "test processOrder"
```

### Use real objects when practical

Mocks should be a last resort, not a first choice. Every mock is an assumption about behavior that may drift from reality.

| Use real objects for | Use mocks/fakes for |
|---------------------|---------------------|
| Database queries (use test DB) | External HTTP APIs |
| Internal services and classes | Payment gateways |
| File system operations (use temp dirs) | Email/SMS delivery |
| Business logic and transformations | Third-party SDKs with rate limits |

**Exception: framework-provided test doubles.** Framework faking mechanisms (Laravel `Queue::fake()`/`Event::fake()`, React test providers, `vi.mock` for API layers) are idiomatic and maintained alongside the framework -- use them. The rule targets hand-rolled mocks that drift, not framework-blessed utilities.

**Where to cut the mock seam.** When a mock is warranted (the right column above -- external APIs, gateways, delivery services, rate-limited SDKs), place it at the last point owned code touches the unowned resource: mock the payment-client wrapper, not `fetch`; the mailer adapter, not the SMTP transport. Mocking below the wrapper re-implements the third party's behavior inside the test suite and leaves the wrapper's own logic untested. Database queries stay on the left column -- a real test DB, not a mocked repository.

### Tests expose bugs, not the reverse

If a test uncovers broken or buggy behavior, fix the source code -- never adjust the test to match incorrect behavior. A test that passes against a bug is worse than no test at all.

### Test edge cases

For every feature, consider:

- Empty input / null / undefined
- Boundary values (0, 1, max, max+1)
- Invalid types (string where number expected)
- Concurrent access (if applicable)
- Error paths (network failure, timeout, permission denied)
- Unicode and special characters in string inputs

### Silent failure coverage

Tests must detect silent failures, not just happy paths. For every code path that catches, logs, or short-circuits on error, add an assertion that proves the failure was observable. Hunt targets during test writing:

- **Empty catch blocks** (`try { ... } catch {}`) — trigger the error; assert the logger (or equivalent signal) received the original exception.
- **Swallowed rejections** (`.catch(() => [])`, `.catch(() => null)`) — trigger the rejection; assert the caller sees a distinguishable signal (specific return value, logged error, re-thrown).
- **Converted errors** (`catch (e) { return defaultValue; }`) — assert the return value AND that the error was recorded where an operator can find it.
- **Missing async handling** — assert a rejected promise inside the function surfaces as a failure, not just an unhandled-rejection warning.
- **No rollback around transactional work** — assert a mid-transaction failure leaves no partial state (row counts match, queue unchanged).
- **Correlated fallbacks feeding an aggregate** — make every item's dependency fail at once and assert the summary reports *unavailable*, not a clean 0% or 100%. A type-valid placeholder (a neutral verdict, a default score) left in the denominator turns a total outage into a confident, precise, entirely wrong number, and it degrades toward a value that reads as real signal. Assert the unavailable state reaches every surface a human reads, including the one-line summary.

Assertion pattern: instead of `expect(result).toBe(null)` (which passes for both "handled gracefully" and "silent drop"), prefer `expect(logger.error).toHaveBeenCalledWith(expect.any(DatabaseError))` — make the observable signal part of the contract.

# test smells

## Anti-Patterns

Extended rationale, fix ladders, and mechanics for the longer items: [anti-patterns-extended.md](./anti-patterns-extended.md).

### Reaching for a default test command

**Symptom:** the bare global runner passes locally, while CI invokes the project-pinned wrapper and fails on a different dependency set or a different runner entirely.

**Fix:** Establish the runner, the checked-in wrapper, and the CI command before writing tests (see "Discover the Test Setup First").

### Host-local wrappers inside tracked test scripts

**Symptom:** a checked-in test script invokes a tool that exists only on the author's machine -- an agent shell wrapper, a personal alias, a locally-installed helper. On a bare CI runner, in a container, or on a colleague's machine every otherwise-correct assertion fails before reaching the code under test.

**Fix:** a tracked test is a portable artifact. Use ordinary POSIX tools inside it and apply any local wrapper to the *outer* invocation instead. Declare genuinely required non-standard dependencies in CI configuration, and grep the test tree for local wrappers before enabling a hosted gate.

### Testing mock behavior instead of real behavior

**Symptom:** Test passes but production breaks. Tests assert that mocks were called correctly, not that the actual system works.

**Fix:** Replace mocks with real objects for internal code (see "Use real objects when practical").

### Sleeping instead of waiting on a condition

**Symptom:** `sleep(2)` / `setTimeout` / `time.sleep()` before asserting on async work. A sleep is a race condition with a timer attached: too short flakes under load, long enough is wasted wall-clock in every run forever.

**Fix:** Wait on the observable condition with a deadline -- poll for the record, the event, or the state change (framework helpers: `waitFor`, `assertEventually`, polling with timeout). The deadline bounds the wait; the condition ends it. A sleep placed to *reproduce* a race is the same mistake pointed the other way -- see "Synchronous adapters hide timing-dependent races" for the barrier form. Write the readiness predicate so it cannot match mid-stream: an alternation that ORs the real marker with a cheap one (a blank line, a token the producer can print more than once) is satisfied on the first poll, and the half-written artifact then reads as a wrong answer rather than an incomplete one.

### Asserting elapsed wall-clock time

**Symptom:** the test calls the real timer and asserts `now() - started >= 100`. That tests the runtime clock and scheduler, not the code's delay policy -- millisecond rounding reports 99 on a run that plainly took longer, and a re-run goes green without any code change.

**Fix:** inject the sleep boundary and assert the policy: the exact delay requested, the cap applied (6000 becomes 5000), and the ordering (the wait resolves before the dependent call). Keep a real-timer test only where integration with the runtime timer is itself the contract, and then use a monotonic clock with a documented tolerance, never a one-millisecond lower bound.

### Re-running a flaky test to green

**Symptom:** A test fails intermittently and the response is re-run until it passes. Each re-run silences a detector -- the flake is a real race, ordering dependency, or shared-state bug in the test or the code.

**Fix:** Treat flaky as red: fix it now, or skip it visibly with a reason and an owner (a linked issue, a named TODO) so it cannot quietly rot. Never leave it in the suite passing-by-retry.

### Test-only methods in production code

**Symptom:** Methods like `reset()`, `clearState()`, `setTestMode()` that exist only because tests need them.

**Fix:** If tests need to reset state, the code has a design problem. Refactor to make state explicit and injectable.

### Snapshot tests as the only test

**Symptom:** All tests are snapshots that get bulk-updated whenever anything changes.

**Fix:** Snapshots catch unintended changes but don't verify correctness. Add behavioral assertions alongside snapshots.

### Change detector

**Symptom:** the test fails only when an intentional decision changes -- a constant's value, exact wording, private structure -- so it fires on every redesign and sleeps through real bugs.

**Fix:** assert the consumer-visible outcome the decision drives, not the decision's literal value -- same fix as Implementation-echo assertions: assert the consumer-visible outcome.

### Regenerating expected output to obtain green

**Symptom:** A snapshot, golden, fixture, or generated expectation is replaced wholesale after a failure, with no review of what behavior changed.

**Fix:** Treat expected-output changes as specification changes. Inspect the semantic diff, explain why the new output is intended, and verify the behavior with an independent assertion or exercised entry point. Follow any repository-specific approval marker for golden changes. If the implementation is wrong, fix the implementation instead of regenerating the oracle.

### Testing the framework

**Symptom:** Tests verify that the ORM saves records, the router routes requests, or the framework does what its docs say.

**Fix:** Trust the framework. Test the project's own logic -- the business rules, transformations, and decisions the code makes.

### Incomplete mocks

**Symptom:** Mock only includes the fields the test author knows about. Downstream code consumes other fields and gets undefined.

**Fix:** Mock the COMPLETE data structure as it exists in reality -- check what fields the real API/type contains and include everything consumed downstream. Prefer real objects or factory fixtures with all fields populated; if mocking is unavoidable, generate from the real type/schema.

### Mocking without understanding

Before mocking any method, ask: (1) What side effects does the real method have? (2) Does this test depend on any of those side effects? (3) Mock at the lowest level that removes the slow/external part -- not higher.

### AI-generated test smells

LLM-written tests (including self-written) fail in predictable ways. **Before committing, scan every test for these six smells:**

- **Mock of the system under test** — mocking the very function being tested, so the test asserts what the mock returned. Always a mistake. Delete the mock; call the real function.
- **Circular assertion** — computing the expected value the same way the code computes the actual value (`expect(sum(a,b)).toBe(a+b)`). The test passes even when both are wrong. Replace with a hand-computed expected value or a known fixture.
- **Snapshot of unreviewed output** — first-run snapshot committed without reading it. The snapshot enshrines whatever the code happened to emit, bugs included. Hand-write the first snapshot or diff it line by line before accepting.
- **Assertion-free exercise** — test calls the function, checks nothing, passes because nothing threw. Every test needs at least one `expect(...)` / `assert ...` tied to the behavior under test.
- **Over-broad matchers** — `expect(result).toBeTruthy()` on a function that returns an object. Passes for `{}`, `true`, `"anything"`, all equally. Pin to the specific shape.
- **Implementation-echo assertions** — `expect(repo.save).toHaveBeenCalledTimes(1)` when the real contract is "the user exists in the database afterward." Assert on outcomes (row exists, response body contains expected fields), not call counts or internal method invocations.