{"slug":"audit-tests","title":"audit-tests","summary":"Audit whether tests earn their maintenance cost and which suite owns each contract. Use when pruning redundant tests, investigating implementation coupling or test-only production hooks, reviewing the value of proposed coverage, or auditing an entire subsystem. Use write-tests to","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-10-14T21:49:24.065359Z","repo":{"url":"https://github.com/dzhng/skills","stars":1333,"forks":81,"license":"MIT","updatedAt":"2026-10-14T20:39:18Z"},"bodyHtml":"<hr>\n<h2>name: audit-tests\ndescription: Audit whether tests earn their maintenance cost and which suite owns each contract. Use when pruning redundant tests, investigating implementation coupling or test-only production hooks, reviewing the value of proposed coverage, or auditing an entire subsystem. Use write-tests to implement the resulting test changes.</h2>\n<h1>Audit Tests</h1>\n<p>Audit <strong>independent proof</strong>: what bug would escape if this test disappeared?\nThe result is a map of contracts to their surviving tests, with evidence for\nwhat to keep, repair, combine, or remove. Fewer tests is not the objective;\nless maintenance for the same or better confidence is.</p>\n<p><a href=\"../write-tests/SKILL.md\">write-tests</a> owns how to write and falsify a test.\nThis skill owns whether that test adds proof and where that proof belongs.\n<a href=\"../review/SKILL.md\">review</a> owns closing out the resulting changes.</p>\n<h2>Workflow</h2>\n<ol>\n<li><strong>Bound the audit.</strong> Read the repository and scoped instructions, then pick\na production responsibility. Find its tests, fixtures, shared suites, and\nCI selection rules. Record the starting revision, local changes, and test\nresults before editing. Distinguish existing failures from checks that\ncould not run. For a focused audit, choose a few substantiated candidates;\nfor a subsystem sweep, use the completeness rules below.</li>\n<li><strong>Trace the claim.</strong> Read each candidate in full, including parameter rows,\nbeside the production entry point and the code it reaches. Inspect callers,\nsibling implementations, overlapping coverage, and the history explaining\nwhy the test exists. Inspect dependency code or types when the claim rests\non dependency behavior. Write down what the assertions actually detect,\neven when the test's name promises something else.</li>\n<li><strong>Record a disposition before editing.</strong> Use the evidence record below.\nShare findings as they become clear; the workflow must not hold back a\nmaterial discovery. An audit-only request ends with this evidence and a\nproposed batch, not with deletions.</li>\n<li><strong>Choose the surviving owner.</strong> Look across the records for whole redundant\nlayers, not just individual weak assertions. Name the suite that will own\neach contract and every unique case it must absorb. Prefer a real path with\ncontrolled external dependencies over replaying internal collaborators.\nAnother layer earns coverage only for a failure the chosen owner cannot\nexpose, such as routing or lifecycle behavior.</li>\n<li><strong>Change one coherent boundary.</strong> When implementation is in scope, carry\nunique cases into the surviving suite using <a href=\"../write-tests/SKILL.md\">write-tests</a>\nbefore removing their old home. Extend existing cases and fixtures instead\nof cloning setup. Remove newly unneeded test hooks and support code in the\nsame batch; verify production consumers before deleting an export, wrapper,\nreset function, global, or injection parameter. Update test discovery and\nCI registration when moving suites.</li>\n<li><strong>Prove preservation.</strong> Compare the removed assertions with the survivors,\nlooking for lost contracts and assertions that cannot fail for the intended\nreason. Falsify repaired or transferred regression coverage using\n<a href=\"../write-tests/SKILL.md\">write-tests</a>. Isolate temporary production changes,\nrestore the original bytes without disturbing other work, and confirm the\ntargeted proof passes. Report baseline failures separately; investigate any\nnew failures. Run owner and sibling suites, then required repository checks.\nWhen replacing static checks, exercise the actual command or dry-run whose\ncontract they claimed. Keep files stable while a runner reads them.</li>\n<li><strong>Close the batch.</strong> Run <a href=\"../review/SKILL.md\">review</a>, then reconcile the\nevidence record with what shipped. Report unresolved failures and unrun\nchecks explicitly. A clean smaller suite does not prove removed coverage\nwas redundant; the contract map and preservation evidence do.</li>\n</ol>\n<h2>Evidence record</h2>\n<p>For each candidate, record its location and name, actual detectable failure,\nreason for existing, production consumers of any supporting hook, and decision:</p>\n<ul>\n<li><strong>Keep:</strong> identify the independently protected contract. A file move alone\ndoes not change this decision.</li>\n<li><strong>Repair:</strong> preserve the contract but replace an assertion that is weak,\nmisleading, or attached to the wrong boundary.</li>\n<li><strong>Combine:</strong> identify the destination suite and the unique cases it must\nprove before the original goes away.</li>\n<li><strong>Remove:</strong> identify stronger surviving proof, or demonstrate that there is\nno contract to protect. Suspicion of duplication is insufficient.</li>\n</ul>\n<p>Include the supporting source/history evidence, cleanup unlocked, risk, and\nvalidation command. Missing evidence stays an open question, not a deletion.</p>\n<h2>False confidence</h2>\n<p>Use these as investigation prompts, not automatic removal rules:</p>\n<ul>\n<li><strong>Circular evidence.</strong> Expected output comes from the same helper being\ntested; a fixture copies an inventory from production; a test compares a\nvalue with itself. Find an expectation independent of the implementation.</li>\n<li><strong>An exercise without a claim.</strong> Executing lines, copying identities, or\nlisting exports may raise coverage without detecting a credible defect.\nDetermine whether execution itself enforces anything meaningful.</li>\n<li><strong>The harness does the job.</strong> A mock implements the promised behavior, a\nfixture supplies the acknowledgement or ordering production must generate,\nor a persistence assertion reads a store the real path never writes. Using\none identical fake for different dependency APIs can hide their differences.</li>\n<li><strong>The wrong cause goes red.</strong> A negative case hits a different guard or never\nreaches the intended branch. Verify the failing condition, not merely that\nsomething rejected the input. Check that every parameter row reaches its claim.</li>\n<li><strong>Declarations posing as behavior.</strong> Source greps, import lists, capability\nflags, and internal call shapes can certify the declaration while the user\npath is broken. Trace the behavior the declaration promises.</li>\n<li><strong>One proof repeated.</strong> Private helper tests, per-provider replays of shared\nlogic, and multiple layers exercising the same regression can all protect\none contract. Keep distinct transport and lifecycle risks, not copies of the\nsame scenario under different filenames.</li>\n<li><strong>Tests keeping code alive.</strong> A helper or production escape hatch exists only\nbecause tests call it. Move proof to the real entry point and remove the\nabandoned mechanism; do not replace it with another test-only abstraction.</li>\n</ul>\n<h2>What earns retention</h2>\n<p>Behavior includes compatibility and operational contracts: API and SDK shape,\nwire bytes, prompts, defaults, configuration, migrations, storage, security,\nplatform behavior, generated cross-language agreement, package contents,\nrelease rules, and architectural boundaries. A static assertion can be their\ncheapest independent proof. Observable ordering also belongs here.</p>\n<p>Ask whether the test survives an internal rename while detecting a broken\nexternal promise. Exact bytes may be essential to a protocol; exact private\nidentifiers usually are not. Neither slowness nor resemblance to implementation\nis sufficient evidence for deletion. Do not add refactor-sensitive coverage\nwhen the real contract can be exercised directly.</p>\n<p>An existing failure may be a product defect. Reproduce and investigate it;\nnever remove a valid claim to make the audit green. Repair in-scope defects\nseparately, with a failing control and passing fix on the same harness. Record\nunrelated defects as follow-ups and keep their coverage.</p>\n<h2>Whole-subsystem sweeps</h2>\n<p>Scale the accounting before scaling the edits:</p>\n<ul>\n<li>Inventory every owned test, including cases in shared suites and integration,\nQA, or live harnesses. Assign each to one production responsibility, even\nwhen file prefixes disagree. Record baseline outcomes and test/support size.</li>\n<li>Give every declaration a disposition and evidence. A parameter table can\nshare one record only if all rows deserve the same decision. Then make a\nsecond pass across the records to choose surviving suites and retired layers.</li>\n<li>Delegate read-only discovery by responsibility when agents are available.\nKeep edits to shared fixtures under one owner. Each batch must introduce no\nunexplained failures; tracked baseline failures remain visible for repair.</li>\n<li>Use an independent reviewer to compare removed coverage with its new owners.\nResolve every reported gap with restored proof or source evidence rejecting\nit. Demonstrate that each restored contract catches a deliberate defect.</li>\n<li>If the base advances, inspect new cases in any file being removed and carry\ntheir contracts forward. Follow the repository's integration policy, then\nrerun the subsystem and required integration checks on the resulting revision.\nRefresh discovery before starting the next batch.</li>\n</ul>\n<h2>Done</h2>\n<p>Hand off the contract map, decisions and reasons, production simplifications,\nvaluable tests deliberately retained, preservation gaps and their resolution,\nand verification actually run. For broad sweeps, include before/after size for\nproduction, tests, and support separately, plus remaining batches and integration\nstate. Size describes the outcome; it never sets the deletion target. Claim a\ncomplete subsystem audit only when the inventory has no unexamined entries.</p>\n","files":[{"path":"SKILL.md","sizeBytes":9423,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-10-14T21:49:33.25104Z","sha256":"C1906D462AC9082D7A84AA82949EAD9EDEC552C4EAFA10A9AEA074E5DB24E1DF","sizeBytes":4127},"review":null,"source":{"repositoryUrl":"https://github.com/dzhng/skills","path":"skills/engineering/audit-tests","license":"MIT","commit":"f2fbf92f131fb08bbf3e400a1ef99b7be85bb2bc","subtreeSha":"61DF21BC468BB0210867B5AA901E6FCD054633BC67E01CCEDFF28DFE236266DB","lastSyncedAt":"2026-10-14T21:49:23.275319Z"},"reviewedAt":"2026-10-14T21:50:00.836709Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/dzhng/skills/tree/main/skills/engineering/audit-tests"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install dzhng-skills@llmmart"},{"target":"git","command":"git clone https://github.com/dzhng/skills.git"}]}