# Testing Agents With Subagents — Templates, Examples, and Error Handling

## Phase 1: Test Plan Template

```markdown
## Test Plan: {agent-name}

**Agent Purpose:** {what the agent does}
**Agent File:** agents/{agent-name}.md
**Date:** {date}

**Test Cases:**

| ID | Input | Expected Output | Validates |
|----|-------|-----------------|-----------|
| T1 | {input} | {expected} | Happy path |
| T2 | {input} | {expected} | Error handling |
| T3 | {input} | {expected} | Edge case |
```

## Phase 1: Subagent Dispatch Template

```
Task(
  prompt="""
  [Test input for the agent]

  Context: [Any required context]

  {Include the actual problem/request the agent should handle}
  """,
  subagent_type="{agent-name}"
)
```

## Phase 1: Verbatim Result Capture Template

```markdown
## Test T1: Happy Path

**Input:**
{exact input provided}

**Expected Output:**
{what you expected}

**Actual Output:**
{verbatim output from agent — record verbatim}

**Result:** PASS / FAIL
**Failure Reason:** {if FAIL, exactly what was wrong}
```

## Phase 2: Failure Severity Triage

| Severity | Description | Priority |
|----------|-------------|----------|
| Critical | Agent produces wrong answers or harmful output | Fix first |
| High | Agent missing required output sections | Fix second |
| Medium | Agent formatting or structure issues | Fix third |
| Low | Agent phrasing or style inconsistencies | Fix last |

## Phase 2: Root Cause → Fix Approach

| Failure Type | Fix Approach |
|--------------|--------------|
| Missing output section | Add explicit instruction to include section |
| Wrong format | Add output schema with examples |
| Missing context handling | Add instructions for handling missing info |
| Incorrect classification | Add calibration examples |
| Hallucinated content | Add constraint to only use provided info |
| Agent asks questions instead of answering | Provide required context in prompt or add default handling |

## Phase 2: Fix Log Template

```markdown
## Fix Log

| Iteration | Change Made | Tests Passed | Tests Failed | Action |
|-----------|------------|-------------|-------------|--------|
| 1 | Added output schema | T1, T2 | T3 | Continue |
| 2 | Added error handling instruction | T1, T2, T3 | — | Green |
```

## Phase 3: Edge Case Categories

| Category | Test Cases |
|----------|------------|
| Empty Input | Empty string, whitespace only, no context |
| Large Input | Very long content, deeply nested structures |
| Unusual Input | Malformed data, unexpected formats |
| Ambiguous Input | Cases where correct behavior is unclear |

## Phase 3: Test Report Template

```markdown
## Test Report: {agent-name}

| Metric | Result |
|--------|--------|
| Test Cases Run | N |
| Passed | N |
| Failed | N |
| Pass Rate | N% |

## Verdict
READY FOR DEPLOYMENT / NEEDS FIXES / REQUIRES REVIEW
```

---

## Examples

### Example 1: Testing a New Reviewer Agent
User says: "Test the new reviewer-security agent"
Actions:
1. Define 6 test cases: 2 real issues, 2 clean code, 1 edge case, 1 ambiguous (RED)
2. Dispatch subagent for each, capture verbatim outputs (RED)
3. Fix agent definition for any failures, re-run all tests (GREEN)
4. Add edge cases (empty input, malformed code), verify consistency (REFACTOR)
Result: Agent passes all tests, report documents pass rate and verdict

### Example 2: Testing After Agent Modification
User says: "I updated the golang-general-engineer, make sure it still works"
Actions:
1. Run existing test cases against modified agent (RED)
2. Compare outputs to previous baseline (RED)
3. Fix any regressions introduced by the modification (GREEN)
4. Test edge cases to verify robustness not degraded (REFACTOR)
Result: Agent modification validated, no regressions confirmed

### Example 3: Testing Routing Logic
User says: "Verify the /do router sends Go requests to the right agent"
Actions:
1. Define test cases: "Review this Go code", "Fix this .go file", "Write a goroutine" (RED)
2. Dispatch each through router, verify correct agent handles it (RED)
3. Fix routing triggers if wrong agent selected (GREEN)
4. Test ambiguous inputs like "Review this code" with mixed-language context (REFACTOR)
Result: Routing validated for all trigger phrases, ambiguous cases documented

---

## Error Handling

### Error: "Agent type not found"
Cause: Agent not registered or name misspelled
Solution:
1. Verify agent file exists: `ls agents/{agent-name}.md`
2. Check YAML frontmatter has correct `name` field
3. Restart Claude Code to pick up new agents

### Error: "Inconsistent outputs across runs"
Cause: Agent produces different results for same input
Solution:
1. Document the inconsistency — this is a valid finding
2. Add more explicit instructions to agent definition
3. Re-test consistency after fix
4. Determine if variation is acceptable (phrasing) or problematic (structure/findings)

### Error: "Subagent timeout"
Cause: Agent taking too long to respond
Solution:
1. Simplify test input to reduce processing
2. Check agent isn't in an infinite loop or excessive tool use
3. Increase timeout if agent legitimately needs more time

### Error: "Agent asks questions instead of answering"
Cause: Agent needs clarification that test input did not provide
Solution:
1. This may be correct behavior — agent properly requesting context
2. Update test input to provide the required context
3. Or update agent definition to handle ambiguity with defaults
4. Document whether questioning behavior is acceptable for this agent type

# Testing Patterns Reference

Detailed patterns for agent testing using subagent dispatch.

## Pattern 1: Dispatch and Capture

Basic pattern for running a single test case:

```
# 1. Dispatch agent with test input
Task(
  prompt="""
  Review this Python function for issues:

  ```python
  def get_user(id):
      return db.execute(f"SELECT * FROM users WHERE id = {id}")
  ```
  """,
  subagent_type="reviewer-security"
)

# 2. Capture output verbatim

# 3. Compare to expected output
Expected: CRITICAL SQL injection finding
Actual: {what agent actually returned}
```

## Pattern 2: Negative Testing

Verify agent handles invalid inputs correctly:

```
# Test with missing required context
Task(
  prompt="""
  Review this code.
  """,
  subagent_type="reviewer-security"
)

# Expected: Agent should request more context or handle gracefully
# NOT: Agent should hallucinate code to review
```

## Pattern 3: Consistency Testing

Verify agent produces consistent outputs:

```
# Run same input 3 times
for i in 1..3:
  Task(prompt=same_input, subagent_type=agent)
  capture output[i]

# Compare outputs
# Structure should be identical
# Key findings should match
# Minor phrasing variation acceptable
```

## Pattern 4: A/B Comparison Testing

Compare agent variants:

```
# Use agent-comparison skill for formal A/B testing
# Or simple side-by-side:

Task(prompt=test_input, subagent_type="agent-v1")
capture output_v1

Task(prompt=test_input, subagent_type="agent-v2")
capture output_v2

# Compare quality, structure, correctness
```

## Pattern 5: Routing Verification

Verify correct agent is selected:

```
# Test routing logic by examining which agent handles request
# Check routing metadata matches behavior

# Example: "Review this Go code" should route to Go expert
# Example: "Check security of this API" should route to security reviewer
```

## Test Scenarios

### Scenario: Testing New Agents

Verify a new agent produces correct outputs for its intended purpose.

| Category | Purpose | Example |
|----------|---------|---------|
| Happy Path | Agent handles ideal input correctly | Valid code for code reviewer |
| Error Cases | Agent handles invalid input gracefully | Malformed input, missing context |
| Edge Cases | Agent handles boundary conditions | Empty input, very large input |
| Output Schema | Agent produces expected structure | Required sections present |

### Scenario: Testing Skill Invocation

- Skill triggers on expected phrases
- Skill produces expected output format
- Skill handles errors appropriately
- Skill follows its documented workflow

### Scenario: Testing Agent Interactions

- Correct agent selected for request
- Agent handoffs work correctly
- No conflicts between parallel agents
- Aggregated results are consistent

### Scenario: Testing Error Handling

- Invalid input handling
- Missing context handling
- Tool failures (simulated)
- Timeout behavior

### Scenario: Testing Output Format

- Required sections present
- Correct markdown formatting
- Expected fields populated
- No placeholder text

## Minimum Test Cases by Agent Type

| Agent Type | Minimum Tests | Required Coverage |
|------------|---------------|-------------------|
| Reviewer agents | 6 | 2 real issues, 2 clean, 1 edge, 1 ambiguous |
| Implementation agents | 5 | 2 typical, 1 complex, 1 minimal, 1 error |
| Analysis agents | 4 | 2 standard, 1 edge, 1 malformed |

## Test Report Template

```markdown
# Agent Test Report: {agent-name}

**Date:** {date}
**Version Tested:** {agent version}
**Tester:** Claude Code (testing-agents-with-subagents skill)

## Summary

| Metric | Result |
|--------|--------|
| Test Cases Run | N |
| Passed | N |
| Failed | N |
| Pass Rate | N% |

## Test Results

### T1: {test name}
- Status: PASS/FAIL
- Input: {input}
- Expected: {expected}
- Actual: {actual}
- Notes: {any observations}

[Continue for all tests]

## Issues Found

1. {Issue description} - Severity: HIGH/MEDIUM/LOW

## Recommendations

1. {Specific fix needed}

## Verdict

READY FOR DEPLOYMENT / NEEDS FIXES / REQUIRES REVIEW
```

## Eval Harness Integration

For agents with YAML-based eval tasks, use the eval harness for automated multi-trial testing:

```bash
# List agents with eval tasks
python evals/harness.py skill-test --list-agents

# Run eval tasks for an agent (default: 3 trials per task)
python evals/harness.py skill-test python-general-engineer

# Run with more trials for higher confidence
python evals/harness.py skill-test python-general-engineer --trials 5

# Output in different formats
python evals/harness.py skill-test python-general-engineer --format json
python evals/harness.py skill-test python-general-engineer --format markdown

# Save results to file
python evals/harness.py skill-test python-general-engineer -o results/agent-test.md
```

**When to use eval harness vs manual testing:**

| Scenario | Approach |
|----------|----------|
| Agent has YAML eval tasks | Use `skill-test` command |
| Agent is new, no eval tasks yet | Manual testing with Task tool |
| Quick iteration during development | Manual testing |
| Pre-deployment validation | Use `skill-test` with `--trials 5` |
| Creating regression tests | Create YAML task, then use harness |

**Creating eval tasks for agents:**

Add task YAML files to `evals/tasks/{category}/` with `execution.agent` set to your agent name. See `evals/task_schema.yaml` for the full schema.

# Async Patterns Reference

> **Scope**: Async coordination patterns for Playwright tests — parallel execution, race conditions, and timing failure modes. Does not cover basic `await` usage or condition-based waiting (see `playwright-patterns.md`).
> **Version range**: Playwright 1.20+ (Node 18+)
> **Generated**: 2026-04-17

---

## Overview

Async bugs are the primary source of intermittent E2E failures. The three failure modes are: (1) missing `await` causes assertions to run before actions complete, (2) `waitForTimeout` hard-codes delays that break on slow CI, and (3) uncoordinated parallel requests cause race conditions. Playwright's `Promise.all` and `waitForResponse` patterns eliminate all three by synchronizing on observable events rather than time.

---

## Pattern Table

| Pattern | Version | Use When | Avoid When |
|---------|---------|----------|------------|
| `Promise.all([waitForResponse, click])` | `1.0+` | Action triggers network request | No network call involved |
| `page.waitForEvent('dialog')` | `1.0+` | Click opens browser dialog | Dialog never appears |
| `test.setTimeout(ms)` | `1.0+` | Single test needs extended timeout | All tests are slow (fix root cause) |
| `expect.poll()` | `1.38+` | Polling an external state change | Element is already in DOM |
| `Promise.race([success, error])` | Node | Two mutually exclusive outcomes | One outcome is always expected |

---

## Correct Patterns

### Parallel Request + Click Coordination

Clicking before registering the `waitForResponse` listener misses the response — it fires during the click, before the listener is set up.

```typescript
// Register listener BEFORE the action that triggers it
const [response] = await Promise.all([
  page.waitForResponse(r => r.url().includes('/api/save') && r.status() === 200),
  page.getByTestId('save-button').click(),
]);
const body = await response.json();
expect(body.id).toBeDefined();
```

**Why**: `waitForResponse` is a promise that resolves on the *next* matching response. If the click fires first, the response arrives before the promise is created and `.waitForResponse` hangs until timeout.

---

### Parallel Independent Actions in Setup

When setup requires multiple independent async steps, run them in parallel:

```typescript
test.beforeEach(async ({ page, request }) => {
  // Sequential (slow) — each waits for the previous
  // await seedUser(request);
  // await seedProducts(request);
  // await page.goto('/dashboard');

  // Parallel (fast) — all start at once
  await Promise.all([
    seedUser(request),
    seedProducts(request),
    page.goto('/dashboard'),
  ]);
});
```

**Why**: Sequential setup multiplies latency. Three 200ms API calls take 600ms sequentially vs 200ms in parallel.

---

### Polling External State with `expect.poll`

For state that changes outside the browser (background job, webhook, database update):

```typescript
// Playwright 1.38+
await expect.poll(
  async () => {
    const res = await request.get('/api/orders/123/status');
    return (await res.json()).status;
  },
  { timeout: 30_000, intervals: [1000, 2000, 5000, 10000] }
).toBe('completed');
```

**Why**: `expect.poll` retries the predicate on the given intervals without blocking the event loop between checks, unlike a sleep loop.

---

### Dialog Handling

Dialogs must be registered before the action that opens them, or they close the browser context:

```typescript
page.on('dialog', async dialog => {
  expect(dialog.message()).toContain('Are you sure?');
  await dialog.accept();
});
await page.getByTestId('delete-button').click();
await expect(page).toHaveURL('/items');
```

**Why**: An unhandled `alert`/`confirm` causes Playwright to auto-dismiss and log a warning. If the dialog is required for the flow, unhandled = broken test.

---

## Pattern Catalog

### Wait for Observable Events Instead of Timeouts

**Detection**:
```bash
grep -rn 'waitForTimeout' --include="*.ts" --include="*.spec.ts"
rg 'waitForTimeout\(' --type ts
```

**Signal**:
```typescript
await page.getByTestId('save-button').click();
await page.waitForTimeout(2000); // "give it time to save"
await expect(page.getByTestId('success-banner')).toBeVisible();
```

**Why this matters**: Hard-coded delays fail silently on slow CI (timeout too short) or waste time on fast machines (timeout too long). `waitForTimeout` is a symptom of not knowing what observable event to wait for.

**Preferred action**:
```typescript
await Promise.all([
  page.waitForResponse(r => r.url().includes('/api/save')),
  page.getByTestId('save-button').click(),
]);
await expect(page.getByTestId('success-banner')).toBeVisible();
```

**Version note**: `waitForTimeout` is deprecated in Playwright 1.39+ — the linter flags it as `no-wait-for-timeout`.

---

### Await Navigation After Click Actions

**Detection**:
```bash
grep -rn '\.click()$' --include="*.spec.ts"
rg '\.click\(\)[^;]' --type ts
```

**Signal**:
```typescript
page.getByTestId('submit').click(); // missing await
await expect(page).toHaveURL('/confirmation'); // races with navigation
```

**Why this matters**: Without `await`, the click fires but the test proceeds immediately. If the URL check runs before navigation completes, it sees the old URL and fails intermittently.

**Preferred action**:
```typescript
await Promise.all([
  page.waitForURL('/confirmation'),
  page.getByTestId('submit').click(),
]);
```

---

### Include Teardown in Every Async Fixture

**Detection**:
```bash
grep -rn "test\.extend" --include="*.ts" -A 10
rg 'test\.extend\b' --type ts -A 10
```

**Signal**:
```typescript
export const test = base.extend<{ db: Database }>({
  db: async ({}, use) => {
    const db = await Database.connect();
    await use(db);
    // Missing: await db.disconnect();  — resource leak
  },
});
```

**Why this matters**: Leaked connections accumulate across the test suite. In CI with 100 tests, this exhausts connection pools and causes later tests to fail with unrelated errors.

**Preferred action**:
```typescript
export const test = base.extend<{ db: Database }>({
  db: async ({}, use) => {
    const db = await Database.connect();
    try {
      await use(db);
    } finally {
      await db.disconnect(); // always runs, even if test throws
    }
  },
});
```

---

### Await Independent Elements in Parallel

**Detection**:
```bash
grep -rn '\.waitFor(' --include="*.spec.ts" -A 1
rg 'waitFor\(' --type ts --include="*.spec.ts"
```

**Signal**:
```typescript
await page.getByTestId('widget-a').waitFor({ state: 'visible' });
await page.getByTestId('widget-b').waitFor({ state: 'visible' });
await page.getByTestId('widget-c').waitFor({ state: 'visible' });
```

**Why this matters**: Each wait is sequential. If each element takes 500ms to render, this adds 1.5s of latency. Elements that load independently should be awaited in parallel.

**Preferred action**:
```typescript
await Promise.all([
  page.getByTestId('widget-a').waitFor({ state: 'visible' }),
  page.getByTestId('widget-b').waitFor({ state: 'visible' }),
  page.getByTestId('widget-c').waitFor({ state: 'visible' }),
]);
```

---

## Error-Fix Mappings

| Error | Root Cause | Fix |
|-------|------------|-----|
| `Timeout 30000ms exceeded waiting for...` | Hard wait wasn't long enough, or wrong observable | Replace `waitForTimeout` with event-based wait |
| `locator.click: Target closed` | Dialog auto-dismissed before handler registered | Register `page.on('dialog', ...)` before the action |
| `browserContext.storageState: Target page, context or browser has been closed` | Parallel tests sharing a single context instance | Use per-test context or `test.use({ storageState })` |
| `Error: page.waitForResponse: page closed` | Navigation completed before `waitForResponse` resolved | Wrap in `Promise.all` with the triggering click |
| `UnhandledPromiseRejection` in fixture | Missing `await` in fixture setup | Add `await` to all async calls inside `test.extend` |

---

## Version-Specific Notes

| Version | Change | Impact |
|---------|--------|--------|
| `1.39` | `waitForTimeout` deprecated | Linter rule `no-wait-for-timeout` flags usage |
| `1.38` | `expect.poll()` added | Use instead of `waitForFunction` for polling external state |
| `1.30` | `waitForResponse` accepts async predicate | Predicate can now `await response.json()` inline |
| `1.25` | Worker-scoped fixtures stable | Share expensive async setup (DB, auth) across tests in a worker |

---

## Detection Commands Reference

```bash
# Find all waitForTimeout usage (replace with event-based waits)
grep -rn 'waitForTimeout' --include="*.ts"

# Find unawaited clicks (potential race conditions)
grep -rn '\.click()$' --include="*.spec.ts"

# Find test.extend fixtures (review for missing teardown)
grep -rn 'test\.extend' --include="*.ts" -A 15

# Find sequential waitFor calls that could be parallelized
grep -rn '\.waitFor(' --include="*.spec.ts"
```

---

## See Also

- `playwright-patterns.md` — condition-based waiting, waitForLoadState, network request patterns
- `flakiness-triage.md` — diagnosing and quarantining intermittent failures

# Authentication Testing Reference

> **Scope**: Patterns for testing authentication flows in Playwright — multi-role state management, OAuth/SSO, session expiry, JWT, and RBAC. The basic `storageState` setup lives in `playwright-patterns.md`; this file covers advanced scenarios.
> **Version range**: Playwright 1.25+ (worker-scoped fixtures required for multi-role)
> **Generated**: 2026-04-17

---

## Overview

Auth testing fails in two common ways: (1) tests share auth state across roles and pollute each other, and (2) tests log in via the UI for every test, spending 70%+ of runtime on login screens. The correct model is: authenticate once per role via the API, save the resulting `storageState`, and inject it into tests that need it. Session expiry, RBAC, and OAuth flows each require separate patterns.

---

## Pattern Table

| Pattern | Version | Use When | Avoid When |
|---------|---------|----------|------------|
| `globalSetup` + `storageState` file | `1.0+` | Single user role, full suite | Multiple roles needed |
| Worker-scoped fixture per role | `1.25+` | Multiple roles, parallel workers | Single role only |
| `request.post('/api/login')` in setup | `1.0+` | App has API login endpoint | UI-only login required |
| `page.addInitScript` for JWT injection | `1.0+` | JWT stored in localStorage | Cookie-based auth |
| `page.route` to mock OAuth callback | `1.0+` | Testing OAuth-gated pages | Testing the OAuth provider itself |

---

## Correct Patterns

### Multi-Role Auth via Worker Fixtures (Playwright 1.25+)

Use one fixture file for all roles. Each fixture authenticates once per worker, not once per test:

```typescript
// fixtures/auth.ts
import { test as base, expect } from '@playwright/test';

type AuthFixtures = {
  adminPage: Page;
  viewerPage: Page;
};

export const test = base.extend<{}, AuthFixtures>({
  // Worker-scoped: runs once per worker process, not per test
  adminPage: [async ({ browser }, use) => {
    const context = await browser.newContext();
    const page = await context.newPage();
    // Authenticate via API — avoid UI login overhead
    const response = await page.request.post('/api/auth/login', {
      data: { email: process.env.ADMIN_EMAIL, password: process.env.ADMIN_PASSWORD },
    });
    expect(response.ok()).toBeTruthy();
    await context.storageState({ path: 'playwright/.auth/admin.json' });
    await use(page);
    await context.close();
  }, { scope: 'worker' }],

  viewerPage: [async ({ browser }, use) => {
    const context = await browser.newContext({
      storageState: 'playwright/.auth/viewer.json',
    });
    const page = await context.newPage();
    await use(page);
    await context.close();
  }, { scope: 'worker' }],
});
```

```typescript
// tests/e2e/admin/dashboard.spec.ts
import { test, expect } from '../../fixtures/auth';

test('admin can delete users', async ({ adminPage }) => {
  await adminPage.goto('/admin/users');
  await adminPage.getByTestId('user-row-42').getByTestId('delete').click();
  await expect(adminPage.getByTestId('success-toast')).toBeVisible();
});

test('viewer cannot see delete button', async ({ viewerPage }) => {
  await viewerPage.goto('/admin/users');
  await expect(viewerPage.getByTestId('user-row-42').getByTestId('delete')).not.toBeVisible();
});
```

**Why**: Worker-scoped fixtures share auth state across all tests in the same worker. Login happens once per worker, not once per test — 10x faster for suites with many authenticated tests.

---

### JWT Injection via `addInitScript`

When the app reads JWT from `localStorage` on page load:

```typescript
// pages/AuthenticatedPage.ts
import { type Page } from '@playwright/test';

export async function injectJWT(page: Page, token: string) {
  await page.addInitScript((jwt) => {
    window.localStorage.setItem('auth_token', jwt);
  }, token);
}
```

```typescript
// tests/e2e/features/profile.spec.ts
import { test, expect } from '@playwright/test';
import { injectJWT } from '../../pages/AuthenticatedPage';
import { generateTestJWT } from '../../helpers/jwt';

test('profile page loads with valid JWT', async ({ page }) => {
  const token = generateTestJWT({ userId: '42', role: 'admin', expiresIn: '1h' });
  await injectJWT(page, token);
  await page.goto('/profile');
  await expect(page.getByTestId('profile-name')).toBeVisible();
});
```

**Why**: `addInitScript` runs before the page's own scripts execute, so the JWT is in `localStorage` when the app initializes. Injecting after navigation misses the initial auth check.

---

### Session Expiry Testing

Test what users see when their session expires mid-flow:

```typescript
test('expired session redirects to login with return URL', async ({ page }) => {
  // Start with valid session
  await page.context().addCookies([{
    name: 'session',
    value: 'valid-session-token',
    domain: 'localhost',
    path: '/',
  }]);
  await page.goto('/dashboard');
  await expect(page.getByTestId('dashboard-content')).toBeVisible();

  // Simulate expiry: intercept next API call and return 401
  await page.route('/api/**', route => {
    route.fulfill({ status: 401, body: JSON.stringify({ error: 'session_expired' }) });
  });

  // Trigger an authenticated action
  await page.getByTestId('load-more').click();

  // App should redirect to login, preserving return URL
  await expect(page).toHaveURL(/\/login\?returnUrl=/);
  await expect(page.getByTestId('session-expired-message')).toBeVisible();
});
```

---

### OAuth / SSO Mock (Bypass the Provider)

Never test against a real OAuth provider in E2E — mock the callback instead:

```typescript
test('OAuth login flow completes', async ({ page }) => {
  // Intercept the OAuth redirect back to the app
  await page.route('/auth/callback*', async route => {
    const url = new URL(route.request().url());
    // Simulate a successful OAuth callback with a test code
    await route.fulfill({
      status: 302,
      headers: {
        Location: '/dashboard',
        'Set-Cookie': 'session=test-oauth-session; Path=/; HttpOnly',
      },
    });
  });

  await page.goto('/login');
  await page.getByTestId('login-with-google').click();
  // Playwright follows the redirect chain; our route intercepts the callback
  await expect(page).toHaveURL('/dashboard');
  await expect(page.getByTestId('nav-user-avatar')).toBeVisible();
});
```

**Why**: Real OAuth providers add network latency, require live credentials, and may rate-limit CI. Mocking the callback tests your app's OAuth handling without testing Google/GitHub.

---

## Pattern Catalog

### Authenticate via API and Reuse storageState

**Detection**:
```bash
grep -rn 'getByTestId.*login\|getByTestId.*password\|getByTestId.*signin' --include="*.spec.ts"
rg '(login-email|login-password|login-submit)' --type ts --include="*.spec.ts" -l
```

**Signal**:
```typescript
test.beforeEach(async ({ page }) => {
  await page.goto('/login');
  await page.getByTestId('login-email').fill('user@test.com');
  await page.getByTestId('login-password').fill('password');
  await page.getByTestId('login-submit').click();
  await page.waitForURL('/dashboard');
});
```

**Why this matters**: A 10-test suite with a 3-second login flow spends 30 seconds on login alone. `storageState` eliminates this: authenticate once, reuse the session.

**Preferred action**: Use `globalSetup` with `storageState` (single role) or worker-scoped fixtures (multiple roles). See `playwright-patterns.md` for the single-role pattern.

---

### Use Separate storageState Per Role

**Detection**:
```bash
grep -rn 'storageState' --include="playwright.config.ts" -A 5
rg 'storageState.*admin.*viewer|storageState.*viewer.*admin' --type ts
```

**Signal**:
```typescript
// playwright.config.ts — wrong: one storageState for all tests
use: {
  storageState: 'playwright/.auth/user.json', // which user? admin or viewer?
},
```

**Why this matters**: When tests for different roles share one auth file, they run with the same permissions. An admin-privilege test passes for a viewer because the session is still admin. RBAC bugs go undetected.

**Preferred action**: Use per-role `projects` in `playwright.config.ts`:
```typescript
projects: [
  { name: 'admin-tests', use: { storageState: 'playwright/.auth/admin.json' } },
  { name: 'viewer-tests', use: { storageState: 'playwright/.auth/viewer.json' } },
  { name: 'unauthenticated-tests' }, // no storageState
],
```

---

### Load Credentials from Environment Variables

**Detection**:
```bash
grep -rn 'password.*:.*"[^"]\+"\|fill.*"password[^"]*"' --include="*.spec.ts"
rg '(password|secret|token)\s*[:=]\s*["'"'"'][^"'"'"']+["'"'"']' --type ts --include="*.spec.ts"
```

**Signal**:
```typescript
await page.getByTestId('login-password').fill('MyP@ssw0rd123'); // hardcoded
```

**Why this matters**: Credentials in source code appear in git history permanently. They also break when rotated.

**Preferred action**:
```typescript
await page.getByTestId('login-password').fill(process.env.TEST_USER_PASSWORD!);
```
Set `TEST_USER_PASSWORD` in `.env.test` (gitignored) and in CI secrets.

---

### Mock OAuth Callbacks with page.route

**Detection**:
```bash
grep -rn 'accounts\.google\.com\|github\.com/login/oauth\|login\.microsoftonline' --include="*.spec.ts"
rg '(google|github|microsoft|auth0)\.com' --type ts --include="*.spec.ts"
```

**Signal**:
```typescript
await page.goto('https://accounts.google.com/o/oauth2/auth?...');
await page.getByLabel('Email').fill(process.env.GOOGLE_TEST_EMAIL!);
```

**Why this matters**: Hits a live external service. Flaky on network issues. Requires a real test account. May trigger bot detection. Rate-limited in CI.

**Preferred action**: Mock the OAuth callback with `page.route` as shown in the Correct Patterns section.

---

## Error-Fix Mappings

| Error | Root Cause | Fix |
|-------|------------|-----|
| `401 Unauthorized` in all API calls after test 50 | `storageState` session expired mid-suite | Re-generate auth state in `globalSetup`, or use shorter-lived tokens |
| `storageState: path does not exist` | `globalSetup` didn't run, or path mismatch | Verify `playwright.config.ts` `globalSetup` is wired; check path matches exactly |
| `Cannot read properties of undefined (reading 'token')` | JWT not in localStorage when page loaded | Use `addInitScript`, not `evaluate`, to inject JWT before page scripts run |
| `page.route` callback not triggered | Route registered after navigation started | Register routes before `page.goto()` |
| Tests pass locally, fail in CI with `403` | CI uses different `BASE_URL` hitting real auth | Check that CI env has `PLAYWRIGHT_BASE_URL` pointing to the test instance |

---

## Version-Specific Notes

| Version | Change | Impact |
|---------|--------|--------|
| `1.25` | Worker-scoped fixtures stable | Multi-role auth fixtures now safe for parallel workers |
| `1.31` | `request` context available in fixtures | Can use `fixture.request.post()` for API login without launching a browser |
| `1.35` | `storageState` supports `origins` filter | Can scope saved auth to specific domains, preventing cross-domain cookie leaks |

---

## Detection Commands Reference

```bash
# Find tests doing UI login in beforeEach (migrate to storageState)
grep -rn 'beforeEach' --include="*.spec.ts" -A 10 | grep -i 'login\|password\|signin'

# Find hardcoded credentials
rg '(password|secret)\s*[:=]\s*["'"'"'][^"'"'"']+["'"'"']' --type ts --include="*.spec.ts"

# Find real OAuth provider URLs in tests
grep -rn 'accounts\.google\.com\|github\.com/login\|auth0\.com' --include="*.spec.ts"

# Find tests without any auth setup (may need storageState)
grep -rn 'test(' --include="*.spec.ts" -l | xargs grep -rL 'storageState\|adminPage\|viewerPage\|login'
```

---

## See Also

- `playwright-patterns.md` — basic `storageState` setup, single-role global auth
- `errors.md` — auth-related error symptom/cause/fix matrix

# Financial Flow Patterns

Production skip guards and async confirmation wait patterns for financial E2E tests.

---

## Production Skip Guards

Always guard destructive financial tests with environment checks:

```typescript
// At describe level — skips entire suite
test.describe('Payment Processing', () => {
  test.skip(
    process.env.NODE_ENV === 'production',
    'Payment tests must not run in production'
  );

  test.skip(
    !['localhost', 'e2e.', 'test.'].some(h =>
      process.env.BASE_URL?.includes(h)
    ),
    'Payment tests only run on local, e2e, or test environments'
  );

  // ... tests
});
```

```typescript
// At individual test level
test('refund processes correctly', async ({ page }) => {
  test.skip(
    process.env.PAYMENT_PROVIDER !== 'sandbox',
    'Refund test requires sandbox payment provider'
  );
  // ...
});
```

---

## Polling for Async Confirmation

Financial operations often involve async backend confirmation. Use `toPass` to poll:

```typescript
import { expect } from '@playwright/test';

async function waitForPaymentConfirmed(page: Page, orderId: string) {
  await expect(async () => {
    const statusEl = page.getByTestId(`order-status-${orderId}`);
    await expect(statusEl).toHaveText('Payment confirmed');
  }).toPass({
    timeout: 30_000,
    intervals: [1000, 2000, 5000, 10000],
  });
}
```

---

## Stripe Test Cards

Use Stripe test card numbers in sandbox environments:

| Scenario | Card Number |
|----------|-------------|
| Success | 4242 4242 4242 4242 |
| Insufficient funds | 4000 0000 0000 9995 |
| 3D Secure required | 4000 0025 0000 3155 |
| Generic decline | 4000 0000 0000 0002 |

Always use expiry `12/34`, CVC `123`, ZIP `00000` for test cards.

# Playwright Patterns Reference

Detailed patterns for the `e2e-testing` skill. These are progressive-disclosure
supplements — the main SKILL.md covers the core workflow; this file covers
patterns you reach for once the basics are in place.

---

## Full POM Class Example

```typescript
// pages/CheckoutPage.ts
import { type Page, type Locator } from '@playwright/test';

export class CheckoutPage {
  readonly page: Page;
  readonly cartSummary: Locator;
  readonly promoCodeInput: Locator;
  readonly applyPromoButton: Locator;
  readonly promoSuccessMsg: Locator;
  readonly promoErrorMsg: Locator;
  readonly placeOrderButton: Locator;
  readonly orderConfirmation: Locator;

  constructor(page: Page) {
    this.page             = page;
    this.cartSummary      = page.getByTestId('checkout-cart-summary');
    this.promoCodeInput   = page.getByTestId('checkout-promo-input');
    this.applyPromoButton = page.getByTestId('checkout-promo-apply');
    this.promoSuccessMsg  = page.getByTestId('checkout-promo-success');
    this.promoErrorMsg    = page.getByTestId('checkout-promo-error');
    this.placeOrderButton = page.getByTestId('checkout-place-order');
    this.orderConfirmation = page.getByTestId('checkout-order-confirmation');
  }

  async goto() {
    await this.page.goto('/checkout');
    await this.page.waitForLoadState('networkidle');
  }

  async applyPromoCode(code: string) {
    await this.promoCodeInput.fill(code);
    await this.applyPromoButton.click();
  }

  async placeOrder() {
    await this.placeOrderButton.click();
    // Wait for confirmation element, not arbitrary timeout
    await this.orderConfirmation.waitFor({ state: 'visible' });
  }
}
```

```typescript
// tests/e2e/features/checkout.spec.ts
import { test, expect } from '@playwright/test';
import { CheckoutPage } from '../../../pages/CheckoutPage';
import { LoginPage } from '../../../pages/LoginPage';

test.describe('Checkout Flow', () => {
  test.beforeEach(async ({ page }) => {
    const login = new LoginPage(page);
    await login.goto();
    await login.login('test@example.com', 'password');
  });

  test('valid promo code applies discount', async ({ page }) => {
    const checkout = new CheckoutPage(page);
    await checkout.goto();
    await checkout.applyPromoCode('SAVE10');
    await expect(checkout.promoSuccessMsg).toBeVisible();
    await expect(checkout.promoSuccessMsg).toContainText('10% off');
  });

  test('invalid promo code shows error', async ({ page }) => {
    const checkout = new CheckoutPage(page);
    await checkout.goto();
    await checkout.applyPromoCode('BADCODE');
    await expect(checkout.promoErrorMsg).toBeVisible();
  });

  test('order completes and shows confirmation', async ({ page }) => {
    const checkout = new CheckoutPage(page);
    await checkout.goto();
    await checkout.placeOrder();
    await expect(checkout.orderConfirmation).toBeVisible();
  });
});
```

---

## Condition-Based Waiting Patterns

### Wait for network response

Use when an action triggers an API call and you need to assert on the result:

```typescript
// Wait for a specific API response before asserting
const [response] = await Promise.all([
  page.waitForResponse(resp =>
    resp.url().includes('/api/orders') && resp.status() === 200
  ),
  checkout.placeOrderButton.click(),
]);
const data = await response.json();
expect(data.orderId).toBeDefined();
```

### Wait for element state

```typescript
// Wait for element to appear
await page.getByTestId('loading-spinner').waitFor({ state: 'hidden' });
await page.getByTestId('results-list').waitFor({ state: 'visible' });

// Wait for element to contain text
await expect(page.getByTestId('status-badge')).toContainText('Complete');

// Wait for URL change
await expect(page).toHaveURL(/\/dashboard/);
```

### Wait for load state

```typescript
// After navigation or form submit
await page.waitForLoadState('networkidle');   // No pending XHR/fetch
await page.waitForLoadState('domcontentloaded'); // DOM parsed
await page.waitForLoadState('load');          // All resources loaded
```

### Wait for element count

```typescript
// Wait until a list has items
await expect(page.getByTestId('product-card')).toHaveCount(5);
// Wait until empty
await expect(page.getByTestId('cart-item')).toHaveCount(0);
```

---

## Multi-Browser Configuration

### Full matrix (default)

```typescript
// playwright.config.ts
projects: [
  { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
  { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
  { name: 'Mobile Chrome', use: { ...devices['Pixel 5'] } },
  { name: 'Mobile Safari', use: { ...devices['iPhone 12'] } },
],
```

### CI-only subset (cost/time trade-off)

```typescript
// playwright.config.ts
projects: process.env.CI
  ? [
      { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
      { name: 'Mobile Chrome', use: { ...devices['Pixel 5'] } },
    ]
  : [
      { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
      { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
      { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
    ],
```

### Browser-specific test

```typescript
import { test, expect } from '@playwright/test';

// Run only on webkit — use sparingly, prefer cross-browser tests
test('safari-specific rendering', async ({ page, browserName }) => {
  test.skip(browserName !== 'webkit', 'WebKit only');
  // ...
});
```

---

## Financial / Production Skip Guards

Protect against running destructive financial flows in production or shared staging:

```typescript
// Skip entire describe block in production
test.describe('Payment Flow', () => {
  test.skip(
    process.env.NODE_ENV === 'production',
    'Payment tests skipped in production environment'
  );

  test('credit card charge succeeds', async ({ page }) => {
    // ...
  });
});
```

```typescript
// Skip based on BASE_URL to protect staging shared with real users
test.beforeEach(async () => {
  test.skip(
    !process.env.BASE_URL?.includes('localhost') &&
    !process.env.BASE_URL?.includes('e2e.'),
    'Skipping destructive test: not on local or dedicated E2E environment'
  );
});
```

```typescript
// Guard for blockchain/async confirmation waits
// Use polling for confirmation state instead of `waitForTimeout`
async function waitForTransactionConfirmed(page: Page, txId: string) {
  await expect(async () => {
    const status = await page.getByTestId(`tx-status-${txId}`).textContent();
    expect(status).toBe('Confirmed');
  }).toPass({ timeout: 30_000, intervals: [1000, 2000, 5000] });
}
```

---

## Shared Authentication State

Reuse sessions with `storageState` instead of logging in for every test:

```typescript
// tests/e2e/auth/setup.ts (global setup)
import { chromium, type FullConfig } from '@playwright/test';

async function globalSetup(config: FullConfig) {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('http://localhost:3000/login');
  await page.getByTestId('login-email').fill('test@example.com');
  await page.getByTestId('login-password').fill('password');
  await page.getByTestId('login-submit').click();
  await page.waitForURL('/dashboard');
  // Save storage state (cookies, localStorage)
  await page.context().storageState({ path: 'playwright/.auth/user.json' });
  await browser.close();
}

export default globalSetup;
```

```typescript
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  globalSetup: require.resolve('./tests/e2e/auth/setup'),
  projects: [
    {
      name: 'authenticated',
      use: {
        storageState: 'playwright/.auth/user.json',
      },
    },
    {
      name: 'unauthenticated',
      // No storageState — tests run without session
    },
  ],
});
```

---

## Viewport and Responsive Testing

```typescript
test('mobile navigation menu opens', async ({ page }) => {
  await page.setViewportSize({ width: 375, height: 667 }); // iPhone SE
  await page.goto('/');
  await page.getByTestId('mobile-menu-toggle').click();
  await expect(page.getByTestId('mobile-nav')).toBeVisible();
});
```

---

## Accessibility Assertions

```typescript
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('checkout page has no accessibility violations', async ({ page }) => {
  await page.goto('/checkout');
  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa'])
    .analyze();
  expect(results.violations).toEqual([]);
});
```

## Templates

### playwright.config.ts Template

```typescript
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [
    ['html', { outputFolder: 'playwright-report' }],
    ['json', { outputFile: 'playwright-results.json' }],
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  outputDir: 'artifacts/',
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
    {
      name: 'Mobile Chrome',
      use: { ...devices['Pixel 5'] },
    },
  ],
});
```

The multi-browser matrix (Chromium, Firefox, WebKit) is the default because cross-browser bugs caught in CI are cheaper than cross-browser bugs caught in production. Remove browsers only when the project explicitly constrains the target set.

### POM Pattern

```typescript
// pages/LoginPage.ts
import { type Page, type Locator } from '@playwright/test';

export class LoginPage {
  readonly page: Page;
  readonly emailInput: Locator;
  readonly passwordInput: Locator;
  readonly submitButton: Locator;
  readonly errorMessage: Locator;

  constructor(page: Page) {
    this.page = page;
    this.emailInput    = page.getByTestId('login-email');
    this.passwordInput = page.getByTestId('login-password');
    this.submitButton  = page.getByTestId('login-submit');
    this.errorMessage  = page.getByTestId('login-error');
  }

  async goto() {
    await this.page.goto('/login');
    await this.page.waitForLoadState('networkidle');
  }

  async login(email: string, password: string) {
    await this.emailInput.fill(email);
    await this.passwordInput.fill(password);
    await this.submitButton.click();
  }
}
```

```typescript
// tests/e2e/auth/login.spec.ts
import { test, expect } from '@playwright/test';
import { LoginPage } from '../../../pages/LoginPage';

test.describe('Login Flow', () => {
  let loginPage: LoginPage;

  test.beforeEach(async ({ page }) => {
    loginPage = new LoginPage(page);
    await loginPage.goto();
  });

  test('successful login redirects to dashboard', async ({ page }) => {
    await loginPage.login('user@example.com', 'password123');
    await expect(page).toHaveURL('/dashboard');
  });

  test('invalid credentials shows error message', async () => {
    await loginPage.login('bad@example.com', 'wrong');
    await expect(loginPage.errorMessage).toBeVisible();
    await expect(loginPage.errorMessage).toContainText('Invalid credentials');
  });
});
```

### data-testid Convention

- **Format**: `<component>-<element>` -- e.g., `login-email`, `checkout-submit`, `nav-profile-link`
- **Scope**: Add `data-testid` to interactive elements and status regions the tests need to assert on
- **Stability**: `data-testid` attributes must not change with styling or refactoring -- they are a testing contract

### Waiting and Timing

Never use `waitForTimeout()` or `setTimeout()` in tests. Arbitrary waits pass slowly on fast machines and fail on slow ones -- they encode a guess about timing instead of observing the actual condition. Use condition-based waiting instead:

| Instead of | Use |
|-----------|-----|
| `await page.waitForTimeout(2000)` | `await expect(locator).toBeVisible()` or `await page.waitForResponse(...)` |
| `await page.waitForTimeout(0)` to "flush" | `await page.waitForLoadState('networkidle')` |
| `page.click('button')` without waiting | `locator.click()` -- Playwright auto-waits for actionability |

Each test must own its own setup in `beforeEach`. Tests sharing state via global variables break parallel execution because Playwright runs specs concurrently by default.

### Flaky Test Quarantine Protocol

When a test fails intermittently:

1. **Reproduce**: `npx playwright test <file> --repeat-each=5` -- if it fails at least once in 5 runs, it is flaky.
2. **Quarantine**: Replace `test(` with `test.fixme(` and add a comment with the symptom and a tracking reference.
3. **Do not delete**: Deleted tests leave coverage gaps. Quarantined tests are visible debt.
4. **Fix criteria**: Before removing `test.fixme`, the test must pass 10/10 with `--repeat-each=10`.

```typescript
// Before
test('checkout completes successfully', async ({ page }) => { ... });

// After quarantine
test.fixme('checkout completes successfully', async ({ page }) => {
  // FLAKY: intermittent race on payment confirmation response
  // TODO: #456 -- investigate network timing in checkout flow
  ...
});
```

### e2e-report.md Template

```markdown
# E2E Test Report

**Date**: YYYY-MM-DD
**Playwright version**: X.X.X
**Base URL**: http://...
**Browsers tested**: Chromium, Firefox, WebKit

## Summary

| Status | Count |
|--------|-------|
| Passed | N |
| Failed | N |
| Flaky (quarantined) | N |
| Skipped | N |
| **Total** | N |

## Failed Tests

### <test name>
- **File**: `tests/e2e/.../file.spec.ts`
- **Error**: <assertion or timeout message>
- **Category**: broken-assertion | selector-mismatch | timing | app-bug
- **Action**: fix | quarantine | investigate

## Quarantined (test.fixme)

| Test | Issue | Tracking |
|------|-------|----------|
| <name> | <symptom> | <issue link or TODO> |

## Artifacts

| Type | Path |
|------|------|
| HTML Report | `playwright-report/index.html` |
| JSON Results | `playwright-results.json` |
| Screenshots | `artifacts/screenshots/` |
| Traces | `artifacts/traces/` |
| Videos | `artifacts/videos/` |

## Next Actions

- [ ] Fix broken assertions in: ...
- [ ] Investigate app bugs: ...
- [ ] Unquarantine after fix: ...
```

### CI/CD Integration

#### GitHub Actions Workflow Template

```yaml
name: E2E Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Start application
        run: npm run build && npm run start &
        env:
          NODE_ENV: test

      - name: Wait for application
        run: npx wait-on http://localhost:3000 --timeout 60000

      - name: Run E2E tests
        run: npx playwright test
        env:
          BASE_URL: http://localhost:3000
          CI: true

      - name: Upload test artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-artifacts
          path: |
            playwright-report/
            playwright-results.json
            artifacts/
          retention-days: 30
```

# Wallet Testing Patterns (Web3 / MetaMask)

Patterns for testing Web3 wallet interactions via `addInitScript`.
Used when your application integrates with MetaMask or similar browser-extension wallets.

---

## Mock MetaMask with addInitScript

```typescript
// pages/Web3Page.ts
import { type Page, type Locator } from '@playwright/test';

export class Web3Page {
  readonly page: Page;
  readonly connectWalletButton: Locator;
  readonly walletAddress: Locator;
  readonly txStatus: Locator;

  constructor(page: Page) {
    this.page = page;
    this.connectWalletButton = page.getByTestId('wallet-connect');
    this.walletAddress       = page.getByTestId('wallet-address');
    this.txStatus            = page.getByTestId('tx-status');
  }

  async injectMockEthereum(address = '0xABC123...') {
    await this.page.addInitScript((addr) => {
      (window as any).ethereum = {
        isMetaMask: true,
        selectedAddress: addr,
        request: async ({ method }: { method: string }) => {
          if (method === 'eth_requestAccounts') return [addr];
          if (method === 'eth_accounts') return [addr];
          if (method === 'eth_chainId') return '0x1';
          if (method === 'eth_sendTransaction') return '0xMOCK_TX_HASH';
          return null;
        },
        on: () => {},
        removeListener: () => {},
      };
    }, address);
  }

  async goto() {
    await this.injectMockEthereum();
    await this.page.goto('/app');
    await this.page.waitForLoadState('networkidle');
  }
}
```

```typescript
// tests/e2e/features/wallet.spec.ts
import { test, expect } from '@playwright/test';
import { Web3Page } from '../../../pages/Web3Page';

test.describe('Wallet Connection', () => {
  test('connects wallet and displays address', async ({ page }) => {
    const web3 = new Web3Page(page);
    await web3.goto();
    await web3.connectWalletButton.click();
    await expect(web3.walletAddress).toContainText('0xABC');
  });
});
```

---

## Notes

- `addInitScript` runs before page scripts — the mock is available when the app initialises.
- For transaction hash assertions, use the mock return value `0xMOCK_TX_HASH`.
- Run wallet tests against mocks or local hardhat/anvil nodes so the suite stays deterministic and avoids real-network side effects.

# Test Blind Spot Taxonomy

Structured categories of what high-coverage test suites commonly miss. Use this taxonomy to move testing from "did you write tests?" to "did you test the right things?"

## Category 1: Concurrency & Race Conditions

| Blind Spot | Example Scenario |
|------------|-----------------|
| Double-submit | Button clicked twice rapidly — two orders created |
| Concurrent mutations | Two requests modifying same resource simultaneously |
| Webhook replay | Same event delivered twice — duplicate side effects |
| Read-after-write inconsistency | Cache returns stale data immediately after update |
| Lock contention under load | Mutex held too long causes timeouts at scale |
| Goroutine / async task leak | Fire-and-forget task never completes, accumulates |

## Category 2: State & Data Integrity

| Blind Spot | Example Scenario |
|------------|-----------------|
| Cache invalidation | Update succeeds but cached copy served to next request |
| Partial transaction failure | Step 3 of 5 fails — what state are we left in? |
| Ordering assumptions | Events arrive out of order — handler assumes sequence |
| Idempotency | Same operation applied twice produces different results |
| State machine invalid transitions | Order goes from "shipped" back to "pending" |
| Orphaned references | Parent deleted but child records remain with dangling FK |

## Category 3: Boundary & Extreme Values

| Blind Spot | Example Scenario |
|------------|-----------------|
| Zero and negative values | Quantity: 0, price: -1, count: -999 |
| Integer overflow | MAX_INT + 1 wraps to negative |
| Empty inputs | Empty string, empty array, null, undefined, nil |
| Unicode edge cases | RTL text, emoji, zero-width characters, surrogate pairs |
| Huge payloads | 1MB JSON body, 10K element arrays, deeply nested objects |
| Date boundaries | Midnight, DST transitions, leap seconds, year 2038, Feb 29 |
| Floating point | 0.1 + 0.2 != 0.3, currency rounding errors |

## Category 4: Security Edge Cases

| Blind Spot | Example Scenario |
|------------|-----------------|
| XSS in user content | Markdown renderer executes injected script tags |
| IDOR | Accessing resource by incrementing ID in URL |
| Mass assignment | Extra fields in request body modify protected attributes |
| JWT edge cases | Algorithm confusion (none/HS256), expired but cached tokens |
| Path traversal | `../../etc/passwd` in file upload filename |
| SSRF | User-provided URL causes server to fetch internal resources |
| Rate limit bypass | Distributed requests from multiple IPs evade per-IP limits |

## Category 5: Integration & External Dependencies

| Blind Spot | Example Scenario |
|------------|-----------------|
| Timeout handling | Third-party API takes 30s — what happens? |
| Malformed responses | External service returns invalid JSON or wrong schema |
| Rate limiting (429) | API returns 429 — retry? backoff? fail? queue? |
| DNS resolution failure | Domain unreachable — is error surfaced or swallowed? |
| TLS issues | Certificate expired, hostname mismatch, self-signed cert |
| Partial success | Batch operation: 8 of 10 items succeed, 2 fail |
| Version skew | API v2 response consumed by client expecting v1 shape |

## Category 6: Error Recovery & Resilience

| Blind Spot | Example Scenario |
|------------|-----------------|
| Disk full during write | File write succeeds partially — corrupt data on disk |
| Connection pool exhausted | All DB connections in use — new requests hang or fail |
| Memory pressure | Large result set loaded into memory — OOM kill |
| Graceful shutdown | In-flight requests during SIGTERM — completed or dropped? |
| Retry storms | Failed retries trigger more retries — exponential load |
| Circuit breaker state | Breaker opens but never half-opens to test recovery |
| Crash recovery | Process restarts mid-operation — consistent state restored? |

## How to Use This Taxonomy

1. **During test planning**: For each feature, scan the 6 categories and ask "does this apply here?"
2. **During test review**: Score existing tests against relevant categories — which are covered, which are gaps?
3. **During TDD RED phase**: Before writing implementation, consider which blind spots should have failing tests
4. **Post-incident**: After a production bug, identify which taxonomy category it falls into and add tests for the entire category

Not every category applies to every feature. Apply judgment — a CLI tool doesn't need SSRF tests, and a batch processor doesn't need double-click protection.

# Fix Strategies by Language

Practical tooling and patterns for resolving testing failure modes in Go, Python, and JavaScript/TypeScript.

---

## Go

### Flaky Test Detection
```bash
# Run a test 100 times to detect flakiness
go test -count=100 -run TestSuspectTest ./...

# Run with race detector
go test -race ./...
```

### Parallelization
```go
// Add t.Parallel() to independent tests
func TestFeatureA(t *testing.T) {
    t.Parallel()
    // ...
}
```

### Fresh State per Test
```go
// Use t.Cleanup for deterministic teardown
func setupTestDB(t *testing.T) *Database {
    t.Helper()
    db := NewTestDatabase()
    t.Cleanup(func() { db.Close() })
    return db
}
```

### Error Checking
```go
// Use require for setup, assert for verification
func TestExample(t *testing.T) {
    result, err := SetupDependency()
    require.NoError(t, err, "setup must succeed")  // Fails test immediately

    output := result.Process()
    assert.Equal(t, expected, output)  // Reports but continues
}
```

### Table-Driven Tests for Edge Cases
```go
func TestParseNumber(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    int
        wantErr bool
    }{
        {"positive", "42", 42, false},
        {"negative", "-1", -1, false},
        {"zero", "0", 0, false},
        {"empty", "", 0, true},
        {"letters", "abc", 0, true},
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := ParseNumber(tt.input)
            if tt.wantErr {
                assert.Error(t, err)
            } else {
                assert.NoError(t, err)
                assert.Equal(t, tt.want, got)
            }
        })
    }
}
```

---

## Python

### Flaky Test Detection
```bash
# Install and run with pytest-repeat
pip install pytest-repeat
pytest --count=20 tests/test_suspect.py

# Run with random order
pip install pytest-randomly
pytest -p randomly
```

### Fixture Scoping for Speed
```python
import pytest

# Expensive setup shared across session
@pytest.fixture(scope="session")
def database():
    db = create_database()
    db.run_migrations()
    yield db
    db.cleanup()

# Per-test isolation via transaction rollback
@pytest.fixture
def db_session(database):
    database.begin_transaction()
    yield database
    database.rollback()
```

### Controlling Time
```python
from unittest.mock import patch
from freezegun import freeze_time

# Deterministic time in tests
@freeze_time("2024-01-15 10:30:00")
def test_expiration():
    token = create_token(expires_in=3600)
    assert token.expires_at == datetime(2024, 1, 15, 11, 30, 0)
```

### Skipped Test Audit
```bash
# Find all skipped tests
grep -rn "@pytest.mark.skip\|@unittest.skip\|pytest.skip" tests/

# Find tests with no assertions
grep -rn "def test_" tests/ | while read line; do
    file=$(echo "$line" | cut -d: -f1)
    func=$(echo "$line" | grep -oP 'def \K\w+')
    if ! grep -A 20 "def $func" "$file" | grep -q "assert\|raises\|pytest.raises"; then
        echo "NO ASSERTION: $line"
    fi
done
```

---

## JavaScript / TypeScript

### Flaky Test Detection
```bash
# Run with Jest repeat
npx jest --testPathPattern=suspect.test.ts --repeat=20

# Vitest retry to identify flaky tests
# vitest.config.ts: test.retry = 3
```

### Replacing sleep with waitFor
```javascript
// BAD
await sleep(500);
expect(element).toBeVisible();

// GOOD (React Testing Library)
await waitFor(() => {
  expect(screen.getByText('loaded')).toBeInTheDocument();
});

// GOOD (Playwright)
await expect(page.getByText('loaded')).toBeVisible();
```

### Semantic Selectors over CSS
```javascript
// BAD: Brittle CSS selector
document.querySelector('div.card > span:nth-child(2)');

// GOOD: Testing Library queries (priority order)
screen.getByRole('button', { name: /submit/i });    // 1. Role
screen.getByLabelText('Email');                       // 2. Label
screen.getByPlaceholderText('Enter email');            // 3. Placeholder
screen.getByText('Submit');                            // 4. Text
screen.getByTestId('submit-btn');                      // 5. Test ID (last resort)
```

### Mock Cleanup
```javascript
// Ensure mocks don't leak between tests
afterEach(() => {
  jest.restoreAllMocks();
});

// Or use Vitest's automatic cleanup
// vitest.config.ts: test.restoreMocks = true
```

---

## Cross-Language Patterns

### The Mock Boundary Rule
Mock at the **architectural boundary**, not at every dependency:
- Mock: HTTP clients, databases, file systems, external APIs
- Prefer real executions for pure functions, value objects, and internal collaborators; reserve mocks for external boundaries

### The Assertion Density Rule
Each test should have **1-3 focused assertions** on the behavior under test:
- 0 assertions = no test (just execution)
- 1-3 assertions = focused test
- 10+ assertions = probably over-specified, split into multiple tests

### The Naming Convention
Consistent naming across languages:
- Go: `TestComponent_Condition_ExpectedResult`
- Python: `test_component_condition_expected_result`
- JS: `describe('Component') > it('expected result when condition')`

# Load Test Scenario Reference

Structured taxonomy of load testing approaches. Use when generating test
scripts (k6, Artillery, Locust) or reviewing load test coverage.

## Scenario Types

### 1. Smoke Test
**Purpose**: Verify the system works at all under minimal load.
**Configuration**: 1-2 virtual users, 1-2 minutes.
**When to use**: After every deployment. Quick sanity check.

```
Duration: 1-2 minutes
Users: 1-2
Thresholds:
  - All requests succeed (error rate = 0%)
  - p95 response time < baseline + 50%
```

### 2. Load Test
**Purpose**: Verify the system handles expected production load.
**Configuration**: Target production user count, 10-15 minutes sustained.
**When to use**: Before releases. Baseline performance validation.

```
Duration: 10-15 minutes
Users: Expected production peak (from analytics)
Ramp-up: 30 seconds per 10% of target
Thresholds:
  - Error rate < 1%
  - p95 response time < SLO target
  - p99 response time < 3x SLO target
```

### 3. Stress Test
**Purpose**: Find the breaking point. How far past normal can we go?
**Configuration**: Ramp beyond expected load until failures appear.
**When to use**: Capacity planning. Finding bottlenecks.

```
Duration: 15-20 minutes
Users: Ramp from 0 to 200-300% of expected peak
Ramp-up: Gradual increase every 2 minutes
Watch for:
  - Error rate spike (which endpoint breaks first?)
  - Response time degradation curve (linear or exponential?)
  - Resource exhaustion (CPU, memory, connections, disk I/O)
```

### 4. Spike Test
**Purpose**: How does the system handle sudden traffic bursts?
**Configuration**: Normal load → instant jump to peak → back to normal.
**When to use**: Systems exposed to viral traffic, marketing campaigns, flash sales.

```
Duration: 10 minutes
Pattern: 2min normal → instant spike to 500% → hold 3min → drop to normal → 3min recovery
Watch for:
  - Recovery time after spike subsides
  - Error rate during spike
  - Queue depth behavior
  - Auto-scaling response time (if applicable)
```

### 5. Soak Test (Endurance)
**Purpose**: Find memory leaks, connection pool exhaustion, log disk fill.
**Configuration**: Normal load sustained for hours.
**When to use**: Before major releases. Detecting slow-burn issues.

```
Duration: 2-8 hours
Users: Expected production average (not peak)
Thresholds:
  - Memory usage doesn't trend upward over time
  - Response times don't degrade over time
  - Error rate stays flat
  - No file descriptor / connection pool exhaustion
```

### 6. Breakpoint Test
**Purpose**: Find exact capacity ceiling with precision.
**Configuration**: Slow, controlled ramp until SLO violation.
**When to use**: Capacity planning with specific numbers.

```
Duration: Until SLO breach
Users: Start at 50%, increase by 5% every 2 minutes
Record:
  - Exact user count where p95 exceeds SLO
  - Exact user count where error rate exceeds threshold
  - Resource utilization at breakpoint (CPU%, memory%, connections)
```

## Critical Endpoints to Test

For any web service, prioritize these in load tests:

| Priority | Endpoint Type | Why |
|----------|--------------|-----|
| 1 | Authentication (login/token refresh) | Gate to everything else. If auth breaks, nothing works. |
| 2 | Most-hit read endpoint | Highest traffic volume. Often the first to show degradation. |
| 3 | Write endpoints (create/update) | Database contention, lock conflicts surface here. |
| 4 | Search/query with filters | Complex queries degrade fastest under load. |
| 5 | File upload/download | I/O bound, different bottleneck profile than compute. |
| 6 | Webhook receivers | External systems don't respect your capacity limits. |

## Common Mistakes in Load Tests

| Mistake | Why It's Wrong | Fix |
|---------|---------------|-----|
| Testing from same machine as server | Network isn't tested, localhost is unrealistic | Use separate load generator machine or cloud service |
| Single endpoint only | Real traffic hits many endpoints concurrently | Create realistic user journey scenarios |
| No think time between requests | Real users pause between actions | Add 1-5 second think time between requests |
| Ignoring ramp-up | Instant full load triggers different failures than gradual | Always ramp up over 30-60 seconds minimum |
| Not checking server-side metrics | Client-side metrics miss server resource exhaustion | Monitor CPU, memory, DB connections, disk I/O during test |
| Testing against production | Risk of outage, data corruption | Use staging with production-like data volume |

## How to Use This Reference

1. **Start with smoke**: Every deployment gets a smoke test
2. **Baseline with load**: Establish normal performance baseline before optimizing
3. **Find limits with stress**: Know your breaking point before production tells you
4. **Test resilience with spike**: Validate auto-scaling and recovery
5. **Catch leaks with soak**: Run overnight before major releases

# Testing Patterns to Detect and Fix Catalog

Detailed code examples for all 10 failure modes. Each section shows BAD (what to avoid) and GOOD (what to do instead) in Go, Python, and JavaScript where applicable.

---

## 1. Testing Implementation Details (Not Behavior)

Tests that assert on internal state, private methods, or implementation specifics break on any refactoring, even when behavior is unchanged.

**Detection heuristic:** If the test would break from a pure refactoring (no behavior change), it tests implementation.

### BAD

```go
// Testing internal state
func TestParser_InternalRegex(t *testing.T) {
    parser := NewParser()
    assert.Equal(t, `\d{3}-\d{3}-\d{4}`, parser.phoneRegex)
}
```

```javascript
// Testing that a specific method was called
test('validates user', () => {
  const spy = jest.spyOn(validator, '_checkEmailFormat');
  validateUser({ email: 'test@example.com' });
  expect(spy).toHaveBeenCalled();  // Tests HOW, not WHAT
});
```

```python
# Asserting on private attributes
def test_cache_internal_storage():
    cache = Cache()
    cache.set("key", "value")
    assert cache._storage["key"] == "value"
```

### GOOD

```go
// Test observable behavior
func TestParser_ValidPhoneNumber_Parses(t *testing.T) {
    parser := NewParser()
    result, err := parser.Parse("123-456-7890")
    assert.NoError(t, err)
    assert.Equal(t, "1234567890", result.Digits())
}
```

```javascript
// Test the outcome
test('validates user with valid email', () => {
  const result = validateUser({ email: 'test@example.com' });
  expect(result.isValid).toBe(true);
});
```

```python
# Test through public interface
def test_cache_stores_and_retrieves():
    cache = Cache()
    cache.set("key", "value")
    assert cache.get("key") == "value"
```

---

## 2. Fragile Tests (Over-Mocking, Brittle Selectors)

When mock setup is longer than test logic, you are testing that mocks work, not that code works.

**Rule of thumb:** If mock setup is >50% of test code, consider integration testing.

### BAD

```javascript
// Over-mocking destroys test value
test('user service creates user', async () => {
  const mockDb = { insert: jest.fn().mockResolvedValue({ id: 1 }) };
  const mockLogger = { info: jest.fn() };
  const mockValidator = { validate: jest.fn().mockReturnValue(true) };
  const mockNotifier = { send: jest.fn() };
  const mockCache = { set: jest.fn() };

  const service = new UserService(mockDb, mockLogger, mockValidator, mockNotifier, mockCache);
  const user = await service.create({ name: 'Alice' });
  expect(mockDb.insert).toHaveBeenCalled();  // Testing mock, not behavior
});

// Brittle CSS selectors
test('shows user name', () => {
  render(<UserProfile user={testUser} />);
  expect(document.querySelector('div.user-card > span.name-text:nth-child(2)')).toHaveTextContent('Alice');
});
```

### GOOD

```javascript
// Mock only external boundaries
test('user service creates user in database', async () => {
  const db = await createTestDatabase();
  const service = new UserService(db);
  const user = await service.create({ name: 'Alice' });

  expect(user.id).toBeDefined();
  expect(user.name).toBe('Alice');
  const retrieved = await db.findById(user.id);
  expect(retrieved.name).toBe('Alice');
});

// Use semantic selectors
test('shows user name', () => {
  render(<UserProfile user={testUser} />);
  expect(screen.getByRole('heading', { name: /Alice/ })).toBeInTheDocument();
});
```

---

## 3. Test Interdependence (Order-Dependent Tests)

Tests that share state and must run in a specific order. They fail when run in isolation or parallel.

**Verification:** Run tests in random order. If they fail, you have interdependence.

### BAD

```python
# Tests share state and must run in order
class TestUserWorkflow:
    user_id = None  # Shared state

    def test_1_create_user(self):
        response = client.post('/users', json={'name': 'Alice'})
        TestUserWorkflow.user_id = response.json()['id']
        assert response.status_code == 201

    def test_2_get_user(self):
        response = client.get(f'/users/{TestUserWorkflow.user_id}')  # Depends on test_1
        assert response.json()['name'] == 'Alice'
```

```go
// Global state mutated by tests
var globalDB *Database

func TestCreateUser(t *testing.T) {
    globalDB.Insert(User{Name: "Alice"})
}

func TestListUsers(t *testing.T) {
    users := globalDB.List()
    assert.Contains(t, users, User{Name: "Alice"})  // Depends on TestCreateUser
}
```

### GOOD

```python
# Each test is self-contained
class TestUserWorkflow:
    def test_create_user(self):
        response = client.post('/users', json={'name': 'Alice'})
        assert response.status_code == 201
        assert response.json()['name'] == 'Alice'

    def test_get_user(self):
        create_response = client.post('/users', json={'name': 'Bob'})
        user_id = create_response.json()['id']
        response = client.get(f'/users/{user_id}')
        assert response.json()['name'] == 'Bob'
```

```go
// Fresh state per test
func TestCreateUser(t *testing.T) {
    db := setupTestDB(t)
    defer db.Cleanup()
    db.Insert(User{Name: "Alice"})
    // ...
}

func TestListUsers(t *testing.T) {
    db := setupTestDB(t)
    defer db.Cleanup()
    db.Insert(User{Name: "Test User"})
    users := db.List()
    assert.Len(t, users, 1)
}
```

---

## 4. Incomplete Assertions (Testing Too Little)

Tests that check existence but not correctness. They pass with completely wrong output.

**Heuristic:** After writing assertions, ask "Could this test pass with obviously wrong output?"

### BAD

```python
def test_fetch_user():
    result = fetch_user(user_id=123)
    assert result is not None  # What if it's an error object?
```

```go
func TestCalculate(t *testing.T) {
    result, err := Calculate(10, 20)
    assert.NoError(t, err)
    // Never checks what result actually is!
}
```

### GOOD

```python
def test_fetch_user():
    result = fetch_user(user_id=123)
    assert result['id'] == 123
    assert result['name'] == 'Alice'
    assert result['email'] == 'alice@example.com'
```

```go
func TestCalculate(t *testing.T) {
    result, err := Calculate(10, 20)
    assert.NoError(t, err)
    assert.Equal(t, 30, result.Value)
    assert.Equal(t, "addition", result.Operation)
}
```

---

## 5. Over-Specification (Testing Too Much)

Asserting on every field including defaults, exact timestamps, and implementation artifacts. Obscures what behavior actually matters.

**Rule:** Each test should have a clear purpose. If you cannot name what behavior it tests, it is over-specified.

### BAD

```javascript
test('creates user', async () => {
  const user = await createUser({ name: 'Alice', email: 'alice@test.com' });
  expect(user.id).toBe(1);  // Why must it be exactly 1?
  expect(user.name).toBe('Alice');
  expect(user.email).toBe('alice@test.com');
  expect(user.createdAt).toBe('2024-01-15T10:30:00Z');  // Exact timestamp
  expect(user.updatedAt).toBe('2024-01-15T10:30:00Z');
  expect(user.version).toBe(1);
  expect(user.isActive).toBe(true);
  expect(user.loginCount).toBe(0);
  expect(user.lastLoginAt).toBeNull();
  expect(user.preferences).toEqual({});
  expect(user.roles).toEqual(['user']);
});
```

### GOOD

```javascript
test('creates user with provided name and email', async () => {
  const user = await createUser({ name: 'Alice', email: 'alice@test.com' });
  expect(user.id).toBeDefined();
  expect(user.name).toBe('Alice');
  expect(user.email).toBe('alice@test.com');
  expect(user.createdAt).toBeInstanceOf(Date);
});

// Separate test for defaults if that behavior matters
test('new user has correct defaults', async () => {
  const user = await createUser({ name: 'Bob', email: 'bob@test.com' });
  expect(user.isActive).toBe(true);
  expect(user.roles).toContain('user');
});
```

---

## 6. Ignored Failures (Skipped Tests, Swallowed Errors)

Skipped tests are broken tests waiting to cause production issues. Swallowed errors hide real failures.

**Action:** Audit skipped tests quarterly. Delete or fix them.

### BAD

```python
@pytest.mark.skip("TODO: fix later")  # Been here 2 years
def test_payment_processing():
    ...

@pytest.mark.skip("Flaky on CI")  # Hiding a real problem
def test_concurrent_updates():
    ...
```

```javascript
test('processes data', async () => {
  try {
    await processData(invalidInput);
    expect(true).toBe(true);  // Always passes
  } catch (e) {
    // Swallowed -- test "passes" even when it fails
  }
});
```

```go
func TestSaveUser(t *testing.T) {
    user := User{Name: "Alice"}
    _ = db.Save(user)  // Error ignored
    retrieved, _ := db.Find(user.ID)  // Another ignored error
    assert.Equal(t, "Alice", retrieved.Name)
}
```

### GOOD

```javascript
test('rejects invalid input', async () => {
  await expect(processData(invalidInput)).rejects.toThrow('Invalid format');
});
```

```go
func TestSaveUser(t *testing.T) {
    user := User{Name: "Alice"}
    err := db.Save(user)
    require.NoError(t, err, "save should succeed")
    retrieved, err := db.Find(user.ID)
    require.NoError(t, err, "find should succeed")
    assert.Equal(t, "Alice", retrieved.Name)
}
```

---

## 7. Poor Test Naming (Unclear What Is Tested)

When test names do not explain behavior, failures do not explain what broke.

**Template:** `Test{Component}_{Condition}_{ExpectedResult}` or `{component} {does what} when {condition}`

### BAD

```go
func TestUser(t *testing.T) { ... }
func TestUserFunc(t *testing.T) { ... }
func TestUserFunc2(t *testing.T) { ... }
```

```python
def test_call_validate_then_save(): ...
def test_with_mock(): ...
def test_new(): ...
```

### GOOD

```go
func TestUser_EmptyName_ReturnsValidationError(t *testing.T) { ... }
func TestUser_DuplicateEmail_ReturnsConflictError(t *testing.T) { ... }
func TestUser_ValidInput_CreatesAndReturnsUser(t *testing.T) { ... }
```

```python
def test_user_with_empty_name_raises_validation_error(): ...
def test_user_with_duplicate_email_raises_conflict_error(): ...
def test_creating_user_with_valid_data_returns_user(): ...
```

---

## 8. Missing Edge Cases

Only testing the happy path gives false confidence. Real users hit edge cases.

**Edge case checklist:**
- Empty input (null, empty string, empty array)
- Boundary values (0, 1, -1, max, min)
- Invalid types
- Unicode / special characters
- Very large / small values
- Concurrent access
- Error conditions

### BAD

```python
def test_parse_number():
    assert parse_number("42") == 42
    assert parse_number("100") == 100
    # What about: "", "abc", None, "-1", "1.5", "999999999999999"?
```

### GOOD

```python
class TestParseNumber:
    def test_positive_integer(self):
        assert parse_number("42") == 42

    def test_negative_integer(self):
        assert parse_number("-42") == -42

    def test_zero(self):
        assert parse_number("0") == 0

    def test_empty_string_raises(self):
        with pytest.raises(ValueError):
            parse_number("")

    def test_none_raises(self):
        with pytest.raises(TypeError):
            parse_number(None)

    def test_non_numeric_raises(self):
        with pytest.raises(ValueError):
            parse_number("abc")

    def test_whitespace_is_trimmed(self):
        assert parse_number("  42  ") == 42

    def test_very_large_number(self):
        assert parse_number("999999999999") == 999999999999
```

---

## 9. Slow Test Suites (Heavy Setup, No Parallelization)

When setup time dominates test time, developers stop running tests.

**Optimization strategies:**
- Use `t.Parallel()` (Go), `pytest-xdist` (Python), concurrent runners (JS)
- Share expensive fixtures at session/module scope
- Use transactions with rollback instead of database recreation
- Mock external services (network calls are slow)

### BAD

```python
class TestUserAPI:
    def setup_method(self):
        self.db = create_database()       # 2 seconds
        self.db.run_migrations()
        self.db.seed_all_fixtures()
        self.app = create_app(self.db)
        self.client = TestClient(self.app)

    def test_get_user(self):
        # 50ms test, 2000ms setup
        response = self.client.get('/users/1')
        assert response.status_code == 200
```

### GOOD

```python
@pytest.fixture(scope="session")
def db():
    """Database connection shared across all tests."""
    database = create_database()
    database.run_migrations()
    yield database
    database.cleanup()

@pytest.fixture
def client(db):
    """Fresh test client with transaction rollback."""
    db.begin_transaction()
    yield TestClient(create_app(db))
    db.rollback()  # Instant cleanup

def test_get_user(client):
    response = client.get('/users/1')
    assert response.status_code == 200
```

```go
func TestGetUser(t *testing.T) {
    t.Parallel()
    db := setupTestDB(t)
    client := newTestClient(db)
    resp := client.Get("/users/1")
    assert.Equal(t, 200, resp.StatusCode)
}
```

---

## 10. Flaky Tests (Race Conditions, Timing Issues)

Tests that fail randomly erode trust. Developers start ignoring failures.

**Flaky test fixes:**
- Replace `sleep()` with explicit waits/conditions
- Inject clocks for time-dependent logic
- Use synchronization primitives (WaitGroups, channels)
- Run in loop to reproduce: `go test -count=100`

### BAD

```javascript
test('shows loading then content', async () => {
  render(<DataLoader />);
  expect(screen.getByText('Loading...')).toBeInTheDocument();
  await sleep(500);  // Might not be enough
  expect(screen.getByText('Data loaded')).toBeInTheDocument();
});
```

```go
func TestCounter(t *testing.T) {
    counter := NewCounter()
    for i := 0; i < 100; i++ {
        go counter.Increment()
    }
    // Race! Goroutines might not be done
    assert.Equal(t, 100, counter.Value())
}
```

### GOOD

```javascript
test('shows loading then content', async () => {
  render(<DataLoader />);
  expect(screen.getByText('Loading...')).toBeInTheDocument();
  await waitFor(() => {
    expect(screen.getByText('Data loaded')).toBeInTheDocument();
  });
});
```

```python
def test_rate_limiter():
    clock = FakeClock()
    limiter = RateLimiter(max_requests=5, window_seconds=1, clock=clock)
    for i in range(5):
        assert limiter.allow() == True
    assert limiter.allow() == False
    clock.advance(1.0)  # Deterministic
    assert limiter.allow() == True
```

```go
func TestCounter(t *testing.T) {
    counter := NewCounter()
    var wg sync.WaitGroup
    for i := 0; i < 100; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            counter.Increment()
        }()
    }
    wg.Wait()
    assert.Equal(t, 100, counter.Value())
}
```

## Pattern Quality Catalog

This section documents the domain-specific failure modes this skill detects and fixes.

### Pattern 1: Test Observable Behavior

**What it looks like:** Tests assert on private fields, internal regex patterns, or spy on private methods.

**Why it's problematic:** Tests coupled to implementation details break whenever the implementation changes, even if public behavior is identical. This creates brittle tests that fail to reflect real-world usage.

**Example signals:**
- Test accesses `obj._privateField`
- Test mocks or spies on `_internalMethod()`
- Test asserts the exact regex used internally

**Fix:** Test the public behavior that those implementation details enable. If private fields matter, they matter because they affect what users see or experience.

### Pattern 2: Mock Only at Boundaries

**What it looks like:** Mock setup spans more than 50% of the test code. CSS selectors use nth-child or rely on brittle DOM structure.

**Why it's problematic:** Over-mocked tests verify mock wiring, not actual behavior. They miss real integration issues and break whenever the mocking structure changes.

**Example signals:**
- Test has 15 lines of setup and 5 lines of assertion
- Test uses `.querySelector('div:nth-child(3) > span')`
- Test mocks every dependency instead of using real implementations at I/O boundaries

**Fix:** Mock only at architectural boundaries (HTTP, DB, external services). Use real implementations for internal logic. For UI tests, select by semantic attributes (data-testid, role) instead of DOM structure.

### Pattern 3: Isolate Test State

**What it looks like:** Tests share mutable state, use class-level variables, or have numbered test names (test1, test2) suggesting sequence dependency.

**Why it's problematic:** Tests that pass in sequence but fail in parallel or random order hide bugs. The suite becomes unreliable — developers can't trust "all tests pass" locally if they fail in CI.

**Example signals:**
- Multiple tests modify a shared class-level variable
- Database is populated by test1, test2 depends on that state
- Test names: `test1_setup`, `test2_verify`, `test3_cleanup`

**Fix:** Each test owns its data. Use setup/teardown or test fixtures to isolate state. Run suite with `--shuffle` or `-random-order` to catch dependencies.

### Pattern 4: Assert Specific Values

**What it looks like:** Tests use assertions like `!= nil`, `> 0`, `toBeTruthy()` without checking specific values.

**Why it's problematic:** Incomplete assertions pass for many wrong reasons. A function that returns 999 (wrong) passes an `> 0` assertion. This gives false confidence — tests pass but miss bugs.

**Example signals:**
- `assert result != nil` (passes for any non-nil value)
- `assert response.status > 0` (passes for 404, 500, etc.)
- `expect(user).toBeTruthy()` (passes for any truthy user, even with wrong name)

**Fix:** Assert specific expected values:
- `assert.equal(result.name, "Alice")`
- `assert.equal(response.status, 200)`
- `expect(user.name).toBe("Alice")`

### Pattern 5: Assert Only What Matters

**What it looks like:** Tests assert on default values, exact timestamps, hardcoded IDs, or every field in a response.

**Why it's problematic:** Over-specified tests are fragile. When a default changes (legitimately), dozens of tests break even though behavior didn't change. Tests should specify only what matters for this test case.

**Example signals:**
- `assert.equal(user.createdAt, "2024-01-15T10:30:00Z")` (timestamp brittle to test time)
- `assert.equal(post.id, "uuid-1234-5678")` (hardcoded ID specific to this test)
- Test asserts `status`, `message`, `timestamp`, `userId`, `metadata` when only `status` matters

**Fix:** Assert only what matters. Use flexible matchers for timestamps and IDs:
- `expect(user.createdAt).toBeDefined()` or `toBeWithin(now, 1000ms)`
- `assert.truthy(post.id)` (just verify it exists)

### Pattern 6: Address or Remove Skipped Tests

**What it looks like:** Tests use `@skip`, `.skip`, `xit`, empty catch blocks, or `_ = err` (ignore error).

**Why it's problematic:** Skipped tests become permanent blind spots. Nobody remembers why they were skipped. Empty catch blocks hide real errors.

**Example signals:**
- `@skip` or `.skip()` with no expiration date
- `try { ...test code... } catch (e) {}` (silently ignore errors)
- `err := doSomething(); _ = err` (acknowledge but ignore)

**Fix:** Delete the test if no longer relevant, or unskip and fix it. Add a reason annotation with a date if skipping is truly necessary:
```go
t.Skip("TODO: fix timing issue (2024-01-15)")
```

### Pattern 7: Use Descriptive Test Names

**What it looks like:** Test names use sequential numbers (`test1`, `test2`), vague names (`testFunc`, `test_new`), or generic descriptions (`it('works')`, `it('handles case')`).

**Why it's problematic:** Poor names hide intent. Developers reading test output see `test1 failed` but have no idea what behavior broke. Good test names document expected behavior.

**Example signals:**
- `TestCreateUser1`, `TestCreateUser2`
- `test_new`, `testFunc`, `test_handle`
- `it('works')`, `it('handles case')`, `it('does something')`

**Fix:** Use descriptive names that describe the scenario and expected outcome:
- Go: `Test_CreateUser_WithValidEmail_ReturnsNewUser`
- Python: `test_create_user_with_valid_email_returns_new_user`
- JS: `it('creates a user when given a valid email')`

### Pattern 8: Cover Boundaries and Errors

**What it looks like:** Test suite covers only the happy path. No tests for empty inputs, null values, boundary conditions, errors, or large datasets.

**Why it's problematic:** Missing edge cases cause production bugs. The happy path works, but the code crashes on empty input, null reference, or boundary values.

**Example signals:**
- Only tests with valid input; no tests with empty/null
- No tests for negative numbers, zero, or max values
- No tests for error conditions (timeout, connection failure)

**Fix:** Add tests for:
- **Empty**: empty string, empty array, empty object
- **Null**: null input, missing required field
- **Boundary**: zero, max value, min value, off-by-one
- **Error**: timeout, network failure, permission denied
- **Large**: very large arrays, deep nesting

### Pattern 9: Optimize Test Speed

**What it looks like:** Full database reset between every test. No parallelization. Fixture data shared instead of created per-test. Tests wait on actual time.

**Why it's problematic:** Slow tests discourage running locally. Developers skip tests before committing, bugs slip through. CI builds take hours, slowing iteration.

**Example signals:**
- Each test: `DROP TABLE users; INSERT INTO users ...` (30s per test)
- Sequential execution with no parallelization
- Tests use `time.Sleep(1000)` to wait for something

**Fix:**
- Use transactions that rollback instead of dropping tables
- Run tests in parallel: `go test -parallel 8`, `pytest -n auto`
- Create fixtures once, reference per-test: fixture factories, test-specific data builders
- Replace waits with condition checks: `waitFor(() => element.textContent)` instead of `sleep(1000)`

### Pattern 10: Ensure Deterministic Tests

**What it looks like:** Tests use `sleep()`, `time.Sleep()`, `setTimeout()` or unsynchronized goroutines. Tests pass locally but fail randomly in CI.

**Why it's problematic:** Flaky tests erode trust in the test suite. Developers cannot tell if a failure is real or just timing. Teams start ignoring test failures — the worst outcome.

**Example signals:**
- `time.Sleep(100 * time.Millisecond)` to wait for goroutine
- `setTimeout(() => { ...assert... }, 500)` hoping it's ready
- Tests pass locally but fail in CI (slower machines, resource contention)

**Fix:**
- Replace `sleep()` with explicit waits: `waitFor()`, `sync.WaitGroup`, channels
- Inject fake clocks or time control: `time.Now()` should be mockable
- Synchronize goroutines with channels or `sync.WaitGroup`, not timing
- Tests must be deterministic: same input → same output, regardless of machine speed

# Test-Driven Development Examples

This file contains complete TDD examples for Go, Python, and JavaScript showing the full RED-GREEN-REFACTOR cycle.

## Go TDD Example: String Reversal Function

### Iteration 1: Basic Reversal

#### RED Phase - Write Failing Test

**File: `strings/reverse_test.go`**
```go
package strings

import "testing"

func TestReverse_SimpleString_ReturnsReversed(t *testing.T) {
    // Arrange
    input := "hello"
    expected := "olleh"

    // Act
    result := Reverse(input)

    // Assert
    if result != expected {
        t.Errorf("Reverse(%q) = %q; want %q", input, result, expected)
    }
}
```

**Run test (RED):**
```bash
$ go test ./strings -v -run TestReverse
# strings [strings.test]
./reverse_test.go:9:14: undefined: Reverse
FAIL    strings [build failed]
```

✅ **Test fails for RIGHT reason**: Function doesn't exist yet

#### GREEN Phase - Minimum Implementation

**File: `strings/reverse.go`**
```go
package strings

func Reverse(s string) string {
    runes := []rune(s)
    for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
        runes[i], runes[j] = runes[j], runes[i]
    }
    return string(runes)
}
```

**Run test (GREEN):**
```bash
$ go test ./strings -v -run TestReverse
=== RUN   TestReverse_SimpleString_ReturnsReversed
--- PASS: TestReverse_SimpleString_ReturnsReversed (0.00s)
PASS
ok      strings    0.002s
```

✅ **Test passes**: Minimum implementation successful

#### REFACTOR Phase - Improve Code

**No refactoring needed yet** - implementation is clean and simple

**Commit:**
```bash
$ git add strings/
$ git commit -m "Add Reverse function for simple strings

- Add test for basic string reversal
- Implement rune-based reversal for Unicode support
- Passes all tests"
```

### Iteration 2: Handle Empty Strings

#### RED Phase - Add Edge Case Test

**File: `strings/reverse_test.go`**
```go
func TestReverse_EmptyString_ReturnsEmpty(t *testing.T) {
    // Arrange
    input := ""
    expected := ""

    // Act
    result := Reverse(input)

    // Assert
    if result != expected {
        t.Errorf("Reverse(%q) = %q; want %q", input, result, expected)
    }
}
```

**Run test (GREEN - already passes):**
```bash
$ go test ./strings -v -run TestReverse_EmptyString
=== RUN   TestReverse_EmptyString_ReturnsEmpty
--- PASS: TestReverse_EmptyString_ReturnsEmpty (0.00s)
PASS
ok      strings    0.001s
```

✅ **Test already passes**: Implementation handles edge case correctly

### Iteration 3: Table-Driven Tests (Refactor)

#### REFACTOR Phase - Consolidate Tests

**File: `strings/reverse_test.go`**
```go
package strings

import "testing"

func TestReverse(t *testing.T) {
    tests := []struct {
        name     string
        input    string
        expected string
    }{
        {
            name:     "simple string",
            input:    "hello",
            expected: "olleh",
        },
        {
            name:     "empty string",
            input:    "",
            expected: "",
        },
        {
            name:     "single character",
            input:    "a",
            expected: "a",
        },
        {
            name:     "unicode characters",
            input:    "Hello, 世界",
            expected: "界世 ,olleH",
        },
        {
            name:     "palindrome",
            input:    "racecar",
            expected: "racecar",
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            // Act
            result := Reverse(tt.input)

            // Assert
            if result != tt.expected {
                t.Errorf("Reverse(%q) = %q; want %q", tt.input, result, tt.expected)
            }
        })
    }
}
```

**Run refactored tests:**
```bash
$ go test ./strings -v
=== RUN   TestReverse
=== RUN   TestReverse/simple_string
=== RUN   TestReverse/empty_string
=== RUN   TestReverse/single_character
=== RUN   TestReverse/unicode_characters
=== RUN   TestReverse/palindrome
--- PASS: TestReverse (0.00s)
    --- PASS: TestReverse/simple_string (0.00s)
    --- PASS: TestReverse/empty_string (0.00s)
    --- PASS: TestReverse/single_character (0.00s)
    --- PASS: TestReverse/unicode_characters (0.00s)
    --- PASS: TestReverse/palindrome (0.00s)
PASS
ok      strings    0.002s
```

✅ **All tests pass after refactoring**

---

## Python TDD Example: Email Validator

### Iteration 1: Basic Email Validation

#### RED Phase - Write Failing Test

**File: `tests/test_validator.py`**
```python
import pytest
from validator import EmailValidator

def test_validate_email_valid_format_returns_true():
    # Arrange
    validator = EmailValidator()
    email = "user@example.com"

    # Act
    result = validator.validate(email)

    # Assert
    assert result is True
```

**Run test (RED):**
```bash
$ pytest tests/test_validator.py::test_validate_email_valid_format_returns_true -v
================================ test session starts =================================
collected 0 items / 1 error

======================================= ERRORS =======================================
________________ ERROR collecting tests/test_validator.py ___________________________
tests/test_validator.py:2: in <module>
    from validator import EmailValidator
E   ModuleNotFoundError: No module named 'validator'
```

✅ **Test fails for RIGHT reason**: Module doesn't exist yet

#### GREEN Phase - Minimum Implementation

**File: `validator.py`**
```python
import re

class EmailValidator:
    def __init__(self):
        self.pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

    def validate(self, email: str) -> bool:
        return bool(re.match(self.pattern, email))
```

**Run test (GREEN):**
```bash
$ pytest tests/test_validator.py::test_validate_email_valid_format_returns_true -v
================================ test session starts =================================
collected 1 item

tests/test_validator.py::test_validate_email_valid_format_returns_true PASSED [100%]

================================= 1 passed in 0.01s ==================================
```

✅ **Test passes**: Minimum implementation successful

### Iteration 2: Invalid Email Format

#### RED Phase - Add Failure Case

**File: `tests/test_validator.py`**
```python
def test_validate_email_invalid_format_returns_false():
    # Arrange
    validator = EmailValidator()
    email = "invalid-email"

    # Act
    result = validator.validate(email)

    # Assert
    assert result is False
```

**Run test (GREEN - already passes):**
```bash
$ pytest tests/test_validator.py::test_validate_email_invalid_format_returns_false -v
================================ test session starts =================================
collected 1 item

tests/test_validator.py::test_validate_email_invalid_format_returns_false PASSED [100%]

================================= 1 passed in 0.01s ==================================
```

✅ **Test already passes**: Regex handles invalid format

### Iteration 3: Parametrized Tests (Refactor)

#### REFACTOR Phase - Use pytest.mark.parametrize

**File: `tests/test_validator.py`**
```python
import pytest
from validator import EmailValidator

@pytest.mark.parametrize("email,expected", [
    # Valid emails
    ("user@example.com", True),
    ("test.user@example.co.uk", True),
    ("user+tag@example.com", True),
    ("user_name@example-domain.com", True),

    # Invalid emails
    ("invalid-email", False),
    ("@example.com", False),
    ("user@", False),
    ("user@.com", False),
    ("user name@example.com", False),
    ("user@example", False),
])
def test_validate_email(email, expected):
    # Arrange
    validator = EmailValidator()

    # Act
    result = validator.validate(email)

    # Assert
    assert result is expected, f"validate({email!r}) should return {expected}"
```

**Run refactored tests:**
```bash
$ pytest tests/test_validator.py -v
================================ test session starts =================================
collected 10 items

tests/test_validator.py::test_validate_email[user@example.com-True] PASSED    [ 10%]
tests/test_validator.py::test_validate_email[test.user@example.co.uk-True] PASSED [ 20%]
tests/test_validator.py::test_validate_email[user+tag@example.com-True] PASSED [ 30%]
tests/test_validator.py::test_validate_email[user_name@example-domain.com-True] PASSED [ 40%]
tests/test_validator.py::test_validate_email[invalid-email-False] PASSED      [ 50%]
tests/test_validator.py::test_validate_email[@example.com-False] PASSED       [ 60%]
tests/test_validator.py::test_validate_email[user@-False] PASSED              [ 70%]
tests/test_validator.py::test_validate_email[user@.com-False] PASSED          [ 80%]
tests/test_validator.py::test_validate_email[user name@example.com-False] PASSED [ 90%]
tests/test_validator.py::test_validate_email[user@example-False] PASSED       [100%]

================================= 10 passed in 0.02s =================================
```

✅ **All tests pass after refactoring**

#### REFACTOR Phase - Extract Pattern to Constant

**File: `validator.py`**
```python
import re
from typing import ClassVar

class EmailValidator:
    EMAIL_PATTERN: ClassVar[str] = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

    def __init__(self):
        self._compiled_pattern = re.compile(self.EMAIL_PATTERN)

    def validate(self, email: str) -> bool:
        """Validate email format using RFC-compliant regex pattern."""
        if not email:
            return False
        return bool(self._compiled_pattern.match(email))
```

**Run tests after refactoring:**
```bash
$ pytest tests/test_validator.py -v
================================= 10 passed in 0.02s =================================
```

✅ **Tests still pass after refactoring**

**Commit:**
```bash
$ git add validator.py tests/
$ git commit -m "Add email validator with comprehensive tests

- Add EmailValidator class with regex pattern
- Parametrized tests for valid/invalid emails
- Extract pattern to class constant
- All 10 test cases passing"
```

---

## JavaScript TDD Example: Shopping Cart

### Iteration 1: Add Items to Cart

#### RED Phase - Write Failing Test

**File: `tests/cart.test.js`**
```javascript
import { describe, it, expect } from 'vitest'
import { ShoppingCart } from '../src/cart'

describe('ShoppingCart', () => {
  it('should add item to cart', () => {
    // Arrange
    const cart = new ShoppingCart()
    const item = { id: 1, name: 'Widget', price: 9.99 }

    // Act
    cart.addItem(item)

    // Assert
    expect(cart.getItems()).toHaveLength(1)
    expect(cart.getItems()[0]).toEqual(item)
  })
})
```

**Run test (RED):**
```bash
$ npm test -- cart.test.js

 FAIL  tests/cart.test.js
  ShoppingCart
    ✕ should add item to cart (2 ms)

  ● ShoppingCart › should add item to cart

    Cannot find module '../src/cart' from 'tests/cart.test.js'
```

✅ **Test fails for RIGHT reason**: Module doesn't exist yet

#### GREEN Phase - Minimum Implementation

**File: `src/cart.js`**
```javascript
export class ShoppingCart {
  constructor() {
    this.items = []
  }

  addItem(item) {
    this.items.push(item)
  }

  getItems() {
    return this.items
  }
}
```

**Run test (GREEN):**
```bash
$ npm test -- cart.test.js

 PASS  tests/cart.test.js
  ShoppingCart
    ✓ should add item to cart (2 ms)

Test Files  1 passed (1)
     Tests  1 passed (1)
```

✅ **Test passes**: Minimum implementation successful

### Iteration 2: Calculate Total

#### RED Phase - Add Total Calculation Test

**File: `tests/cart.test.js`**
```javascript
it('should calculate total price', () => {
  // Arrange
  const cart = new ShoppingCart()
  cart.addItem({ id: 1, name: 'Widget', price: 9.99 })
  cart.addItem({ id: 2, name: 'Gadget', price: 14.99 })

  // Act
  const total = cart.getTotal()

  // Assert
  expect(total).toBe(24.98)
})
```

**Run test (RED):**
```bash
$ npm test -- cart.test.js

 FAIL  tests/cart.test.js
  ShoppingCart
    ✓ should add item to cart (1 ms)
    ✕ should calculate total price (3 ms)

  ● ShoppingCart › should calculate total price

    TypeError: cart.getTotal is not a function
```

✅ **Test fails for RIGHT reason**: Method doesn't exist

#### GREEN Phase - Implement Total Calculation

**File: `src/cart.js`**
```javascript
export class ShoppingCart {
  constructor() {
    this.items = []
  }

  addItem(item) {
    this.items.push(item)
  }

  getItems() {
    return this.items
  }

  getTotal() {
    return this.items.reduce((sum, item) => sum + item.price, 0)
  }
}
```

**Run test (GREEN):**
```bash
$ npm test -- cart.test.js

 PASS  tests/cart.test.js
  ShoppingCart
    ✓ should add item to cart (1 ms)
    ✓ should calculate total price (1 ms)

Test Files  1 passed (1)
     Tests  2 passed (2)
```

✅ **Test passes**: Total calculation working

### Iteration 3: Handle Quantities

#### RED Phase - Add Quantity Test

**File: `tests/cart.test.js`**
```javascript
it('should handle item quantities', () => {
  // Arrange
  const cart = new ShoppingCart()
  const item = { id: 1, name: 'Widget', price: 9.99 }

  // Act
  cart.addItem(item, 3)

  // Assert
  expect(cart.getItems()).toHaveLength(1)
  expect(cart.getItems()[0].quantity).toBe(3)
  expect(cart.getTotal()).toBe(29.97)
})
```

**Run test (RED):**
```bash
$ npm test -- cart.test.js

 FAIL  tests/cart.test.js
  ● ShoppingCart › should handle item quantities

    expect(received).toBe(expected) // Object.is equality

    Expected: 3
    Received: undefined
```

✅ **Test fails for RIGHT reason**: Quantity not tracked

#### GREEN Phase - Add Quantity Support

**File: `src/cart.js`**
```javascript
export class ShoppingCart {
  constructor() {
    this.items = []
  }

  addItem(item, quantity = 1) {
    this.items.push({ ...item, quantity })
  }

  getItems() {
    return this.items
  }

  getTotal() {
    return this.items.reduce(
      (sum, item) => sum + item.price * item.quantity,
      0
    )
  }
}
```

**Run test (GREEN):**
```bash
$ npm test -- cart.test.js

 PASS  tests/cart.test.js
  ShoppingCart
    ✓ should add item to cart (1 ms)
    ✓ should calculate total price (1 ms)
    ✓ should handle item quantities (1 ms)

Test Files  1 passed (1)
     Tests  3 passed (3)
```

✅ **All tests pass**: Quantity support working

#### REFACTOR Phase - Round Total to 2 Decimals

**File: `src/cart.js`**
```javascript
export class ShoppingCart {
  constructor() {
    this.items = []
  }

  addItem(item, quantity = 1) {
    this.items.push({ ...item, quantity })
  }

  getItems() {
    return this.items
  }

  getTotal() {
    const total = this.items.reduce(
      (sum, item) => sum + item.price * item.quantity,
      0
    )
    // Round to 2 decimal places to avoid floating point errors
    return Math.round(total * 100) / 100
  }
}
```

**Run tests after refactoring:**
```bash
$ npm test -- cart.test.js

 PASS  tests/cart.test.js
  ShoppingCart
    ✓ should add item to cart (1 ms)
    ✓ should calculate total price (1 ms)
    ✓ should handle item quantities (1 ms)

Test Files  1 passed (1)
     Tests  3 passed (3)
```

✅ **Tests still pass after refactoring**

**Commit:**
```bash
$ git add src/cart.js tests/cart.test.js
$ git commit -m "Add shopping cart with item management

- Add ShoppingCart class with addItem and getTotal
- Support item quantities with default of 1
- Round totals to 2 decimal places
- All tests passing (3/3)"
```

---

## Advanced TDD Patterns

### Pattern 1: Property-Based Testing (Go)

**Test random inputs to find edge cases:**

```go
package strings

import (
    "testing"
    "testing/quick"
)

func TestReverse_PropertyBased_DoublReverse(t *testing.T) {
    // Property: Reverse(Reverse(s)) == s
    f := func(s string) bool {
        return Reverse(Reverse(s)) == s
    }

    if err := quick.Check(f, nil); err != nil {
        t.Error(err)
    }
}
```

### Pattern 2: Test Fixtures with Cleanup (Python)

**Use pytest fixtures for setup/teardown:**

```python
import pytest
import tempfile
import os

@pytest.fixture
def temp_database():
    """Create temporary database for testing."""
    # Setup
    db_fd, db_path = tempfile.mkstemp()
    db = Database(db_path)

    yield db  # Provide to test

    # Teardown
    db.close()
    os.close(db_fd)
    os.unlink(db_path)

def test_database_insert(temp_database):
    temp_database.insert({'id': 1, 'name': 'test'})
    assert temp_database.count() == 1
```

### Pattern 3: Mocking External Dependencies (JavaScript)

**Use vitest mocks to isolate code under test:**

```javascript
import { vi, describe, it, expect, beforeEach } from 'vitest'
import { UserService } from '../src/userService'
import { ApiClient } from '../src/apiClient'

// Mock the API client
vi.mock('../src/apiClient')

describe('UserService', () => {
  let mockApiClient
  let userService

  beforeEach(() => {
    mockApiClient = {
      get: vi.fn()
    }
    userService = new UserService(mockApiClient)
  })

  it('should fetch user by id', async () => {
    // Arrange
    const mockUser = { id: 1, name: 'John' }
    mockApiClient.get.mockResolvedValue(mockUser)

    // Act
    const result = await userService.getUserById(1)

    // Assert
    expect(mockApiClient.get).toHaveBeenCalledWith('/users/1')
    expect(result).toEqual(mockUser)
  })
})
```

---

## TDD Patterns to Detect and Fix

### Signal 1: Writing Tests After Implementation

**Wrong:**
```javascript
// 1. Write implementation first
function calculateDiscount(price, percent) {
  return price * (percent / 100)
}

// 2. Then write test
it('should calculate discount', () => {
  expect(calculateDiscount(100, 10)).toBe(10)
})
```

**Right:**
```javascript
// 1. Write test first (RED)
it('should calculate discount', () => {
  expect(calculateDiscount(100, 10)).toBe(10)
})

// 2. Then implement (GREEN)
function calculateDiscount(price, percent) {
  return price * (percent / 100)
}
```

### Signal 2: Testing Implementation Details

**Wrong:**
```python
def test_user_service_uses_cache():
    service = UserService()
    service.get_user(1)
    # Testing HOW it works (implementation detail)
    assert service._cache_hits == 1
```

**Right:**
```python
def test_user_service_returns_user():
    service = UserService()
    user = service.get_user(1)
    # Testing WHAT it does (behavior)
    assert user.id == 1
    assert user.name is not None
```

### Signal 3: Overly Broad Tests

**Wrong:**
```go
func TestEverything(t *testing.T) {
    // Tests too many things at once
    user := CreateUser("john", "john@example.com")
    if user.Name != "john" {
        t.Error("name wrong")
    }
    if user.Email != "john@example.com" {
        t.Error("email wrong")
    }
    user.UpdateEmail("new@example.com")
    if user.Email != "new@example.com" {
        t.Error("email update wrong")
    }
}
```

**Right:**
```go
func TestCreateUser_ValidData_ReturnsUser(t *testing.T) {
    user := CreateUser("john", "john@example.com")

    if user.Name != "john" {
        t.Errorf("expected name %q, got %q", "john", user.Name)
    }
}

func TestUpdateEmail_ValidEmail_UpdatesSuccessfully(t *testing.T) {
    user := CreateUser("john", "john@example.com")

    user.UpdateEmail("new@example.com")

    if user.Email != "new@example.com" {
        t.Errorf("expected email %q, got %q", "new@example.com", user.Email)
    }
}
```

---

## TDD Workflow Summary

**The cycle for EVERY feature:**

1. **RED**: Write failing test → Run test → Verify failure reason
2. **GREEN**: Write minimum code → Run test → Verify pass
3. **REFACTOR**: Improve code → Run tests → Verify still pass
4. **COMMIT**: Save working test + implementation

**Key principles:**
- Test first, always
- Verify test fails for the RIGHT reason
- Implement minimum code to pass
- Refactor incrementally with green tests
- Commit atomic units of working code

# TDD Phase Guidance

Detailed rationale, steps, and code examples for each phase of the RED-GREEN-REFACTOR cycle. Loaded on demand; the SKILL.md body holds the lean phase skeleton and gates.

## Phase 1: Write a Failing Test (RED)

The test MUST exist and fail before any implementation code is written, because seeing the test fail first proves it can actually detect the bug or missing feature. A test that has never been seen failing provides no evidence that it tests anything meaningful.

**Steps:**

1. **Understand the requirement** -- clarify what behavior needs to be implemented
2. **Write the test first** -- create a test that describes the desired behavior
3. **Use descriptive test names** -- the test name should read as a specification of behavior (e.g., `TestCalculateTotal_WithEmptyCart_ReturnsZero`), because vague names like `TestCalc` make failures impossible to diagnose without reading the test body
4. **Write minimal test setup** -- only create fixtures/mocks needed for THIS test
5. **Assert expected behavior** -- use specific assertions (not just "no error"), because weak assertions like `assert result != nil` pass for wrong reasons and provide false confidence

Use specific assertions:
- `assert result == 42` (specific value)
- `assert error.message.contains("invalid")` (specific content)
- NOT `assert result != nil` (too weak -- passes even when result is garbage)
- NOT `assert len(result) > 0` (not specific enough -- passes with wrong data)

Test one concept per test. If the test name needs "and", split into multiple tests, because multi-assertion tests produce ambiguous failures.

Follow the Arrange-Act-Assert pattern:

```python
def test_feature():
    # Arrange: Set up test data
    input_data = create_test_data()

    # Act: Execute the code under test
    result = function_under_test(input_data)

    # Assert: Verify expected behavior
    assert result.status == "success"
```

**Optional techniques** (use when explicitly requested):
- **Property-based testing**: Generate tests with random/fuzzed inputs (Go: `testing/quick`, Python: `hypothesis`)
- **Table-driven tests**: Convert multiple similar tests to data-driven approach when 3+ tests share the same structure

**Run the test:**
```bash
go test ./... -v -run TestNewFeature          # Go
pytest tests/test_feature.py::test_name -v    # Python
npm test -- --testNamePattern="new feature"   # JavaScript
```

Show the full test runner output -- never summarize test results, because summarization hides warnings, partial failures, and unexpected output that reveal problems early.

## Phase 2: Verify Failure Reason (RED Verification)

The test must fail because the feature is not implemented, NOT because of syntax errors, import errors, wrong test setup, or unrelated failures. A test that fails for the wrong reason proves nothing about the missing feature and will pass for the wrong reason after implementation.

1. **Execute test command** and show the complete output
2. **Verify failure reason** -- confirm the error matches expected missing-implementation patterns:
   - Go: `--- FAIL: TestFeatureName` with expected vs actual mismatch
   - Python: `AssertionError` or `AttributeError: module has no attribute`
   - JavaScript: `Expected X but received undefined`

**If the test fails for the WRONG reason:**
- Fix the test setup/syntax
- Re-run until it fails for the RIGHT reason (missing implementation)
- Proceed only when the failure clearly indicates "this feature does not exist yet"

## Phase 3: Implement Minimum Code (GREEN)

Write ONLY enough code to make the failing test pass. Implement nothing beyond what the test demands, because untested code paths are invisible liabilities -- they cannot be verified, they rot silently, and they complicate future refactoring.

1. **Minimal implementation** -- the simplest code that satisfies the test
2. **No extra features** -- do not implement behavior not covered by tests. First make it work, then make it right
3. **Hardcoded values are OK initially** -- a hardcoded return that passes the test is better than a generic algorithm that also handles untested cases

Wrong (over-engineering in GREEN phase):
```go
// Test only requires simple addition
func TestCalculator_AddTwoNumbers(t *testing.T) {
    calc := NewCalculator()
    result := calc.Add(2, 3)
    assert.Equal(t, 5, result)
}

// But implementation adds unnecessary complexity
type Calculator struct {
    operations map[string]func(float64, float64) float64
    precision  int
    history    []Operation
}
```

Correct (implement only what is tested):
```go
type Calculator struct{}

func (c *Calculator) Add(a, b int) int {
    return a + b
}
// Add complexity ONLY when a test requires it
```

## Phase 4: Verify Test Passes (GREEN Verification)

Run the test and show the complete output. Never summarize -- the full output reveals warnings, deprecation notices, and timing issues that summaries hide.

1. **Execute test command** and display all output
2. **Verify PASS status**
3. **Run the full test suite** -- not just the new test, because a change that makes one test pass while breaking another is not progress. Run tests after every code modification to catch regressions immediately

```bash
go test ./... -v                    # Go - all tests
pytest -v                           # Python - all tests
npm test                            # JavaScript - all tests
```

**If the test still fails:**
- Review implementation logic
- Check test assertions are correct
- Debug until the test passes

## Phase 5: Refactor (REFACTOR)

Improve code quality without changing behavior. Run the full test suite before refactoring to establish a green baseline, because you need proof that any future failure was caused by your refactoring, not by a pre-existing issue.

**Refactoring decision criteria** (evaluate each):

| Criterion | Check | Action if YES |
|-----------|-------|---------------|
| Duplication | Same logic in 2+ places? | Extract to shared function |
| Naming | Names unclear or misleading? | Rename for clarity |
| Length | Function >20 lines? | Extract sub-functions |
| Complexity | Nested conditionals >2 deep? | Simplify or extract |
| Reusability | Could other code use this? | Extract to module |

1. **Run full test suite BEFORE refactoring** -- establish green baseline
2. **Refactor incrementally** -- extract functions, rename for clarity, remove duplication
3. **Run tests after EACH refactoring step** -- ensure tests stay green after every individual change, because large refactoring batches make it impossible to identify which change broke the test
4. **Refactor tests too** -- improve test readability and maintainability. Suggest better assertions, edge cases, and test organization where they would strengthen coverage

Test behavior, not implementation details. Tests coupled to internals break on refactoring and defeat its purpose:

Wrong (testing internals):
```go
func TestParser_UsesCorrectRegex(t *testing.T) {
    parser := NewParser()
    // Testing internal regex pattern -- breaks on refactor
    assert.Equal(t, `\d{3}-\d{3}-\d{4}`, parser.phoneRegex)
}
```

Correct (testing behavior):
```go
func TestParser_ValidPhoneNumber_ParsesCorrectly(t *testing.T) {
    parser := NewParser()
    result, err := parser.ParsePhone("123-456-7890")
    assert.NoError(t, err)
    assert.Equal(t, "1234567890", result.Digits())
}

func TestParser_InvalidPhoneNumber_ReturnsError(t *testing.T) {
    parser := NewParser()
    _, err := parser.ParsePhone("invalid")
    assert.Error(t, err)
    assert.Contains(t, err.Error(), "invalid phone format")
}
```

Track which code paths are tested and suggest missing coverage, because untested paths are invisible to the refactoring safety net.

**Optional techniques** (use when explicitly requested):
- **Mutation testing**: Verify test quality by introducing bugs -- if mutating code does not break a test, that test is too weak
- **Benchmark tests**: Performance regression testing to ensure refactoring does not degrade speed
- **Test parallelization**: Run independent tests concurrently for speed

## Phase 6: Commit

Commit the test and implementation together as an atomic unit, because separating them creates a window where the repository is in an inconsistent state -- either tests exist for unimplemented code, or code exists without its test coverage.

1. **Review changes** -- verify test + implementation are complete
2. **Run full test suite** -- ensure nothing broke
3. **Commit with descriptive message**

After committing, clean up any temporary test files, coverage reports, or debug outputs created during the TDD cycle. Keep only files explicitly needed for the project.

Report facts without self-congratulation. Show command output rather than describing it.

## Cycle Discipline

Each feature gets its own RED-GREEN-REFACTOR cycle. Do not batch multiple features into one cycle:

Wrong (implementing everything at once):
```javascript
// Implementing many features at once without tests
class UserManager {
  createUser(data) { /* complex logic */ }
  updateUser(id, data) { /* complex logic */ }
  deleteUser(id) { /* complex logic */ }
  validateUser(user) { /* complex logic */ }
}
// Then one giant test for everything
```

Correct (one cycle per feature):
```javascript
// Cycle 1: Create user (RED -> GREEN -> REFACTOR)
it('should create user with valid data', () => {
    const manager = new UserManager()
    const user = manager.createUser({ name: 'Alice', email: 'alice@example.com' })
    expect(user.id).toBeDefined()
    expect(user.name).toBe('Alice')
})
// Implement createUser() to pass, then move to next cycle

// Cycle 2: Validate user (RED -> GREEN -> REFACTOR)
it('should reject user with invalid email', () => {
    const manager = new UserManager()
    expect(() => manager.createUser({ name: 'Bob', email: 'invalid' }))
      .toThrow('Invalid email format')
})
// Add validation to make test pass
```

## Language-Specific Testing Commands

| Language | Run One Test | Run All | With Coverage |
|----------|-------------|---------|---------------|
| Go | `go test -v -run TestName ./pkg` | `go test ./...` | `go test -cover ./...` |
| Python | `pytest tests/test_file.py::test_fn -v` | `pytest` | `pytest --cov=src` |
| JavaScript | `npm test -- --testNamePattern="name"` | `npm test` | `npm test -- --coverage` |

# Adversarial Artifact Verification Methodology

> **Core Principle**: Verify what ACTUALLY exists in the codebase. The verification question is not "did the executor say it's done?" but "does the codebase prove it's done?"

This methodology goes deeper than test/build/lint checks: it verifies that artifacts are real implementations (not stubs), actually integrated (not orphaned), and processing real data (not hardcoded empties). Apply after Steps 1-7 pass, focusing on artifacts that are part of the stated goal.

**Why four levels**: Existence checks (L1) catch forgotten writes. Substance checks (L2) catch stubs. Wiring checks (L3) catch orphaned files. Data flow checks (L4) catch integration that exists structurally but passes no real data. Each level catches a distinct class of premature-completion failure.

## Goal-Backward Framing

**Replace this question**: "Were all tasks completed?"
**Instead ask**: "What must be TRUE for the goal to be achieved?"

This framing prevents task-forward verification that invites executors to confirm their own narrative. Goal-backward verification derives conditions independently from the goal itself, then checks whether the codebase satisfies them. This structural approach counteracts confirmation bias.

**Procedure:**

1. **State the goal as a testable condition**: Express what the user asked for as a concrete, verifiable outcome.
   - Example: "Users can create a PR with quality scoring that blocks merges below threshold"

2. **Decompose into must-be-true conditions**: Break the goal into independent conditions that must ALL hold.
   - "A scoring function exists" (L1)
   - "It contains real scoring logic, not stubs" (L2)
   - "It is called by the PR pipeline" (L3)
   - "It receives actual PR data and its score affects the merge gate" (L4)

3. **Verify each condition independently** at the appropriate level using the 4-Level system below.

4. **Report unverified conditions** as blockers — not "you missed a task" but "this condition is not yet true in the codebase."

## The Four Levels of Artifact Verification

Each artifact produced during the task is verified at four progressively deeper levels. Higher levels subsume lower ones — an artifact at Level 4 has passed Levels 1-3 by definition.

### Level 1: EXISTS — File is present on disk

**Check**: Use Glob or Bash (`ls`, `test -f`) to confirm the file exists.

**What this catches**: Claims about files that were planned but not written to disk (forgotten Write calls, planned-but-not-executed steps).

**What this misses**: Everything else. Existence is necessary but nowhere near sufficient.

---

### Level 2: SUBSTANTIVE — File contains real logic, not placeholder implementations

**Check**: Scan for stub indicators using Grep against changed files. See the **Stub Detection Patterns** table below. A match does not automatically mean failure — `return []` is sometimes correct — but each match requires investigation to confirm the empty return or placeholder is intentional.

**What this catches**: Files that exist but contain no real implementation — the most common form of premature completion claim. This catches stubs disguised as code.

**What this misses**: Code that has logic but wrong logic, or logic that handles only the happy path.

---

### Level 3: WIRED — The artifact is imported AND used by other code in the codebase

**Check**:
1. Search for import/require statements referencing the artifact
2. Verify the imported symbols are actually called (not just imported)
3. Check that the call sites pass real arguments (not empty objects or nil)

```bash
# Example: Check if scoring.py is imported anywhere
grep -r "from.*scoring import\|import.*scoring" --include="*.py" .

# Example: Check if the imported function is actually called
grep -r "calculate_score\|score_package" --include="*.py" .
```

**What this catches**: Orphaned files that were created but left unintegrated. Wiring gaps indicate the component exists structurally but is not active in the system.

**What this misses**: Circular or dead-end wiring where the integration exists but the code path is unreachable at runtime.

---

### Level 4: DATA FLOWS — Real data reaches the artifact and real results come out

**Check**:
1. Trace the call chain from entry point to the artifact
2. Verify inputs are not hardcoded empty values (`[]`, `{}`, `""`, `0`)
3. Verify outputs are consumed by downstream code (not discarded)
4. If tests exist, verify test inputs exercise meaningful cases (not just empty-input tests)

**What this catches**: Integration that exists structurally but passes no real data — functions wired in but fed empty arrays, handlers registered but inactive. Data flow verification confirms the entire chain is active end-to-end.

**What this misses**: Semantic correctness (the data flows but produces wrong results). That is the domain of testing, not verification.

## Stub Detection Patterns for Level 2 (SUBSTANTIVE)

Scan changed files for these patterns to verify they contain real logic, not placeholder implementations:

| Pattern | Language | Indicates |
|---------|----------|-----------|
| `return []` | Python, JS/TS | Empty list return — may be stub if function should compute results |
| `return {}` | Python, JS/TS | Empty dict/object return — may be stub if function should build a structure |
| `return None` | Python | Sole return in non-optional function — likely stub |
| `return nil, nil` | Go | Returning no value and no error — likely stub |
| `return nil` | Go | Single nil return in a function expected to produce a value |
| `pass` (as sole body) | Python | Empty function body — definite stub |
| `...` (Ellipsis as body) | Python | Protocol/abstract stub — should not appear in concrete implementations |
| `() => {}` | JS/TS | Empty arrow function — no-op handler |
| `onClick={() => {}}` | JSX/TSX | Empty click handler — UI wired but non-functional |
| `throw new Error("not implemented")` | JS/TS | Explicit "not done" marker |
| `panic("not implemented")` | Go | Explicit "not done" marker |
| `raise NotImplementedError` | Python | Explicit "not done" marker |
| `TODO`, `FIXME`, `HACK`, `XXX` | Any | Markers for incomplete work (in non-test files) |
| `PLACEHOLDER`, `stub`, `mock` | Any | Self-described placeholder code (in non-test files) |
| `"coming soon"`, `"not yet implemented"` | Any | Placeholder UI/API text |

**Automated scan command** (run against files changed in the current task):

```bash
# Get changed files relative to base branch
changed_files=$(git diff --name-only main...HEAD)

# Scan for stub patterns (adjust base branch as needed)
grep -n -E "(return \[\]|return \{\}|return None|return nil|pass$|raise NotImplementedError|panic\(\"not implemented\"\)|throw new Error\(\"not implemented\"\)|TODO|FIXME|HACK|XXX|PLACEHOLDER)" $changed_files
```

**Review methodology**: Each match requires investigation. If the pattern is intentional (e.g., a function that genuinely returns an empty list), note it in the verification report with rationale. If it is a stub, flag it as a blocker — resolve stubs before declaring task complete.

## Completion Shortcut Scan (Level 2 Supplement)

Beyond stub detection, scan for patterns that indicate premature completion claims:

**Log-only functions** — functions whose entire body is a log/print statement with no real logic:
```bash
# Python: functions that only log
grep -A2 "def " $changed_files | grep -B1 "logging\.\|print(" | grep "def "
```

**Empty handlers** — event handlers that prevent default but do nothing else:
```bash
grep -n "onSubmit.*preventDefault" $changed_files
grep -n "handler.*{\\s*}" $changed_files
```

**Placeholder text** in non-test files:
```bash
grep -n -i "(placeholder|example data|test data|lorem ipsum)" $changed_files
```

**Dead imports** — modules imported but unused:
```bash
# Python: imported but not referenced later in the file
# (manual check — read the file and verify each import is used)
```

---

## Verification Report Format

After completing 4-level verification, produce a structured report. This replaces the simpler verification statement in Step 7 when adversarial verification applies:

```markdown
## Verification Report

### Goal
[Stated goal as a testable condition]

### Conditions

| Condition | L1 | L2 | L3 | L4 | Status |
|-----------|----|----|----|----|--------|
| [condition 1] | Y/N | Y/N | Y/N | Y/N/- | VERIFIED / INCOMPLETE — [reason] |
| [condition 2] | Y/N | Y/N | Y/N | Y/N/- | VERIFIED / INCOMPLETE — [reason] |

### Blockers
- [Any condition not verified at the required level]

### Stub Scan Results
- [N matches found, M confirmed intentional, K flagged as blockers]

### Verdict
**COMPLETE** / **NOT COMPLETE** — [summary]
```

Use `-` in a level column when that level does not apply (e.g., a configuration file does not need L3 wiring checks).

---

## When to Apply Each Level

Not every artifact needs Level 4 verification. Apply only the minimum level required, avoiding unnecessary overhead on trivial changes:

| Artifact Type | Minimum Level | Rationale |
|---------------|---------------|-----------|
| Core feature code (new modules, handlers, logic) | Level 4 | Must prove data flows end-to-end |
| Configuration files, YAML, env | Level 1 | Existence is sufficient — content verified by build/tests |
| Test files | Level 2 | Must be substantive (not empty test stubs), but wiring is implicit |
| Documentation, README, comments | Level 1 | Existence check only |
| Integration glue (imports, routing, wiring) | Level 3 | Must be wired, but data flow verified through the module it connects |
| Bug fixes to existing code | Level 2 + tests | Substance verified, plus tests must cover the fix |

# Verification Checklists

Comprehensive verification checklists for different domains and scenarios.

## Universal Verification Checklist

Use this checklist for ANY code change, regardless of language or domain:

### Core Checks (Required)
- [ ] **Tests executed**: Ran relevant test suite with verbose output
- [ ] **Tests passed**: All tests completed successfully (output shown)
- [ ] **Build succeeded**: Project builds without errors (output shown)
- [ ] **Files reviewed**: Used Read tool on all changed files
- [ ] **Syntax validated**: Code parses correctly (syntax checker run)
- [ ] **No debug code**: Removed console.log, print statements, debug flags
- [ ] **Diff checked**: Reviewed git diff for unintended changes
- [ ] **Dependencies resolved**: All imports/requires work correctly

### Extended Checks (Recommended)
- [ ] **Documentation updated**: README, docstrings, comments reflect changes
- [ ] **Error handling**: Edge cases and errors properly handled
- [ ] **Backwards compatibility**: Existing functionality not broken
- [ ] **Performance check**: No obvious performance regressions
- [ ] **Security review**: No credentials, secrets, or vulnerabilities introduced
- [ ] **Type safety**: Type annotations correct and checked (where applicable)
- [ ] **Resource cleanup**: Proper cleanup of files, connections, locks
- [ ] **Logging appropriate**: Important operations logged, no excessive logging

---

## Python Project Checklist

### Python-Specific Verification
- [ ] **pytest executed**: `pytest -v` run with full output shown
- [ ] **Test coverage**: Coverage report generated and reviewed (aim for >80%)
- [ ] **Syntax check**: `python -m py_compile` on all changed .py files
- [ ] **Import validation**: All imports resolve correctly
- [ ] **Type hints**: Type annotations correct (if using mypy/pyright)
- [ ] **Linting**: `ruff` or `flake8` passes without new warnings
- [ ] **Formatting**: `black` or `ruff format` applied consistently
- [ ] **Security scan**: `bandit` run for security issues (if using python-quality-gate)
- [ ] **Dependencies**: No new vulnerable dependencies added

### Flask Application Specific
- [ ] **App starts**: Flask app runs without errors
- [ ] **Routes accessible**: Test endpoints with curl/requests
- [ ] **Database migrations**: Migrations up-to-date (if using Flask-Migrate)
- [ ] **Templates render**: HTML templates don't have syntax errors
- [ ] **Static files**: JavaScript/CSS files load correctly

### Django Application Specific
- [ ] **Migrations created**: `python manage.py makemigrations --check` passes
- [ ] **Migrations applied**: `python manage.py migrate` succeeds
- [ ] **Admin registered**: Models show correctly in admin (if applicable)
- [ ] **URL patterns**: `python manage.py check --deploy` passes
- [ ] **Static collected**: `python manage.py collectstatic` works

### FastAPI Application Specific
- [ ] **OpenAPI docs**: `/docs` endpoint shows correct schema
- [ ] **Pydantic validation**: Request/response models validate correctly
- [ ] **Dependency injection**: Dependencies resolve properly
- [ ] **Async operations**: No blocking calls in async functions

---

## Go Project Checklist

### Go-Specific Verification
- [ ] **Tests executed**: `go test ./... -v` run with full output
- [ ] **Race detection**: `go test -race ./...` passes
- [ ] **Build succeeds**: `go build ./...` completes without errors
- [ ] **Linting**: `golangci-lint run ./...` passes
- [ ] **Go modules**: `go mod tidy` doesn't change go.mod/go.sum
- [ ] **Formatting**: `gofmt -l .` shows no unformatted files
- [ ] **Vet passed**: `go vet ./...` shows no issues

### Go Best Practices
- [ ] **Error wrapping**: Errors wrapped with context (fmt.Errorf with %w)
- [ ] **Resource cleanup**: defer statements for Close/Cancel operations
- [ ] **Thread safety**: Concurrent code uses proper synchronization
- [ ] **Context propagation**: Context passed through call chains
- [ ] **Structured logging**: Uses structured logging (not fmt.Println)

### Go Service/API Specific
- [ ] **Service builds**: Binary builds without errors
- [ ] **Service starts**: Can start and handle graceful shutdown
- [ ] **Health endpoint**: `/healthz` or equivalent responds correctly
- [ ] **Metrics exposed**: Prometheus metrics endpoint accessible
- [ ] **API contract**: OpenAPI/Swagger spec matches implementation

---

## JavaScript/TypeScript Project Checklist

### JavaScript-Specific Verification
- [ ] **Tests executed**: `npm test` run with full output shown
- [ ] **Build succeeds**: `npm run build` completes without errors
- [ ] **Syntax check**: `node -c` on all changed .js files
- [ ] **Linting**: `npm run lint` passes without new warnings
- [ ] **Type checking**: `npx tsc --noEmit` passes (if TypeScript)
- [ ] **Dependencies installed**: `npm install` completes successfully

### React Application Specific
- [ ] **Development server**: `npm start` runs without errors
- [ ] **Production build**: `npm run build` creates optimized bundle
- [ ] **Bundle size**: Build output shows reasonable file sizes
- [ ] **Components render**: No React errors in console
- [ ] **Tests pass**: Jest/React Testing Library tests pass

### Node.js Backend Specific
- [ ] **Server starts**: Application starts and listens on port
- [ ] **API endpoints**: Routes respond with expected status codes
- [ ] **Database connection**: Database connects successfully
- [ ] **Environment variables**: All required env vars documented
- [ ] **Error handling**: Unhandled promise rejections caught

---

## Database Change Checklist

### Schema Changes
- [ ] **Migration created**: Database migration file generated
- [ ] **Migration tested**: Migration runs successfully on test database
- [ ] **Rollback tested**: Down migration works correctly
- [ ] **Data preserved**: Existing data not lost during migration
- [ ] **Indexes created**: Appropriate indexes added for new columns
- [ ] **Constraints valid**: Foreign keys, unique constraints work correctly

### Query Changes
- [ ] **Queries tested**: New/modified queries execute successfully
- [ ] **Performance checked**: EXPLAIN ANALYZE shows reasonable plan
- [ ] **Indexes used**: Queries use appropriate indexes
- [ ] **N+1 prevented**: No new N+1 query patterns introduced
- [ ] **Transactions proper**: Transaction boundaries correct

### Schema Verification Gate

Run when a diff touches migration files or schema definitions. Verify schema state before and after the change, not just that the migration command exited 0.

**Before the change** — capture the baseline schema:
- [ ] **SQLite**: `sqlite3 <db> "SELECT sql FROM sqlite_master WHERE type IN ('table','index')"` -- save output
- [ ] **Django**: `python manage.py showmigrations` -- confirm current applied state
- [ ] **Rails**: `rails db:migrate:status` -- confirm current applied state
- [ ] **Raw SQL migrations**: dump schema (`pg_dump --schema-only`, `mysqldump --no-data`, or equivalent) -- save output

**After the change** — diff against the baseline:
- [ ] **SQLite**: re-run `SELECT sql FROM sqlite_master ...`, diff against before-output
- [ ] **Django**: `showmigrations` shows the new migration applied; `makemigrations --check` reports no pending changes
- [ ] **Rails**: `db:migrate:status` shows the new migration `up`
- [ ] **Raw SQL migrations**: re-dump schema, diff against before-output -- confirm only the intended tables/columns changed

**Duplicate/collision check**:
- [ ] **No existing equivalent**: grep the schema dump / model definitions for a table or column that already covers this data under another name (renamed field, near-duplicate table) before adding a new one
- [ ] **Existing queries compatible**: grep the codebase for references to the changed table/column (raw SQL strings, ORM model fields, serializers) -- confirm none break against the new schema

---

## Infrastructure/DevOps Checklist

### Configuration Changes
- [ ] **Config validated**: Configuration files parse correctly
- [ ] **Secrets secured**: No credentials in version control
- [ ] **Environment tested**: Changes work in target environment
- [ ] **Backwards compatible**: Old deployments still function
- [ ] **Documentation updated**: Deployment docs reflect changes

### Kubernetes/Helm Changes
- [ ] **YAML valid**: `kubectl apply --dry-run` succeeds
- [ ] **Helm lint**: `helm lint` passes without errors
- [ ] **Helm template**: `helm template` generates correct manifests
- [ ] **Resources defined**: CPU/memory limits set appropriately
- [ ] **Probes configured**: Liveness/readiness probes working

### Docker Changes
- [ ] **Image builds**: `docker build` completes successfully
- [ ] **Image size**: Image size reasonable (not bloated)
- [ ] **Container starts**: `docker run` starts container without errors
- [ ] **Layers optimized**: Dockerfile uses layer caching effectively
- [ ] **Security scanned**: Image scanned for vulnerabilities

---

## Quality Gate Checklist

Use this when running language-specific quality gates:

### Go Quality Gate (go-patterns)
- [ ] **golangci-lint**: All linters pass without violations
- [ ] **go test**: All tests pass with `-v` flag
- [ ] **Race detector**: `go test -race` passes without issues
- [ ] **go build**: Project builds successfully
- [ ] **go vet**: Static analysis passes
- [ ] **gofmt**: All files properly formatted
- [ ] **Coverage**: Test coverage meets project standards (typically >70%)

### Python Quality Gate (python-quality-gate)
- [ ] **ruff**: Linting passes without violations
- [ ] **pytest**: All tests pass with coverage report
- [ ] **mypy**: Type checking passes (if configured)
- [ ] **bandit**: Security scan passes without HIGH severity issues
- [ ] **Coverage**: Test coverage meets standards (typically >80%)

### Universal Quality Gate (universal-quality-gate)
- [ ] **Language detection**: All project languages detected correctly
- [ ] **Per-language linting**: Each language's linter passes
- [ ] **Per-language tests**: Each language's tests pass
- [ ] **Build verification**: All languages build successfully
- [ ] **Summary report**: Overall quality report shows all green

---

## Documentation Change Checklist

### Documentation Verification
- [ ] **Markdown valid**: Markdown syntax correct (no broken formatting)
- [ ] **Links work**: All links point to valid URLs/files
- [ ] **Code examples**: Code snippets are syntactically correct
- [ ] **Examples tested**: Code examples actually run successfully
- [ ] **Spelling checked**: No obvious typos or misspellings
- [ ] **Formatting consistent**: Consistent style throughout

### API Documentation Specific
- [ ] **Examples accurate**: API examples match actual API behavior
- [ ] **Parameters documented**: All parameters listed with types
- [ ] **Responses documented**: Response formats and status codes listed
- [ ] **Errors documented**: Error conditions and codes documented
- [ ] **Authentication described**: Auth requirements clearly stated

---

## CI/CD Pipeline Checklist

### Pipeline Changes
- [ ] **Syntax valid**: CI config file parses correctly
- [ ] **Pipeline runs**: Pipeline executes without errors
- [ ] **Tests execute**: Test jobs run successfully
- [ ] **Build succeeds**: Build jobs complete without errors
- [ ] **Deploy tested**: Deployment steps work in staging
- [ ] **Rollback possible**: Rollback mechanism still functional

---

## Security Change Checklist

### Security Verification
- [ ] **Secrets excluded**: No credentials in code or logs
- [ ] **Input validated**: User input properly sanitized
- [ ] **Authentication checked**: Auth mechanisms still work
- [ ] **Authorization verified**: Permission checks correct
- [ ] **Dependencies scanned**: No new vulnerable dependencies
- [ ] **Security tests**: Security-related tests pass

---

## Performance Change Checklist

### Performance Verification
- [ ] **Benchmarks run**: Performance benchmarks executed
- [ ] **No regression**: Performance not significantly worse
- [ ] **Memory usage**: Memory consumption reasonable
- [ ] **Load tested**: Handles expected load (if applicable)
- [ ] **Profiling done**: CPU/memory profiling shows no issues

---

## Hotfix/Emergency Change Checklist

### Minimal Viable Verification (Emergency Only)
- [ ] **Core tests pass**: Critical path tests executed and passed
- [ ] **Build succeeds**: Application builds without errors
- [ ] **Smoke test**: Basic functionality verified
- [ ] **Rollback ready**: Can rollback quickly if needed
- [ ] **Monitoring active**: Can detect issues in production

**Note**: Document any skipped verification steps and plan comprehensive testing post-deployment.

---

## Verification Report Template

After completing verification, provide a report in this format:

```
✅ Verification Complete

**Domain**: [Python Flask / Go Service / React App / etc.]

**Tests Executed**:
```
[paste complete test output]
```

**Build Status**:
```
[paste complete build output]
```

**Files Verified**:
- `path/to/file1.py`: ✅ Reviewed, syntax valid, logic correct
- `path/to/file2.go`: ✅ Reviewed, syntax valid, logic correct
- `path/to/file3.js`: ✅ Reviewed, syntax valid, logic correct

**Checklist Status**:
- Core checks: 8/8 passed ✅
- Extended checks: 5/5 passed ✅
- Domain-specific checks: 7/7 passed ✅

**Total**: 20/20 verification checks passed

**Verification Evidence**:
- Tests: [link to test output or paste above]
- Build: [link to build output or paste above]
- Changed files reviewed with Read tool: [Yes/No]

**Next Steps**:
Test if this addresses the issue. Please verify the changes work for your specific use case.
```

---

## Completion Checks

### Patterns That Fail Verification
- Say "tests pass" without showing output
- Skip verification because "it's a small change"
- Assume tests exist without checking
- Mark complete based on code review alone
- Say "should work" or "should be fixed"
- Summarize test results instead of showing them

### ✅ ALWAYS Do This
- Show complete, unabbreviated test output
- Run verification even for one-line changes
- Check if tests exist, acknowledge if they don't
- Run actual tests, not just read code
- Say "test if this addresses the issue"
- Display full verification evidence

---

## Verification Levels

### Level 1: Quick Verification (< 2 minutes)
- Syntax check
- Build check
- Quick smoke test
- Changed files reviewed

**Use when**: Documentation changes, config tweaks, very small code changes

### Level 2: Standard Verification (2-5 minutes)
- Full test suite for affected modules
- Build verification
- Changed files reviewed
- Diff checked for unintended changes
- Basic integration check

**Use when**: Feature changes, bug fixes, refactoring

### Level 3: Comprehensive Verification (5-15 minutes)
- Full test suite (all tests)
- Integration tests
- Performance benchmarks
- Security scan
- Cross-platform testing
- Load testing (if applicable)

**Use when**: Major changes, API changes, security-sensitive code, pre-release

### Level 4: Production Verification (15+ minutes)
- All of Level 3
- Staging deployment
- Production smoke test
- Monitoring validation
- Rollback drill

**Use when**: Production deployments, critical hotfixes, infrastructure changes

---

## Success Criteria

Verification is complete when:
1. **All applicable checklists** are satisfied (boxes checked)
2. **Full test output** is shown (not summarized)
3. **Build output** is shown (not summarized)
4. **Changed files** are reviewed with Read tool
5. **Verification report** is provided using template above
6. **User is asked to test** with phrase "Test if this addresses the issue"

**Remember**: The goal is not to check boxes, but to have **confidence** that the changes work correctly and don't break anything.

# Verification Examples: Good vs Bad

This document shows real-world examples of good and bad verification practices to illustrate the defense-in-depth approach.

## Example 1: Python Bug Fix

### Insufficient Verification

**User**: Fix the login bug where passwords aren't being validated

**Assistant**: I've fixed the password validation in auth.py. The bug was that we weren't checking password length. Should be working now!

**Problems**:
- No tests run
- No code shown
- No verification of the fix
- Says "should be working" without evidence
- User has no confidence the fix works

---

### Thorough Verification

**User**: Fix the login bug where passwords aren't being validated

**Assistant**: I'll fix the password validation issue. Let me identify the problem and verify the fix.

**Step 1: Review the code**
```bash
$ cat -n /home/project/auth.py
    15  def validate_password(password):
    16      # TODO: Add validation
    17      return True
```

**Issue found**: Password validation always returns True

**Step 2: Implement fix**
```python
def validate_password(password):
    if len(password) < 8:
        return False
    if not any(c.isupper() for c in password):
        return False
    if not any(c.isdigit() for c in password):
        return False
    return True
```

**Step 3: Run tests**
```bash
$ pytest tests/test_auth.py -v
============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.0
collected 5 tests

tests/test_auth.py::test_password_too_short PASSED                       [ 20%]
tests/test_auth.py::test_password_no_uppercase PASSED                    [ 40%]
tests/test_auth.py::test_password_no_digit PASSED                        [ 60%]
tests/test_auth.py::test_password_valid PASSED                           [ 80%]
tests/test_auth.py::test_login_with_invalid_password PASSED             [100%]

============================== 5 passed in 0.23s ==============================
```

**Step 4: Verify syntax**
```bash
$ python -m py_compile auth.py
$ echo $?
0
```

**Step 5: Check for unintended changes**
```bash
$ git diff auth.py
--- a/auth.py
+++ b/auth.py
@@ -14,7 +14,12 @@

 def validate_password(password):
-    # TODO: Add validation
-    return True
+    if len(password) < 8:
+        return False
+    if not any(c.isupper() for c in password):
+        return False
+    if not any(c.isdigit() for c in password):
+        return False
+    return True
```

**Verification Complete**:
- ✅ Tests pass (5/5)
- ✅ Syntax valid
- ✅ Only intended changes
- ✅ Password validation now enforces: min 8 chars, uppercase, digit

Test if this addresses the issue.

**Why this is better**:
- Shows actual test output (5 specific tests)
- Verifies syntax
- Shows git diff to prove only intended changes
- User can see exactly what was fixed
- No ambiguous "should work" language

---

## Example 2: Go Refactoring

### Insufficient Verification

**User**: Refactor the user service to use the new database package

**Assistant**: Done! I've refactored user.go to use the new db package. All the imports are updated and it should compile fine.

**Problems**:
- No build verification
- No test execution
- "Should compile" is speculation
- No evidence provided

---

### Thorough Verification

**User**: Refactor the user service to use the new database package

**Assistant**: I'll refactor the user service to use the new database package and verify everything still works.

**Step 1: Review changes**
```bash
$ git diff --name-only
internal/user/service.go
internal/user/repository.go
```

**Step 2: Run tests**
```bash
$ go test ./internal/user/... -v
=== RUN   TestUserService_Create
--- PASS: TestUserService_Create (0.01s)
=== RUN   TestUserService_Get
--- PASS: TestUserService_Get (0.01s)
=== RUN   TestUserService_Update
--- PASS: TestUserService_Update (0.01s)
=== RUN   TestUserService_Delete
--- PASS: TestUserService_Delete (0.01s)
=== RUN   TestUserRepository_FindByEmail
--- PASS: TestUserRepository_FindByEmail (0.02s)
PASS
ok      github.com/example/internal/user        0.156s
```

**Step 3: Run race detector**
```bash
$ go test -race ./internal/user/...
PASS
ok      github.com/example/internal/user        0.312s
```

**Step 4: Verify build**
```bash
$ go build ./...
$ echo $?
0
```

**Step 5: Check imports**
```bash
$ head -20 internal/user/service.go
package user

import (
    "context"
    "github.com/example/internal/db"  // New import
    "github.com/example/internal/models"
)
```

**Step 6: Verify no print statements**
```bash
$ grep -r "fmt.Println" internal/user/
$ echo $?
1
```
(Exit code 1 means no matches found - good!)

**Verification Complete**:
- ✅ Tests pass (5/5) with race detector
- ✅ Build succeeds
- ✅ New db package imported correctly
- ✅ No debug print statements
- ✅ Files reviewed: service.go, repository.go

Test if this addresses the issue.

---

## Example 3: JavaScript Feature Addition

### Insufficient Verification

**User**: Add a "Clear All" button to the dashboard

**Assistant**: I've added the Clear All button to Dashboard.jsx. It's wired up to call clearAllItems(). Everything looks good!

**Problems**:
- No build verification
- No tests run
- No visual confirmation
- "Looks good" is subjective

---

### Thorough Verification

**User**: Add a "Clear All" button to the dashboard

**Assistant**: I'll add the Clear All button and verify it works correctly.

**Step 1: Review changes**
```bash
$ git diff --name-only
src/components/Dashboard.jsx
src/components/Dashboard.test.jsx
```

**Step 2: Check syntax**
```bash
$ node -c src/components/Dashboard.jsx
$ echo $?
0
```

**Step 3: Run tests**
```bash
$ npm test -- Dashboard.test.jsx
 PASS  src/components/Dashboard.test.jsx
  Dashboard Component
    ✓ renders dashboard with items (45 ms)
    ✓ displays Clear All button (12 ms)
    ✓ calls clearAllItems when button clicked (23 ms)
    ✓ disables button when no items (8 ms)

Test Suites: 1 passed, 1 total
Tests:       4 passed, 4 total
Snapshots:   0 total
Time:        1.234 s
```

**Step 4: Build verification**
```bash
$ npm run build
> app@1.0.0 build
> react-scripts build

Creating an optimized production build...
Compiled successfully.

File sizes after gzip:

  52.3 kB  build/static/js/main.a1b2c3d4.js
  1.8 kB   build/static/css/main.e5f6g7h8.css

The build folder is ready to be deployed.
```

**Step 5: Check for console.log**
```bash
$ grep -n "console.log" src/components/Dashboard.jsx
$ echo $?
1
```
(No console.log statements found)

**Step 6: Review actual code**
```jsx
// Dashboard.jsx excerpt
<button
  className="btn btn-danger"
  onClick={clearAllItems}
  disabled={items.length === 0}
>
  Clear All
</button>
```

**Verification Complete**:
- ✅ Syntax valid
- ✅ Tests pass (4 new tests)
- ✅ Build succeeds (52.3 kB bundle)
- ✅ No console.log statements
- ✅ Button properly wired to clearAllItems()
- ✅ Button disabled when no items

Test if this addresses the issue.

---

## Example 4: Database Migration

### Insufficient Verification

**User**: Add an index on the email column in the users table

**Assistant**: Index added! I created the migration file and the index should improve query performance.

**Problems**:
- Migration not tested
- No verification it actually runs
- "Should improve" is speculation
- Doesn't check if index was created

---

### Thorough Verification

**User**: Add an index on the email column in the users table

**Assistant**: I'll create the migration and verify it works correctly.

**Step 1: Create migration**
```bash
$ ls migrations/
20250129_add_email_index.sql
```

**Step 2: Review migration content**
```sql
-- migrations/20250129_add_email_index.sql
-- Add index on users.email for faster lookups

BEGIN;

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_email
ON users(email);

COMMIT;
```

**Step 3: Test migration on test database**
```bash
$ psql test_db -f migrations/20250129_add_email_index.sql
BEGIN
CREATE INDEX
COMMIT
```

**Step 4: Verify index exists**
```bash
$ psql test_db -c "\d users"
Table "public.users"
 Column |  Type   | Collation | Nullable | Default
--------+---------+-----------+----------+---------
 id     | integer |           | not null |
 email  | varchar |           |          |
Indexes:
    "users_pkey" PRIMARY KEY, btree (id)
    "idx_users_email" btree (email)  <--- NEW INDEX
```

**Step 5: Test query performance**
```bash
$ psql test_db -c "EXPLAIN SELECT * FROM users WHERE email = 'test@example.com';"
                                   QUERY PLAN
---------------------------------------------------------------------------------
 Index Scan using idx_users_email on users  (cost=0.42..8.44 rows=1 width=100)
   Index Cond: ((email)::text = 'test@example.com'::text)
```
(Using index scan - good!)

**Step 6: Test rollback**
```sql
-- migrations/20250129_add_email_index_down.sql
BEGIN;
DROP INDEX CONCURRENTLY IF EXISTS idx_users_email;
COMMIT;
```

```bash
$ psql test_db -f migrations/20250129_add_email_index_down.sql
BEGIN
DROP INDEX
COMMIT

$ psql test_db -c "\d users" | grep idx_users_email
$ echo $?
1
```
(Index successfully removed by rollback)

**Verification Complete**:
- ✅ Migration file created
- ✅ Migration runs successfully
- ✅ Index actually exists (verified with \d)
- ✅ Query uses index (verified with EXPLAIN)
- ✅ Rollback works correctly
- ✅ Uses CONCURRENTLY to avoid locks

Test if this addresses the issue.

---

## Example 5: Configuration Change

### Insufficient Verification

**User**: Update nginx config to add rate limiting

**Assistant**: Updated nginx.conf with rate limiting. The config should prevent abuse now.

**Problems**:
- Config not syntax-checked
- Not tested
- "Should prevent" is speculation
- No verification nginx accepts it

---

### Thorough Verification

**User**: Update nginx config to add rate limiting

**Assistant**: I'll update the nginx configuration and verify it's valid.

**Step 1: Review changes**
```bash
$ git diff nginx.conf
@@ -10,6 +10,9 @@ http {
     include       mime.types;
     default_type  application/octet-stream;

+    # Rate limiting
+    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
+
     server {
         listen 80;
         server_name example.com;
@@ -18,6 +21,8 @@ http {
             proxy_pass http://localhost:8000;
             proxy_set_header Host $host;
             proxy_set_header X-Real-IP $remote_addr;
+
+            limit_req zone=api_limit burst=20 nodelay;
         }
     }
 }
```

**Step 2: Syntax validation**
```bash
$ nginx -t -c /etc/nginx/nginx.conf
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
```

**Step 3: Check rate limit zone**
```bash
$ grep -A 1 "limit_req_zone" nginx.conf
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
```

**Step 4: Dry-run reload**
```bash
$ nginx -s reload
2025/11/29 10:30:00 [notice] signal process started
```
(No errors - successful reload)

**Step 5: Test rate limiting**
```bash
$ for i in {1..15}; do curl -w "%{http_code}\n" -o /dev/null -s http://localhost/api/test; done
200
200
200
200
200
200
200
200
200
200
503
503
503
503
503
```
(First 10 requests succeed, next 5 rate-limited with 503)

**Verification Complete**:
- ✅ Syntax valid (nginx -t passed)
- ✅ Configuration reload successful
- ✅ Rate limit zone created (10m size, 10r/s limit)
- ✅ Rate limiting actually works (tested with curl)
- ✅ Burst handling configured (burst=20)

Test if this addresses the issue.

---

## Key Patterns in Good Verification

### 1. Multiple Independent Checks
Good verification uses multiple different validation methods:
- Syntax checkers (py_compile, node -c, gofmt)
- Tests (pytest, go test, npm test)
- Build verification (go build, npm run build)
- Runtime checks (start server, query database)
- Manual inspection (Read tool, git diff)

### 2. Show Actual Output
Never summarize - always show:
- Complete test results with test names
- Full build output
- Actual command output
- Exit codes

### 3. Evidence-Based Claims
Replace speculation with evidence:
- ❌ "Should work now"
- ✅ "Tests pass: [show output]"
- ❌ "Looks good"
- ✅ "Syntax valid: [show command]"
- ❌ "Index improves performance"
- ✅ "EXPLAIN shows index scan: [show output]"

### 4. Check for Unintended Changes
Always verify:
```bash
git diff                    # Review all changes
grep "console.log"          # No debug code
grep "TODO"                 # No leftover TODOs
grep "password\|secret"     # No credentials
```

### 5. Domain-Appropriate Verification
Different domains need different checks:
- **Python**: pytest + syntax + imports
- **Go**: tests + race detector + build + gofmt
- **JavaScript**: tests + syntax + build + bundle size
- **Database**: migration test + rollback test + EXPLAIN
- **Config**: syntax check + dry-run + actual reload

### 6. User-Friendly Format
Good verification provides:
- Clear step numbers
- Actual commands with $ prefix
- Complete output (not summaries)
- Checklist summary at end
- "Test if this addresses the issue" invitation

---

## Red Flags in Bad Verification

Watch for these warning signs:

1. **Speculation Language**:
   - "Should work"
   - "Should fix"
   - "Should prevent"
   - "Looks good"
   - "Seems fine"

2. **Missing Evidence**:
   - "Tests pass" (without showing output)
   - "Build succeeds" (without showing command)
   - "No errors" (without showing what was checked)

3. **Skipped Verification**:
   - "Small change, no need to test"
   - "Just config, won't break anything"
   - "Only comments changed"

4. **Incomplete Checks**:
   - Only syntax check, no tests
   - Only read code, no execution
   - Only manual review, no automation

5. **No User Confidence**:
   - User must still test blind
   - No evidence changes work
   - No visibility into verification process

---

## Verification Mindset

**Bad Mindset**: "I changed the code, it should work"

**Good Mindset**: "I changed the code, let me prove it works through multiple independent verification layers"

**Defense-in-Depth Principle**:
- **Layer 1**: Syntax validation
- **Layer 2**: Unit tests
- **Layer 3**: Integration tests
- **Layer 4**: Build verification
- **Layer 5**: Manual code review
- **Layer 6**: Runtime verification

Each layer catches different types of errors. Never rely on just one layer.

---

## Final Checklist

Before saying "done":
- [ ] Tests run (output shown)
- [ ] Build successful (output shown)
- [ ] Syntax validated (command shown)
- [ ] Files reviewed (Read tool used)
- [ ] Diff checked (git diff shown)
- [ ] No debug code (grep verified)
- [ ] No speculation (only evidence-based statements)
- [ ] User invited to test ("Test if this addresses the issue")

**Remember**: The goal isn't to check boxes - it's to have genuine confidence the changes work correctly.