# Reference Decomposition — Headless Prompt

You are running as an autonomous process to improve the toolkit's progressive disclosure architecture. This is the inverse of enrichment: instead of ADDING reference files, you EXTRACT content from bloated SKILL.md and agent files into properly structured reference files.

The goal is to keep agent/skill bodies lean (workflow, phases, routing) while domain knowledge lives in reference files loaded on demand.

## Context

- **Date:** ${DECOMP_DATE}
- **Run ID:** ${DECOMP_RUN_ID}
- **Repository:** ${DECOMP_REPO_DIR}
- **Worktree:** ${DECOMP_WORKTREE}
- **Targets:** ${DECOMP_TARGETS}
- **Max targets:** ${DECOMP_MAX_TARGETS}
- **Dry-run mode:** ${DECOMP_DRY_RUN_MODE}

These targets were identified by the detection script as containing extractable content blocks (catalogs, examples, specs, rosters) that belong in reference files rather than the main body. Each target includes file paths, content types, line ranges, and suggested reference filenames.

## If Dry-Run Mode is "yes"

Only perform the audit analysis. For each target:
1. Read the target file and measure its current line count
2. Identify which content blocks are extractable (catalogs, example tables, failure mode lists, spec sections, rosters)
3. Check for existing reference files that could absorb the content
4. Report what WOULD be decomposed: block types, estimated line counts, suggested reference filenames
5. Report findings only — file creation happens in a later phase

Print a summary and exit.

## If Dry-Run Mode is "no"

### Phase 1: Setup

1. Check for existing open decomposition PRs: `gh pr list --search "decomp/refs" --state open --json number | python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d))"`
2. If 3 or more decomposition PRs are already open, log "Too many open decomposition PRs (>=3), skipping to avoid accumulation" and exit. Decomposition changes are larger than enrichment, so the threshold is lower.
3. Verify you are running inside the worktree path (should contain /tmp/decomp-worktree):
   - Run `pwd` to confirm you're in the worktree path
   - Run `git log --oneline -1` to confirm you're at the expected HEAD
4. Create a feature branch: `git checkout -b decomp/refs-${DECOMP_RUN_ID}`

### Phase 2: Decompose Each Target

Process up to ${DECOMP_MAX_TARGETS} targets from the targets list. For each target:

1. **Save a snapshot** before any modifications:
   ```bash
   cp {path} /tmp/decomp-before-{component}.md
   ```

2. **Read the current file** (SKILL.md or agent body) and note its line count.

3. **Read the detection script output** in the target data to understand what content blocks are extractable: their types, line ranges, and suggested reference filenames.

4. **Read any existing reference files** in the component's `references/` directory to avoid creating duplicates. If a reference file with related content already exists, you will MERGE into it rather than creating a new file.

5. **Read the reference file template** at `skills/analysis/data/references/reference-file-template.md` for the standard structure that new reference files must follow.

6. **For each extractable block**, perform the extraction:

   a. **Determine the reference filename.** Use the detection script's suggestion as a starting point. Follow the naming convention: `{domain}-{topic}.md` (e.g., `go-preferred-patterns.md`, `voice-banned-patterns.md`).

   b. **Check for merge targets.** If a reference file with related content already exists, MERGE into it instead of creating a duplicate. Two files covering the same sub-domain is worse than one longer file.

   c. **Create or update the reference file** with the extracted content. Ensure it follows the reference file template structure: scope header, overview, pattern tables, failure mode catalog, etc. Maximum 500 lines per reference file. If content would exceed 500 lines, split into two reference files covering narrower sub-domains.

   d. **Remove the content from the SKILL.md body.** The content is being MOVED, not copied. Do not leave the original content in place.

   e. **Add a loading table entry** that maps task signals to the new reference file. The loading table tells the agent/skill when to load each reference. Format:
      ```
      | Signal/Task | Reference File | When to Load |
      |-------------|---------------|--------------|
      | {signal pattern} | `references/{filename}` | {condition} |
      ```

7. **Verify retained structure.** After extraction, confirm the SKILL.md still retains these essential sections:
   - Frontmatter (title, description, etc.)
   - Brief overview / purpose statement
   - Phase workflow (if applicable)
   - Loading table (with entries for all reference files, including new ones)
   - Error handling / safety section (if applicable)

8. **Run validation:**
   ```bash
   python3 scripts/validate-decomposition.py --before /tmp/decomp-before-{component}.md --after {path} --refs {refs_dir}/
   ```

9. **Run structural check** (if the validation script exists):
   ```bash
   python3 scripts/validate-references.py --skill {name}
   ```
   Or for agents: `python3 scripts/validate-references.py --agent {name}`

10. **Run depth audit:**
    ```bash
    python3 scripts/audit-reference-depth.py --skill {name} --verbose
    ```
    Or for agents: `python3 scripts/audit-reference-depth.py --agent {name} --verbose`

### Phase 2.5: Validate Decomposition (Keep-or-Revert Gate)

For each decomposed target, evaluate whether the decomposition is valid:

1. **Run validate-decomposition.py** and check the exit code:

   - **Exit code 0 (PASS):** The decomposition is valid. KEEP all changes for this target.

   - **Exit code 1 (FAIL):** The decomposition has issues.
     - Check the error output. Common fixable issues:
       - Missing loading table entry: add the entry and re-run validation
       - Reference file missing scope header: add the header
     - Attempt to fix the issue, then re-run validation
     - If validation still fails after the fix attempt: REVERT by restoring from the snapshot:
       ```bash
       cp /tmp/decomp-before-{component}.md {path}
       ```
       Remove any reference files that were created for this target.

2. **Check the resulting SKILL.md line count.** If the file is still above 500 lines after decomposition, log a warning:
   ```
   [WARN] {name}: still {N} lines after decomposition (target: <=500)
   ```
   This is a warning, not a failure. Some skills have legitimate large workflows that cannot be decomposed further.

3. **Log the decision** for each target:
   - `[KEEP] {name}: {N} blocks extracted, {before} -> {after} lines`
   - `[REVERT] {name}: validation failed — {reason}`
   - `[SKIP] {name}: {reason}`

### Phase 3: Commit and PR

If any targets were successfully decomposed (KEEP decisions exist):

1. **Stage only modified and created files.** This includes:
   - Modified SKILL.md / agent body files
   - New or updated reference files in `references/` directories
   - Stage only files related to this decomposition

2. **Commit** with a descriptive message:
   ```
   refactor(refs): {component} — extract {N} content blocks into references ({before_lines}->{after_lines} lines)
   ```
   If multiple components were decomposed:
   ```
   refactor(refs): decompose {N} components — extract content blocks into references
   ```

3. **Push:**
   ```bash
   git push -u origin decomp/refs-${DECOMP_RUN_ID}
   ```

4. **Create PR:**
   ```bash
   gh pr create --title "refactor(refs): {component} — extract references ({before}->{after} lines)" --body "$(cat <<'EOF'
   ## Reference Decomposition

   Extracted content blocks from bloated skill/agent bodies into structured reference files.

   ### Changes
   - **{component}**: {before_lines} -> {after_lines} lines ({N} blocks extracted)
     - `references/{filename1}`: {description}
     - `references/{filename2}`: {description}

   ### Validation
   - validate-decomposition.py: PASS
   - validate-references.py: PASS
   - audit-reference-depth.py: Level {N}

   *Automated decomposition run ${DECOMP_RUN_ID}*
   EOF
   )"
   ```

5. **Auto-merge:**
   ```bash
   gh pr merge --squash --auto --delete-branch
   ```

6. Stay in the worktree — the primary checkout owns the main branch.

## Safety Constraints

- **Never modify skill/agent LOGIC.** Only move content and add routing. Workflow phases, gates, and decision points STAY in the SKILL.md. Only catalogs, examples, specs, and rosters move to reference files.
- **Never delete content without first putting it in a reference file.** The operation is MOVE, not DELETE. Every line removed from a SKILL.md must appear in a reference file.
- **Maximum 500 lines per reference file.** Split into narrower sub-domain files if needed.
- **Keep `<!-- DO NOT OPTIMIZE -->` blocks in place.** These blocks are explicitly protected from decomposition.
- **No force-push, no commits to main.** Everything goes through a PR.
- **If anything fails, restore from snapshot and continue with the next target.** One failure should not abort the entire run.
- **Preserve loading table integrity.** Every reference file must have a corresponding loading table entry in the parent skill/agent body.
- **Validation gate is mandatory.** Never skip Phase 2.5, even if the decomposition "looks correct".

## Output

End your session with a summary:

```
=== Reference Decomposition Summary ===
Date: {date}
Targets processed: N/M
  - {name}: {before_lines} -> {after_lines} lines ({N} blocks extracted to {M} reference files)
  - {name}: SKIPPED ({reason})
  - {name}: REVERTED ({reason})
PR: {url or "none (dry-run)"}
===
```

# Reference Enrichment — Headless Prompt

You are running as an autonomous hourly process to improve the toolkit's domain knowledge depth. This is ADR-173: Reference Enrichment.

## Context

- **Date:** ${ENRICH_DATE}
- **Run ID:** ${ENRICH_RUN_ID}
- **Repository:** ${ENRICH_REPO_DIR}
- **Targets:** ${ENRICH_TARGETS}
- **Max targets:** ${ENRICH_MAX_TARGETS}
- **Dry-run mode:** ${ENRICH_DRY_RUN_MODE}

These targets were identified by `scripts/audit-reference-depth.py` as having Level 0-2 reference depth (missing, thin, or incomplete domain knowledge). Your job is to enrich them to Level 3+ by generating concrete, domain-specific reference files.

## If Dry-Run Mode is "yes"

Only perform the audit analysis. For each target:
1. Run `python3 scripts/audit-reference-depth.py --agent {name} --verbose` (or `--skill {name}`)
2. Run `python3 skills/analysis/data/scripts/gap-analyzer.py --agent {name}` (or `--skill {name}`)
3. Report what WOULD be enriched: domains, gaps, recommended reference files
4. Report findings only — file creation happens in a later phase

Print a summary and exit.

## If Dry-Run Mode is "no"

### Phase 1: Setup

1. Check for existing open enrichment PRs: `gh pr list --search "enrich/refs" --state open --json number | python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d))"`
2. If 5 or more enrichment PRs are already open, log "Too many open enrichment PRs (>=5), skipping to avoid accumulation" and exit
3. For each target in the targets list, check for an existing enrichment PR for that specific agent/skill:
   `gh pr list --search "enrich/refs" --state open --json title --jq '[.[] | select(.title | test("{name}"))] | length'`
   If any exist, skip that target — it already has pending enrichment work.
4. You are running inside a git worktree based on the latest origin/main. Verify you are NOT on the main branch of the primary checkout:
   - Run `git log --oneline -1` to confirm you're at the expected HEAD
   - Run `pwd` to confirm you're in the worktree path (should contain /tmp/enrichment-worktree)
5. Create a feature branch: `git checkout -b enrich/refs-${ENRICH_RUN_ID}`

### Phase 2: Enrich Each Target

For each target name in the targets list:

1. **Audit before:** Run `python3 scripts/audit-reference-depth.py --agent {name} --verbose` to record the starting level
2. **Gap analysis:** Run `python3 skills/analysis/data/scripts/gap-analyzer.py --agent {name}` to identify specific knowledge gaps
3. **Read exemplars:** Read one high-quality reference file as a model (e.g., `~/.claude/agents/golang-general-engineer/references/go-preferred-patterns.md`). This shows what Level 3 depth looks like.
4. **Read the quality rubric:** Read `skills/analysis/data/references/quality-rubric.md` for Level classification criteria
5. **Read the template:** Read `skills/analysis/data/references/reference-file-template.md` for the standard structure
6. **Generate reference files:** For the top 2-3 gaps identified, create reference files that include:
   - Concrete pattern tables with version ranges (e.g., "Python 3.10+: use match/case")
   - Failure mode catalog with detection commands (grep/rg patterns)
   - Code examples in fenced blocks
   - Error-fix mappings from common issues
   - Maximum 500 lines per file
7. **Update the agent/skill body:** Add a loading table that maps task types to the new reference files (or update existing table)
8. **Audit after:** Run `python3 scripts/audit-reference-depth.py --agent {name} --verbose` to verify improvement
9. **Lint:** Run `ruff check . --config pyproject.toml` and `ruff format --check . --config pyproject.toml` if any .py files were created

### Phase 2.5: Validate Enrichment Quality (Keep-or-Revert Gate)

For each enriched target, run a quick validation before committing:

1. **Generate 3 domain-specific test queries** for the agent/skill. Examples:
   - For threejs-builder: "Create a holographic shader effect", "Optimize a scene with 10,000 instances", "Set up morph target animation"
   - For phaser-gamedev: "Add screen shake on enemy hit", "Set up Matter.js collision categories"
   - Queries should specifically exercise the domains covered by the NEW reference files

2. **Score the agent's knowledge** by checking if the agent would produce output that uses patterns from the new references:
   - Read each new reference file and identify 3-5 key patterns it teaches (specific function calls, specific parameter values, specific failure modes to detect)
   - For each test query, check: would the reference loading table's signal detection correctly trigger loading this reference?
   - Verify the loading table entry exists and has correct file path

3. **Keep-or-revert decision:**
   - If the loading table correctly maps signals to the new references AND the references contain concrete, actionable patterns (not generic advice): **KEEP**
   - If the loading table is missing entries for the new references: **FIX the loading table, then KEEP**
   - If the reference file contains only generic patterns that don't add domain-specific value beyond what the agent already knows: **REVERT** (delete the file, remove loading table entry)

4. **Log the decision** for each target: `[KEEP] {name}: {reason}` or `[REVERT] {name}: {reason}`

This gate prevents reference bloat — only references that add concrete, signal-matched domain knowledge survive. The enrichment system should get better, not just bigger.

### Phase 3: Commit and PR

1. Stage only the files you created/modified (reference files, agent/skill body updates)
2. Commit with: `feat(refs): {agent-name} — {brief description of what was added} (Level {before}→{after})`
   - Example: `feat(refs): prometheus-grafana-engineer — PromQL patterns, alerting rules, cardinality management (Level 0→3)`
3. Push: `git push -u origin enrich/refs-${ENRICH_RUN_ID}`
4. Create PR and auto-merge: `gh pr create --title "feat(refs): {agent-name} (Level {before}→{after})" --body "..."` then `gh pr merge --squash --auto --delete-branch`
   - PR title should describe WHAT was enriched, not just the date
   - PR body should include: targets processed, level before/after for each, list of new reference files
   - The `--auto` flag merges once CI passes — no human review needed for reference-only changes
5. Stay in the worktree — the primary checkout owns the main branch.

## Safety Constraints

- **Never modify existing code** — only create/modify files in `references/` directories and loading tables in agent/skill bodies
- **Never modify agent/skill logic** — only add knowledge, not change behavior
- **Maximum 500 lines per reference file** — progressive disclosure principle
- **If a target already has Level 3+ references** when you actually check (audit data may be stale), skip it
- **No force-push, no commits to main** — everything goes through a PR
- **If anything fails, continue with the next target** — don't abort the whole run
- **Validation gate is mandatory** — never skip Phase 2.5, even if the references "look good"

## Output

End your session with a summary:

```
=== Reference Enrichment Summary ===
Date: {date}
Targets processed: N/M
  - {name}: Level {before} → Level {after} ({new_files} new reference files)
  - {name}: SKIPPED (already Level 3+)
  - ...
PR: {url or "none (dry-run)"}
===
```

# Output Templates

Templates for common analysis types. Each template specifies the structure for `analysis-report.md` tailored to the analysis pattern.

---

## A/B Test Evaluation

Use when comparing two (or more) experimental variants with a defined control.

```markdown
# A/B Test Analysis Report

## Headline
[One sentence: does the test variant win, lose, or is the result inconclusive?]

## Test Design
- **Hypothesis**: [What was being tested]
- **Control (A)**: [Description]
- **Variant (B)**: [Description]
- **Primary metric**: [Metric name and definition]
- **Sample**: Control N=[X], Variant N=[Y]
- **Test duration**: [Start - End]
- **Minimum detectable effect**: [MDE used in test design]

## Results

### Primary Metric
| Group | Value | 95% CI | N |
|-------|-------|--------|---|
| Control (A) | [value] | [lower - upper] | [N] |
| Variant (B) | [value] | [lower - upper] | [N] |
| **Difference** | **[value]** | **[lower - upper]** | |

- Statistical significance: [p-value]
- Effect size: [Cohen's d or relative change]
- Practical significance: [Above/Below minimum actionable threshold]

### Secondary Metrics (if applicable)
| Metric | Control | Variant | Difference | 95% CI | Significant? |
|--------|---------|---------|------------|--------|-------------|
| [name] | [value] | [value] | [diff] | [CI] | [Yes/No] |

### Segment Breakdown (if applicable)
| Segment | Control | Variant | Difference | N (per group) |
|---------|---------|---------|------------|---------------|
| [name] | [value] | [value] | [diff] | [N] |

*Note: [N] segments tested. [Correction method] applied. [N] significant after correction.*

## Rigor Checks
- Sample adequacy: [PASS/FAIL]
- Group balance: [Ratio and assessment]
- Time window: [Complete/Gaps noted]
- Multiple testing: [Method if applicable]

## Limitations
- [Limitation 1]
- [Limitation 2]

## Recommendation
[Ship / Do not ship / Extend test -- with rationale tied to evidence thresholds]

## What Would Increase Confidence
- [Suggestion 1]
- [Suggestion 2]
```

---

## Trend Analysis

Use when examining how a metric changes over time.

```markdown
# Trend Analysis Report

## Headline
[One sentence: what is the trend and what does it mean for the decision?]

## Trend Summary
- **Metric**: [Name and definition]
- **Period**: [Start - End]
- **Granularity**: [Daily / Weekly / Monthly]
- **Overall direction**: [Increasing / Decreasing / Flat / Volatile]
- **Rate of change**: [Average period-over-period change with CI]

## Trend Data

### Overall
| Period | Value | Change | Change % |
|--------|-------|--------|----------|
| [period] | [value] | [abs change] | [pct change] |

### By Segment (if applicable)
| Segment | Start Value | End Value | Overall Change | Trend |
|---------|-------------|-----------|----------------|-------|
| [name] | [value] | [value] | [change] | [direction] |

## Pattern Detection
- **Seasonality**: [Detected / Not detected -- pattern description]
- **Structural breaks**: [Any abrupt changes in trend? When?]
- **Outliers**: [Periods that deviate significantly from trend]

## Drivers
[What is driving the trend? Segment decomposition showing which segments contribute most to the overall change.]

| Segment | Contribution to Overall Change |
|---------|-------------------------------|
| [name] | [pct of total change] |

## Limitations
- [Limitation 1: e.g., "Seasonality not fully controlled -- only 1 year of data"]
- [Limitation 2: e.g., "External factors (pricing change in March) may confound trend"]

## Recommendation
[Action tied to the decision frame from Phase 1]

## What Would Increase Confidence
- [More time periods to control for seasonality]
- [Additional segmentation to isolate drivers]
```

---

## Distribution Profiling

Use when understanding the shape and characteristics of a metric's distribution.

```markdown
# Distribution Analysis Report

## Headline
[One sentence: what does the distribution tell us about the decision?]

## Distribution Summary
- **Metric**: [Name and definition]
- **Sample**: N=[count]
- **Period**: [Time window]

## Key Statistics
| Statistic | Value |
|-----------|-------|
| Mean | [value] |
| Median | [value] |
| Std Dev | [value] |
| Min | [value] |
| Max | [value] |
| p25 | [value] |
| p75 | [value] |
| p90 | [value] |
| p95 | [value] |
| p99 | [value] |
| Skewness | [value -- interpretation] |

## Distribution Shape
- **Type**: [Normal / Right-skewed / Bimodal / Long-tailed / Uniform]
- **Key characteristic**: [What is notable about this distribution?]
- **Mean vs Median gap**: [If large, indicates skew -- report which is more appropriate]

## Percentile Analysis
[For performance/latency data, focus on tail behavior]

| Percentile | Value | Within Threshold? |
|------------|-------|-------------------|
| p50 | [value] | [Yes/No] |
| p90 | [value] | [Yes/No] |
| p95 | [value] | [Yes/No] |
| p99 | [value] | [Yes/No] |

## Segments (if applicable)
| Segment | Mean | Median | p99 | N |
|---------|------|--------|-----|---|
| [name] | [value] | [value] | [value] | [count] |

## Outlier Analysis
- **Outlier definition**: [Method: >3 std dev / IQR / domain-specific]
- **Count**: [N outliers of M total ([pct]%)]
- **Impact on mean**: [Mean with outliers vs without]
- **Investigation**: [What outliers represent -- errors? genuine extremes?]

## Limitations
- [Limitation 1]
- [Limitation 2]

## Recommendation
[Action tied to the decision frame]
```

---

## Cohort Comparison

Use when comparing groups defined by a shared characteristic (sign-up date, plan tier, geography).

```markdown
# Cohort Comparison Report

## Headline
[One sentence: how do cohorts differ and what does it mean?]

## Cohort Definitions
| Cohort | Definition | N |
|--------|-----------|---|
| [name] | [selection criteria] | [count] |

## Comparison Fairness Assessment
- Same time window: [Yes/No -- details]
- Same population base: [Yes/No -- details]
- Known confounders: [List]
- Survivorship bias: [Risk level and mitigation]

## Results

### Primary Metric
| Cohort | Value | 95% CI | N |
|--------|-------|--------|---|
| [name] | [value] | [lower - upper] | [count] |

### Pairwise Comparisons
| Comparison | Difference | 95% CI | Significant? | Practically Significant? |
|-----------|-----------|--------|-------------|------------------------|
| A vs B | [diff] | [CI] | [Yes/No] | [Yes/No -- vs threshold] |

*[N] pairwise comparisons performed. [Correction method] applied.*

## Cohort Behavior Over Time (if temporal)
| Period | Cohort A | Cohort B | Gap |
|--------|----------|----------|-----|
| [period] | [value] | [value] | [diff] |

[Is the gap widening, narrowing, or stable?]

## Limitations
- [Limitation 1]
- [Limitation 2]

## Recommendation
[Action tied to the decision frame]
```

---

## Funnel Analysis

Use when measuring drop-off through a sequence of steps.

```markdown
# Funnel Analysis Report

## Headline
[One sentence: where is the biggest drop-off and what does it mean?]

## Funnel Definition
- **Entry**: [First step and its definition]
- **Exit**: [Last step / conversion event]
- **Time window**: [Period analyzed]
- **Population**: [Who is included]

## Funnel Steps
| Step | Count | Rate from Previous | Rate from Entry | Drop-off |
|------|-------|-------------------|-----------------|----------|
| [Step 1] | [N] | -- | 100% | -- |
| [Step 2] | [N] | [pct] | [pct] | [pct dropped] |
| [Step 3] | [N] | [pct] | [pct] | [pct dropped] |
| [Final] | [N] | [pct] | [pct] | [pct dropped] |

**Overall conversion**: [Entry to Final pct]

## Biggest Drop-off
- **Step**: [Where the largest drop occurs]
- **Drop rate**: [Percentage]
- **Volume**: [Absolute number of users lost]
- **Context**: [Why this might be happening -- if data supports it]

## By Segment (if applicable)
| Segment | Entry | Final | Conversion | vs Overall |
|---------|-------|-------|------------|-----------|
| [name] | [N] | [N] | [pct] | [+/- vs overall] |

## Limitations
- [Limitation 1: e.g., "Cannot distinguish intentional exits from errors"]
- [Limitation 2]

## Recommendation
[Where to focus optimization efforts, tied to decision frame]
```

---

## Phase Artifact Templates

### analysis-frame.md (Phase 1 output)

```markdown
# Analysis Frame

## Decision
[What decision is being supported]

## Decision-Maker
[Who will act on this analysis]

## Options
- Option A: [description]
- Option B: [description]
- Default (no action): [what happens if we take no action]

## Evidence Requirements
- Favors Option A if: [condition]
- Favors Option B if: [condition]
- Minimum threshold: [what bar must be cleared]

## Deal-Breakers
- [condition that forces a specific option regardless]
```

### metric-definitions.md (Phase 2 output)

```markdown
# Metric Definitions

## Metrics
### [Metric Name]
- Formula: [exact computation]
- Population: [inclusion/exclusion criteria]
- Time window: [start - end, granularity]
- Segments: [how data is sliced]

## Comparison Groups (if applicable)
### Group A: [Name]
- Selection: [criteria]
### Group B: [Name]
- Selection: [criteria]
- Fairness: [same population? same time window?]

## Success Criteria
- Minimum meaningful effect: [threshold]
- Minimum sample per segment: [N]
- Test type: [one-tailed / two-tailed / descriptive only]
```

### data-quality-report.md (Phase 3 output)

```markdown
# Data Quality Report

## Dataset Overview
- Source: [file path / description]
- Rows: [N]
- Columns: [N]
- Date range: [start - end]

## Column Profiles
| Column | Type | Non-null | Missing % | Unique | Notes |
|--------|------|----------|-----------|--------|-------|
| [name] | [type] | [count] | [pct] | [count] | [flags] |

## Quality Assessment
- [ ] Sample adequate (N=[count], population=[est])
- [ ] Time window complete (gaps: [none / list])
- [ ] Segment minimums met ([list segments below 30])
- [ ] Missing values acceptable ([list columns above 20%])

## Quality Issues
[List any issues that affect planned analysis]

## Data Ready: [YES / NO - with reason]
```

### analysis-results.md (Phase 4 output)

```markdown
# Analysis Results

## Metrics
### [Metric Name]
- Value: [point estimate]
- 95% CI: [lower - upper]
- Sample: N=[count]

## Comparisons (if applicable)
### [Group A] vs [Group B]
- Group A: [metric] = [value] (N=[count])
- Group B: [metric] = [value] (N=[count])
- Difference: [absolute] ([relative]%)
- 95% CI of difference: [lower - upper]
- Practical significance: [above/below minimum threshold]

## Rigor Gate Results
- [ ] Sample Adequacy: [PASS / FAIL - details]
- [ ] Comparison Fairness: [PASS / FAIL / N/A - details]
- [ ] Multiple Testing: [PASS / FAIL / N/A - details]
- [ ] Practical Significance: [PASS / FAIL - details]

## Rigor Violations (if any)
[List violations and their impact on conclusions]
```

### analysis-report.md (Phase 5 output)

```markdown
# Analysis Report

## Headline
[One sentence: what the data says about the decision]

## Decision Context
[Recap from Phase 1 frame]

## Key Findings
1. [Primary finding with CI]
2. [Supporting finding]
3. [Qualifying finding or important segment variation]

## Limitations
- [Limitation 1]
- [Limitation 2]

## Recommendation
[Action recommendation with confidence level]

## What Would Increase Confidence
- [Additional data or analysis that would help]

---

## Appendix: Methodology
- Data source: [file]
- Rows analyzed: [N]
- Time window: [range]
- Tools: [pandas/stdlib]
- Metrics: See metric-definitions.md
- Quality: See data-quality-report.md
- Detailed results: See analysis-results.md
```

---

## Anomaly Investigation

Use when investigating unexpected spikes, drops, or deviations in metrics.

```markdown
# Anomaly Investigation Report

## Headline
[One sentence: what happened, when, and what caused it?]

## Anomaly Summary
- **Metric**: [Name]
- **Normal range**: [Expected value or range]
- **Anomaly value**: [Observed value]
- **Deviation**: [How far from normal -- Z-score or percentage]
- **When**: [Date/time range]
- **Duration**: [How long the anomaly lasted]

## Timeline
| Time | Value | Status |
|------|-------|--------|
| [before] | [normal value] | Normal |
| [onset] | [value] | Anomaly begins |
| [peak] | [value] | Peak deviation |
| [recovery] | [value] | Returns to normal |

## Root Cause Analysis
### Correlated Events
| Event | Timing | Correlation |
|-------|--------|-------------|
| [deployment/change/incident] | [time] | [Likely/Possible/Unlikely] |

### Segment Isolation
| Segment | Affected? | Severity |
|---------|-----------|----------|
| [name] | [Yes/No] | [Magnitude] |

[Which segments are affected narrows the possible causes.]

## Impact Assessment
- **Duration**: [How long]
- **Magnitude**: [Quantified impact -- revenue lost, users affected, etc.]
- **Scope**: [How widespread -- single endpoint vs system-wide]

## Limitations
- [Limitation 1]

## Recommendation
[Action to prevent recurrence, tied to decision frame]
```

# Data Analysis Patterns to Detect and Fix

Extended catalog of patterns beyond the top 5 in the main SKILL.md. Each entry includes the signal, why it matters, and what to do instead.

---

## Methodology Patterns to Fix

### Data-First Analysis (also in SKILL.md)

**What it looks like**: Jumping straight to `pd.read_csv()` and running `.describe()` before establishing the decision context.

**Why wrong**: Produces technically correct summaries that answer the wrong question. The analyst presents "interesting findings" but the decision-maker cannot act because the findings do not map to their options.

**Do instead**: Complete Phase 1 (FRAME) first. Even 5 minutes of framing prevents hours of wasted analysis.

---

### Confirmation Bias in Extraction

**What it looks like**: Loading data and immediately computing the metric the analyst expects to find, without profiling the data first.

```python
# Bad: Jump straight to the metric
df = pd.read_csv('data.csv')
print(f"Conversion improved: {df[df.group=='B'].converted.mean() - df[df.group=='A'].converted.mean():.1%}")
```

**Why wrong**: Skipping data profiling misses quality issues that invalidate the metric. Missing values, date gaps, or population mismatches silently distort the result.

**Do instead**: Separate extraction (Phase 3) from analysis (Phase 4). Profile data quality before computing any metric.

```python
# Good: Profile first
df = pd.read_csv('data.csv')
print(f"Rows: {len(df)}")
print(f"Missing values:\n{df.isnull().sum()}")
print(f"Date range: {df.date.min()} to {df.date.max()}")
print(f"Group sizes: {df.group.value_counts().to_dict()}")
# THEN compute metrics
```

---

### Moving the Goalposts

**What it looks like**: Defining success as "5% lift" in Phase 2, then declaring success at 3% lift because "3% is still meaningful."

**Why wrong**: Post-hoc threshold adjustment is a form of p-hacking. The threshold was set before data was seen for a reason -- changing it after invalidates the pre-registration. If 3% is truly meaningful, that should have been the threshold from the start.

**Do instead**: If the result falls below the pre-defined threshold, report it as not meeting the threshold. If the stakeholder believes the threshold was wrong, document the revised threshold and note that it was changed after seeing results.

---

### Survivorship Bias

**What it looks like**: Analyzing only "active users" or "successful transactions" without accounting for those who churned or failed.

```
"Our average customer lifetime value is $500"
(Computed only from customers still active -- ignoring the 40% who churned in month 1)
```

**Why wrong**: Excluding failures inflates metrics. The real average CLV includes the customers who paid $0 after churning. Survivorship bias makes everything look better than it is.

**Do instead**: Define the population before filtering. Start with ALL users/events, then apply filters with explicit disclosure:
```
"Average CLV: $500 for retained customers (60% of cohort).
 Including churned customers: $300."
```

---

### Simpson's Paradox Ignorance

**What it looks like**: Reporting an aggregate trend without checking if the trend reverses within segments.

```
"Overall conversion improved from 3% to 4%"
(But: desktop went from 5% to 4%, mobile went from 1% to 1.5%.
 The "improvement" is entirely from more traffic shifting to higher-converting desktop.)
```

**Why wrong**: The aggregate trend can be the opposite of every segment's trend when segment sizes shift. Acting on the aggregate would mean doing more of what is not working.

**Do instead**: Always check at least one level of segmentation. If segment trends contradict the aggregate, report the segment-level finding as the primary result.

---

## Statistical Patterns to Fix

### Point Estimates Without Uncertainty (also in SKILL.md)

**What it looks like**: "The conversion rate is 4.2%."

**Why wrong**: No sample size, no confidence interval, no context for reliability. 4.2% from 50 users is noise. 4.2% from 50,000 users is signal.

**Do instead**: "Conversion rate: 4.2% (95% CI: 3.8-4.6%, N=12,400)."

---

### Relative Change Without Base Rate (also in SKILL.md)

**What it looks like**: "Revenue increased 200%!" or "Error rate dropped 50%!"

**Why wrong**: 200% of $10 is $30. 50% drop from 0.1% is 0.05%. Relative numbers without base rates mislead by making small changes sound large (or large changes sound small).

**Do instead**: Always include the base rate: "Revenue increased from $10K to $30K (+200%)" or "Error rate dropped from 0.10% to 0.05% (-50%)."

---

### P-Value Worship

**What it looks like**: "p < 0.05 therefore the effect is real and we should ship it."

**Why wrong**: Statistical significance tells you the probability of seeing this data if there were no effect. It does NOT tell you:
- How large the effect is (could be trivially small)
- Whether the effect matters for your business
- Whether your sample was representative
- Whether there are confounders

**Do instead**: Report p-value as ONE input alongside effect size, confidence interval, practical significance threshold, and known limitations.

---

### Multiple Comparisons Without Correction (also in SKILL.md)

**What it looks like**: Testing 20 user segments, finding one with p < 0.05, and reporting it as a significant finding.

**Why wrong**: At alpha=0.05, testing 20 segments gives a 64% chance of at least one false positive (1 - 0.95^20). The "finding" is likely noise.

**Do instead**: Apply Bonferroni correction (divide alpha by number of tests) or label the analysis as exploratory requiring confirmation.

---

### Confusing Correlation with Causation

**What it looks like**: "Users who complete onboarding have 3x higher retention. We should force all users to complete onboarding."

**Why wrong**: Users who complete onboarding may be more motivated to begin with. Forcing unmotivated users through onboarding will not make them motivated. The correlation may reflect selection, not treatment.

**Do instead**: Report as correlation. Note that causation requires an experiment (A/B test) or quasi-experimental methods. If the user needs causal claims, recommend an experiment design.

---

## Communication Patterns to Fix

### Methods-First Communication (also in SKILL.md)

**What it looks like**: Leading with statistical methodology instead of business insight.

**Why wrong**: The decision-maker does not need to know you used OLS regression. They need to know revenue is declining.

**Do instead**: Lead with the insight. Put methodology in the appendix.

---

### Data Dump

**What it looks like**: Presenting every computed metric in a giant table without highlighting what matters.

**Why wrong**: The decision-maker drowns in numbers. Key findings are buried. Analysis without prioritization is not analysis -- it is a spreadsheet.

**Do instead**: Lead with the 1-3 most important findings. Use the headline/evidence/limitations structure. Reference the full data in an appendix.

---

### Certainty Theater

**What it looks like**: Presenting conclusions with absolute confidence when the data has significant limitations.

```
"We have definitively proven that the new pricing model increases revenue."
```

**Why wrong**: "Definitively proven" implies controlled experiment with no confounders. Most business analyses have significant caveats. Overstating certainty sets up the decision-maker for surprise when reality diverges.

**Do instead**: State confidence level explicitly:
```
"Revenue increased 12% after the pricing change (95% CI: 5-19%).
 However, the comparison period included a holiday season,
 which may account for 3-5% of the increase."
```

---

### Hiding Inconclusive Results

**What it looks like**: Omitting analyses that showed no significant effect, only reporting the ones that "worked."

**Why wrong**: Publication bias at the individual analysis level. The decision-maker needs to know that 4 of 5 metrics showed no effect -- that IS a finding. It means the intervention probably does not work, despite one metric being marginally significant.

**Do instead**: Report all planned analyses. "We tested 5 metrics. Only email open rate showed significance (p=0.04). The other 4 metrics showed no significant change. Given the multiple testing context, the email open rate result should be treated with caution."

---

## Process Patterns to Fix

### Silent Definition Changes (also in SKILL.md)

**What it looks like**: Changing how a metric is computed after seeing the data without documenting the change.

**Why wrong**: Invalidates the pre-registration. Makes it impossible to audit whether the analyst cherry-picked a favorable definition.

**Do instead**: Return to Phase 2, update the definition with a changelog entry explaining why.

---

### No Artifact Trail

**What it looks like**: Performing analysis entirely in conversation without saving intermediate artifacts.

**Why wrong**: When context compresses or the session ends, all work is lost. No one can audit the methodology. Reproducibility is zero.

**Do instead**: Save artifacts at every phase: analysis-frame.md, metric-definitions.md, data-quality-report.md, analysis-results.md, analysis-report.md.

---

### Scope Creep in Analysis

**What it looks like**: "While I was analyzing conversion, I also looked at retention, engagement, revenue per user, session length, and feature adoption."

**Why wrong**: Each additional metric dilutes focus and increases multiple testing risk. The original decision was about conversion -- other metrics are interesting but not actionable without their own decision frame.

**Do instead**: Answer the question that was asked. If other findings emerge, note them as "exploratory observations for future investigation" -- do not present them as findings of this analysis.

---

### Copy-Paste Methodology

**What it looks like**: Applying the same analytical approach to every dataset regardless of the data characteristics or decision context.

**Why wrong**: A/B test evaluation requires different methods than trend analysis. Distribution profiling requires different methods than cohort comparison. Using the wrong template produces misleading output.

**Do instead**: Select the analysis template based on the decision question:
- "Which option is better?" -> A/B Test / Cohort Comparison
- "Is it getting better or worse?" -> Trend Analysis
- "What does the distribution look like?" -> Distribution Profiling
- "Where do we lose users?" -> Funnel Analysis
- "What happened?" -> Anomaly Investigation

# Reference File Quality Rubric

Classification criteria for Level 0-3. Used by Phase 4 Tier 2 self-assessment and by
`scripts/audit-reference-depth.py` for deterministic scoring.

---

## Level 0 — No References

**Criteria**: No `references/` directory exists for the component.

**What this means**: The agent or skill carries only what the base model already knows.
Generic domain knowledge is available but nothing project-specific, version-specific, or
pattern-specific has been codified.

**Example state**: A new agent created from template with only the body prompt. No references/
directory at all.

**Upgrade path**: Run Phase 1 DISCOVER to identify gaps, then proceed through the full pipeline.

---

## Level 1 — Thin References

**Criteria**: Reference files exist but average fewer than 50 lines, or content is dominated
by generic phrases with no version specificity, no code blocks, and no detection commands.

**What thin looks like**:
```
## Error Handling
Follow best practices for error handling in Go. Ensure errors are handled appropriately
and comprehensively. Consider edge cases and handle them thoroughly.
```

**Why it fails**: Adds no information the model doesn't already have. "Follow best practices"
is not a practice. "Handle appropriately" gives no guidance on what appropriate means here.

**Scoring signals that indicate Level 1**:
- Generic phrases: "best practices", "common issues", "review carefully", "ensure quality"
- No version ranges (`1.22+`, `Python 3.11+`)
- No fenced code blocks (` ``` `)
- No grep/rg/find detection commands
- No function names, package names, or import paths
- Fewer than 5 concrete pattern hits across all reference files

**Upgrade path**: Phase 2 RESEARCH — rewrite thin sections with version-specific content.
Each claim needs either a code example, a version qualifier, or a grep command.

---

## Level 2 — Domain-Specific References

**Criteria**: Reference files contain domain-specific patterns: version numbers, function names,
import paths, fenced code blocks, or enough concrete pattern hits to be useful.

**What Level 2 looks like**:
```
## Error Wrapping (Go 1.13+)

Use `fmt.Errorf("context: %w", err)` to wrap errors. Unwrap with `errors.Is()` and
`errors.As()`. Reserve `errors.New()` for fresh sentinel errors; keep the original error
when downstream handling depends on it.

```go
// Good
if err := db.QueryRow(query).Scan(&id); err != nil {
    return fmt.Errorf("fetch user %d: %w", userID, err)
}
// Bad — loses original error
return errors.New("database error")
```
```

**Why it passes Level 2**: Version qualifier (1.13+), function names (`fmt.Errorf`, `errors.Is`),
code block showing correct and incorrect patterns.

**What's still missing for Level 3**: No detection command to find `errors.New()` misuse in
the codebase. No error-fix mapping for common error messages.

**Scoring signals that indicate Level 2**:
- At least one fenced code block
- OR at least 5 concrete pattern hits with specificity score >= 0.4
- Average line count >= 30 lines per file

---

## Level 3 — Deep References

**Criteria**: Detection commands present AND substantial concrete content (10+ concrete pattern
hits, average 80+ lines per file).

**What Level 3 looks like**:

```markdown
## Preferred Pattern: Wrapping Errors with Context

**Detection**:
```bash
grep -rn 'errors\.New(' --include="*.go" | grep -v "_test.go"
rg 'return errors\.New\(' --type go
```

**What it looks like**:
```go
func fetchUser(id int) (*User, error) {
    u, err := db.QueryRow(...).Scan(&u)
    if err != nil {
        return nil, errors.New("database error")  // loses context
    }
}
```

**Why wrong**: `errors.New()` creates a fresh error with no link to the original. Callers
using `errors.Is(err, sql.ErrNoRows)` will get false even when the root cause is ErrNoRows.
Breaks error chain inspection in production debugging.

**Fix**: `return nil, fmt.Errorf("fetchUser %d: %w", id, err)`

**Version note**: `%w` verb available since Go 1.13. For Go 1.12 and earlier, use
`github.com/pkg/errors` Wrap().
```

**Why it's Level 3**: Detection command with two alternatives (grep + rg), code example showing
the exact bad pattern, explanation of *why* it breaks (not just that it's wrong), fix with
version qualifier.

**Scoring signals that indicate Level 3**:
- At least one grep/rg/find detection command
- AND 10+ total concrete pattern hits across all reference files
- AND average 80+ lines per file

---

## Level 3 Checklist (Per Pattern Entry)

Use this checklist in Phase 4 Tier 2 self-assessment. Each pattern entry should pass
at least 3 of 5:

- [ ] Detection command that finds the pattern in a real codebase (`grep`, `rg`, `find`)
- [ ] Code block showing the bad pattern (copy-pasteable, not pseudocode)
- [ ] Code block showing the correct fix
- [ ] Explanation of *why* the pattern is wrong (behavioral consequence, not style opinion)
- [ ] Version qualifier (when did this change? what version introduced the fix?)

---

## Common Failure Modes

| Failure | Symptom | Fix |
|---------|---------|-----|
| Generic advice | No code blocks, phrases like "handle errors properly" | Replace with specific pattern + code example |
| No detection | Pattern entries listed without detection commands | Add `grep -rn` or `rg` command for each pattern entry |
| Version-free | Patterns stated without version context | Add version ranges where behavior changed |
| No error-fix mapping | Common errors listed without root cause | Add "Error: X → Root cause: Y → Fix: Z" rows |
| Too short | < 50 lines average across files | Each sub-domain warrants at least 80 lines of concrete content |
| Too long | > 500 lines in one file | Split into focused sub-topic files |

---

## Quick Test

To check if a reference file is Level 3, answer these three questions:

1. **Can you detect it?** Pick the most important failure mode in the file. Is there a grep
   command that would find it in a real codebase? If no: Level 1.

2. **Is it version-aware?** Does the file mention at least one specific version number where
   behavior changed? If no: likely Level 1-2.

3. **Does it fix, not just flag?** For each failure mode, is the correct replacement shown
   in a code block? If no: Level 2 at best.

All three "yes": Level 3. Two "yes": Level 2. Fewer: Level 1.

# Reference File Template

Use this template when generating new reference files in Phase 3 COMPILE.
Replace `{{PLACEHOLDER}}` markers with actual content. Delete sections that don't apply
to the domain — a focused 80-line file beats a padded 300-line file.

---

# {{DOMAIN}} Reference

> **Scope**: {{ONE-SENTENCE description of what this file covers and what it doesn't}}
> **Version range**: {{e.g., "Go 1.18+", "Python 3.11+", "all versions"}}
> **Generated**: {{YYYY-MM-DD}} — verify version-specific content against current release notes

---

## Overview

{{2-4 sentences explaining the sub-domain: what problem it solves, why it matters for this
agent's work, and what the most common failure mode is. Be specific — mention the language,
framework, or tool by name.}}

---

## Pattern Table

| Pattern | Version | Use When | Prefer When |
|---------|---------|----------|------------|
| `{{function_or_construct}}` | `{{X.Y+}}` | {{condition}} | {{counter-condition}} |
| `{{function_or_construct}}` | `{{X.Y+}}` | {{condition}} | {{counter-condition}} |

*Delete this table if the domain has no version-specific API surface.*

---

## Correct Patterns

### {{Pattern Name}}

{{One sentence: what this pattern does and why it's correct.}}

```{{language}}
// {{comment explaining the key point}}
{{correct code example — copy-pasteable, not pseudocode}}
```

**Why**: {{Behavioral explanation — what breaks if you don't follow this.}}

---

### {{Pattern Name}}

```{{language}}
{{correct code example}}
```

**Why**: {{Explanation.}}

---

## Pattern Catalog: Detection and Fixes

### {{Pattern Name}}

**Detection**:
```bash
grep -rn '{{pattern}}' --include="*.{{ext}}"
rg '{{pattern}}' --type {{lang}}
```

**Signal**:
```{{language}}
{{bad code example — should be something grep above would find}}
```

**Why it matters**: {{Behavioral consequence — what actually breaks, not just "it's bad practice".
Mention the specific failure mode: data loss, silent error, performance degradation, etc.}}

**Preferred action**:
```{{language}}
{{corrected code example}}
```

**Version note**: {{If behavior changed in a specific version, state it here. Otherwise delete.}}

---

### {{Pattern Name}}

**Detection**:
```bash
grep -rn '{{pattern}}' --include="*.{{ext}}"
```

**Signal**:
```{{language}}
{{bad code}}
```

**Why it matters**: {{Consequence.}}

**Preferred action**:
```{{language}}
{{fix}}
```

---

## Error-Fix Mappings

| Error Message | Root Cause | Fix |
|---------------|------------|-----|
| `{{exact error text or regex}}` | {{why it occurs}} | {{what to change}} |
| `{{exact error text or regex}}` | {{why it occurs}} | {{what to change}} |

*Delete this table if the domain has no common error messages.*

---

## Version-Specific Notes

| Version | Change | Impact |
|---------|--------|--------|
| `{{X.Y}}` | {{What changed}} | {{How it affects code using this pattern}} |
| `{{X.Y}}` | {{What changed}} | {{How it affects code using this pattern}} |

*Delete this table if no significant version changes exist for this domain.*

---

## Detection Commands Reference

Quick collection of all grep/rg commands from this file:

```bash
# {{Pattern 1 name}}
grep -rn '{{pattern1}}' --include="*.{{ext}}"

# {{Pattern 2 name}}
rg '{{pattern2}}' --type {{lang}}
```

---

## See Also

- `{{other-reference.md}}` — {{what it covers}}
- {{Official docs URL if applicable}}

---

## Template Usage Notes

**Minimum for Level 2**: Overview + one correct pattern with code block + one failure mode entry.

**Minimum for Level 3**: All of the above + at least one detection command per failure mode +
error-fix mappings table (if the domain has common errors) + version notes (if API changed).

**Line count target**: 80-200 lines for a focused sub-domain. If you need more than 300 lines,
split into two files covering narrower sub-domains.

**What NOT to include**:
- Generic advice ("follow best practices", "be careful with...")
- Patterns that apply to every language (not domain-specific)
- Content that duplicates what's already in the agent body
- Aspirational content ("in the future, consider...")

# Statistical Rigor Gates

Detailed documentation for the four statistical rigor gates applied during Phase 4 (ANALYZE). These are hard gates -- analysis that fails a gate must either remediate or explicitly document the violation as a limitation in the conclusion.

---

## Gate 1: Sample Adequacy

**Purpose**: Verify the data is sufficient to draw meaningful conclusions before computing any summary statistic.

**Why this gate exists**: Small or incomplete samples produce wide confidence intervals that cannot support decisions. Computing metrics on inadequate samples gives false confidence -- the number looks precise but is not.

### Checks

| Check | Minimum | Action if Failed |
|-------|---------|------------------|
| Row count vs. population | Report sample fraction | State "N of M" and warn if <5% coverage |
| Time window completeness | No gaps >10% of window | Identify gaps, adjust window or note limitation |
| Segment minimums | 30+ observations per segment | Merge small segments or exclude with disclosure |
| Missing value rate | <20% per critical column | Impute with disclosure or exclude column |

### Row Count Assessment

```python
import math

def assess_sample(sample_size, population_size=None, confidence=0.95, margin=0.05):
    """Assess whether sample size is adequate."""
    z = 1.96  # 95% confidence
    # Minimum sample for proportion estimation at given margin
    min_sample = math.ceil((z**2 * 0.25) / (margin**2))

    result = {
        'sample_size': sample_size,
        'min_for_5pct_margin': min_sample,
        'adequate': sample_size >= min_sample,
    }

    if population_size:
        result['coverage'] = sample_size / population_size
        result['coverage_warning'] = result['coverage'] < 0.05
        # Finite population correction
        adjusted_min = math.ceil(
            min_sample / (1 + (min_sample - 1) / population_size)
        )
        result['adjusted_min'] = adjusted_min
        result['adequate'] = sample_size >= adjusted_min

    return result
```

### Time Window Completeness

Check for gaps in temporal data:

```python
from datetime import datetime, timedelta

def check_time_gaps(dates, expected_granularity='daily'):
    """Identify gaps in time series data."""
    sorted_dates = sorted(set(dates))
    gaps = []
    delta = timedelta(days=1) if expected_granularity == 'daily' else timedelta(weeks=1)

    for i in range(1, len(sorted_dates)):
        actual_gap = sorted_dates[i] - sorted_dates[i-1]
        if actual_gap > delta * 1.5:  # Allow small tolerance
            gaps.append({
                'start': sorted_dates[i-1],
                'end': sorted_dates[i],
                'duration': actual_gap,
            })

    total_window = sorted_dates[-1] - sorted_dates[0]
    gap_duration = sum((g['duration'] for g in gaps), timedelta())
    gap_pct = gap_duration / total_window if total_window.days > 0 else 0

    return {
        'gaps': gaps,
        'gap_count': len(gaps),
        'gap_percentage': gap_pct,
        'passes': gap_pct <= 0.10,  # Pass if gaps <= 10% of window
    }
```

### Segment Minimums

The 30-observation minimum per segment comes from the Central Limit Theorem -- below ~30, the sampling distribution of the mean is not reliably normal, which invalidates standard confidence interval calculations.

For segments below 30:
1. **Merge**: Combine related small segments (e.g., "Northeast" + "Southeast" -> "East")
2. **Exclude**: Remove the segment with explicit disclosure ("Excluded segments with <30 observations: [list]")
3. **Accept**: Keep with disclosure that confidence intervals may be unreliable for small segments

### Missing Value Assessment

```python
def assess_missing(data, columns, critical_columns=None):
    """Assess missing value rates per column."""
    if critical_columns is None:
        critical_columns = columns

    report = {}
    for col in columns:
        total = len(data)
        missing = sum(1 for row in data if not row.get(col) or row[col] in ('', 'NA', 'null', 'None'))
        pct = missing / total if total > 0 else 0
        is_critical = col in critical_columns

        report[col] = {
            'missing': missing,
            'total': total,
            'pct': pct,
            'critical': is_critical,
            'passes': pct <= 0.20 or not is_critical,
        }

    return report
```

**Handling missing values**:
- **<5%**: Safe to drop rows or ignore. Minimal impact on analysis.
- **5-20%**: Analyze with and without missing rows. If results differ materially, investigate the missingness pattern (is it random or systematic?).
- **>20% in critical column**: The column is unreliable. Either find an alternative data source, impute with explicit disclosure of method, or exclude with documentation.

---

## Gate 2: Comparison Fairness

**Purpose**: Verify that group comparisons are apples-to-apples before drawing conclusions.

**Why this gate exists**: Unfair comparisons produce misleading results. Comparing Q1 to Q3 confounds seasonality with the variable of interest. Comparing active users to all users conflates engagement with the treatment effect. These errors are easy to make and hard to detect after the fact.

### Checks

| Check | Requirement | Common Violation |
|-------|-------------|------------------|
| Same time window | Groups cover identical periods | Comparing Q1 of year A to Q3 of year B |
| Same population definition | Groups drawn from same base | Comparing active users to all users |
| Confounding variables | Identify and document known confounders | Attributing outcome to single variable without controls |
| Survivorship bias | Check if selection criteria exclude failures | Analyzing "successful" cohorts without the failures |

### Time Window Alignment

```python
def check_time_alignment(group_a_dates, group_b_dates):
    """Verify groups cover the same time period."""
    a_start, a_end = min(group_a_dates), max(group_a_dates)
    b_start, b_end = min(group_b_dates), max(group_b_dates)

    overlap_start = max(a_start, b_start)
    overlap_end = min(a_end, b_end)

    a_range = (a_end - a_start).days
    b_range = (b_end - b_start).days
    overlap = max(0, (overlap_end - overlap_start).days)

    alignment = overlap / max(a_range, b_range) if max(a_range, b_range) > 0 else 0

    return {
        'group_a_range': f"{a_start} to {a_end}",
        'group_b_range': f"{b_start} to {b_end}",
        'overlap_pct': alignment,
        'passes': alignment >= 0.90,  # 90% overlap minimum
        'warning': f"Groups cover different periods" if alignment < 0.90 else None,
    }
```

### Population Definition Check

Verify groups are drawn from the same base population:
- Both groups should have the same inclusion/exclusion criteria except for the variable being tested
- Check for self-selection bias: did users choose which group they are in?
- Check for survivorship bias: does one group exclude users who churned/failed/left?

### Confounding Variable Documentation

For each comparison, list known confounders:

```markdown
### Confounders
| Variable | Affects | Controlled? | Method |
|----------|---------|-------------|--------|
| Seasonality | Conversion rate | Yes | Same time window |
| Device type | Page load time | Partially | Reported by segment |
| User tenure | Retention | No | Documented as limitation |
```

If a confounder is not controlled, it must appear in the Limitations section of the final report.

---

## Gate 3: Multiple Testing Correction

**Purpose**: Prevent false discoveries when testing multiple hypotheses simultaneously.

**Why this gate exists**: At a 5% significance level, 1 in 20 tests will be "significant" by chance. If you test 20 segments, you expect one false positive. Without correction, the analyst reports this false positive as a real finding. Multiple testing correction prevents this.

### Decision Table

| Scenario | Number of Tests | Correction | Example |
|----------|----------------|------------|---------|
| Single hypothesis | 1 | None needed | "Did conversion improve?" |
| Few comparisons | 2-5 | Report all p-values, note unadjusted | A/B test with primary + 2 secondary metrics |
| Many comparisons | 6+ | Bonferroni: alpha / N | Analyzing 20 user segments for significance |
| Exploratory sweep | Any | Label as exploratory | "Which of 50 features correlates with churn?" |

### Bonferroni Correction

The simplest and most conservative correction:

```python
def bonferroni_correction(p_values, alpha=0.05):
    """Apply Bonferroni correction to multiple p-values."""
    n_tests = len(p_values)
    adjusted_alpha = alpha / n_tests

    results = []
    for name, p in p_values:
        results.append({
            'test': name,
            'p_value': p,
            'adjusted_alpha': adjusted_alpha,
            'significant_after_correction': p < adjusted_alpha,
            'significant_unadjusted': p < alpha,
        })

    return {
        'n_tests': n_tests,
        'original_alpha': alpha,
        'adjusted_alpha': adjusted_alpha,
        'results': results,
    }
```

### When to Apply

- **Pre-specified primary metric**: No correction needed for the single primary outcome
- **Secondary metrics**: Report as secondary; note they are not corrected unless 6+
- **Subgroup analyses**: Almost always require correction (or labeling as exploratory)
- **Data dredging / feature sweeps**: MUST be labeled exploratory with no causal claims

### Reporting Pattern

When multiple tests are performed, report transparently:

```markdown
## Multiple Testing Note
- Tests performed: [N]
- Correction method: [Bonferroni / None (reported as unadjusted) / Exploratory label]
- Adjusted significance threshold: [alpha / N]
- Tests significant after correction: [list]
- Tests significant before correction only: [list -- interpret with caution]
```

---

## Gate 4: Practical Significance

**Purpose**: Ensure statistically significant results are also meaningful in practice.

**Why this gate exists**: With enough data, tiny effects become statistically significant. A 0.01% conversion lift with p=0.001 is statistically significant but practically meaningless if your minimum actionable threshold is 1%. Practical significance bridges the gap between "real effect" and "worth acting on."

### Requirements

| Metric | Requirement | Why |
|--------|-------------|-----|
| Effect size | Report alongside p-value | "5% lift (p=0.03)" not just "p=0.03" |
| Confidence interval | Report range, not point estimate | "3-7% lift" not "5% lift" |
| Business threshold | Compare to minimum actionable threshold | "5% lift exceeds our 2% threshold for shipping" |
| Base rate context | Show absolute numbers, not just relative | "2.1% to 2.3%" not just "+10% lift" |

### Effect Size Calculation

```python
import math

def cohens_d(mean_a, mean_b, std_a, std_b, n_a, n_b):
    """Calculate Cohen's d for two independent groups."""
    pooled_std = math.sqrt(
        ((n_a - 1) * std_a**2 + (n_b - 1) * std_b**2)
        / (n_a + n_b - 2)
    )
    d = (mean_a - mean_b) / pooled_std if pooled_std > 0 else 0

    # Interpretation
    if abs(d) < 0.2:
        interpretation = "negligible"
    elif abs(d) < 0.5:
        interpretation = "small"
    elif abs(d) < 0.8:
        interpretation = "medium"
    else:
        interpretation = "large"

    return {
        'cohens_d': d,
        'interpretation': interpretation,
    }

def relative_and_absolute_change(baseline, treatment):
    """Report both relative and absolute change."""
    absolute = treatment - baseline
    relative = (treatment - baseline) / baseline if baseline != 0 else float('inf')
    return {
        'baseline': baseline,
        'treatment': treatment,
        'absolute_change': absolute,
        'relative_change_pct': relative * 100,
        'summary': f"{baseline:.2%} to {treatment:.2%} (absolute: {absolute:+.2%}, relative: {relative:+.1%})"
    }
```

### Base Rate Context

Always provide base rate context to prevent misleading relative claims:

| Misleading | Informative |
|------------|-------------|
| "+50% lift!" | "Conversion rose from 0.2% to 0.3%" |
| "-30% reduction in errors" | "Error rate dropped from 3.0% to 2.1%" |
| "2x improvement" | "p99 latency improved from 800ms to 400ms" |

The absolute numbers tell the decision-maker whether the change matters. A 50% lift sounds huge; going from 0.2% to 0.3% may not justify any action.

### Decision Mapping

At the end of Phase 4, map each finding to the decision:

```markdown
## Practical Significance Assessment

| Finding | Effect Size | CI | Threshold | Actionable? |
|---------|------------|-----|-----------|-------------|
| Conversion lift | +0.3% | [-0.1%, 0.7%] | 1.0% | No -- below threshold and CI includes zero |
| Churn reduction | -2.1% | [-3.5%, -0.7%] | 1.0% | Yes -- exceeds threshold, CI excludes zero |
| Revenue impact | +$12K/mo | [$3K, $21K] | $10K/mo | Marginal -- point estimate exceeds but CI lower bound does not |
```

---

## Applying Gates: Decision Tree

```
START: Metric computed
  |
  v
Gate 1: Is sample adequate?
  |-- NO --> Document limitation. Can remediate?
  |            |-- YES --> Merge segments / narrow window / impute
  |            |-- NO  --> Proceed with "insufficient data" caveat
  |
  |-- YES --> Continue
  |
  v
Gate 2: Is comparison fair? (if comparing groups)
  |-- NO --> Document unfairness. Can fix?
  |            |-- YES --> Align windows / match populations
  |            |-- NO  --> Proceed with "unfair comparison" caveat
  |
  |-- YES / N/A --> Continue
  |
  v
Gate 3: Multiple testing? (if >1 hypothesis)
  |-- YES, 2-5 --> Report all, note unadjusted
  |-- YES, 6+  --> Apply Bonferroni, report adjusted
  |-- Exploratory --> Label as exploratory, no causal claims
  |-- NO --> Continue
  |
  v
Gate 4: Is effect practically significant?
  |-- Below threshold --> "Statistically significant but not actionable"
  |-- Above threshold --> "Significant and actionable"
  |-- CI includes zero --> "Inconclusive"
  |
  v
DONE: Gate results documented in analysis-results.md
```

#!/usr/bin/env python3
"""
Gap Analyzer — deterministic gap detection for reference file enrichment.

Reads an agent or skill's .md file, extracts stated domains from the description,
triggers, and body content, compares against existing reference file coverage, and
outputs a JSON report of sub-domains missing reference coverage.

Usage:
    python3 skills/meta/reference-enrichment/scripts/gap-analyzer.py --agent golang-general-engineer
    python3 skills/meta/reference-enrichment/scripts/gap-analyzer.py --skill systematic-code-review
    python3 skills/meta/reference-enrichment/scripts/gap-analyzer.py --agent python-general-engineer --verbose

Exit code: always 0 (analysis tool, not a gate).
"""

from __future__ import annotations

import argparse
import json
import re
import sys
from dataclasses import dataclass, field
from pathlib import Path

# ─── Paths ────────────────────────────────────────────────────

_HOME = Path.home()
_CLAUDE_AGENTS_DIR = _HOME / ".claude" / "agents"
_CLAUDE_SKILLS_DIR = _HOME / ".claude" / "skills"
_REPO_ROOT = Path(__file__).parent.parent.parent.parent
_REPO_AGENTS_DIR = _REPO_ROOT / "agents"
_REPO_SKILLS_DIR = _REPO_ROOT / "skills"

# ─── Domain Extraction ─────────────────────────────────────────

# Technology terms to extract as candidate domains.
# Each tuple: (pattern, canonical_domain_name)
_DOMAIN_PATTERNS: list[tuple[re.Pattern[str], str]] = [
    # Languages
    (re.compile(r"\bGo\b(?:lang)?", re.IGNORECASE), "go"),
    (re.compile(r"\bPython\b", re.IGNORECASE), "python"),
    (re.compile(r"\bTypeScript\b", re.IGNORECASE), "typescript"),
    (re.compile(r"\bJavaScript\b", re.IGNORECASE), "javascript"),
    (re.compile(r"\bRust\b", re.IGNORECASE), "rust"),
    (re.compile(r"\bKotlin\b", re.IGNORECASE), "kotlin"),
    (re.compile(r"\bSwift\b", re.IGNORECASE), "swift"),
    (re.compile(r"\bRuby\b", re.IGNORECASE), "ruby"),
    (re.compile(r"\bPHP\b", re.IGNORECASE), "php"),
    (re.compile(r"\bJava\b", re.IGNORECASE), "java"),
    # Concurrency / async
    (re.compile(r"\bconcurren(?:cy|t)\b", re.IGNORECASE), "concurrency"),
    (re.compile(r"\basync(?:hronous)?\b", re.IGNORECASE), "async"),
    (re.compile(r"\bgoroutine\b", re.IGNORECASE), "goroutines"),
    (re.compile(r"\bchannel\b", re.IGNORECASE), "channels"),
    (re.compile(r"\bTaskGroup\b"), "task-groups"),
    (re.compile(r"\bactor\b", re.IGNORECASE), "actors"),
    # Testing
    (re.compile(r"\btesting\b|\btest\b", re.IGNORECASE), "testing"),
    (re.compile(r"\bpytest\b", re.IGNORECASE), "pytest"),
    (re.compile(r"\bJUnit\b", re.IGNORECASE), "junit"),
    (re.compile(r"\bmock(?:ing)?\b", re.IGNORECASE), "mocking"),
    # Error handling
    (re.compile(r"\berror.handl\w+\b", re.IGNORECASE), "error-handling"),
    (re.compile(r"\bexception\b", re.IGNORECASE), "exceptions"),
    # Performance
    (re.compile(r"\bperformance\b|\boptimiz\w+\b", re.IGNORECASE), "performance"),
    (re.compile(r"\bbenchmark\b", re.IGNORECASE), "benchmarking"),
    (re.compile(r"\bprofil\w+\b", re.IGNORECASE), "profiling"),
    # Security
    (re.compile(r"\bsecurity\b", re.IGNORECASE), "security"),
    (re.compile(r"\bSQL injection\b", re.IGNORECASE), "sql-injection"),
    (re.compile(r"\bXSS\b", re.IGNORECASE), "xss"),
    (re.compile(r"\bauth(?:entication|orization)?\b", re.IGNORECASE), "auth"),
    # Data / types
    (re.compile(r"\btype.hint\b|\btype.safe\b|\btyping\b", re.IGNORECASE), "type-hints"),
    (re.compile(r"\bPydantic\b", re.IGNORECASE), "pydantic"),
    (re.compile(r"\bdataclass\b", re.IGNORECASE), "dataclasses"),
    (re.compile(r"\bgenerics?\b", re.IGNORECASE), "generics"),
    (re.compile(r"\binterface\b", re.IGNORECASE), "interfaces"),
    # Infrastructure
    (re.compile(r"\bKubernetes\b|\bk8s\b", re.IGNORECASE), "kubernetes"),
    (re.compile(r"\bDocker\b", re.IGNORECASE), "docker"),
    (re.compile(r"\bSQL\b", re.IGNORECASE), "sql"),
    (re.compile(r"\bPostgres\b", re.IGNORECASE), "postgres"),
    (re.compile(r"\bMySQL\b", re.IGNORECASE), "mysql"),
    # Frameworks
    (re.compile(r"\bFastAPI\b", re.IGNORECASE), "fastapi"),
    (re.compile(r"\bDjango\b", re.IGNORECASE), "django"),
    (re.compile(r"\bFlask\b", re.IGNORECASE), "flask"),
    (re.compile(r"\bReact\b", re.IGNORECASE), "react"),
    (re.compile(r"\bNext\.js\b|\bNextJS\b", re.IGNORECASE), "nextjs"),
    # Toolkit-specific
    (re.compile(r"\banti.pattern\b", re.IGNORECASE), "anti-patterns"),
    (re.compile(r"\bcode.review\b", re.IGNORECASE), "code-review"),
    (re.compile(r"\brefactor\b", re.IGNORECASE), "refactoring"),
    (re.compile(r"\blogging\b|\bstructured.log\b", re.IGNORECASE), "logging"),
    (re.compile(r"\bmetrics?\b", re.IGNORECASE), "metrics"),
    (re.compile(r"\btracing\b", re.IGNORECASE), "tracing"),
    (re.compile(r"\bmodule\b", re.IGNORECASE), "modules"),
    (re.compile(r"\bpackage.manag\w+\b", re.IGNORECASE), "package-management"),
]

# Coverage mapping: reference filename keywords → domain names they cover.
# A reference file "go-concurrency.md" is treated as covering "concurrency", "goroutines", etc.
_COVERAGE_MAP: dict[str, list[str]] = {
    "concurren": ["concurrency", "goroutines", "channels", "task-groups", "actors", "async"],
    "async": ["async", "concurrency", "task-groups"],
    "goroutine": ["goroutines", "concurrency", "channels"],
    "channel": ["channels", "concurrency"],
    "testing": ["testing", "mocking", "pytest", "junit"],
    "test": ["testing", "mocking"],
    "mock": ["mocking", "testing"],
    "error": ["error-handling", "exceptions"],
    "exception": ["exceptions", "error-handling"],
    "performance": ["performance", "benchmarking", "profiling"],
    "benchmark": ["benchmarking", "performance"],
    "profil": ["profiling", "performance"],
    "security": ["security", "sql-injection", "xss", "auth"],
    "auth": ["auth", "security"],
    "anti.pattern": ["anti-patterns"],
    "pattern": ["anti-patterns"],
    "type": ["type-hints", "generics"],
    "typing": ["type-hints"],
    "dataclass": ["dataclasses"],
    "pydantic": ["pydantic"],
    "generic": ["generics"],
    "interface": ["interfaces"],
    "logging": ["logging"],
    "log": ["logging"],
    "metric": ["metrics"],
    "tracing": ["tracing"],
    "sql": ["sql", "sql-injection"],
    "module": ["modules", "package-management"],
    "package": ["package-management", "modules"],
    "modern": [],  # "modern" alone doesn't indicate a specific covered domain
    "feature": [],
    "idiom": [],
    "review": ["code-review"],
    "refactor": ["refactoring"],
}


def _extract_domains_from_text(text: str) -> set[str]:
    """Extract candidate domain names from free text using pattern matching."""
    found: set[str] = set()
    for pattern, domain in _DOMAIN_PATTERNS:
        if pattern.search(text):
            found.add(domain)
    return found


def _domains_covered_by_filename(filename: str) -> set[str]:
    """Infer which domains a reference file covers based on its filename."""
    name_lower = filename.lower().replace("-", " ").replace("_", " ")
    covered: set[str] = set()
    for keyword, domains in _COVERAGE_MAP.items():
        if re.search(keyword, name_lower):
            covered.update(domains)
    # Also add the stem words themselves as covered domains
    stem = Path(filename).stem.lower()
    parts = re.split(r"[-_]", stem)
    covered.update(parts)
    return covered


def _filename_for_domain(domain: str) -> str:
    """Suggest a reference filename for a domain gap."""
    return f"{domain}.md"


# ─── Data Classes ─────────────────────────────────────────────


@dataclass
class RecommendedRef:
    """A recommended reference file to create for a gap."""

    filename: str
    domain: str
    reason: str


@dataclass
class GapReport:
    """Gap analysis result for a single component."""

    component: str
    kind: str  # "agent" or "skill"
    current_level: int
    existing_references: list[str] = field(default_factory=list)
    stated_domains: list[str] = field(default_factory=list)
    covered_domains: list[str] = field(default_factory=list)
    gaps: list[str] = field(default_factory=list)
    recommended_references: list[RecommendedRef] = field(default_factory=list)


# ─── Scanning ─────────────────────────────────────────────────


def _find_component(name: str, kind: str) -> tuple[Path | None, Path | None]:
    """Locate a component's .md file and its references/ directory.

    Returns (md_path, ref_dir) — either may be None if not found.
    """
    search_dirs: list[Path] = []
    if kind == "agent":
        search_dirs = [_CLAUDE_AGENTS_DIR, _REPO_AGENTS_DIR]
    else:
        search_dirs = [_CLAUDE_SKILLS_DIR, _REPO_SKILLS_DIR]

    for base in search_dirs:
        # Flat .md file: agents/foo.md
        flat = base / f"{name}.md"
        if flat.is_file():
            ref_dir_candidate = flat.parent / name / "references"
            ref_dir = ref_dir_candidate if ref_dir_candidate.is_dir() else None
            return flat, ref_dir

        # Named directory: agents/foo/foo.md  OR  skills/foo/SKILL.md
        named_dir = base / name
        if named_dir.is_dir():
            inner_md = named_dir / f"{name}.md"
            if not inner_md.is_file():
                # Skills use SKILL.md convention
                inner_md = named_dir / "SKILL.md"
            if inner_md.is_file():
                ref_dir_inner = named_dir / "references"
                ref_dir = ref_dir_inner if ref_dir_inner.is_dir() else None
                return inner_md, ref_dir

    return None, None


def _read_md(md_path: Path) -> str:
    """Read a markdown file, returning empty string on error."""
    try:
        return md_path.read_text(encoding="utf-8", errors="replace")
    except OSError:
        return ""


def _current_level(ref_dir: Path | None) -> int:
    """Return a rough depth level for the component's current state.

    This is a lightweight approximation — for authoritative scoring, run
    scripts/audit-reference-depth.py.
    """
    if ref_dir is None or not ref_dir.is_dir():
        return 0

    ref_files = list(ref_dir.glob("*.md"))
    if not ref_files:
        return 0

    total_lines = 0
    has_code = False
    has_commands = False
    concrete_hits = 0

    for f in ref_files:
        try:
            text = f.read_text(encoding="utf-8", errors="replace")
        except OSError:
            continue
        lines = text.splitlines()
        total_lines += len(lines)
        if "```" in text:
            has_code = True
        if re.search(r"\b(?:grep|rg|find)\s+", text):
            has_commands = True
        concrete_hits += len(re.findall(r"\b\d+\.\d+\+", text))
        concrete_hits += len(re.findall(r"```", text))

    avg_lines = total_lines / len(ref_files) if ref_files else 0

    if has_commands and concrete_hits >= 10 and avg_lines >= 80:
        return 3
    if has_code or (concrete_hits >= 5 and avg_lines >= 30):
        return 2
    return 1


def _build_gap_report(name: str, kind: str) -> GapReport | None:
    """Build a gap report for the named component."""
    md_path, ref_dir = _find_component(name, kind)
    if md_path is None:
        return None

    md_text = _read_md(md_path)
    level = _current_level(ref_dir)

    # Collect existing reference filenames
    existing: list[str] = []
    if ref_dir and ref_dir.is_dir():
        existing = sorted(f.name for f in ref_dir.glob("*.md"))

    # Extract domains from the .md content
    stated = _extract_domains_from_text(md_text)

    # Determine which domains are already covered by existing reference filenames
    covered: set[str] = set()
    for ref_filename in existing:
        covered.update(_domains_covered_by_filename(ref_filename))

    # Gaps: stated domains with no coverage
    gaps = sorted(stated - covered)

    # Build recommendations — filter out domains too vague to act on
    _LOW_SIGNAL_DOMAINS = {"go", "python", "typescript", "javascript", "rust", "kotlin", "swift", "ruby", "php", "java"}

    recommendations: list[RecommendedRef] = []
    for domain in gaps:
        if domain in _LOW_SIGNAL_DOMAINS:
            # The primary language domain — too broad for a single reference file;
            # skip and let sub-domain gaps (concurrency, testing, etc.) drive files instead.
            continue
        filename = _filename_for_domain(domain)
        reason = (
            f"Component mentions '{domain}' but no reference file covers it. "
            f"Add concrete patterns, anti-patterns with detection commands, and version notes."
        )
        recommendations.append(RecommendedRef(filename=filename, domain=domain, reason=reason))

    return GapReport(
        component=name,
        kind=kind,
        current_level=level,
        existing_references=existing,
        stated_domains=sorted(stated),
        covered_domains=sorted(covered),
        gaps=gaps,
        recommended_references=recommendations,
    )


# ─── Reporting ─────────────────────────────────────────────────


def _to_dict(report: GapReport) -> dict:
    """Convert a GapReport to a JSON-serialisable dict."""
    return {
        "component": report.component,
        "type": report.kind,
        "current_level": report.current_level,
        "existing_references": report.existing_references,
        "stated_domains": report.stated_domains,
        "covered_domains": report.covered_domains,
        "gaps": report.gaps,
        "recommended_references": [
            {
                "filename": r.filename,
                "domain": r.domain,
                "reason": r.reason,
            }
            for r in report.recommended_references
        ],
    }


def _format_text(report: GapReport) -> str:
    """Render a human-readable gap report."""
    lines: list[str] = []
    lines.append(f"GAP ANALYSIS: {report.component} ({report.kind})")
    lines.append("=" * 50)
    lines.append(f"  Current level : {report.current_level}")
    lines.append(f"  Existing refs : {len(report.existing_references)}")
    if report.existing_references:
        for ref in report.existing_references:
            lines.append(f"    - {ref}")
    lines.append("")

    lines.append(f"  Stated domains ({len(report.stated_domains)}):")
    for d in report.stated_domains:
        covered = " [covered]" if d in report.covered_domains else " [GAP]"
        lines.append(f"    - {d}{covered}")
    lines.append("")

    if report.recommended_references:
        lines.append(f"  Recommended reference files ({len(report.recommended_references)}):")
        for rec in report.recommended_references:
            lines.append(f"    → {rec.filename}")
            lines.append(f"      {rec.reason}")
    else:
        lines.append("  No gaps found — references cover all stated domains.")

    return "\n".join(lines)


# ─── Main ──────────────────────────────────────────────────────


def main() -> int:
    """Entry point."""
    parser = argparse.ArgumentParser(
        description="Analyze reference file gaps for an agent or skill.",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=__doc__,
    )
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument(
        "--agent",
        metavar="NAME",
        help="Analyze gaps for an agent by name",
    )
    group.add_argument(
        "--skill",
        metavar="NAME",
        help="Analyze gaps for a skill by name",
    )
    parser.add_argument(
        "--json",
        dest="as_json",
        action="store_true",
        help="Output machine-readable JSON",
    )
    parser.add_argument(
        "--verbose",
        action="store_true",
        help="Show additional detail in text output",
    )
    args = parser.parse_args()

    name = args.agent or args.skill
    kind = "agent" if args.agent else "skill"

    report = _build_gap_report(name, kind)
    if report is None:
        print(f"[error] {kind} '{name}' not found in agents/ or skills/ directories.", file=sys.stderr)
        return 1

    if args.as_json:
        print(json.dumps(_to_dict(report), indent=2))
    else:
        print(_format_text(report))

    return 0


if __name__ == "__main__":
    raise SystemExit(main())