# Generating and Maintaining the Thesaurus

Read this when the user asks to create, generate, update, or audit the project thesaurus.
For naming consultation (the frequent case), the main SKILL.md has everything you need.

## Creating a New Thesaurus

**Derive the thesaurus from the codebase** — not from docs or specs.

### Step 0: Determine the thesaurus path

1. If the user specified a path — use it
2. Search the repo for an existing `THESAURUS.md`: `find . -name "THESAURUS.md" -not -path "*/node_modules/*"`
3. If found — use the existing location
4. Default: `docs/THESAURUS.md` (create `docs/` if it doesn't exist)

Use this resolved path throughout all subsequent steps.

### Step 1: Scan high-signal hubs (do NOT read the whole codebase)

An agent cannot scan an entire repository without exhausting context. Instead, target
**structural files that are dense with domain nouns** — 5-10 tool calls covers 80-90%:

1. **DB schemas and migrations** — ORM model definitions, schema files, migration scripts
2. **API contracts** — OpenAPI/Swagger specs, GraphQL schemas, route/controller definitions
3. **Domain layer** — aggregate/entity/value-object type declarations
4. **Directory structure** — `ls` top-level dirs to map product areas (cheap, zero-read)
5. **Symbol extraction** — grep for type/class/interface/struct declarations across source

**Stop conditions:**
- If the repo has multiple product areas, ask the user which to catalog first
- Stop when new scans mostly return terms already seen (diminishing returns)
- Default to one bounded area at a time, not the entire monorepo

**Do NOT rely on docs** — they're often outdated. If a README says "User" but the code
says "Customer" everywhere, the canonical term is "Customer".

**Scope the thesaurus to the problem.** Don't catalog every noun in the codebase —
catalog only terms that matter for the system's purpose. Include a term if: domain
experts use it, it appears in invariants/commands/events, ambiguity about it has caused
bugs, or multiple synonyms exist. Exclude purely technical infrastructure terms
(LogLevel, RetryPolicy, ConnectionPool). **Target: 15-40 terms per bounded context.**
Past 60 you're likely including infrastructure; fewer than 10 you're missing concepts.

### Step 2: Separate active from legacy/obsolete

Codebases accumulate dead weight. When scanning, classify each term:

- **Active**: Used in current code paths, referenced by live features
- **Legacy**: Still in codebase but deprecated, behind feature flags, or in migration layers. Mark with `[LEGACY]` prefix and note what replaces it
- **Obsolete**: Dead code, unused classes, abandoned tables. Don't add to thesaurus — just note for cleanup

**How to detect legacy/obsolete:**
- Classes/tables with `Legacy`, `Old`, `Deprecated`, `V1`, `V2` prefixes/suffixes
- Code behind `if (featureFlag)` guards or `#if LEGACY` preprocessor directives
- Methods marked `@Deprecated`, `[Obsolete]`, or with deprecation comments
- Database tables with zero recent writes (check with user)
- Modules that nothing imports anymore (check import graph)
- Names that only appear in test fixtures or migration scripts

**Ask the user** when classification is ambiguous: "I found `UserProfile` and `CustomerProfile` — which is the active concept? Is the other legacy?"

### Step 3: Cluster, identify conflicts, collect ambiguities

- Group synonyms and variants (same concept, different names)
- Identify polysemy (same name, different concepts in different places)
- Flag naming inconsistencies between active code and its tests/docs

**Don't stop on each ambiguity** — collect them all into the `## Unresolved` section
of the thesaurus. This lets you scan the entire codebase in one pass and gives the
user a complete picture to prioritize, rather than answering questions one by one.

For each ambiguity, record: what term, where it's found, what the question is,
how many files are affected, and possible resolutions if obvious.

### Step 4: Write the thesaurus

Write `THESAURUS.md` in the grep-first layout (Bootstrap Template below):
- YAML frontmatter first: `thesaurus-format: "2.0"`, `skill: ubiquitous-language`
- Keep the template's "Reconstructed, not authored" line in the header — this document
  was mined from code, not written by domain experts; names are evidence, definitions are
  reconstruction, and readers must know the difference
- Active terms → one `## Index` line each **and** one `### Term` entry under `## Terms`
- Synonyms, abbreviations, competing names → the `avoid:` list of their concept's line
- Weasel words and jargon you actually saw in the code → `## Forbidden` lines
- Deprecated names → `## Legacy` lines (one line each; no prose entries)
- All ambiguities → `## Unresolved` entries

**Write the Index first**, then the entries in the same alphabetical order. The Index is
what agents read on every naming task; the entries are what they grep into. Before
finishing, confirm the registry invariant: every name appears in exactly one registry
line — Index line, Forbidden line, Legacy line, or Unresolved header. A name that is both
under `avoid:` and in Legacy, or has an entry but no Index line, is a defect.

**Start with a flat thesaurus** — no `ctx:` tokens, no context sections. Most
projects don't need context separation. Only introduce bounded contexts later if Step 6
or Step 8 reveals genuine polysemy.

### Step 5: Surface unresolved issues

**This step is mandatory** — always runs right after writing the file.

After creating `THESAURUS.md`, explicitly present the `## Unresolved` section to the
user. Frame it as: "The thesaurus is ready, but there are N naming conflicts that
need your input — without resolving them, the thesaurus quality will suffer."

List each issue with its impact. Example output:

```
THESAURUS.md created with 24 terms, 3 legacy terms, and 5 unresolved issues:

1. `Account` — used as financial entity (billing/) AND user identity (auth/) — 18 files affected
2. `User` vs `Customer` — synonym drift between API and domain layers — 31 files
3. `Process` — means workflow in scheduler/, means OS process in runtime/ — 7 files
4. `Status` — enum with 12 values, some overlap with `State` enum — 15 files
5. `Service` — bare word used 40+ times, needs polysemy unpacking

Which would you like to resolve first?
```

As the user resolves each item, promote it from `## Unresolved` to an Index line + entry
(or a `## Legacy` line). Items the user defers stay as documented naming debt.

**Then offer Step 6** — before the user starts answering item by item, offer to mine the
git history first. Offer it once, as an option, never as a blocking question.

### Step 6 (optional): Resolve ambiguities from git history

**Offer this whenever `## Unresolved` is non-empty and the repo has real history.**
It is the one step that can answer unresolved items *without* the user answering them:
git history records which name came first, which one replaced which, and which is dying.

Ask, don't assume:

```
5 unresolved naming issues remain. Before you answer them one by one, I can mine the
git history — when each name was born, which replaced which, which is growing vs dying.
It builds a temporary index outside the repo (~1-3 min here, deleted afterwards) and
comes back with evidence-backed proposals per item. Want me to try that first?
```

Skip the offer when the repo has < ~50 commits, has no `.git`, starts from a squashed
import, or when the unresolved items are `[WHITE-SPOT]` tags (a concept nobody has named
yet leaves no trace in history).

If the user accepts, read [git-history-mining.md](git-history-mining.md) and follow its
protocol. In short:

1. `python3 scripts/git_term_index.py build --repo-dir . --content` — a throwaway index of
   commit messages, paths, renames, and diff-level identifiers, written to `$TMPDIR`,
   never into the working tree.
2. `query` each candidate name and `pair` the competing ones; confirm the few decisive
   commits with `git show` / `git log -S`.
3. Report one batch of **proposals with citations and confidence** — never silent edits.
4. Apply only what the user approves; cite the commit in the Legacy note or an entry
   `- **History**:` line; leave the rest in `## Unresolved` with what was ruled out.
5. `python3 scripts/git_term_index.py clean` and say the index is gone.

History is evidence of what happened, not authority on what the term *should* be. It
ranks candidates; the user decides.

### Step 7: Update project instructions

**This step is mandatory** — the thesaurus is only useful if the agent knows about it.

After creating the thesaurus, add a reference to it in **all** agent instruction files
found in the project. Check for and update whichever exist:

- `CLAUDE.md` (Claude Code)
- `GEMINI.md` (Gemini CLI)
- `AGENTS.md` (multi-agent)
- `.cursorrules` (Cursor)
- `.github/copilot-instructions.md` (GitHub Copilot)

Propose adding a section like (use the resolved path from Step 0):

```markdown
## Domain Language

This project maintains a domain thesaurus at `docs/THESAURUS.md`. It is grep-first:
one `## Index` line per concept — ``- **Term** `Identifier` kind:… avoid: `synonyms` ``.

- **Before naming anything** (class, method, variable, DB table, API endpoint, file),
  search `rg -n -i '<word>' docs/THESAURUS.md` for each name you are considering.
  - Hit in an Index line → use that line's `Identifier`, even if your word was under `avoid:`.
  - Hit in a `` - `word` use: `` (Forbidden) or `` - `word` → `` (Legacy) line → do not use it;
    the line names the replacement.
  - Hit in `## Unresolved` → open question; ask before deciding.
  - No hit → new concept: add an Index line and a `### Term` entry **before** using it in code.
- Useful: `rg 'kind:event'` · ``rg 'avoid:.*`Word`'`` · `rg -F '**Term**'` · `rg '^### Term( \(|$)'`.
- Never introduce a synonym for an existing Index term.
```

This ensures every agent session — even without the ubiquitous-language skill installed —
knows the thesaurus exists and should consult it.

### Step 8 (optional): Detect polysemy

**Do NOT pre-assign bounded contexts.** The agent cannot reliably determine context
boundaries — this is an architectural decision that requires deep domain knowledge.

Instead, look for **evidence of polysemy** — the same word meaning different things:
- Same class name in different modules/packages with different fields/methods
- Same DB column name with different semantics in different tables
- Same API term used inconsistently across endpoints
- User/team disagreement about what a term means

**When you find evidence**, don't decide — report it to the user:
"I found `Account` used in two different ways: as a financial entity in `billing/`
and as a user identity in `auth/`. Should these be separate bounded contexts, or is
one of them a legacy naming mistake?"

Only add `ctx:` tokens and context sections to the thesaurus after the user
confirms the separation. A wrong context boundary is worse than no boundary.

**The invariant test** (from FPF A.1.1): A bounded context is justified only when you
can name **at least one rule (invariant)** that is true inside the context but not
outside. Example: "An Order in Sales context can be cancelled; an Order in Fulfillment
context cannot be cancelled once shipped." If you can't name such a rule — it's just
a module, not a bounded context.

### Bootstrap Template

```markdown
---
thesaurus-format: "2.0"
skill: ubiquitous-language
---

# Project Thesaurus

> Domain glossary following DDD ubiquitous language. Every name in code, APIs, docs,
> and conversations comes from here. Add the term here BEFORE using it in code.
>
> **Reconstructed, not authored.** An AI agent mined this vocabulary from the codebase —
> the names are evidence found in code and are binding; the definitions are a
> reconstruction of what the code seems to mean and may be wrong until a domain expert
> confirms them. Maintained by the `ubiquitous-language` skill
> (https://github.com/CodeAlive-AI/ai-driven-development/tree/main/skills/ubiquitous-language).
>
> **How to use (grep-first):** `rg -n -i '<word>' THESAURUS.md` — the shape of the hit
> line tells you what to do:
> - ``- **Term** `Identifier` kind:… avoid: … `` (Index) → use the Identifier, even if
>   your word was under `avoid:`
> - ``- `Word` use: `X` `` (Forbidden) → banned; use `X`
> - ``- `Old` → `New` in: … `` (Legacy) → use `New` in new code
> - `### Term — …` under Unresolved → open question; ask before deciding
> - no hit → new concept: add an Index line + `### Term` entry first
> Handy: `rg 'kind:event'` · `rg 'ctx:Billing'` · `rg -F '**Term**'` · `rg '^### Term( \(|$)'`
>
> **Rules:** one canonical Identifier per concept; every name lives in exactly one
> registry line; `avoid:`/Forbidden/Legacy names never appear in new code; on rename,
> update code, docs, API, DB — and add a Legacy line.

## Index

- **[Term]** `[PascalCase]` kind:[aggregate|entity|value|event|command|query|service|role|process|state|policy|concept] avoid: `[synonym]`, `[abbrev]`

## Terms

### [Term]
- **Definition**: [What this means in the business domain — one sentence]
- **NOT**: [What this does NOT mean — the neighbouring term it is confused with]
- **Related**: [Other Index Terms, written exactly as in the Term field]

## Forbidden

> Words that MUST NOT appear in domain-layer names: implementation details, weasel
> words, bundle-collapse terms. `use:` always points at an Index Identifier.

- `Manager` use: [specific activity] — vague, hides responsibility
- `Handler` use: [what it handles] — generic
- `Service` use: [specific facet] — overloaded, see Polysemy Unpacking
- `Info`, `Data` use: [the term itself] — meaningless suffix

## Legacy

> Names still present in the codebase but deprecated. New code MUST use the name after
> `→`. `A + B` = the old name was split; `→ —` = retired with no single successor.

- `[OldName]` → `[Identifier]` in: [modules/files] — [what it meant; since when]

## Unresolved

> Naming ambiguities, contradictions, and open questions. Each needs a human decision
> before the name can enter the Index. Resolve top-down by impact.

### [Term] — [short description of the problem]
- **Found in**: [where this name appears with different meanings/usage]
- **Question**: [what needs to be decided]
- **Impact**: [how many files/modules are affected]
- **Options**: [possible resolutions, if known]

<!-- ONLY when confirmed polysemy requires bounded contexts:
  1. Add `ctx:<Context>` to every Index line that belongs to a context (after kind:)
  2. Headers become `### Term (Context)`; group entries under `## Terms: [Context]`
  3. Add the bridges section (one line per bridge, SKOS mapping vocabulary):

## Cross-Context Bridges

- **Account** Billing ↔ Identity: distinct — service accounts have no ledger; guest checkout has a ledger but no login
- **Customer** Sales ↔ Identity: closeMatch — one Customer may own several Identity Accounts; never join on spelling

  Mapping is one of: exactMatch · closeMatch · broadMatch · narrowMatch · relatedMatch · distinct
  (SKOS mapping properties + explicit `distinct` for homonyms). Never `sameAs`.
-->
```

## Migrating an Existing Thesaurus

Read the format stamp first: `rg -n '^thesaurus-format:' THESAURUS.md`. No stamp, or
`1.x`, means **format 1.0** — `### Term` entries carrying `**Synonyms to AVOID**` lines,
no `## Index`. An unstamped file that already has ``- **Term** `Id` kind: `` Index lines was
written by plugin 9.2.0 — it is format 2.0, only step 0 applies. (An unstamped file whose
`## Index` is a Markdown table is an interim pre-2.0 draft; migrate it like 1.0.) Migrate in one pass — the content is the same,
only the shape changes:

0. **Stamp** the result: YAML frontmatter at the top of the file —
   `thesaurus-format: "2.0"`, `skill: ubiquitous-language`
   (replace an older stamp if present; merge into existing frontmatter if the file has any).

1. **Build the Index lines** from the entries (or table rows): Term = header text;
   Identifier = PascalCase of the term (or the name the code actually uses — grep to
   confirm); `kind:` = infer from the definition, default `concept`; `avoid:` = the
   `Synonyms to AVOID` list (or Avoid column), each in backticks.
2. **Strip** `**Synonyms to AVOID**` lines from the entries; rename `**Related terms**`
   to `**Related**`.
3. **Convert `## Legacy Terms` entries (or Legacy table rows) to `## Legacy` lines**:
   `` `Old` → `Replacement` in: <still found in> — <note> ``.
4. **Rename** `## Forbidden Lexicon` → `## Forbidden`; one `` `Word` use: `X` — why ``
   line per word.
5. **Contexts**: `## Bounded Context: X` sections → `## Terms: X`, headers `### Term (X)`,
   `ctx:X` on the Index lines; a Bridges table → one line per bridge with a SKOS mapping.
6. **Check the registry invariant** (see the audit's Check 0) and show the user the
   diff before writing — migration must not change a single definition.

### Format history

| Format | Skill | What changed |
|--------|-------|--------------|
| 1.0 | ≤ 1.x (plugin ≤ 9.1.1) | Prose entries with `Synonyms to AVOID`, `## Legacy Terms` entries, `## Forbidden Lexicon` table; no stamp |
| 2.0 | 2.0.0 (plugin 9.3.0) | Grep-first: `## Index` lines (`kind:` `ctx:` `avoid:`), `use:` Forbidden lines, `→` Legacy lines, SKOS bridge mappings, format stamp |

Bump the format **major** only when 1.x readers would misparse the file (a changed line
grammar or section set). Adding an optional token is a **minor** bump: stamp `2.1`, keep
every 2.0 line valid.

## Polysemy Unpacking

Some terms are "bundle-collapse" words — they silently stand in for multiple distinct
concepts. The word "service" is the canonical example: it can mean a promise, a system,
an endpoint, a commitment, a delivery method, or a work episode — all at once.

**When you encounter an overloaded term**, unpack it into its facets:

1. **Identify the facets** — what distinct things does this word refer to?
2. **Create separate thesaurus entries** for each facet with qualified names
3. **Add the bare word to `## Forbidden`** — it must always be qualified
4. **Document which facet is meant** in each code location

### Example: Unpacking "Service"

The bare word "service" collapses at least these facets:

| Facet | What it means | Qualified name |
|-------|--------------|----------------|
| Promise | What is offered/contracted | ServiceOffering |
| Provider | Who is accountable | ServiceProvider |
| Endpoint | What you can call/address | ServiceEndpoint |
| Delivery System | What performs the work | ServiceSystem |
| Commitment | The binding obligation (SLA) | ServiceCommitment |
| Delivery Work | A fulfillment episode | ServiceRun |

**The "can you X it?" tests:**
- "Can you call/restart it?" → it's an **endpoint**, not a promise
- "Can it guarantee/must it?" → it's a **commitment**, not an endpoint
- "How does it work?" → it's a **system** or **method**, not a promise
- "Is it down/slow?" → it's an **endpoint** or **work episode**, with evidence

### When to Unpack

Flag a term for polysemy unpacking when:
- The same word appears as subject of incompatible verbs ("the X is deployed" AND "the X promises")
- Different team members mean different things by the same word
- Code uses the term in structurally different ways across modules
- You can't answer "what type is this?" with a single answer

## Bounded Contexts and Polysemy

The same word can mean different things in different bounded contexts. This is correct
DDD — don't fight it, document it.

> "Cross-context sameness is never inferred from spelling; cross-context alignment is
> represented only via explicit Bridges." — FPF A.1.1

### When the Same Word Means Different Things

Example: "Account" across three contexts:
- **Payment Context**: Financial account with a balance
- **Customer Context**: User login credentials and profile
- **Accounting Context**: Ledger entry in chart of accounts

**Rules:**
- Each context owns its own definition in the thesaurus
- One Index line per (Term, Context) pair; group entries by context, alphabetical within
- If code has `if` statements checking "which context am I in?" — the boundary is wrong
- Use Anti-Corruption Layers at context boundaries for term translation
- **Never assume sameness from spelling** — "Account" in Payment and "Account" in Customer are different concepts that happen to share a label

### Recognizing Context Boundaries

You've found a boundary when:
- Domain experts disagree on what a term means
- Translation logic between modules keeps growing
- The same class name appears with different structures in different packages
- Teams use different words for the same concept (this is a signal, not a problem)

### Cross-Context Bridges

When terms appear in multiple contexts, document the **bridge** explicitly — one line
per bridge under `## Cross-Context Bridges`:

```
- **<Term>** <Context A> ↔ <Context B>: <mapping> — <loss notes>
```

- **mapping** uses the SKOS mapping vocabulary, plus an explicit `distinct` for homonyms:
  `exactMatch` (interchangeable in practice) · `closeMatch` (interchangeable in some
  uses) · `broadMatch` / `narrowMatch` (A is more general / more specific than B) ·
  `relatedMatch` (associated, neither subsumes) · `distinct` (same spelling, different
  concept). Never `sameAs` — cross-context identity is never inferred from spelling.
- **loss notes** — what breaks if you treat them as the same. The most important field.

```markdown
## Cross-Context Bridges

- **Account** Billing ↔ Identity: distinct — service accounts have no ledger; guest checkout has a ledger but no login
- **Order** Sales ↔ Fulfillment: narrowMatch — Fulfillment Order adds prep steps and timing, loses pricing
```

### Documenting Cross-Context Terms

```markdown
## Index

- **Account** `Account` kind:aggregate ctx:Billing avoid: `Wallet`, `Purse`, `Balance`
- **Account** `Account` kind:aggregate ctx:Identity avoid: `User`, `Profile`, `Login`

## Terms: Billing

### Account (Billing)
- **Definition**: Financial account with balance, used for charging and refunds
- **NOT**: User identity or login credentials (that's Account in Identity)

## Terms: Identity

### Account (Identity)
- **Definition**: User's login identity — email, password, profile
- **NOT**: Financial balance (that's Account in Billing)
```

`rg -n '^### Account( \(|$)'` returns both entries with their context in the header;
`rg -n 'ctx:Billing'` lists everything one context owns.

## Term Relationships

Use these relationship types to connect terms (based on ISO 25964 / SKOS):

| Relationship | Meaning | Example |
|-------------|---------|---------|
| **Broader** | More general concept | Repository is broader than GitHubRepository |
| **Narrower** | More specific concept | GitHubRepository is narrower than Repository |
| **Part-of** | Composition | Commit is part-of Repository |
| **Related** | Associated, not hierarchical | Repository is related to Branch |
| **Synonym** | Same concept, different word (pick one, avoid the other) | Codebase = Repository (avoid Codebase) |

Synonyms go to the Index `avoid:` list; the hierarchical relations are optional entry
lines, added only when they carry real information. Write each relation on one side
only — `rg` gives the inverse; mirrored copies drift. SKOS rule: a pair is never both
`Broader` and `Related`, and `Broader` chains never cycle.
```markdown
### Repository
- **Definition**: A version-controlled code storage location
- **Broader**: Version Control System
- **Narrower**: GitHub Repository, GitLab Repository, Monorepo
- **Has parts**: Branch, Commit, File
- **Related**: Code Source, Indexed Analysis
```
Write related names exactly as in the Index Term field, so `grep -n 'Branch'` finds
both the Branch Index line and every entry that points at it.

## Consistency Audit

For a full naming audit protocol (9 checks, severity levels, report format), see
[naming-audit.md](naming-audit.md).

## Brownfield Language Adoption

When introducing ubiquitous language to a project that already has an established
(but imprecise) vocabulary:

1. **Don't try to change how people talk overnight.** Language habits are ingrained.
   Correcting colleagues mid-conversation creates friction, not alignment.
2. **Control what you can first:** new code uses thesaurus terms, docs get updated,
   new API endpoints use canonical names, tests use domain language.
3. **Let conversational language follow.** As people read correct terms in code and
   PRs, spoken language shifts gradually. This takes weeks, not days.
4. **Watch for technical terms masquerading as domain language.** Stakeholders using
   DB table names as domain terms ("the users table" instead of "Customer") is a
   common brownfield pattern. Map these as `## Legacy` lines.
5. **Pick battles by frequency.** Fix terms used 200 times across the codebase before
   terms used in 3 files.

## Legacy Code Migration

When migrating from inconsistent legacy naming:

### Phase 1: Anti-Corruption Layer
Keep legacy code as-is. Create adapter layer with correct naming:
```
Legacy: class UserManager → New: class Customer (domain) + LegacyUserAdapter (boundary)
```

### Phase 2: Gradual Rename
- New code always uses thesaurus terms
- Old classes get "Legacy" prefix only at migration boundaries
- Use interfaces to decouple: `IOrderRepository` stays stable while implementation changes
- Wire command handlers to new implementation first

### Phase 3: Cleanup
- Delete legacy classes once all consumers migrated
- Remove "Legacy" prefixes
- Final audit against thesaurus

**Rules:**
- Never rename across all layers at once
- Use interfaces to decouple
- Config flags to toggle old vs new implementation during migration

# Resolving Ambiguities via Git History Mining

Read this when the `## Unresolved` section has entries and the user wants them
resolved **without** answering every question by hand — or when a naming decision
hinges on "which of these two names came first / replaced which".

This is an **optional last step** of thesaurus generation (Step 6 in
[generating-thesaurus.md](generating-thesaurus.md)) and can also be run on its own
against an existing thesaurus.

## What history can and cannot decide

Git history is **evidence of what happened**, never authority on what *should* be.
It answers questions of fact that no static scan of the current tree can:

| Question | History answers it | Signal |
|----------|-------------------|--------|
| Which of two synonyms is newer? | yes | first commit that introduced each identifier |
| Did `A` replace `B`, or do they coexist by design? | usually | one commit removes `A` and adds `B` in the same files |
| Is a term dying or growing? | yes | additions vs deletions per year |
| Was this rename deliberate? | often | the commit message / PR that did it |
| Why do two modules spell the same concept differently? | sometimes | the two names were born in different commits by different authors, never reconciled |
| What does the domain expert call it? | **no** | ask the user |
| Which name *should* be canonical? | **no** | ask the user; history only ranks the candidates |

**Rule:** history mining produces *ranked hypotheses with citations*, not decisions.
Every resolution it proposes is presented to the user with the commits behind it.
An `## Unresolved` entry is promoted to the Index only after the user agrees — the
one exception is a rename so unambiguous (single commit, message says "rename X to Y",
no later reappearance of X in new code) that it can be proposed as a `## Legacy` line
and confirmed in a single yes/no.

## When to offer it

Offer once, right after presenting `## Unresolved` to the user (Step 5), phrased as
an option and never as a blocking question:

```
5 unresolved naming issues remain. Before you answer them one by one, I can mine
the git history — when each name was born, which one replaced which, and which is
growing vs dying. It builds a temporary index outside the repo (~1-3 min for this
repo's size, deleted afterwards) and comes back with evidence-backed proposals for
each item. Want me to try that first?
```

Skip the offer entirely when any of these hold:

- The repository has < ~50 commits, or its history starts with a single "initial import"
  squash — there is nothing to mine.
- There is no `.git` (a vendored copy, an export, a fresh `mkdir`).
- The `## Unresolved` items are white spots (`[WHITE-SPOT]` tag) rather than conflicts —
  a concept nobody has named yet cannot appear in history.
- The user already answered the questions.

## Tooling

Two layers. Use the index for anything term-shaped; drop to raw git for the last mile.

### 1. The throwaway index — `scripts/git_term_index.py`

Dependency-free Python 3.9+ (stdlib `sqlite3`), needs only `git`. Writes **one SQLite
file outside the working tree** (`$TMPDIR/ubiquitous-language-git-index/<repo>-<hash>.sqlite3`)
— never into the repo, never committed. Safe to delete at any moment.

```bash
S=path/to/skills/ubiquitous-language/scripts/git_term_index.py

# messages + paths + renames — seconds
python3 $S build --repo-dir .

# + every identifier in the FULL diff history — the mode that answers naming questions
python3 $S build --repo-dir . --content

python3 $S query Account Customer      # per-term evidence report
python3 $S pair User Customer Account  # competing names, head to head
python3 $S contexts Account            # where the name lives — the polysemy check
python3 $S search 'rename account'     # BM25-ranked commit-message search
python3 $S status                      # what is indexed, how big
python3 $S clean                       # delete the index
```

Schema:

| Table | Content | Answers |
|-------|---------|---------|
| `commits` + `commits_fts` (FTS5) | sha, date, author, subject, body | "what did people *say* about this name?" — ranked by BM25, not by recency |
| `files` | commit, status, path, oldpath | "where did the name live?" |
| `renames` | detected renames (`-M`) | "was the file itself renamed?" |
| `tokens` (`tok`, `norm`) | every spelling, plus a casing-independent normal form | `OrderLineItem`, `order_line_item` and `ORDER_LINE_ITEM` are one concept — so a PascalCase thesaurus Identifier finds a snake_case codebase |
| `token_sub` | camelCase/snake_case subwords | `--family` widens `Order` to `OrderLineItem`, `order_id` |
| `paths` | files touched | — |
| `tc` | identifier × commit × **file**, with add/delete counts | birth, death, volume, co-occurrence, *and* which files a name lives in |

**Why identifier × file and not identifier × commit.** One commit touches many files, so
commit-level co-occurrence cannot localise a name: measured on git.git, the directory
distribution of one identifier was 74% "wherever the code is". Per-file rows buy the two
things the protocol actually needs — the **polysemy split** (does any file contain both
names? which directories does each occupy?) and **same-file exchanges**, which are much
stronger rename evidence than "both names appear somewhere in one commit".

**The whole history is indexed, not a recent window.** That matters: a windowed index
reports "first seen in the window" and an agent reads it as a birth date. Measured on
git.git, a 3000-commit window covered 10 months of a 21-year history and dated
`oid_array` to 2025 instead of 2017.

**Defaults worth knowing:**

- **HEAD only.** Side branches carry status files, release notes and vendored trees whose
  vocabulary is not the project's. `--all-refs` opts in. (On git.git, `--all` dated
  `oid_array` to a "What's cooking" status email three days before the real rename commit.)
- **Vendored and generated paths excluded** — `vendor/`, `third_party/`, `node_modules/`,
  `testdata/`, `*.pb.go`, `zz_generated*`, lockfiles, minified assets. They dominate token
  counts and hold no domain vocabulary. `--no-default-excludes` turns this off.
- **Merges excluded** (`--no-merges`), so a squash-merge is counted once, not twice.
- **Concept, not family.** `pair Account BillingAccount` compares two concepts; it does not
  let the `Account` set swallow `BillingAccount` (which would cancel the deletions against
  the additions and hide the very exchange being measured). `--family` opts into the wider
  match, and identifiers shared by both sides are dropped from each.
- **Scope is part of the index identity.** An index built with `--pathspec src/` is not
  reused for a whole-repo question, and every report prints the scope it was built with.

**Cost.** Measured on an M-series laptop:

| Repo | Indexed commits | `build` | `build --content` | Index size | `query` / `pair` / `search` |
|------|-----------------|---------|-------------------|-----------|------------------------------|
| small skill repo | 238 | 1 s | 4 s | 11 MB | instant |
| git/git (21 years) | 60 751 | 5 s | **111 s** | 232 MB | 0.1 s |
| kubernetes (12 years, 140 k on HEAD) | 82 704 | 11 s | **249 s** | 1 040 MB | 0.1–1.3 s |

Per-file granularity costs +18% on disk on git.git and no extra time; on kubernetes, whose
commits touch far more files, it is the dominant cost (15.7 M identifier×commit×file rows,
1 GB). If a repo of that size is too heavy, `--pathspec` is the lever — it cuts the corpus
and sharpens the signal at once.

The diff pass streams at roughly 12 MB/s of diff text and `git` itself is most of that —
tokenising is nearly free, so the cost scales with how much diff the repo has, not with how
many terms you ask about. That is the whole argument for the index: `git log -S` costs 4 s
per term on git.git and 84–98 s per term on kubernetes, *every time*.

Levers when a repo is too big or too noisy: `--pathspec pkg/ src/` (the strongest — it cuts
the corpus *and* sharpens the signal by dropping docs and scripts), `--since '5 years ago'`,
and `--content-max-commits N` (a hard cap; it makes the index truncated and every report
then says so). Tell the user the scope you chose. Always `clean` when done.

### 2. Raw git for the last mile

The index narrows to a handful of commits; these confirm them. **Always confirm a rename
candidate by reading the diff** — the index tells you a commit removed one name and added
another, not that they mean the same thing.

```bash
git show <sha> -- path/to/file                        # read the actual change — do this
git log -S'OrderLineItem' --oneline --name-status     # pickaxe, straight from git
git log -G'Order(Line)?Item' --oneline                # regex over diff content
git log --follow --oneline -- src/billing/Account.ts  # a file's life across renames
git log --grep='rename' -i --oneline                  # deliberate rename commits
git shortlog -sn -- src/billing/                      # who owns this vocabulary
```

Note on `git log -S` vs the index: pickaxe needs no index and is exact, but it walks the
whole history *per term* — measured at 4 s per term on git.git and 84–98 s per term on
kubernetes. The SQLite index pays that cost once and then answers in ~0.1 s, which is why
it wins as soon as you have more than a couple of names to settle. `--pickaxe-regex` (for
whole-identifier matching) is another ~40× slower again; don't reach for it.

The agent may also use the [investigating-repository-history](../../investigating-repository-history/)
skill when an item needs PR discussion and review comments, not just commits — that
is where the human reasoning behind a rename usually lives.

## Protocol

Run per `## Unresolved` entry, batched — collect all findings, report once.

### Step 1: Scope and build

1. Confirm the repo has usable history: `git rev-list --count HEAD`, and
   `git log --reverse --oneline -1` (how far back does it actually go?).
2. Choose the scope. Default to the full history — that is the point of the index. Reach
   for `--pathspec` when the repo is large or when the unresolved entries name specific
   modules; it is both the cheapest and the sharpest lever.
3. `build --content`. Report what you indexed, over what window, and how long it took.

### Step 2: Gather evidence per entry

For each entry, extract the candidate names from its `**Found in**` / `**Options**`
lines and run `query` on each, then `pair` on the competing set. Collect:

- **Birth order** — which identifier is older; the older one is usually the incumbent,
  the newer one is either a replacement or an unreconciled fork of vocabulary. Read the
  birth commit's subject: a rename usually announces itself there.
- **Dormancy** — the last commit that touched the name. A name nothing has changed in
  years is retired vocabulary, whatever the current tree still contains. `pair` flags this
  and states a displacement hypothesis.
- **Trajectory** — adds vs dels, and mentions per year. Heavy deletions with no recent
  additions means dying: `## Legacy` or an `avoid:` list, not the Index.
- **Swap commits** — `pair` counts the commits that **remove one name while adding the
  other** and lists them first. These are the rename candidates; open them with `git show`.
- **Path split** — `pair` ends with `files: A in N, B in M, both in K`. `both in 0` means no
  file has ever contained both names: two concepts sharing a vocabulary, a candidate bounded
  context — not a rename. `contexts <name>` does the same for a single word and is the
  polysemy check: one word split across two directories is the `Account` billing-vs-auth case.
  **Only claim a path split when one of those commands printed it** — never infer it.
- **Stated intent** — commit messages, PR/issue numbers, and `BREAKING`/`rename`/`migrate`
  wording. Use `search` (BM25) to find the commits that *explain* rather than merely mention.
  A message that says why beats any inference from counts.
- **Ownership** — different authors owning the two spellings, never touching each other's
  files, is drift, not design.

**Worked example 1 — a rename.** `pair sha1_array oid_array` on git.git (60 751 commits, 0.1 s):

```
### `sha1_array`
- born: 2009-05-09 6212b1aae — bisect: use "sha1_array" to store skipped revisions
- last grown:   2017-03-31 910650d2f — Rename sha1_array to oid_array
- signal: dormant ~9.4 years — nothing has *added* it since 2017

- verdict: **RENAME — strong.** `sha1_array` → `oid_array` — net exchange in 1 commit,
  announced in 1 subject.
- subjects that announce it (both names present):
  - 2017-03-31 910650d2f Rename sha1_array to oid_array
```

What carried it: the *last grown* date (not last touched — a stale comment naming the old
identifier was deleted in 2020, long after anyone stopped writing it), plus one commit that
both net-exchanges the names and announces the rename with both names present.

**Worked example 2 — the same machinery at scale.** `pair Minion Node` on kubernetes
(82 704 commits indexed, 1.3 s):

```
### `Minion`
- born: 2014-06-06 2c4b3a562 — First commit
  - caveat: present in the repository's oldest indexed commit — import artefact or a
    common English word, not a naming event
  - signal: dormant ~3.8 years — nothing has *added* it since 2022-11-10

- shared commits: 526 · net exchanges `Minion`→`Node`: 39 (reverse: 2)
  · same-file exchanges: 35 · subjects naming a rename: 6
- verdict: **RENAME — strong.**
- strongest exchanges (same-file ones first):
  - 2014-12-07 19379b5a3 (net -144 `Minion` / +144 `Node`, 35 files swapped in place)
    Internal rename api.Minion -> api.Node
  - 2016-05-05 9d5bac633 (net -146 / +141, 24 files swapped in place) Change minion to node
- files: `Minion` in 525, `Node` in 5574, **both in 363**
  - heavily shared (69% of the smaller set) — they live in the same code, so a rename
    or synonym drift is plausible
```

Neither birth date carried this verdict — both words appear in the repo's first commit and
are flagged as such. What carried it: dormancy, 35 commits that swap the names *inside the
same files*, and six subjects naming both sides. For contrast, the five unrelated k8s pairs
tested alongside it (`Kubelet`/`Scheduler`, `Pod`/`Deployment`, …) reach at most 6 same-file
exchanges in 1 193 shared commits and zero naming subjects — all correctly COEXISTENCE.

**Worked example 3 — polysemy, the case the protocol exists for.** `contexts Account` on a
repo where `billing/` and `auth/` both define an `Account`:

```
## `Account`
- directories (2 total):
  - billing        67%  (8 additions)
  - auth           33%  (4 additions)
- **possible polysemy**: the name is split across `billing` (67%) and `auth` (33%).
  Read one file from each before assuming they mean the same thing.
```

and `pair account_balance account_email`:

```
- verdict: **COEXISTENCE.** No exchange, both still growing.
- files: `account_balance` in 4, `account_email` in 4, **both in 0**
  - **disjoint** — no file has ever contained both. Two concepts that share a
    vocabulary, not two names for one thing. Candidate bounded-context split.
  - `account_balance` by directory: billing 100%
  - `account_email` by directory: auth 100%
```

COEXISTENCE plus a disjoint path split is a real finding — it is the evidence for two
bounded contexts. COEXISTENCE with *shared* files is not: that is synonym drift, and history
cannot settle it. Do not report one as the other.

### What `pair` decides, and how far to trust it

`pair` emits one labelled verdict per pair. The label already encodes the confidence —
**do not upgrade it in your report.**

| Verdict | Emitted when | What to do |
|---------|--------------|------------|
| **RENAME — strong** | Net exchange *and* at least one subject naming **both** sides with rename wording | Propose the Legacy line, citing the announcing commit |
| **RENAME — probable** | Commits swap the names **inside the same files** (with or without the old name having stopped growing earlier), but no subject says so | Read the top commits with `git show` before proposing |
| **RENAME — possible** | Net exchange and the old name stopped growing ≥2 years earlier, but never within one file | Weak. Verify before it goes near the thesaurus |
| **DRIFT / PARTIAL MIGRATION — weak** | Net exchange, never inside one file, both names still being added | Not a settled rename; ask whether the migration should finish |
| **NOT A RENAME** | No meaningful exchange; one name merely stopped growing earlier | An abandoned concept, not a replaced name |
| **COEXISTENCE** | No exchange, both still growing | Two live concepts or synonym drift — settle it with the path split below, not by guessing |

Direction is inferred from the evidence, not from argument order: `pair Cart Basket` and
`pair Basket Cart` both conclude `Basket` → `Cart`. When there is no exchange evidence at
all, the pair is oriented by dormancy so the label cannot flip with argument order either.

**How the thresholds were set** — by falsification, not taste. Each was added after a
measured false positive or false negative on a real repository:

- **Net direction.** The first version flagged "A deleted at all, B added at all". On a real
  repo that labelled **105 of 174** commits touching two unrelated integrations (`jira`,
  `telegram`) as rename candidates — including commits where *both* names were net-removed
  and commits where `jira` actually grew. A swap now needs A to net-shrink by ≥3 and B to
  net-grow by ≥3 in the same commit. That single change took 105 → 2.
- **Noise floor.** Two exchanges out of 145 shared commits (`bisect`/`rebase` in git.git) is
  churn. Exchanges must be ≥2 commits and ≥5% of shared commits to carry a verdict; below
  that they print as "below the noise floor", explicitly not as evidence. Same-file exchanges
  are stronger and clear half that bar — but not no bar: making any same-file swap sufficient
  put `bisect`/`rebase` straight back to RENAME on 2 swaps in 111 shared commits.
- **Both names in the subject.** Accepting one name let "Rename Telegram meeting wrapper"
  certify `club` → `meeting`. Every true announcement names both sides — "Rename sha1_array
  to oid_array", "Change minion to node", "Internal rename api.Minion -> api.Node".
- **Last *grown*, not last touched.** A retired name keeps being deleted for years after
  nobody adds it. git.git's `get_sha1` was last *touched* in 2026 by a commit deleting a
  stale comment; it was last *grown* in 2017.
- **Casing-independent identity.** `query OrderLineItem` returned "never appears" on a
  snake_case repository — a false negative on this skill's own canonical input, since
  thesaurus Identifiers are PascalCase by format. Identifiers now carry a normal form
  computed *before* lowercasing, so all spellings of one concept share it.
- **Concepts, not families, when comparing.** The `Account` set contained `BillingAccount`,
  so B's additions cancelled A's deletions and masked the very exchange being measured.
  Identifiers shared by both sides are dropped from each; `--family` opts into wider matching.
- **Locale and changelog files excluded.** `.po` files made "l10n: zh_CN …" the top rename
  candidate for two unrelated git.git terms; changelog churn did the same on kubernetes.

Measured after those fixes across three repositories: **18 negative controls → zero false
rename verdicts** (the worst reaches 6 same-file exchanges in 1 193 shared commits and no
naming subject), while `sha1_array`→`oid_array`, `get_sha1`→`get_oid`, `Minion`→`Node` and
two synthetic ground-truth repos all still land as RENAME — strong.

**What it cannot do.** `pair` compares *identifiers*. A rename that only moved files
(`sha1_file.c` → `object-file.c`) shows up in `query`'s **file renames** section, not here —
run both. And history never says which name *should* be canonical: it ranks candidates, the
user decides.

### Step 3: Classify each entry

| Evidence pattern | Proposal | Thesaurus effect |
|------------------|----------|------------------|
| One commit removes `A`, adds `B`, message says rename | **Rename**, high confidence | Index line keeps `B`; `` `A` → `B` `` Legacy line |
| `A` dying (dels ≫ adds, no recent adds), `B` growing | **Deprecate `A`**, medium | `B` in the Index; `A` in its `avoid:` or a Legacy line with `in:` paths |
| COEXISTENCE **and** `both in 0` files (disjoint directories) | **Two concepts**, medium | Two Index lines; propose `ctx:` only if the user confirms the invariant test |
| COEXISTENCE with files shared between the names | **Synonym drift**, low | History cannot separate these; take it to the user |
| Both alive, same paths, interchangeable in diffs | **Synonym drift**, medium | One Index line; the loser joins `avoid:` |
| `A` born once, never touched again, only in tests/migrations | **Obsolete**, high | Not in the Index at all; note for cleanup |
| Nothing conclusive | **Unresolved**, stays | Keep the entry; append a `**History**` line with what was found |

Confidence is part of every proposal. Say `low` when the only evidence is counts.

### Step 4: Report and let the user decide

One batch report, ordered by impact (files affected), each item citing its commits:

```
History mining — 5 unresolved items, 3 resolvable:

1. `User` vs `Customer` — RENAME, high confidence
   `Customer` born 2023-11-04 (a41f2c9 "rename User→Customer in domain layer");
   `User` has 0 additions since, 41 deletions. 3 files still use it: api/v1/*.
   → Propose: Index `Customer`; Legacy line `User` → `Customer` in: `api/v1/`.

2. `Account` billing/ vs auth/ — TWO CONCEPTS, medium confidence
   Born in unrelated commits (2021-03 by @alice in billing/, 2022-08 by @bob in
   auth/), never co-edited, no shared fields. Reads as parallel vocabularies.
   → Propose: BillingAccount + UserAccount, or contexts Billing / Identity.
     Needs the invariant test — an invariant true inside one and not the other?

3. `Status` vs `State` — INCONCLUSIVE
   Both grow steadily, same modules, no rename commit. History cannot settle this.
   → Still needs your call. Recorded what was found under the entry.

Apply 1 and 2 as proposed?
```

Apply only what the user approves. Then delete the index (`clean`) and say so.

### Step 5: Record the provenance

The thesaurus is "reconstructed, not authored" — a resolution mined from history is
still a reconstruction, and readers must be able to check it. When a history-mined
decision lands in the thesaurus, cite it:

- On a `## Legacy` line, the note field carries the commit:
  `` - `User` → `Customer` in: `api/v1/` — renamed in a41f2c9 (2023-11-04) ``
- On an entry, an optional line:
  `- **History**: split from `Account` in 8e21f0b (2022-08); billing/ and auth/ never co-edited`
- On an entry that stays in `## Unresolved`, the same `**History**` line records what was
  ruled out, so the next run does not repeat the work.

These are prose lines inside entries and the free-text tail of Legacy lines — they add
no new tokens and do **not** change the thesaurus format (still `2.0`).

## Failure modes to state out loud

- **Squashed / imported history.** If most files trace back to one "initial commit", birth
  dates are meaningless. Check with `git log --diff-filter=A --oneline -- <path> | tail -1`
  before trusting any "born" date, and say so in the report. A term whose birth commit is
  the repo's first commit is almost always an import artefact, not a naming event.
- **A common English word is not an identifier.** `Node`, `Item`, `State`, `Order` occur in
  comments, docs and unrelated code. Their "birth" is usually noise; their *dormancy* and
  *swap commits* are still meaningful. Prefer the compound form (`OrderLineItem`,
  `oid_array`) when one exists, and scope with `--pathspec` to the domain layer.
- **Bulk reformatting and mass renames** (a linter run, a directory move) inflate token
  counts across the board. A commit touching hundreds of files is not vocabulary evidence —
  discount it.
- **Vendored, generated and minified files** dominate token counts. They are excluded by
  default; if you pass `--no-default-excludes`, expect the signal to degrade.
- **Side branches** carry release notes, status files and imported trees. The index walks
  HEAD by default for that reason — `--all-refs` is opt-in and noisier.
- **Shallow clones** (`git rev-parse --is-shallow-repository` → `true`) have no early
  history. The build prints a warning and every report repeats it; treat every date as a
  lower bound rather than a birth.
- **A truncated diff index lies about birth.** If `--content-max-commits` was used, every
  report says so — pass that caveat on to the user instead of quoting the date as a birth.
- **The index is a snapshot** keyed to the HEAD sha *and the build scope*; rebuild with
  `--refresh` after new commits land. Changing `--pathspec`/`--since`/`--all-refs` rebuilds
  automatically rather than silently reusing a narrower index.
- **One index per repository path.** Two agents mining the same checkout at once will delete
  each other's index; pass `--index-file` to give a second session its own.
- **Never resolve silently.** History moves an item from "unknown" to "probable". The user
  moves it to "decided".

# Naming & Ubiquitous Language Audit

Read this when the user asks to audit, review, or check naming consistency in the codebase.
This is a systematic protocol — run it top to bottom, report findings at the end.

## When to Run

- User asks: "audit naming", "check naming consistency", "review ubiquitous language"
- Before a major refactoring or architecture change
- When onboarding to a new codebase
- Periodically (quarterly) to catch drift

## Prerequisites

- `THESAURUS.md` must exist. If it doesn't, run thesaurus generation first
  (see [generating-thesaurus.md](generating-thesaurus.md))
- Read the `## Index` of `THESAURUS.md` before starting — it is the audit's work-list

## Audit Protocol

Run all 9 checks. Collect findings without stopping. Present the full report at the end.

The thesaurus's `## Index` is the work-list for most checks: each line gives the
Identifier to look for and the `avoid:` names to look against. Extract it once:

```bash
# Index lines only
awk '/^## Index/{f=1;next} /^## /{f=0} f && /^- \*\*/' docs/THESAURUS.md
# one avoid-name per line, paired with its canonical Identifier
awk '/^## Index/{f=1;next} /^## /{f=0} f && /avoid:/' docs/THESAURUS.md \
  | sed -E 's/^- \*\*[^*]+\*\* `([^`]+)`.*avoid: (.*)$/\1\t\2/'
```

### Check 0: Thesaurus Integrity

Before auditing code against the thesaurus, check the thesaurus against itself —
a broken registry makes every later check lie.

- **Index ↔ Terms**: every Index line has exactly one `### Term` header and vice versa.
  Header count inside `## Terms` (not `## Unresolved`) must equal the Index line count:
  `awk '/^## Terms/{f=1;next} /^## [^T]/{f=0} f && /^### /' THESAURUS.md | wc -l`
- **Line grammar**: every Index line matches
  ``^- \*\*[^*]+\*\* `[A-Z][A-Za-z0-9]*` kind:[a-z]+( ctx:[A-Za-z]+)?( avoid: .+)?$``;
  Forbidden lines contain `` use: ``; Legacy lines contain `` → ``.
- **Registry invariant**: no name appears in two registry lines. Collect all backticked
  names from Index `avoid:` lists, `## Forbidden`, `## Legacy`; sort; report duplicates.
  Also: no `avoid:` name equals another line's Identifier (SKOS: an altLabel is never
  anyone's prefLabel).
- **Anchor grammar**: every header under `## Terms` matches `^### [^(]+( \([^)]+\))?$`.
- **Hierarchy sanity** (SKOS): the same pair is never both `Broader` and `Related`;
  `Broader` chains have no cycles (A broader B broader A).
- **Bridges vocabulary**: every bridge's mapping is one of
  `exactMatch` `closeMatch` `broadMatch` `narrowMatch` `relatedMatch` `distinct`.
- **Format stamp**: `rg -n '^thesaurus-format:'` must return exactly one line, inside the
  YAML frontmatter at the top of the file, with a quoted `2.x` version. Missing/`1.x` → stop and offer migration (see
  generating-thesaurus.md "Migrating an Existing Thesaurus"); missing but the Index is
  already in 2.0 line grammar → just add the stamp and continue. Stamp says `2.x` but the
  file still has `Synonyms to AVOID` lines or a table Index → stamp lies; report it.
  A stamp newer than this skill's `thesaurus-format` → read-only audit; tell the user to
  update the skill.

**Severity: HIGH** — thesaurus cannot be trusted until fixed.

### Check 1: Synonym Violations

For each Index line, grep the codebase for every name in its `avoid:` list:

```
Line: - **Order** `Order` kind:aggregate avoid: `Purchase`, `Transaction`, `Buy`
→ rg -n -i -w 'purchase|transaction|buy' src/ -t ts -t cs -t py -t java -t go -t ruby
→ filter: only class names, method names, variable names, DB columns (not comments/strings)
```

Report as: `<avoid name>` in `<file:line>` — Index says `<Identifier>`.

**Severity: HIGH** — direct contradiction of the thesaurus.

### Check 2: Forbidden Words in Domain Layer

For each `## Forbidden` line, scan domain-layer code for the word:

```
Work-list: the backticked words of ## Forbidden (project-specific), plus the baseline
  Manager, Handler, Service (bare), Info, Data, Base, Util, Helper, Object, Obj, Record, Model
In: domain layer classes, interfaces, method names (NOT infrastructure layer)
```

If a baseline word is hit but missing from `## Forbidden`, propose adding the line.

**Severity: MEDIUM** — vague naming that hides domain concepts.

### Check 3: Technical Jargon Leak

Scan domain-layer code for implementation-specific prefixes/suffixes:

```
Scan for: Mongo*, Sql*, Http*, Redis*, Kafka*, Elastic*, *Dto, *Entity, *Model, *Record
In: domain layer only (NOT infrastructure/persistence/API layers)
```

**Severity: HIGH** — infrastructure leaking into domain.

### Check 4: Synonym Drift (same concept, multiple names)

Look for groups of identifiers that likely refer to the same domain concept:

- Same-shaped classes in different modules (similar fields, different names)
- API endpoints that use different terms for the same resource
- Database tables/columns with overlapping semantics
- Tests that use different names than the code they test

**Detection heuristics (grep-friendly, no pairwise comparison):**
- Same role suffix with different noun stems: `UserController` vs `CustomerController`,
  `UserRepository` vs `CustomerRepository` — grep for common suffixes, compare stems
- API route vs domain code drift: `/users/...` in routes but `Customer` in domain
- DB column vs code drift: `user_id` in schema but `customerId` in code
- Test vs implementation drift: test descriptions say "user" but code says `Customer`
- Competing stems in the same module: file that imports both `User` and `Customer`

Treat synonym drift as a **local-cluster problem**: same module, same role, different noun.
Do NOT attempt global pairwise class comparison.

**Severity: HIGH** — the ubiquitous language is fractured.

### Check 5: Polysemy (same name, different meanings)

Look for the same identifier used with structurally different meanings:

- Same class name in different packages with different fields/methods
- Same enum name with different values in different modules
- Same method name doing fundamentally different things in different classes
- Same API parameter meaning different things in different endpoints

**The incompatible-verbs test:** If the same word appears as subject of incompatible
verbs in different parts of the code ("Account is charged" vs "Account is logged in"),
it's polysemy.

**Severity: HIGH** — silent bugs waiting to happen.

### Check 6: Translation Chain

Compare terminology across artifact layers:

```
Layer mapping (local artifacts only — do NOT assume access to Jira/Linear/Notion):
  Local docs/ADRs/OpenAPI specs → API controllers → Domain code → DB schema → Test descriptions → Git commit messages

For each major domain concept, trace the name through available local layers.
If external specs are needed, ask the user to paste the relevant text.
```

**Detection method:**
- Pick 3-10 major domain terms from the thesaurus or API surface
- Grep their stems across available local layers
- Check test descriptions (`describe()`/`it()`/`test()` strings) — tests often use
  the domain expert's term while code uses an abbreviation
- Check `git log --oneline -50` — commit messages reveal human intent vs code naming
- Flag any layer where the term changes

**Example finding:**
```
"Campaign" in docs/architecture.md
→ "Promotion" in openapi.yaml
→ `marketing_push` in domain code
→ `promotions` table in DB
→ "advertising effort" in test descriptions
= Translation chain with 4 breaks
```

If some layers are missing locally, report them as `not auditable from local files`.

**Severity: HIGH** — information loss at every translation.

### Check 7: Abbreviation & Naming Inconsistency

Scan for inconsistent forms of the same term:

- Abbreviated vs full: `usr` / `user` / `customer` / `acct` / `account`
- Casing inconsistency: `orderId` in one file, `order_id` in another (within same language)
- Plural inconsistency: `Order` class but `order_items` table vs `orderItem` field

**Severity: LOW** — cosmetic but creates cognitive load.

### Check 8: Orphan Terms

Check for Index terms that no longer appear in code, and code concepts missing from the Index:

```
For each Index line → grep the codebase for the Identifier (also its snake_case form)
If zero matches → term may be obsolete, or not yet implemented — ask

For each `## Legacy` line → grep for the legacy name
If zero matches → the legacy line is done; propose removing it
```

Reverse direction: grep type/class/interface declarations in the domain layer, strip
technical suffixes, and list names that hit neither an Index Identifier, an Avoid
name, nor a Legacy name — candidates for new Index lines.

**Severity: LOW** — thesaurus drift from codebase.

## Report Format

After running all 9 checks, present findings grouped by severity:

```
## Ubiquitous Language Audit Report

**Codebase**: [project name]
**Date**: [date]
**Thesaurus**: [N terms, M legacy, K unresolved]

### Critical (fix now)

1. **Synonym violation**: `fetchPurchases()` in src/api/orders.ts:12
   — Thesaurus says "Order", not "Purchase" — 8 files affected

2. **Polysemy**: `Account` used as financial entity (billing/)
   AND user identity (auth/) — 18 files affected

3. **Translation chain**: "Campaign" → "Promotion" → `marketing_push`
   — 4 translation breaks across 31 files

### Warning (plan to fix)

4. **Weasel word**: `OrderManager` in src/domain/OrderManager.ts
   — "Manager" hides responsibility. What does it actually do?

5. **Technical leak**: `MongoOrder` in src/domain/MongoOrder.ts
   — Infrastructure prefix in domain layer

### Info (track as debt)

6. **Abbreviation**: `usr` in 3 files, `user` in 12, `customer` in 8
   — All refer to the same concept

7. **Orphan term**: "ShippingLabel" in thesaurus, 0 matches in code
   — May be obsolete or not yet implemented

### Stats

| Check | Findings |
|-------|----------|
| Thesaurus integrity | 0 |
| Synonym violations | 3 |
| Forbidden words | 5 |
| Technical leaks | 2 |
| Synonym drift | 4 clusters |
| Polysemy | 1 |
| Translation chains | 2 |
| Abbreviation issues | 6 |
| Orphan terms | 3 |
| **Total** | **26** |

### Recommended Priority

1. Fix polysemy first (silent bug risk)
2. Fix translation chains (information loss)
3. Fix synonym violations (thesaurus credibility)
4. Clean up weasel words (naming quality)
5. Track the rest as naming debt
```

## After the Audit

- **Critical findings**: suggest immediate fixes or add to `## Unresolved` in thesaurus
- **Warnings**: create tech debt tickets or note in thesaurus
- **Info**: note for next audit cycle
- **Update the thesaurus**: add missing Index lines + entries, remove finished Legacy
  lines, extend `avoid:` lists with the synonyms you found in the wild — keeping the
  registry invariant (a name lives under `avoid:` *or* Legacy *or* Forbidden, never two)
- **If no thesaurus existed**: the audit findings ARE the input for thesaurus generation —
  feed them into the generating-thesaurus.md workflow
- **Offer history mining** for the findings the code alone can't settle — synonym drift
  clusters (Check 5), polysemy candidates (Check 6), and anything you had to file under
  `## Unresolved`. Git history says which spelling came first, which commit replaced which,
  and which one is dying: see [git-history-mining.md](git-history-mining.md). Offer it once,
  as an option; skip it when the repo has no usable history

#!/usr/bin/env python3
"""Build a throwaway SQLite index of git history and query it for domain-term evidence.

Purpose: resolve naming ambiguities collected in the `## Unresolved` section of a
THESAURUS.md by mining what actually happened in the repository — when a name was
born, when it stopped being touched, which names travelled together in the same
commits, and what the commit messages said about them.

Dependency-free (Python 3.9+ stdlib, git). The index is a single SQLite file in a
temp directory OUTSIDE the working tree, safe to delete at any time (`clean`).

Why SQLite: the whole diff history is indexed, not a recent window, so "born" means
born. Commit messages go into an FTS5 table, so message search is ranked by BM25
relevance instead of recency.

Schema:
  commits(id, sha, date, author, subject, body)   + commits_fts (FTS5/BM25)
  files(commit_id, status, path, oldpath)         changed paths
  renames(commit_id, oldpath, newpath, sim)       detected renames (-M)
  tokens(id, tok, norm)                           identifiers + casing-independent norm
  token_sub(sub, token_id)                        camelCase/snake_case subwords
  paths(id, path)                                 files touched
  tc(token_id, commit_id, path_id, adds, dels)    identifier x commit x FILE occurrences

Commands:
  build    build or refresh the index
  status   show what is indexed
  query    per-term evidence report (life, trajectory, renames, paths, messages)
  contexts where one name lives — directory split, the polysemy check
  pair     compare competing names (birth order, displacement, shared commits)
  search   BM25 full-text search over commit messages
  clean    delete the index

Examples:
  python3 git_term_index.py build --repo-dir .
  python3 git_term_index.py build --repo-dir . --content          # + full diff history
  python3 git_term_index.py query Account Customer
  python3 git_term_index.py pair Minion Node
  python3 git_term_index.py contexts Account
  python3 git_term_index.py search 'rename minion node'
  python3 git_term_index.py clean
"""

from __future__ import annotations

import argparse
import hashlib
import json
import os
import re
import shutil
import sqlite3
import subprocess
import sys
import tempfile
import threading
import time
from collections import Counter
from pathlib import Path
from typing import Dict, Iterator, List, Optional, Sequence, Tuple

REC = "\x01"  # record separator inside git log output
FLD = "\x02"  # field separator inside git log output

TOKEN_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]{2,}")
SHA_RE = re.compile(r"^[0-9a-f]{40}$")
SUBWORD_RE = re.compile(r"[A-Z]+(?![a-z])|[A-Z][a-z0-9]+|[a-z0-9]+")
RENAME_WORD_RE = re.compile(
    r"\b(renam|rename[ds]?|replac|migrat|deprecat|s/\w+/\w+|instead of|switch to|"
    r"convert.{0,12}to|drop\b)", re.I)

# High-precision issue/PR references only. A bare `#12` is deliberately NOT matched:
# real histories are full of `(2026/08 #09)` release-note numbering and `#1` stack-trace
# frames, and that noise is worse than a missed reference.
ISSUE_RE = re.compile(
    r"""(?:
          \(\#(?P<squash>[1-9]\d{0,5})\)                  # GitHub squash-merge subject
        | \bGH-(?P<gh>[1-9]\d{0,5})\b                     # GH-123
        | \b(?:fix(?:e[sd])?|close[sd]?|resolve[sd]?|
               refs?|see|PR|pull\srequest|issue)
          \s*:?\s*\#(?P<kw>[1-9]\d{0,5})\b                # fixes #123, PR #123
        | github\.com/[^\s]+?/(?:pull|issues)/(?P<url>[1-9]\d{0,5})
        )""",
    re.I | re.X,
)

# Vendored, generated and minified paths dominate token counts in large repos and
# carry no domain vocabulary. Excluded from the diff pass unless --no-default-excludes.
DEFAULT_EXCLUDES = [
    ":(exclude)vendor/**", ":(exclude)third_party/**", ":(exclude)node_modules/**",
    ":(exclude)**/testdata/**", ":(exclude)**/*.min.js", ":(exclude)**/*.min.css",
    ":(exclude)**/*.lock", ":(exclude)**/package-lock.json", ":(exclude)**/yarn.lock",
    ":(exclude)**/go.sum", ":(exclude)**/Cargo.lock", ":(exclude)**/*.snap",
    ":(exclude)**/*generated*.go", ":(exclude)**/*.pb.go", ":(exclude)**/zz_generated*",
    ":(exclude)**/*.svg", ":(exclude)**/*.map",
    ":(exclude)**/*.po", ":(exclude)**/*.pot", ":(exclude)**/locale/**", ":(exclude)po/**",
    ":(exclude)**/CHANGELOG*", ":(exclude)**/*.golden",
]

READ_CHUNK = 1 << 20
BATCH = 20000
MIN_NET_SWAP = 3  # net occurrences; below this a 'swap' is line noise
NOISE_FLOOR = 0.05  # exchanges must be this share of shared commits to count

# Tokens that carry no domain signal. Kept deliberately small: this is a naming
# tool, and over-filtering hides real evidence.
STOPWORDS = {
    "the", "and", "for", "not", "with", "this", "that", "from", "into", "out",
    "var", "let", "const", "def", "func", "function", "class", "struct", "enum",
    "interface", "type", "typedef", "public", "private", "protected", "internal",
    "static", "return", "import", "export", "require", "package", "namespace",
    "using", "include", "true", "false", "null", "none", "nil", "void", "int",
    "str", "string", "bool", "float", "double", "list", "dict", "map", "set",
    "new", "self", "async", "await", "try", "catch", "except", "finally",
    "if", "else", "elif", "while", "break", "continue", "pass", "yield",
}


def eprint(*args: object) -> None:
    print(*args, file=sys.stderr)


# --------------------------------------------------------------------------- git


def run(cmd: Sequence[str], cwd: Path, timeout: int = 900) -> str:
    """Run git and return stdout. Only for commands with bounded output."""
    proc = subprocess.run(
        list(cmd), cwd=str(cwd), text=True, errors="replace",
        stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=timeout,
    )
    if proc.returncode != 0:
        raise RuntimeError(f"git failed ({proc.returncode}): {' '.join(cmd)}\n{proc.stderr.strip()}")
    return proc.stdout


def stream_records(cmd: Sequence[str], cwd: Path, timeout: int = 3600) -> Iterator[str]:
    """Stream `git log` output record by record so memory stays flat on huge histories."""
    proc = subprocess.Popen(
        list(cmd), cwd=str(cwd), text=True, errors="replace",
        stdout=subprocess.PIPE, stderr=subprocess.PIPE,
    )
    # A git that wedges emits nothing, so a deadline checked after read() returns would
    # never fire. Kill it from a watchdog thread instead.
    timed_out = threading.Event()

    def watchdog() -> None:
        if proc.wait_event.wait(timeout):  # type: ignore[attr-defined]
            return
        timed_out.set()
        proc.kill()

    proc.wait_event = threading.Event()  # type: ignore[attr-defined]
    threading.Thread(target=watchdog, daemon=True).start()
    deadline = time.monotonic() + timeout
    buf = ""
    try:
        assert proc.stdout is not None
        while True:
            data = proc.stdout.read(READ_CHUNK)
            if not data:
                break
            if time.monotonic() > deadline:
                proc.kill()
                raise subprocess.TimeoutExpired(list(cmd), timeout)
            buf += data
            if REC in buf:
                parts = buf.split(REC)
                buf = parts.pop()
                for part in parts:
                    if part.strip():
                        yield part
        if buf.strip():
            yield buf
    finally:
        if proc.stdout:
            proc.stdout.close()
        err = proc.stderr.read() if proc.stderr else ""
        if proc.stderr:
            proc.stderr.close()
        rc = proc.wait()
        proc.wait_event.set()  # type: ignore[attr-defined]
        if timed_out.is_set():
            raise subprocess.TimeoutExpired(list(cmd), timeout)
        if rc not in (0, -9):
            raise RuntimeError(f"git failed ({rc}): {' '.join(cmd)}\n{err.strip()}")


def repo_root(repo_dir: Path) -> Path:
    try:
        return Path(run(["git", "rev-parse", "--show-toplevel"], repo_dir, timeout=60).strip())
    except Exception as exc:  # noqa: BLE001
        raise SystemExit(f"not a git repository: {repo_dir} ({exc})")


def head_commit_date(root: Path) -> str:
    try:
        return run(["git", "log", "-1", "--date=short", "--format=%ad"], root, timeout=60).strip()
    except Exception:  # noqa: BLE001
        return ""


# --------------------------------------------------------------------------- helpers


def index_path_for(root: Path, override: Optional[str]) -> Path:
    if override:
        return Path(override).expanduser().resolve()
    digest = hashlib.sha1(str(root).encode()).hexdigest()[:10]
    base = Path(os.environ.get("TMPDIR", tempfile.gettempdir()))
    return base / "ubiquitous-language-git-index" / f"{root.name}-{digest}.sqlite3"


def flatten(text: str) -> str:
    return " ".join(text.split())


def subwords(token: str) -> List[str]:
    """camelCase / snake_case / PascalCase -> lowercase word list.

    Must be called on the ORIGINAL token: lowercasing first destroys the camelCase
    boundaries, which is why `OrderLineItem` and `order_line_item` used to index as two
    unrelated identifiers and a PascalCase thesaurus Identifier found nothing in a
    snake_case codebase.
    """
    out = []
    for part in token.split("_"):
        out.extend(w.lower() for w in SUBWORD_RE.findall(part))
    return [w for w in out if w]


def normal_form(token: str) -> str:
    """Casing-independent identity of an identifier: its subwords, underscore-joined.

    `OrderLineItem`, `order_line_item`, `ORDER_LINE_ITEM` and `orderLineItem` all share
    the norm `order_line_item`; `BillingAccount` is `billing_account`, distinct from
    `account`, so the two never contaminate each other's counts.
    """
    return "_".join(subwords(token))


def content_subwords(token: str) -> List[str]:
    """Subwords worth indexing for family lookups (drops noise words)."""
    return [w for w in subwords(token) if len(w) >= 3 and w not in STOPWORDS]


def word_re(term: str) -> re.Pattern:
    """Match the term as a whole identifier, in any casing style."""
    parts = [p for p in SUBWORD_RE.findall(term) if p]
    joined = r"[\s_\-]*".join(re.escape(p) for p in parts) if parts else re.escape(term)
    return re.compile(rf"(?<![A-Za-z0-9]){joined}(?![A-Za-z0-9])", re.I)


def issue_refs(text: str) -> List[str]:
    out = []
    for m in ISSUE_RE.finditer(text):
        num = next((g for g in m.groups() if g), None)
        if num:
            out.append(num)
    return out


def fts_query(text: str) -> str:
    """Turn free user input into a safe FTS5 MATCH expression.

    FTS5 has its own operator grammar (`-`, `:`, `NEAR`, `*`), so raw input like
    `grep-first` is parsed as syntax and fails. Every word becomes a quoted phrase;
    implicit AND still applies between them.
    """
    words = re.findall(r"[0-9A-Za-z_]+", text)
    return " ".join(f'"{w}"' for w in words)


def plural(n: int, noun: str) -> str:
    return f"{n} {noun}" if n == 1 else f"{n} {noun}s"


def dormant_years(last_date: str, head_date: str) -> float:
    try:
        return (int(head_date[:4]) - int(last_date[:4])) + (int(head_date[5:7]) - int(last_date[5:7])) / 12
    except Exception:  # noqa: BLE001
        return 0.0


def connect(path: Path, write: bool = False) -> sqlite3.Connection:
    con = sqlite3.connect(str(path))
    con.row_factory = sqlite3.Row
    if write:
        con.executescript("PRAGMA journal_mode=OFF; PRAGMA synchronous=OFF; "
                          "PRAGMA temp_store=MEMORY; PRAGMA cache_size=-200000;")
    return con


SCHEMA = """
CREATE TABLE meta(k TEXT PRIMARY KEY, v TEXT);
CREATE TABLE commits(id INTEGER PRIMARY KEY, sha TEXT, date TEXT, author TEXT,
                     subject TEXT, body TEXT);
CREATE TABLE files(commit_id INTEGER, status TEXT, path TEXT, oldpath TEXT);
CREATE TABLE renames(commit_id INTEGER, oldpath TEXT, newpath TEXT, sim TEXT);
CREATE TABLE tokens(id INTEGER PRIMARY KEY, tok TEXT, norm TEXT);
CREATE TABLE token_sub(sub TEXT, token_id INTEGER);
CREATE TABLE paths(id INTEGER PRIMARY KEY, path TEXT);
CREATE TABLE tc(token_id INTEGER, commit_id INTEGER, path_id INTEGER,
                adds INTEGER, dels INTEGER);
"""

INDEXES = """
CREATE UNIQUE INDEX commits_sha ON commits(sha);
CREATE INDEX files_path ON files(path);
CREATE INDEX files_commit ON files(commit_id);
CREATE INDEX renames_commit ON renames(commit_id);
CREATE UNIQUE INDEX tokens_tok ON tokens(tok);
CREATE INDEX tokens_norm ON tokens(norm);
CREATE INDEX token_sub_i ON token_sub(sub);
CREATE UNIQUE INDEX paths_path ON paths(path);
CREATE INDEX tc_token ON tc(token_id);
CREATE INDEX tc_commit ON tc(commit_id);
CREATE INDEX tc_path ON tc(path_id);
"""


def has_fts5(con: sqlite3.Connection) -> bool:
    try:
        con.execute("CREATE VIRTUAL TABLE _fts_probe USING fts5(x)")
        con.execute("DROP TABLE _fts_probe")
        return True
    except sqlite3.OperationalError:
        return False


# --------------------------------------------------------------------------- build


def build(args: argparse.Namespace) -> int:
    root = repo_root(Path(args.repo_dir).resolve())
    db_path = index_path_for(root, args.index_file)
    try:
        head = run(["git", "rev-parse", "HEAD"], root, timeout=60).strip()
    except RuntimeError:
        raise SystemExit(f"{root} has no commits yet — nothing to mine")

    if db_path.exists() and not args.refresh:
        meta = read_meta_safe(db_path)
        scope_now = build_scope(args)
        if (meta and meta.get("head") == head
                and (meta.get("content") == "1" or not args.content)
                and meta.get("scope") == scope_now):
            print(f"index up to date: {db_path}\n  {meta.get('commits')} commits, content={meta.get('content')}")
            return 0
    db_path.parent.mkdir(parents=True, exist_ok=True)
    if db_path.exists():
        db_path.unlink()

    if (args.max_commits and args.content_max_commits
            and args.content_max_commits > args.max_commits):
        raise SystemExit(
            f"--content-max-commits ({args.content_max_commits}) exceeds --max-commits "
            f"({args.max_commits}); the diff pass would walk commits the message pass never "
            f"indexed and silently drop them. Lower it or raise --max-commits.")
    started = time.monotonic()
    con = connect(db_path, write=True)
    con.executescript(SCHEMA)
    fts = has_fts5(con)
    if fts:
        con.executescript(
            "CREATE VIRTUAL TABLE commits_fts USING fts5(subject, body, "
            "content='commits', content_rowid='id');")

    shallow = run(["git", "rev-parse", "--is-shallow-repository"], root, timeout=60).strip() == "true"
    n_commits, n_files, n_renames, oldest, newest = pass_commits(con, root, args)
    con.commit()

    n_tokens = n_tc = 0
    diff_stats: Dict[str, object] = {}
    if args.content:
        n_tokens, n_tc, diff_stats = pass_diffs(con, root, args)
        con.commit()

    print("  building indexes …", flush=True)
    con.executescript(INDEXES)
    if fts:
        con.execute("INSERT INTO commits_fts(rowid, subject, body) SELECT id, subject, body FROM commits")
    con.commit()

    meta = {
        "root": str(root), "head": head, "commits": n_commits, "file_changes": n_files,
        "renames": n_renames, "tokens": n_tokens, "token_commits": n_tc,
        "content": "1" if args.content else "0", "fts5": "1" if fts else "0",
        "oldest": oldest, "newest": newest, "shallow": "1" if shallow else "0",
        "since": args.since or "", "max_commits": str(args.max_commits or ""),
        "pathspec": " ".join(args.pathspec or []),
        "scope": build_scope(args),
        "refs": "all" if args.all_refs else "HEAD",
        "excludes": "default" if not args.no_default_excludes else "none",
        "built_seconds": str(round(time.monotonic() - started, 1)),
        **{f"diff_{k}": str(v) for k, v in diff_stats.items()},
    }
    con.executemany("INSERT INTO meta(k, v) VALUES(?, ?)", list(meta.items()))
    con.commit()
    con.execute("PRAGMA optimize")
    con.close()

    size = db_path.stat().st_size / 1024 / 1024
    print(f"index built: {db_path}")
    print(f"  commits {n_commits} ({oldest} … {newest})  file-changes {n_files}  renames {n_renames}")
    if args.content:
        print(f"  identifiers {n_tokens}  identifier×commit rows {n_tc}"
              + (f"  diff window: {diff_stats.get('commits')} commits, "
                 f"{diff_stats.get('oldest')} … {diff_stats.get('newest')}"
                 if diff_stats.get("truncated") else "  (full history)"))
    else:
        print("  (messages + paths only — add --content to index identifiers from diffs)")
    if not fts:
        print("  ** this Python's sqlite3 has no FTS5 — `search` unavailable, messages fall back to LIKE **")
    if shallow:
        print("  ** shallow clone — early history is missing; treat every date as a lower bound **")
    print(f"  {size:.0f} MB, {meta['built_seconds']}s")
    return 0


def build_scope(args: argparse.Namespace) -> str:
    """Everything that changes what the index covers. Part of the up-to-date check: a
    `--pathspec src/` index must not be silently reused for a whole-repo question."""
    return json.dumps({
        "pathspec": args.pathspec or [], "since": args.since or "",
        "max_commits": args.max_commits, "content_max_commits": args.content_max_commits,
        "all_refs": bool(args.all_refs), "excludes": not args.no_default_excludes,
    }, sort_keys=True)


def pass_commits(con: sqlite3.Connection, root: Path,
                 args: argparse.Namespace) -> Tuple[int, int, int, str, str]:
    """Pass A: commit messages, changed paths, renames. Cheap."""
    cmd = ["git", "log", "--no-merges", "-M", "--date=short", "--name-status",
           f"--pretty=format:{REC}%H{FLD}%ad{FLD}%an{FLD}%B{FLD}"]
    if args.all_refs:
        cmd.insert(2, "--all")
    if args.since:
        cmd.append(f"--since={args.since}")
    if args.max_commits:
        cmd.append(f"-n{args.max_commits}")
    if args.pathspec:
        cmd += ["--"] + args.pathspec

    print("  pass 1/2: commit messages, paths, renames …", flush=True)
    cbuf: List[tuple] = []
    fbuf: List[tuple] = []
    rbuf: List[tuple] = []
    n = nf = nr = dropped = 0
    oldest = newest = ""
    for chunk in stream_records(cmd, root, timeout=args.timeout):
        parts = chunk.split(FLD)
        if len(parts) < 4 or not SHA_RE.match(parts[0]):
            dropped += 1
            continue
        sha, date, author, body = parts[0], parts[1], parts[2], parts[3]
        tail = parts[4] if len(parts) > 4 else ""
        lines = body.splitlines()
        subject = lines[0].strip() if lines else ""
        rest = flatten(" ".join(lines[1:]))
        n += 1
        cid = n
        if not newest:
            newest = date
        oldest = date
        cbuf.append((cid, sha, date, flatten(author), flatten(subject), rest))
        for line in tail.splitlines():
            cols = line.rstrip("\n").split("\t")
            if len(cols) < 2 or not cols[0]:
                continue
            status = cols[0]
            if status[0] in ("R", "C") and len(cols) >= 3:
                fbuf.append((cid, status, cols[2], cols[1]))
                rbuf.append((cid, cols[1], cols[2], status))
                nr += 1
            else:
                fbuf.append((cid, status, cols[1], None))
            nf += 1
        if len(cbuf) >= BATCH:
            flush_pass_a(con, cbuf, fbuf, rbuf)
    flush_pass_a(con, cbuf, fbuf, rbuf)
    if dropped:
        print(f"    ** {dropped} commit record(s) unparseable (control bytes in a message?) "
              f"— skipped **", flush=True)
    return n, nf, nr, oldest, newest


def flush_pass_a(con: sqlite3.Connection, cbuf: List[tuple], fbuf: List[tuple], rbuf: List[tuple]) -> None:
    if cbuf:
        con.executemany("INSERT INTO commits(id, sha, date, author, subject, body) "
                        "VALUES(?,?,?,?,?,?)", cbuf)
        cbuf.clear()
    if fbuf:
        con.executemany("INSERT INTO files(commit_id, status, path, oldpath) VALUES(?,?,?,?)", fbuf)
        fbuf.clear()
    if rbuf:
        con.executemany("INSERT INTO renames(commit_id, oldpath, newpath, sim) VALUES(?,?,?,?)", rbuf)
        rbuf.clear()


def pass_diffs(con: sqlite3.Connection, root: Path,
               args: argparse.Namespace) -> Tuple[int, int, Dict[str, object]]:
    """Pass B: identifiers on added/removed diff lines, per FILE, over the FULL history."""
    sha_to_id = {r["sha"]: r["id"] for r in con.execute("SELECT id, sha FROM commits")}

    cmd = ["git", "log", "--no-merges", "-M", "--date=short", "-U0", "--no-color", "-p",
           f"--pretty=format:{REC}%H{FLD}%ad{FLD}"]
    if args.all_refs:
        cmd.insert(2, "--all")
    if args.since:
        cmd.append(f"--since={args.since}")
    if args.content_max_commits:
        cmd.append(f"-n{args.content_max_commits}")
    pathspec = list(args.pathspec or [])
    if not args.no_default_excludes:
        pathspec += DEFAULT_EXCLUDES
    if pathspec:
        cmd += ["--"] + pathspec

    print("  pass 2/2: identifiers from diffs, per file (full history) …", flush=True)
    tok_ids: Dict[str, int] = {}
    path_ids: Dict[str, int] = {}
    sub_rows: List[tuple] = []
    tok_rows: List[tuple] = []
    tc_buf: List[tuple] = []
    n_tc = 0
    seen = 0
    skipped = 0
    oldest = newest = ""
    last_report = time.monotonic()

    def token_id(raw: str) -> int:
        """Intern by lowercase literal, but derive the norm from the ORIGINAL casing."""
        key = raw.lower()
        tid = tok_ids.get(key)
        if tid is None:
            tid = len(tok_ids) + 1
            tok_ids[key] = tid
            tok_rows.append((tid, key, normal_form(raw)))
            for sub in set(content_subwords(raw)):
                if sub != key:
                    sub_rows.append((sub, tid))
        return tid

    def path_id(path: str) -> int:
        pid = path_ids.get(path)
        if pid is None:
            pid = len(path_ids) + 1
            path_ids[path] = pid
        return pid

    for chunk in stream_records(cmd, root, timeout=args.timeout):
        parts = chunk.split(FLD, 2)
        if len(parts) < 3 or not SHA_RE.match(parts[0]):
            skipped += 1
            continue
        sha, date, diff = parts[0], parts[1], parts[2]
        cid = sha_to_id.get(sha)
        if cid is None:
            skipped += 1
            continue
        seen += 1
        if not newest:
            newest = date
        oldest = date

        # (path, token) -> [adds, dels] for this commit
        per_file: Dict[Tuple[str, str], List[int]] = {}
        cur = ""
        for line in diff.splitlines():
            if line.startswith("+++ "):
                p = line[4:].strip()
                cur = p[2:] if p.startswith("b/") else ("" if p == "/dev/null" else p)
                continue
            if line.startswith("--- ") or line.startswith("diff --git ") or len(line) < 2:
                continue
            if line[0] not in "+-":
                continue
            slot = 0 if line[0] == "+" else 1
            for m in TOKEN_RE.finditer(line, 1):
                raw = m.group(0)
                if len(raw) < 3:
                    continue
                rec = per_file.get((cur, raw))
                if rec is None:
                    per_file[(cur, raw)] = rec = [0, 0]
                rec[slot] += 1

        for (path, raw), (a, d) in per_file.items():
            low = raw.lower()
            if low in STOPWORDS:
                continue
            tc_buf.append((token_id(raw), cid, path_id(path), a, d))
        if len(tc_buf) >= BATCH:
            n_tc += len(tc_buf)
            con.executemany(
                "INSERT INTO tc(token_id, commit_id, path_id, adds, dels) VALUES(?,?,?,?,?)", tc_buf)
            tc_buf.clear()
        if time.monotonic() - last_report > 20:
            print(f"    … {seen} commits, {len(tok_ids)} identifiers, {len(path_ids)} files",
                  flush=True)
            last_report = time.monotonic()

    if tc_buf:
        n_tc += len(tc_buf)
        con.executemany(
            "INSERT INTO tc(token_id, commit_id, path_id, adds, dels) VALUES(?,?,?,?,?)", tc_buf)
    con.executemany("INSERT INTO tokens(id, tok, norm) VALUES(?,?,?)", tok_rows)
    con.executemany("INSERT INTO paths(id, path) VALUES(?,?)",
                    ((v, k) for k, v in path_ids.items()))
    con.executemany("INSERT INTO token_sub(sub, token_id) VALUES(?,?)", sub_rows)

    stats = {"commits": seen, "oldest": oldest, "newest": newest,
             "files": len(path_ids), "skipped": skipped,
             "truncated": bool(args.content_max_commits)}
    return len(tok_ids), n_tc, stats


def read_meta_safe(db_path: Path) -> Optional[Dict[str, str]]:
    try:
        con = connect(db_path)
        meta = {r["k"]: r["v"] for r in con.execute("SELECT k, v FROM meta")}
        con.close()
        return meta
    except Exception:  # noqa: BLE001
        return None


# --------------------------------------------------------------------------- query


def require_index(args: argparse.Namespace) -> Tuple[Path, sqlite3.Connection, Dict[str, str]]:
    root = repo_root(Path(args.repo_dir).resolve())
    db_path = index_path_for(root, args.index_file)
    if not db_path.exists():
        raise SystemExit(f"no index at {db_path} — run: {Path(__file__).name} build --repo-dir {root}")
    con = connect(db_path)
    meta = {r["k"]: r["v"] for r in con.execute("SELECT k, v FROM meta")}
    return root, con, meta


def token_ids(con: sqlite3.Connection, term: str, family: bool = False) -> List[int]:
    """Identifiers denoting `term`.

    Default is the *concept*: every spelling whose normal form equals the term's
    (`OrderLineItem` == `order_line_item` == `ORDER_LINE_ITEM`). `family=True` widens to
    identifiers merely containing the term as a subword (`OrderLineItem` for `Order`) —
    useful for exploration, wrong for comparing two names, because the set for `Account`
    would then swallow `BillingAccount` and mask the very exchange being measured.
    """
    norm = normal_form(term)
    ids = {r["id"] for r in con.execute(
        "SELECT id FROM tokens WHERE norm = ? OR tok = ?", (norm, term.lower()))}
    if family:
        for sub in set(content_subwords(term)) or {term.lower()}:
            ids |= {r["token_id"] for r in con.execute(
                "SELECT token_id FROM token_sub WHERE sub = ?", (sub,))}
    return sorted(ids)


def term_life(con: sqlite3.Connection, term: str, family: bool = False) -> Optional[Dict[str, object]]:
    """Birth, last change, last growth and volume of an identifier — one indexed query."""
    ids = token_ids(con, term, family)
    if not ids:
        return None
    ph = ",".join("?" * len(ids))
    row = con.execute(
        f"SELECT COUNT(DISTINCT tc.commit_id) n, SUM(tc.adds) a, SUM(tc.dels) d, "
        f"MIN(c.date) first, MAX(c.date) last "
        f"FROM tc JOIN commits c ON c.id = tc.commit_id WHERE tc.token_id IN ({ph})", ids
    ).fetchone()
    if not row or not row["n"]:
        return None

    def commit_at(date: str, order: str) -> Dict[str, str]:
        r = con.execute(
            f"SELECT c.sha, c.date, c.author, c.subject FROM tc JOIN commits c ON c.id = tc.commit_id "
            f"WHERE tc.token_id IN ({ph}) AND c.date = ? ORDER BY c.id {order} LIMIT 1",
            ids + [date]).fetchone()
        return dict(r) if r else {}

    # A retired name keeps getting deleted (stale comments, leftover tests) for years
    # after nobody adds it any more, so "last touched" overstates how alive it is.
    # Measured: git.git's `get_sha1` was last *touched* in 2026 by a commit that deleted a
    # comment mentioning it — it was last *grown* in 2017.
    grown = con.execute(
        f"SELECT MAX(c.date) d FROM tc JOIN commits c ON c.id = tc.commit_id "
        f"WHERE tc.token_id IN ({ph}) AND tc.adds > tc.dels", ids).fetchone()
    last_grown = grown["d"] if grown and grown["d"] else ""
    spellings = [r["tok"] for r in con.execute(
        f"SELECT tok FROM tokens WHERE id IN ({ph}) ORDER BY tok", ids)]
    return {"birth": commit_at(row["first"], "DESC"), "last": commit_at(row["last"], "ASC"),
            "grown": commit_at(last_grown, "ASC") if last_grown else {},
            "commits": row["n"], "adds": row["a"] or 0, "dels": row["d"] or 0,
            "ids": ids, "spellings": spellings}


def term_paths(con: sqlite3.Connection, ids: Sequence[int], limit: int = 8) -> List[sqlite3.Row]:
    """Where the identifier actually occurs — files, not commits."""
    ph = ",".join("?" * len(ids))
    return list(con.execute(
        f"SELECT p.path, SUM(tc.adds) a, SUM(tc.dels) d, COUNT(DISTINCT tc.commit_id) n "
        f"FROM tc JOIN paths p ON p.id = tc.path_id WHERE tc.token_id IN ({ph}) "
        f"GROUP BY p.id ORDER BY a DESC LIMIT ?", list(ids) + [limit]))


def dir_spread(con: sqlite3.Connection, ids: Sequence[int], depth: int = 2) -> List[Tuple[str, int]]:
    """Directory distribution of an identifier's occurrences, weighted by additions."""
    ph = ",".join("?" * len(ids))
    agg: Counter = Counter()
    for r in con.execute(
            f"SELECT p.path path, SUM(tc.adds) a FROM tc JOIN paths p ON p.id = tc.path_id "
            f"WHERE tc.token_id IN ({ph}) GROUP BY p.id", list(ids)):
        parts = (r["path"] or "").split("/")
        key = "/".join(parts[:depth]) if len(parts) > depth else (
            "/".join(parts[:-1]) or "(root)")
        agg[key] += r["a"] or 0
    return agg.most_common()


def print_life(life: Optional[Dict[str, object]], head_date: str, indent: str = "",
               first_date: str = "") -> None:
    if life is None:
        print(f"{indent}- **history**: this identifier never appears in any indexed diff")
        return
    b, l = life["birth"], life["last"]  # type: ignore[index]
    print(f"{indent}- **born**: {b.get('date')} `{str(b.get('sha'))[:9]}` by {b.get('author')} — "
          f"{str(b.get('subject'))[:100]}")
    print(f"{indent}- **last changed**: {l.get('date')} `{str(l.get('sha'))[:9]}` — "
          f"{str(l.get('subject'))[:100]}")
    g = life.get("grown") or {}
    if g and g.get("date") != l.get("date"):
        print(f"{indent}- **last grown**: {g.get('date')} `{str(g.get('sha'))[:9]}` — "
              f"{str(g.get('subject'))[:100]}")
    print(f"{indent}- **volume**: {life['commits']} commits, +{life['adds']} / -{life['dels']} occurrences")
    if first_date and str(b.get("date")) == first_date:
        print(f"{indent}  - **caveat**: present in the repository's oldest indexed commit — "
              f"this is an import/squash artefact or a common English word, not a naming event")
    basis = (g.get("date") if g else None) or l.get("date")
    gap = dormant_years(str(basis), head_date)
    if gap >= 1.5:
        print(f"{indent}  - **signal**: dormant ~{gap:.1f} years — nothing has *added* it since "
              f"{basis}; reads as retired, not current vocabulary")
    elif int(life["dels"]) > int(life["adds"]) * 2 and int(life["commits"]) > 2:  # type: ignore[arg-type]
        print(f"{indent}  - **signal**: removals far outweigh additions — being phased out")


def meta_notes(meta: Dict[str, str]) -> List[str]:
    out = []
    if meta.get("content") == "1" and meta.get("diff_truncated") == "True":
        out.append(f"**Diff index is truncated** to {meta.get('diff_commits')} commits "
                   f"({meta.get('diff_oldest')} … {meta.get('diff_newest')}) — dates below are "
                   f"first-seen-in-window, not birth dates. Rebuild without --content-max-commits.")
    if meta.get("content") != "1":
        out.append("No diff index — `born`/`volume` unavailable. Rebuild with `--content`.")
    if meta.get("shallow") == "1":
        out.append("**Shallow clone** — early history is absent; every date is a lower bound.")
    scope = meta.get("scope")
    if scope:
        sc = json.loads(scope)
        bits = []
        if sc.get("pathspec"):
            bits.append("paths " + " ".join(sc["pathspec"]))
        if sc.get("since"):
            bits.append("since " + sc["since"])
        bits.append("all refs" if sc.get("all_refs") else "HEAD only")
        out.append("Indexed scope: " + ", ".join(bits))
    if meta.get("excludes") == "default":
        out.append("Vendored/generated paths were excluded from the diff pass "
                   "(vendor/, third_party/, testdata/, *.pb.go, lockfiles, …).")
    return out


def query(args: argparse.Namespace) -> int:
    root, con, meta = require_index(args)
    head_date = meta.get("newest") or head_commit_date(root)
    first_date = meta.get("oldest") or ""
    for note in meta_notes(meta):
        print(f"> {note}")

    for term in args.terms:
        print(f"\n## `{term}`\n")
        pat = word_re(term)

        life = term_life(con, term, args.family) if meta.get("content") == "1" else None
        if meta.get("content") == "1":
            print_life(life, head_date, first_date=first_date)
            if life and len(life.get("spellings") or []) > 1:
                print("- **spellings**: " + ", ".join(f"`{x}`" for x in life["spellings"][:8]))

        msg = message_hits(con, meta, term, pat, args.limit)
        print(f"- **commit messages**: {msg['total']} mention(s)"
              + (" (ranked by BM25)" if msg["ranked"] else ""))
        for r in msg["rows"]:
            print(f"  - {r['date']} `{r['sha'][:9]}` {r['subject'][:120]}")
        if msg["total"] > len(msg["rows"]):
            print(f"  - … {msg['total'] - len(msg['rows'])} more")

        issues = Counter()
        for r in msg["all_text"]:
            for num in issue_refs(r):
                issues[num] += 1
        if issues:
            print("- **referenced issues/PRs**: " + ", ".join(f"#{n}" for n, _ in issues.most_common(8)))

        rn = [r for r in con.execute(
            "SELECT c.sha, c.date, r.oldpath, r.newpath FROM renames r "
            "JOIN commits c ON c.id = r.commit_id ORDER BY c.date DESC")
            if pat.search(r["oldpath"]) or pat.search(r["newpath"])]
        if rn:
            print(f"- **file renames**: {len(rn)}")
            for r in rn[: args.limit]:
                print(f"  - {r['date']} `{r['sha'][:9]}` {r['oldpath']} → {r['newpath']}")

        if life:
            spread = dir_spread(con, life["ids"])
            tot = sum(n for _, n in spread) or 1
            if spread:
                print("- **where it occurs** (by directory): "
                      + ", ".join(f"{d} {n / tot:.0%}" for d, n in spread[:4]))
                if len(spread) >= 2 and spread[0][1] / tot < 0.8:
                    print(f"  - split across {len(spread)} directories — run "
                          f"`contexts {term}` before assuming one meaning")
            files = term_paths(con, life["ids"], args.limit)
            if files:
                print("- **busiest files**:")
                for r in files:
                    print(f"  - {r['path']} (+{r['a']} / -{r['d']}, {r['n']} commits)")
        named_paths = Counter()
        for r in con.execute("SELECT path, COUNT(*) n FROM files GROUP BY path"):
            if pat.search(r["path"]):
                named_paths[r["path"]] = r["n"]
        if named_paths:
            print(f"- **paths whose name carries the term**: {len(named_paths)}")
            for path, n in named_paths.most_common(args.limit):
                print(f"  - {path} ({n} changes)")

        if msg["years"] and len(msg["years"]) > 1:
            print("- **mentions by year**: " + " ".join(f"{y}:{n}" for y, n in sorted(msg["years"].items())))

    con.close()
    print()
    return 0


def message_hits(con: sqlite3.Connection, meta: Dict[str, str], term: str,
                 pat: re.Pattern, limit: int) -> Dict[str, object]:
    """BM25-ranked message hits when FTS5 exists, else a LIKE scan; both verified by regex."""
    rows: List[sqlite3.Row] = []
    ranked = False
    if meta.get("fts5") == "1":
        try:
            # Query the subwords, not the literal: FTS5 tokenises `order_line_item` into
            # three words, so a PascalCase `OrderLineItem` would match nothing. word_re
            # below still filters, and it accepts the spaced form.
            words = subwords(term) or [term]
            rows = list(con.execute(
                "SELECT c.sha, c.date, c.subject, c.body FROM commits_fts f "
                "JOIN commits c ON c.id = f.rowid WHERE commits_fts MATCH ? "
                "ORDER BY bm25(commits_fts, 4.0, 1.0)", (fts_query(" ".join(words)),)))
            ranked = True
        except sqlite3.OperationalError:
            rows = []
    if not rows:
        like = f"%{term}%"
        rows = list(con.execute(
            "SELECT sha, date, subject, body FROM commits "
            "WHERE subject LIKE ? OR body LIKE ? ORDER BY date DESC", (like, like)))
    hits = [r for r in rows if pat.search(r["subject"] or "") or pat.search(r["body"] or "")]
    return {
        "total": len(hits), "ranked": ranked, "rows": hits[:limit],
        "all_text": [f"{r['subject']} {r['body']}" for r in hits],
        "years": Counter(r["date"][:4] for r in hits),
    }


def shared_commits(con: sqlite3.Connection, ia: List[int], ib: List[int]) -> List[sqlite3.Row]:
    """Commits touching both identifier sets, with per-side totals and same-file totals.

    `same_file_*` restricts the exchange to paths where BOTH names occur in that commit —
    a rename edits one file to swap one name for the other, whereas ordinary churn merely
    happens to touch both names somewhere in a large commit.
    """
    pa, pb = ",".join("?" * len(ia)), ",".join("?" * len(ib))
    return list(con.execute(
        f"""
        WITH t AS (
          SELECT tc.commit_id cid, tc.path_id pid,
                 SUM(CASE WHEN tc.token_id IN ({pa}) THEN tc.adds ELSE 0 END) aa,
                 SUM(CASE WHEN tc.token_id IN ({pa}) THEN tc.dels ELSE 0 END) ad,
                 SUM(CASE WHEN tc.token_id IN ({pb}) THEN tc.adds ELSE 0 END) ba,
                 SUM(CASE WHEN tc.token_id IN ({pb}) THEN tc.dels ELSE 0 END) bd
          FROM tc WHERE tc.token_id IN ({pa}) OR tc.token_id IN ({pb})
          GROUP BY tc.commit_id, tc.path_id
        )
        SELECT c.sha, c.date, c.subject,
               SUM(t.aa) a_add, SUM(t.ad) a_del, SUM(t.ba) b_add, SUM(t.bd) b_del,
               SUM(CASE WHEN t.ad > t.aa AND t.ba > t.bd THEN t.ad - t.aa ELSE 0 END) sf_shrink,
               SUM(CASE WHEN t.ad > t.aa AND t.ba > t.bd THEN t.ba - t.bd ELSE 0 END) sf_grow,
               SUM(CASE WHEN t.ad > t.aa AND t.ba > t.bd THEN 1 ELSE 0 END) sf_files,
               SUM(CASE WHEN t.bd > t.ba AND t.aa > t.ad THEN 1 ELSE 0 END) sf_files_rev
        FROM t JOIN commits c ON c.id = t.cid
        GROUP BY t.cid
        HAVING SUM(t.aa + t.ad) > 0 AND SUM(t.ba + t.bd) > 0
        ORDER BY c.date""",
        ia + ia + ib + ib + ia + ib))


def score_direction(rows: Sequence[sqlite3.Row], forward: bool,
                    shrink_re: re.Pattern, grow_re: re.Pattern) -> Dict[str, object]:
    """Commits where one name net-shrinks while the other net-grows.

    A swap is a NET exchange. Testing "any deletion of A and any addition of B" fires on
    ordinary churn: measured on a real repo it labelled 105 of 174 commits touching two
    unrelated integrations as rename candidates — including commits where BOTH names were
    net-removed and commits where A actually grew.
    """
    scored = []
    for r in rows:
        if forward:
            net_shrink, net_grow = r["a_del"] - r["a_add"], r["b_add"] - r["b_del"]
            files = r["sf_files"]
        else:
            net_shrink, net_grow = r["b_del"] - r["b_add"], r["a_add"] - r["a_del"]
            files = r["sf_files_rev"]
        if net_shrink >= MIN_NET_SWAP and net_grow >= MIN_NET_SWAP:
            # same-file exchanges outrank whole-commit ones
            scored.append((files * 1000 + min(net_shrink, net_grow), net_shrink, net_grow,
                           files, r))
    scored.sort(key=lambda x: -x[0])
    # A subject announces a rename only if it carries a rename word AND names BOTH sides —
    # "Rename sha1_array to oid_array", "Change minion to node". Accepting one side lets an
    # unrelated "Rename Telegram meeting wrapper" certify `club` -> `meeting`.
    named, hinted = [], []
    for x in scored:
        subj = x[4]["subject"] or ""
        if not RENAME_WORD_RE.search(subj):
            continue
        hits = bool(shrink_re.search(subj)) + bool(grow_re.search(subj))
        (named if hits == 2 else hinted).append(x)
    same_file = [x for x in scored if x[3]]
    return {"scored": scored, "named": named, "hinted": hinted, "same_file": same_file}


def pair(args: argparse.Namespace) -> int:
    root, con, meta = require_index(args)
    head_date = meta.get("newest") or head_commit_date(root)
    first_date = meta.get("oldest") or ""
    for note in meta_notes(meta):
        print(f"> {note}")
    terms = args.terms

    if meta.get("content") != "1":
        print("\n> No diff index — rebuild with `--content` to compare names.")
        con.close()
        return 0

    lives: Dict[str, Optional[Dict[str, object]]] = {}
    print("\n## Life of each name\n")
    for term in terms:
        lives[term] = term_life(con, term, args.family)
        print(f"### `{term}`")
        print_life(lives[term], head_date, first_date=first_date)
        print()

    good = {t: v for t, v in lives.items() if v}
    if len(good) > 1:
        by_birth = sorted(good.items(), key=lambda kv: str(kv[1]["birth"].get("date")))  # type: ignore[index]
        print("> **Birth order**: " + " → ".join(
            f"`{t}` ({v['birth'].get('date')})" for t, v in by_birth))  # type: ignore[index]

    print("\n## Exchange evidence and verdict\n")
    for i, a in enumerate(terms):
        for b in terms[i + 1:]:
            verdict_for(con, a, b, lives, head_date, args.limit, args.family)

    con.close()
    print()
    return 0


def verdict_for(con: sqlite3.Connection, a: str, b: str,
                lives: Dict[str, Optional[Dict[str, object]]], head_date: str,
                limit: int, family: bool) -> None:
    ia, ib = token_ids(con, a, family), token_ids(con, b, family)
    if not ia or not ib:
        missing = a if not ia else b
        print(f"### `{a}` vs `{b}`\n\n- `{missing}` never appears in any indexed diff — "
              f"nothing to compare. Check the spelling, or the name may live only in docs "
              f"or in paths (see `query {missing}`).\n")
        return
    overlap = set(ia) & set(ib)
    if overlap:
        # One name contains the other (`Account` vs `BillingAccount` under --family):
        # keeping the shared identifiers in both sets makes B's additions cancel A's
        # deletions and hides the exchange entirely.
        ia = [i for i in ia if i not in overlap]
        ib = [i for i in ib if i not in overlap]
        if not ia or not ib:
            print(f"### `{a}` vs `{b}`\n\n- these two names resolve to the same identifiers — "
                  f"they are spellings of one concept, not competing names.\n")
            return

    rows = shared_commits(con, ia, ib)
    ra, rb = word_re(a), word_re(b)
    fwd = score_direction(rows, True, ra, rb)
    rev = score_direction(rows, False, rb, ra)

    # Direction is inferred, not taken from argument order — an agent asking about two
    # names rarely knows which way a rename went.
    def strength(d):
        return (len(d["named"]), len(d["same_file"]), len(d["scored"]))

    if strength(rev) > strength(fwd):
        old, new, best, other = b, a, rev, fwd
    else:
        old, new, best, other = a, b, fwd, rev

    def grown(t: str) -> str:
        v = lives.get(t) or {}
        g = v.get("grown") or {}
        return str(g.get("date") or (v.get("last") or {}).get("date") or "")

    scored, named, same_file = best["scored"], best["named"], best["same_file"]
    share = len(scored) / max(1, len(rows))
    # Noise floor: in a busy codebase a couple of incidental exchanges out of hundreds of
    # shared commits means nothing. Measured on git.git, `bisect`/`rebase` — two unrelated
    # concepts — produced 2 exchanges out of 145 shared commits.
    # Same-file exchanges are stronger evidence than whole-commit ones, so they clear a
    # lower bar — but not no bar: measured on git.git, `bisect`/`rebase` (unrelated) produced
    # 2 same-file exchanges in 111 shared commits, which is still churn.
    sf_share = len(same_file) / max(1, len(rows))
    sf_signal = len(same_file) >= 2 and sf_share >= NOISE_FLOOR / 2
    signal = bool(named) or sf_signal or (len(scored) >= 2 and share >= NOISE_FLOOR)

    if not signal:
        # With no exchange evidence the labels must not depend on which name was typed
        # first: orient by dormancy instead.
        if dormant_years(grown(b), head_date) > dormant_years(grown(a), head_date):
            old, new = b, a
        else:
            old, new = a, b
    stale = dormant_years(grown(old), head_date) - dormant_years(grown(new), head_date)
    gap = abs(stale)

    print(f"### `{a}` vs `{b}`\n")
    print(f"- shared commits: {len(rows)} · net exchanges `{old}`→`{new}`: {len(scored)} "
          f"(reverse direction: {len(other['scored'])}) · same-file exchanges: {len(same_file)} "
          f"· subjects naming a rename: {len(named)}")
    print(f"- last grown: `{old}` {grown(old) or '—'} · `{new}` {grown(new) or '—'}")

    if named and scored:
        line = (f"**RENAME — strong.** `{old}` → `{new}` — net exchange in "
                f"{plural(len(scored), 'commit')}, announced in "
                f"{plural(len(named), 'subject')}.")
    elif sf_signal and stale >= 2:
        line = (f"**RENAME — probable.** `{old}` → `{new}` — {plural(len(same_file), 'commit')} "
                f"swap them inside the same files, and `{old}` stopped growing {gap:.1f} years "
                f"earlier. No subject says so — read the top commits before deciding.")
    elif sf_signal:
        line = (f"**RENAME — probable.** `{old}` → `{new}` — {plural(len(same_file), 'commit')} "
                f"swap them inside the same files. Both names are still being added, so the "
                f"migration may be unfinished.")
    elif signal and stale >= 2:
        line = (f"**RENAME — possible.** `{old}` → `{new}` — net exchange in "
                f"{plural(len(scored), 'commit')} and `{old}` stopped growing {gap:.1f} years "
                f"earlier, but never in the same file. Weak: verify with `git show`.")
    elif signal:
        line = (f"**DRIFT or PARTIAL MIGRATION — weak.** {plural(len(scored), 'commit')} shift "
                f"`{old}` → `{new}`, but never within one file and both names are still being "
                f"added. Check whether they occupy the same paths.")
    elif stale >= 2:
        line = (f"**NOT A RENAME.** No meaningful exchange between them. `{old}` merely stopped "
                f"growing {gap:.1f} years before `{new}` — an abandoned concept, not a replaced "
                f"name.")
    else:
        line = ("**COEXISTENCE.** No exchange, both still growing — two live concepts, or synonym "
                "drift. The path split below is the evidence that separates those two cases.")
    print(f"- verdict: {line}")

    if scored:
        if not signal:
            print(f"- below the noise floor: {len(scored)} exchange(s) in {len(rows)} shared "
                  f"commits ({share:.0%}), none inside one file and no subject names a rename — "
                  f"listed for completeness, not as evidence:")
        else:
            print("- strongest exchanges (same-file ones first):")
        for _s, net_old, net_new, files, r in scored[:limit]:
            mark = f", {plural(files, 'file')} swapped in place" if files else ""
            print(f"  - {r['date']} `{r['sha'][:9]}` (net -{net_old} `{old}` / +{net_new} "
                  f"`{new}`{mark}) {r['subject'][:80]}")
        if named:
            print("- subjects that announce it (both names present):")
            for _s, _no, _nn, _f, r in named[:limit]:
                print(f"  - {r['date']} `{r['sha'][:9]}` {r['subject'][:100]}")
        elif best.get("hinted"):
            print("- subjects with rename wording but only one of the two names "
                  "(weaker — may be about something else):")
            for _s, _no, _nn, _f, r in best["hinted"][:limit]:
                print(f"  - {r['date']} `{r['sha'][:9]}` {r['subject'][:100]}")
        if signal and share > 0.4 and not named and not sf_signal:
            print(f"- **caution**: {share:.0%} of shared commits look like exchanges, none inside "
                  f"one file and none names a rename — in a busy codebase that is ordinary churn.")
        print("- confirm before acting: `git show <sha> -- <path>`")

    print_split(con, a, b, ia, ib, limit)
    print()


def print_split(con: sqlite3.Connection, a: str, b: str,
                ia: List[int], ib: List[int], limit: int) -> None:
    """Do the two names live in the same code, or in disjoint parts of the tree?

    This is the polysemy test: one word meaning two things in two modules shows up as two
    directory clusters with no shared files. It needs identifier x FILE granularity —
    commit-level co-occurrence cannot see it, because one commit touches many files.
    """
    pa, pb = ",".join("?" * len(ia)), ",".join("?" * len(ib))
    both = con.execute(
        f"SELECT COUNT(*) n FROM (SELECT tc.path_id FROM tc WHERE tc.token_id IN ({pa}) "
        f"INTERSECT SELECT tc.path_id FROM tc WHERE tc.token_id IN ({pb}))", ia + ib).fetchone()["n"]
    only_a = con.execute(
        f"SELECT COUNT(DISTINCT path_id) n FROM tc WHERE token_id IN ({pa})", ia).fetchone()["n"]
    only_b = con.execute(
        f"SELECT COUNT(DISTINCT path_id) n FROM tc WHERE token_id IN ({pb})", ib).fetchone()["n"]
    print(f"- files: `{a}` in {only_a}, `{b}` in {only_b}, **both in {both}**")
    if both == 0:
        print(f"  - **disjoint** — no file has ever contained both. Two concepts that share a "
              f"vocabulary, not two names for one thing. Candidate bounded-context split.")
    else:
        overlap_share = both / max(1, min(only_a, only_b))
        if overlap_share >= 0.5:
            print(f"  - heavily shared ({overlap_share:.0%} of the smaller set) — they live in the "
                  f"same code, so a rename or synonym drift is plausible.")
    for label, ids in ((a, ia), (b, ib)):
        spread = dir_spread(con, ids)[:4]
        if spread:
            tot = sum(n for _, n in spread) or 1
            print(f"  - `{label}` by directory: "
                  + ", ".join(f"{d} {n / tot:.0%}" for d, n in spread))


def contexts(args: argparse.Namespace) -> int:
    """Where one identifier lives — the polysemy check for a single name."""
    root, con, meta = require_index(args)
    for note in meta_notes(meta):
        print(f"> {note}")
    if meta.get("content") != "1":
        print("\n> No diff index — rebuild with `--content`.")
        con.close()
        return 0
    for term in args.terms:
        ids = token_ids(con, term, args.family)
        print(f"\n## `{term}`\n")
        if not ids:
            print("- never appears in any indexed diff")
            continue
        spellings = [r["tok"] for r in con.execute(
            f"SELECT tok FROM tokens WHERE id IN ({','.join('?' * len(ids))}) ORDER BY tok", ids)]
        print(f"- spellings indexed as this concept: " + ", ".join(f"`{s}`" for s in spellings[:8]))
        spread = dir_spread(con, ids)
        tot = sum(n for _, n in spread) or 1
        print(f"- directories ({len(spread)} total):")
        for d, n in spread[: args.limit]:
            print(f"  - {d:<40} {n / tot:5.0%}  ({n} additions)")
        if len(spread) >= 2 and spread[0][1] / tot < 0.8:
            top = spread[:2]
            print(f"- **possible polysemy**: the name is split across `{top[0][0]}` "
                  f"({top[0][1] / tot:.0%}) and `{top[1][0]}` ({top[1][1] / tot:.0%}). "
                  f"Read one file from each before assuming they mean the same thing.")
        print("- busiest files:")
        for r in term_paths(con, ids, args.limit):
            print(f"  - {r['path']} (+{r['a']} / -{r['d']}, {r['n']} commits)")
    con.close()
    print()
    return 0


def search(args: argparse.Namespace) -> int:
    """BM25-ranked full-text search over commit messages."""
    root, con, meta = require_index(args)
    if meta.get("fts5") != "1":
        raise SystemExit("this Python's sqlite3 was built without FTS5 — `search` unavailable")
    q = fts_query(" ".join(args.terms))
    if not q:
        raise SystemExit("empty search query")
    rows = con.execute(
        "SELECT c.sha, c.date, c.author, c.subject, bm25(commits_fts, 4.0, 1.0) score "
        "FROM commits_fts f JOIN commits c ON c.id = f.rowid "
        "WHERE commits_fts MATCH ? ORDER BY score LIMIT ?", (q, args.limit)).fetchall()
    if not rows:
        print("no matches")
    for r in rows:
        print(f"{r['date']} `{r['sha'][:9]}` {r['subject'][:110]}")
    con.close()
    return 0


def status(args: argparse.Namespace) -> int:
    root, con, meta = require_index(args)
    db_path = index_path_for(root, args.index_file)
    print(f"index: {db_path}  ({db_path.stat().st_size / 1024 / 1024:.0f} MB)")
    print(json.dumps(meta, indent=2))
    con.close()
    return 0


def clean(args: argparse.Namespace) -> int:
    root = repo_root(Path(args.repo_dir).resolve())
    db_path = index_path_for(root, args.index_file)
    removed = False
    if db_path.exists():
        db_path.unlink()
        print(f"removed {db_path}")
        removed = True
    legacy = db_path.with_suffix("")  # TSV directory written by earlier versions
    if legacy.is_dir():
        shutil.rmtree(legacy)
        print(f"removed {legacy}")
        removed = True
    if not removed:
        print(f"nothing to remove at {db_path}")
    parent = db_path.parent
    if parent.exists() and parent.name == "ubiquitous-language-git-index" and not any(parent.iterdir()):
        shutil.rmtree(parent)
    return 0


# --------------------------------------------------------------------------- cli


def main(argv: Optional[Sequence[str]] = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    sub = parser.add_subparsers(dest="cmd", required=True)

    def common(p: argparse.ArgumentParser) -> None:
        p.add_argument("--repo-dir", default=".", help="path inside the git repository (default: .)")
        p.add_argument("--index-file", default=None, help="override the temp index location")

    b = sub.add_parser("build", help="build or refresh the index")
    common(b)
    b.add_argument("--content", action="store_true",
                   help="also index identifiers from diffs over the FULL history (the useful mode)")
    b.add_argument("--since", default=None, help="limit history, e.g. '3 years ago'")
    b.add_argument("--max-commits", type=int, default=None, help="cap commits for the message pass")
    b.add_argument("--content-max-commits", type=int, default=None,
                   help="cap commits for the diff pass (default: no cap — index everything)")
    b.add_argument("--pathspec", nargs="*", default=None, help="limit to these paths")
    b.add_argument("--all-refs", action="store_true",
                   help="walk every ref, not just HEAD — picks up side branches (and their noise)")
    b.add_argument("--no-default-excludes", action="store_true",
                   help="do not exclude vendor/, third_party/, testdata/, generated and lock files")
    b.add_argument("--timeout", type=int, default=3600)
    b.add_argument("--refresh", action="store_true", help="rebuild even if the index looks current")
    b.set_defaults(func=build)

    q = sub.add_parser("query", help="evidence report for one or more terms")
    common(q)
    q.add_argument("terms", nargs="+")
    q.add_argument("--limit", type=int, default=10)
    q.add_argument("--family", action="store_true",
                   help="also match identifiers merely containing the term as a subword")
    q.set_defaults(func=query)

    p = sub.add_parser("pair", help="compare competing names for the same concept")
    common(p)
    p.add_argument("terms", nargs="+")
    p.add_argument("--limit", type=int, default=10)
    p.add_argument("--family", action="store_true",
                   help="widen each name to identifiers containing it (shared ones are dropped)")
    p.set_defaults(func=pair)

    cx = sub.add_parser("contexts", help="where one identifier lives — the polysemy check")
    common(cx)
    cx.add_argument("terms", nargs="+")
    cx.add_argument("--limit", type=int, default=8)
    cx.add_argument("--family", action="store_true")
    cx.set_defaults(func=contexts)

    s = sub.add_parser("search", help="BM25 full-text search over commit messages")
    common(s)
    s.add_argument("terms", nargs="+")
    s.add_argument("--limit", type=int, default=15)
    s.set_defaults(func=search)

    st = sub.add_parser("status", help="show index metadata")
    common(st)
    st.set_defaults(func=status)

    c = sub.add_parser("clean", help="delete the index")
    common(c)
    c.set_defaults(func=clean)

    args = parser.parse_args(argv)
    try:
        return args.func(args)
    except KeyboardInterrupt:
        return 130
    except subprocess.TimeoutExpired:
        eprint("git timed out — narrow the scope with --since / --pathspec / --content-max-commits")
        return 2
    except BrokenPipeError:
        return 0
    except RuntimeError as exc:
        eprint(str(exc))
        return 1


if __name__ == "__main__":
    raise SystemExit(main())