# AI-authorship markers & judgment overlay

The orchestrator's Phase 1 overlay on the raw inventory. These signals risk-rank scope and set severity priors; they are not findings themselves.

## Structural map

For each module, record what it exports, imports, and is called by. Flag modules importing from >5 sources (likely god module) and modules imported by >10 consumers (critical shared dependency — highest audit priority).

## AI-authorship markers

Signals that raise scrutiny on a file or directory:

- Excessive inline comments narrating trivial logic.
- Stale `TODO:`/`FIXME:` never resolved.
- Near-duplicate functions separated by 100+ lines (lost-context duplication).
- Convention switches mid-file — camelCase to snake_case, a pattern used then abandoned.
- Naming-convention drift within a unit.

## Iteration-depth estimate

Inspect git history. A large surface with few commits, or long runs of AI commits without human edits, indicates high AI-generation ratio and a higher feedback-loop-degradation prior — code more secure at step 1 than at the final state. Identify AI commits via `Co-Authored-By: Claude` / AI-tool trailers and characteristic generated message shapes (uniform conventional-commit bodies with bullet lists) — a heuristic signal, not proof. Raise scrutiny on the highest-iteration areas.

## Risk ranking & trust boundaries

Rank directories: highest = untrusted input, money movement, auth, PII, high churn. Mark trust boundaries explicitly — these feed the appsec lens directly.

## Effect on triage

A finding in a high-marker or high-iteration area carries a small upward severity prior at calibration. The prior never manufactures a finding — it only adjusts one that already cleared evidence-or-drop.

# Cost estimate, model recommendation & concurrency

## Sizing commands

Prefer `tokei` or `cloc` if present; else `git ls-files | xargs wc -l`. Capture total LOC, file count, and the top languages. Read `tech-stack.md` for the language set first.

## v0 token estimate (crude, self-calibrating)

An order-of-magnitude band, not a quote. Refined over runs by the `tokens_estimated`/`tokens_actual` telemetry pair.

```
repo_tokens   = LOC * 10                     # ~8-12 tokens/line; 10 default
per_lens      = repo_tokens * 0.5 * 3        # 0.5 coverage fraction * 3 read+reason+write on high-reasoning
lenses_total  = per_lens * active_lens_count
mapping       = repo_tokens * 1.2
total_point   = mapping + lenses_total
band          = [total_point * 0.5, total_point * 2]
```

Present the band, the inputs, and that high-reasoning output tokens dominate. On a large repo this is routinely millions of tokens — say so.

## Model recommendation (state at Phase 0)

Drive this lane with the highest-reasoning model available at high effort. A cheap model inflates false positives, and this lane files real issues. Every lens runs the same agent (`tribunal-lens-reviewer`); the tier varies per LENS at dispatch time:

| Lens (or role) | Model | Effort |
|---|---|---|
| orchestrator (the skill) | Opus 4.8 | high |
| appsec | Opus 4.8 | high |
| reliability | Opus 4.8 | high |
| architecture | Opus 4.8 | high |
| secrets-supply | Sonnet 5 | high |
| migration | Sonnet 5 | high |
| test-fidelity | Sonnet 5 | high |
| performance | Sonnet 5 | medium |
| observability | Sonnet 5 | medium |
| typesafety | Sonnet 5 | medium |
| coverage | Sonnet 5 | medium |
| infra | Sonnet 5 | medium |
| optional mappers (`map-structure`, `map-deps`) | Haiku 4.5 | low |

Advisory only: `tribunal-lens-reviewer` ships `model: inherit`; a dispatch-time override takes an alias (`haiku`/`sonnet`/`opus`/`fable`), not a pinned ID, and there is no per-dispatch effort control. This table guides the orchestrator's dispatch choices — it is not mechanically enforced.

API strings: `claude-opus-4-8`, `claude-sonnet-5`, `claude-haiku-4-5-20251001` — as of authoring; substitute the current flagship. `claude-fable-5` now sits above Opus as the highest-reasoning flagship. On proprietary code all tiers must be approved (Anthropic) models — never an external worker.

## Default wave partition

The default dispatch order (Phase 2), aligned with the model tiers above:

| Wave | Lenses |
|---|---|
| 1 | appsec, architecture, reliability |
| 2 | secrets-supply, migration, test-fidelity |
| 3 | coverage, infra, observability, performance, typesafety |

A lens dropped from the roster at Phase 1 (scope-inapplicable) is simply absent from its wave — no renumbering. Phase 0/1 MAY choose a different partition for cause (e.g. a migration-heavy repo pulling `migration` into wave 1), but whichever partition is used MUST be recorded in `run-started` (`schemas.md`) — resume reads the recorded partition, never re-derives it.

## Concurrency & cost control

Concurrency ≤5 lenses in flight regardless of roster size — the roster is a budget, not a simultaneous-dispatch target. Cost levers offered at Phase 0: narrow scope to a subtree; trim the Tier-2 lenses (`performance`, `observability`, `typesafety`); lower concurrency.

## Optional mappers

On a large/sprawling repo, offload raw file-reading to two cheap mapper subagents so it stays out of the orchestrator's retained context: `map-structure` (tree, languages, entry points, core/shared/test locations, churn) and `map-deps` (manifests, lockfiles, integration surface, env/secret-usage surface). On a small repo, map inline and skip them. Either way, produce the same `inventory.md`. These are the only subagents beyond the lens reviewer, and they carry no `tribunal-` prefix because they are generic extractors, not judges.

# Finding record — the agent write contract

Every tribunal lens agent emits findings in this format; the orchestrator reads them at triage. This is the only schema an agent needs — triage/run schemas are orchestrator-only (`schemas.md`).

## Write rule

Write each finding as its own file — `findings/<lens>/<finding-id>.json` (e.g. `findings/appsec/appsec-001.json`), one `finding/v1` JSON object per file — in the run directory the orchestrator gives you, the moment it is found. Never a batched write at the end. Durability comes from the one-file-one-finding layout: a process killed mid-write risks only the file being written; every previously written finding is already safe on disk. (There is no append tool; a read-then-rewrite of a shared per-lens jsonl would put the lens's whole findings file at risk on every write — that layout is rejected.)

**Mechanism:** use `Write` only. Never use a Bash shell command to write finding content — arbitrary `evidence` text (quotes, backticks, embedded newlines) will corrupt a shell-escaped write. Never `Write` over an existing finding file. This is safe without locking: you are the only writer under `findings/<lens>/` — no other agent touches your lens's directory.

**Numbering on (re-)dispatch:** before your first write, Glob `findings/<lens>/` — files may already exist if a prior attempt at this lens died partway. Continue numbering from the highest existing `NNN`; never renumber, rewrite, or delete an existing finding file. Prior findings stand; the orchestrator's triage dedups any overlap.

## finding/v1

```json
{"schema":"finding/v1","id":"<lens>-NNN","lens":"<lens>","title":"<imperative,specific>","category":"security|reliability|performance|architecture|observability|maintainability|testing|dependency|migration","severity":"critical|high|medium|low","confidence":0.0,"observed":true,"locations":[{"path":"src/...","lines":"42-58"}],"evidence":"<minimal snippet + 1-2 sentences>","impact":"<what breaks / cost>","recommendation":"<remediation shape, not a patch>","acceptance_criteria":["<verifiable close condition>"],"effort":"S|M|L","depends_on":["<id>"],"dedup_key":"<lens>:<normalized-path>:<slug>","created_at":"<iso8601>"}
```

Minimum required: `locations` (path + lines), `evidence`, `recommendation`. Set `lens` to your own lens name; set `category` to the finding's class, which may differ from your lens (the secrets-supply lens may file a `dependency` finding). `severity`/`confidence` are **provisional** — the orchestrator recalibrates at triage; do not treat your own scores as final. `observed` is `true` when the failure/behavior was directly observed or reproduced (test run, executed path, live config) and `false` when inferred from reading code; triage weighs observed findings above inferred ones at the same confidence.

## id & dedup_key

`id`: `<lens>-NNN`, sequential per lens (`appsec-001`, `appsec-002`, …). `dedup_key`: `<lens>:<path-normalized-to-repo-root>:<short-slug-of-title>` — the orchestrator dedups on this plus overlapping `locations`, and it rides into any filed issue body as a searchable comment.

## Evidence discipline

Concrete `path:line` + minimal snippet on every finding. An absence claim — "no handler", "no teardown", "missing validation" — requires reading the whole relevant unit, never a truncated window.

# Issue filing

Findings become GitHub issues only on explicit selection and authorization. Use the tracker command documented in `tech-stack.md` if present; otherwise default to `gh issue create` on a GitHub origin; otherwise STOP. This lane reruns over time, so filing is **idempotent** — never create a duplicate.

## Selection

File only findings the user explicitly selects ("all keep+combine critical/high", or specific ids). Silence or ambiguity → file nothing. Offer `decision-required` findings as a **separate** opt-in (discussion issues), so design questions don't masquerade as fix tickets.

## What is eligible

`keep` (one issue each) and `combine` groups (one issue per `group_id`), only above the confidence gate (defined in `triage.md`). `duplicate` / `false-positive` / `defer` / `accept-risk` / `investigate` never file. `decision-required` files as a discussion/ADR-candidate issue framed as a question — never a fix ticket, and **never by authoring an ADR** (ADRs come only from `/ca-adr`, user-attributed).

## Dedup — before generating any body

1. Skip any finding already carrying an `issue_ref` in `triage.jsonl` (filed on a prior run recorded in this log).
2. For every remaining selected finding, search the tracker for an open issue carrying its `dedup_key` or matching title (e.g. `gh issue list --search "<dedup_key>"`). The `dedup_key` rides in each body as an HTML comment, so it is searchable. If found, skip and record it as a duplicate in the filing report — do not re-file.

## Body — `bodies/<finding-id>.md`, generated lazily, approved-only

```
# <title>

**Severity:** <final_severity>  |  **Confidence:** <final_confidence>  |  **Effort:** <effort>

**Where:** <path:lines, one per line>

**Evidence:** <minimal snippet + what is observed>

**Impact:** <what breaks / what it costs>

**Recommendation:** <remediation shape>

**Acceptance criteria:**
- <verifiable close condition>

<!-- dedup_key: <dedup_key> · finding: <id> -->
```

`decision-required` variant: frame as **Question / Options / Evidence** — the decision and its trade-offs, not a remediation. Anti-slop applies (no em-dash sentence separators, no filler, no fabricated precision).

## Filing procedure

- **Default (no execution):** write `issue-commands.sh` with one line per issue: `gh issue create --title "<title>" --label "<final_severity>" --body-file bodies/<finding-id>.md` (`decision-required` labelled distinctly, e.g. `--label discussion`). Print the list. Stop.
- **On explicit approval:** run each command; capture the issue URL; write it to `triage.jsonl` as `issue_ref`.
- **Report** a table: finding/group id → created URL, or skipped (duplicate), or failed (with the error). Never silently drop a failure.

Findings file as GitHub issues, never `open-tasks.md` — a periodic-review finding must survive PR abandonment.

# Report projection

`report.md` is a projection of the logs, regenerated in Phase 4 — never hand-authored, never a source of truth. Rebuild it fully from `findings/*/*.json` + `triage.jsonl` on every run and on resume; it is deterministic from the logs. Task-list-structured, not prose.

## Structure

- **Header** — run-id, scope, date, models used (from `run.jsonl`), the token estimate from Phase 0 vs. actuals (summed from `lens-completed` `tokens` in `run.jsonl` when present; best-effort, since the orchestrator cannot always observe subagent spend), and a launched/skipped-lens summary with the skip reason per lens.
- **Findings** — grouped by **calibrated** `final_severity` (critical to low), then by lens/type within each severity. Each entry on one line: `id` · `path:line(s)` · one-line description · remediation shape · triage `decision` · link to `plans/phase-<n>.md`. Only `keep`/`combine` findings appear here.
- **Decisions needed** — a separate section for `decision-required` findings, each as its question + options. These need a decision, not a fix; do not fold them into the severity list.
- **Investigate appendix** — medium/low findings below the confidence gate after calibration (defined in `triage.md`; below-gate critical/high land in Decisions needed instead); `id` + `path:line` + one terse line each. Preserved, not filed.
- **Blocking-severity note** — one line: critical/high should block shipping the affected code, but this lane is not a gate and blocks nothing.

## Anti-slop

Apply `core` (no em-dash sentence separators, no filler/AI cadence, no fabricated precision) and `medium-documents`. Every count comes from the logs — never invent a number to make the report "feel precise."

## Relationship to `manifest.yaml`

`manifest.yaml` is the machine-readable run snapshot (a projection of `run.jsonl`); `report.md` is the human view. Both regenerate from the logs; neither is edited by hand.

# Orchestrator schemas & artifact layout

The finding record (what agents emit) is in `finding-record.md`. This file holds the orchestrator-only logs and the run layout. Source of truth = the per-finding files (`findings/<lens>/<finding-id>.json`) plus the two append-only logs (`triage.jsonl`, `run.jsonl`); everything else is a projection regenerable from them. Write each record as it is produced — never batch.

## Artifact tree

```
.codearbiter/reports/<run-id>/        # run-id = <UTC-date>-<scope-slug>
  run.jsonl              # APPEND-ONLY run-state events; resume source of truth
  manifest.yaml          # projection of run.jsonl (regenerable snapshot)
  inventory.md           # map + risk/boundary/marker overlay
  findings/<lens>/<finding-id>.json  # one finding per file, written on discovery
                         # (crash-durable: a kill risks only the in-flight file;
                         # per-lens dirs, so no write contention)
  triage.jsonl           # APPEND-ONLY, one decision/line
  bodies/<finding-id>.md # issue body, lazy, approved-only
  plans/phase-<n>.md     # per-wave path plan (projection)
  report.md              # final human-readable (projection)
  issue-commands.sh      # ready-to-run gh issue create commands
  telemetry.json         # KPI payload, opt-in
```

## triage/v1 — one object per line in `triage.jsonl`

```json
{"schema":"triage/v1","id":"<finding-id>","decision":"keep|combine|duplicate|false-positive|defer|accept-risk|decision-required|investigate","final_severity":"critical|high|medium|low","final_confidence":0.0,"counter_argument":"<steelman; required for critical+high>","rationale":"<why>","group_id":"<when combine>","duplicate_of":"<finding-id, when duplicate>","issue_ref":"<filled after filing>","decided_at":"<iso8601>"}
```

`final_*` override the provisional self-scores everywhere downstream. `issue_ref` closes the finding→issue loop and makes re-runs idempotent.

## run/v1 — one state event per line in `run.jsonl`

```json
{"schema":"run/v1","event":"run-started|lens-launched|lens-skipped|lens-completed|wave-flushed|wave-triaged|report-written|issues-filed|telemetry-sent|run-aborted","wave":1,"lens":"<lens>","detail":"<optional>","surface_seen":0,"findings":0,"model":"<model>","tokens":0,"at":"<iso8601>"}
```

`run-aborted` records a deliberate abandon (optional `detail` = reason) and marks the run terminal.

A `lens-completed` event carries `surface_seen` (int — the lens's Exposure denominator), `findings` (int — count the lens emitted), and `model` (the model the lens ran on, as dispatched); `model` also appears on `lens-launched`. `tokens` (int, optional) records the lens's observed token spend when the orchestrator can see it; null/omitted when unobserved.

The `run-started` event's `detail` carries the chosen wave partition — the lens list per wave (default or repartitioned-for-cause, per `cost-and-models.md`). This is the single record of the partition; nothing else derives or re-derives it.

## Resume — read the cursor, never the finding bodies

`run.jsonl` is the coarse state log — one line per wave/lens transition, tens of lines even across retries, not the per-finding logs. Resume reads the cursor, not the whole run, and never re-hydrates completed work:

0. **No `run-started`.** A run dir present but with no `run-started` event in `run.jsonl` is a Phase-0/1 death — restart Phase 0/1 fresh (`inventory.md` and lens selection are cheap to rebuild); any finding files already on disk stand and are deduped at triage as normal.
1. **Position.** The resume point is fixed by the last triaged wave: `grep '"event":"wave-triaged"' run.jsonl | tail -1` returns it while reading only matching lines. If none, resume at wave 1.
2. **Plan.** Read the wave partition from the `run-started` event's `detail` — the recorded partition, never re-derived — via `manifest.yaml` (a small projection) or, if it is missing or stale, `run.jsonl` directly. Both are bounded reads — never a full-file scan for the plan.
3. **Re-enter** Phase 2/3 for waves after the last triaged one only.
4. **Do not load** already-triaged waves' `findings/<lens>/` files or `triage.jsonl` into context — they are authoritative on disk. A later wave's dedup that needs a specific prior id fetches it by targeted `grep` across `findings/`, never a full read.
5. **Ordering.** `plans/phase-<n>.md` is written before the `wave-triaged` event for that wave is emitted — the event asserts the plan exists.

`manifest.yaml` is a convenience snapshot regenerated from `run.jsonl`; it accelerates the plan read but is never authoritative — a corrupt or stale manifest falls back to the append-only log.

## dedup_key & ids

`id`: `<lens>-NNN`, sequential per lens. `dedup_key`: `<lens>:<path-normalized-to-repo-root>:<short-slug-of-title>`. Dedup matches on `dedup_key` and overlapping `locations`.

# Telemetry (opt-in, KPI-only)

Optional feedback that refines the skill and calibrates the estimator, pooled across contributors. The target is the public codeArbiter repo, so a shared corpus needs a shared destination — which is safe only because the payload is boring by construction: aggregates and per-lens exposure counts, never anything that identifies a codebase or exposes a weakness. Off by default; sent only on explicit per-run authorization.

## Hard scrubbing

The payload carries integers and enums only. It MUST NOT contain code, file paths, finding titles or text, commit hashes, or remote URLs. Repo identity is omitted by default; a contributor who wants to self-tag their submission for their own cross-run tracking may add `--tag <label>`, and only then does `tag` appear. `run_id` is a fresh random value, not derived from the repo.

## Why exposure counts, not raw findings

A raw finding count is uninterpretable alone: it is base-rate x detector-sensitivity x exposure, and you cannot invert the product from one number. "0 SQL-injection findings" reads as both "nobody concatenates SQL" and "our appsec lens is blind" — and those resolve oppositely. The per-lens `surface_seen` denominator disambiguates: 0 against a large surface is a blind-spot alarm; 0 against no surface is correctly uninformative. `false_positives` (from the decision log) separates a silent lens from a noisy one. These fields let the corpus *flag* blind-vs-rare; they cannot *confirm* it — that needs ground truth (a seeded-vulnerability canary or a semgrep/CodeQL cross-check), which is a separate self-test, not a payload field.

Reading the corpus is the maintainer's judgment, not skill behavior — and a suspicious zero in a security lens defaults to suspecting the detector, never to dialing the lens back, because those costs are asymmetric.

## Payload — `telemetry.json`

```json
{"schema":"telemetry/v1","skill_version":"<x.y.z>","run_id":"<random>","at":"<iso8601>","tag":"<omitted unless --tag>","loc_total":0,"loc_by_language":{"<lang>":0},"files_scanned":0,"primary_language":"<lang>","lenses_run":0,"lenses_skipped":0,"model_orchestrator":"<api-string>","models_by_tier":{"opus":0,"sonnet":0,"haiku":0},"tokens_estimated":0,"tokens_actual":null,"lens_exposure":{"<lens>":{"ran":true,"surface_seen":0,"findings":0,"false_positives":0}},"issues_found":0,"severity_breakdown":{"critical":0,"high":0,"medium":0,"low":0},"decision_breakdown":{"keep":0,"combine":0,"duplicate":0,"false_positive":0,"defer":0,"accept_risk":0,"decision_required":0,"investigate":0},"issues_filed":0,"run_duration_sec":0}
```

`lens_exposure` is the field that makes the corpus interpretable; `surface_seen` and `model_orchestrator`/`models_by_tier` are sourced from `run.jsonl` `lens-launched`/`lens-completed` events, not hand-tallied. `tokens_estimated` vs `tokens_actual` calibrates the cost estimate (a guardrail, not a measure of review quality); `tokens_actual` sums `run.jsonl` `lens-completed` `tokens` when present, and is otherwise optional/best-effort — null when the orchestrator could not observe subagent spend. `issues_found` is the post-triage kept count (the `keep` + `combine` decision groups), distinct from the per-lens raw `findings` counts in `lens_exposure`. `skill_version` is read from `plugin.json`, not hand-set.

## Send procedure

- Write `telemetry.json`; show it in full.
- State plainly, in one line: these aggregates post publicly to the codeArbiter repo; they carry no code, paths, or finding text, and no repo identity unless you added `--tag`.
- **Default:** print `gh issue create --repo arbiterForge/codeArbiter --label telemetry --title "run-metrics <at>" --body-file telemetry.json`. Stop.
- **On explicit approval:** run it; record a `telemetry-sent` event in `run.jsonl`.

# Triage & calibration

Triage per wave from disk. The orchestrator's calibrated values are final and override every provisional self-score downstream.

## Dedup

Before calibrating, dedup each new finding against all findings already on disk — match by `dedup_key` and by overlapping locations. A match decides as `duplicate` (`duplicate_of` set), distinct from `combine`.

## Severity rubric (impact x likelihood)

- **critical** — exploitable security hole, data loss/corruption, or an outage path reachable with realistic input.
- **high** — serious correctness/security weakness, latent but plausible; or a systemic architectural defect amplifying other risk.
- **medium** — real defect/debt, limited blast radius or lower likelihood.
- **low** — minor quality, polish, or DX improvement.

## Calibration

For each finding, set `final_severity`/`final_confidence` from the evidence directly — the lens's values are provisional input. For every critical/high, record a `counter_argument` — the strongest case it is lower or a false positive; if compelling, downgrade or reclassify. Calibration is bidirectional: promote under-rated findings too. Optional for criticals: dispatch a fresh-context adversary that sees only the finding + code and tries to refute it, to defeat anchoring.

## Severity priors

Apply as priors on findings that already cleared evidence-or-drop, never to manufacture one: resource-level authz / IDOR → high or critical; injection with reachable user input → high or critical; literal secret → high or critical; async operation with no handler on a critical path → high. A high-marker/high-iteration location (per `ai-markers.md`) nudges one level at most.

## Confidence gate

The bar a finding's `final_confidence` must clear to file, tiered by severity — an uncertain critical is too costly to bury silently, so it gets a lower bar and a softer landing than a low:

| `final_severity` | gate | below the gate |
| --- | --- | --- |
| critical / high | ≥0.5 | → `decision-required`, framed as a question, never dropped silently |
| medium | ≥0.7 | → `investigate` |
| low | ≥0.75 | → `investigate` |

## Low-severity discipline

A `low` is kept only above the confidence gate (≥0.75, see above) with a concrete, actionable remediation. Beyond ~5 lows per lens, aggregate the remainder into a single rollup finding that still lists each `path:line`.

## Decision vocabulary (into `triage.jsonl`)

- **keep** — actionable fix; files as its own issue.
- **combine** — real, merged with siblings under a shared `group_id`; one issue.
- **duplicate** — identical to a recorded finding (`duplicate_of`); distinct from combine.
- **false-positive** — not real; `rationale` required (this tunes future-run noise down — keep it).
- **defer** — real, out of scope/priority now; preserved, not filed this run.
- **accept-risk** — real, consciously not fixing; the risk-acceptance trail.
- **decision-required** — real and significant, but the response is an ADR-grade design choice, not a clear fix; files as a discussion, not a fix ticket.
- **investigate** — undecided, or a medium/low below the confidence gate after calibration; never filed.

Below the confidence gate after calibration: medium/low → `investigate`; critical/high → `decision-required` (see Confidence gate above — never dropped silently). ADR-grade questions also → `decision-required`.

## Per-wave plan

`plans/phase-<n>.md` covers only `keep`/`combine`, grouped by type (lens/category/`group_id`): shared remediation approach, ordered sequence, cross-group `depends_on`, rolled-up acceptance criteria. Roadmap level only — no per-finding code steps. A `decision-required` item gets a one-line "ADR-candidate — resolve via `/ca-adr`" pointer, never an authored ADR.