# Sol brief template

Fill this in and write it to `$SCRATCHPAD/sol-brief.md`, then pipe it into `codex exec` on stdin.
Piping avoids shell-quoting damage to code snippets and multi-line criteria.

Keep it short. Sol is a frontier model — it needs intent, contracts, and fences, not
line-by-line instructions. An over-specified brief produces worse code than a tight
contract, because it substitutes your guesses for the model's search.

## Implementation brief

```xml
<task>
[The user's request, verbatim.]

Plan:
- Goal: [one sentence]
- Likely files: [paths, with a one-line note on what each is for]
- Conventions to follow: [the specific ones that matter here — test framework,
  error-handling style, naming, the module this should mirror]
</task>

<acceptance_criteria>
- [Each item a runnable command or an observable behavior.]
- `pytest tests/test_auth.py` passes, including a case covering lockout after 5 attempts
- `npm run typecheck` is clean
- POST /login with a locked account returns 423, not 401
</acceptance_criteria>

<non_goals>
- [Explicit scope fence. Name the adjacent things you do NOT want touched.]
- Do not change the session-token format
- Do not add new dependencies
</non_goals>

<verification_loop>
Follow the conventions already in this repo. Add or update tests for the behavior you
change. Before finishing, run [the project's actual commands] and fix whatever they
surface. Do not report success on checks you did not run.
</verification_loop>

<action_safety>
No unrelated changes. No drive-by refactors, reformatting, or dependency bumps.
Confine edits to what the task requires.
</action_safety>

<output_contract>
End your final message with:
1. Files changed (paths only)
2. Exact commands you ran
3. The result of each command
</output_contract>
```

## Visual tasks

Two capabilities the plain template does not reach for.

**Attaching images.** List them in a sidecar beside the brief —
`<run-dir>/tasks/NN-<slug>.images`, one path per line, `#` comments ignored, paths
absolute or relative to the repo root. The launcher passes each as `codex exec -i`,
and a missing path fails the run at preflight. (On a direct `codex exec` call, pass
`-i <file>` yourself, repeatable.) This puts images in front of Sol — a screenshot of the broken state, the mockup to match, the
chart that renders wrong. Verified: Sol reads them and describes their actual content.
Reference the attachment in `<task>` so it knows what the image is for:

```xml
<task>
The attached screenshot shows the settings panel overflowing its container at 320px.
Fix the layout so it matches the second attached image, which is the intended design.
</task>
```

**Asking for an asset.** Codex has a built-in `image_gen` tool, so a brief can ask for
a raster asset directly and get AI-generated pixels. Sol routes to code on its own when
the visual is code-native (a geometric shape, an icon belonging to an existing SVG
system), so state the intent, not the method. The criteria have to change shape,
because there is no test command for a picture:

```xml
<acceptance_criteria>
- `assets/title-bg.png` exists and `file` reports a PNG of at least 1024x768.
- It depicts a lighthouse on a cliff at dusk, in the painterly style of the
  attached reference — checked by looking at it, not by a command.
</acceptance_criteria>
```

Review such a run by opening the image, not by reading `git diff`: a binary shows only
`Binary files differ`, which certifies nothing.

## Parallel brief additions

In parallel mode each worker runs in its own isolated git worktree, so no two workers
ever touch the same working directory at once — but they share one repository's
history, and the launcher integrates their branches afterward by cherry-picking each
onto a common base. A scope fence keeps declared file scopes from overlapping, which is
what makes that integration conflict-free and keeps each worker's isolated-green
verification still meaningful once merged. Add this to every parallel brief's
`<non_goals>`, plus a git-operation fence — the launcher, not the worker, owns commits
and branches:

```xml
<non_goals>
- Touch only these paths: [the task's declared file scope]. Another worker's changes
  will be integrated onto the same base branch — staying inside your scope is what
  keeps that integration conflict-free.
- Do not commit, do not create or switch branches, do not run git rebase or merge.
  The launcher commits your work on its own branch.
</non_goals>
```

## Research brief

Only when the user explicitly names Sol as the researcher. Runs `-s read-only`.

```xml
<task>
[The question.] Today's date is [YYYY-MM-DD].
Repo context, if relevant: [what the answer will be used for]
</task>

<research_mode>
Search broadly. Prefer primary sources — official docs, changelogs, source code,
specs — over blog summaries. Prioritize current-year information; note when a source
is older than the window and may be stale.
</research_mode>

<citation_rules>
Every load-bearing claim needs a source URL. Mark each claim as EVIDENCE (directly
supported by a source) or INFERENCE (your reasoning from the evidence). Do not
present inference as evidence.
</citation_rules>

<output_contract>
A structured report of at most 600 words:
- Findings (the answer, ordered by importance)
- Evidence (claim → source URL)
- Open questions (what you could not establish)
No transcript of your search process.
</output_contract>
```

## Correction messages

Corrections resume the same session, so do **not** restate the brief. Send only the
delta, and make it checkable:

```
src/auth/lockout.py:42 — counter resets on every failed attempt, so lockout never
fires. Required: the counter persists across attempts within the window and locks at
5. `pytest tests/test_auth.py::test_lockout_after_five` must pass.
```

Three parts, every time: **where**, **what's wrong and what's required**, **what check must pass**.

# Parallel flow

Followed instead of phases 2–5 when `SKILL.md`'s parallel routing rule fires. Runs N
`codex exec` workers concurrently via `scripts/sol-parallel.sh`, each in its own git
worktree on branch `sol/<slug>`, then integrates and re-verifies the result on one
merged branch. Phase 1 (plan) still applies unchanged; this replaces phases 2–5 with
the twelve steps below.

## 1. When this applies

Only when the user names a worker count: `--workers N`, or plain language like "use 3
workers". Never inferred from a request that merely looks like several independent
tasks — the trigger is a number the user typed, not a judgment about the work. If the
user says `--workers 1` (or "use 1 worker"), that is *not* parallel mode: follow
`SKILL.md` phases 1–5 as normal.

## 2. Resolve the ceiling

The ceiling is `SOL_MAX_WORKERS` if set, else 3 — this is `sol-parallel.sh`'s own
default and it is a hard cap, not a suggestion. If the user asks for more workers than
the ceiling, refuse in one line naming both numbers, e.g.: "You asked for 5 workers;
the ceiling is 3 (`SOL_MAX_WORKERS`). Raise `SOL_MAX_WORKERS` or ask for fewer." The
script itself refuses the same way (exit 2) if you launch anyway — do not silently
clamp the requested count down to the ceiling and proceed.

If there are more tasks than the ceiling, run waves: one `sol-parallel.sh` invocation
per wave. Write each new wave's briefs only after integrating and re-verifying the
previous wave (steps 7–10), so the new wave branches from that merged, checked result
— never from the stale pre-integration base.

## 3. Split and confirm

Print the numbered split before writing anything: task number, slug, one-line goal,
expected file scope. Call out any overlap between two tasks' file scopes explicitly and
propose either merging them into one task or moving one to a later wave. **Wait for the
user's confirmation before writing briefs or invoking the script** — this split is a
plan the user hasn't seen yet, and deserves the same confirmation bar as any other plan.

## 4. Write briefs

One brief per task at `<run-dir>/tasks/NN-<slug>.md` (two-digit `NN`, lowercase-hyphen
`<slug>`), using the "Parallel brief additions" in `brief-template.md` layered on the
normal Implementation brief. `<run-dir>` is a scratch directory you choose, e.g.
`$SCRATCHPAD/sol-run` — create `<run-dir>/tasks/` yourself before writing into it.

`sol-parallel.sh` derives each worker's branch name from this filename: it strips a
leading `NN-`, lowercases, and collapses every character that isn't `a-z0-9` to `-`.
Whatever slug you announced in the split, use the same text in the filename, or the
resulting branch name (`sol/<slug>`) won't match what you told the user.

Reference images for a worker go in a sidecar beside its brief:
`<run-dir>/tasks/NN-<slug>.images`, one path per line, `#` comments ignored, absolute
or repo-root-relative. Each is passed to that worker as `codex exec -i`; a path that
does not resolve fails the whole run at preflight, before any worktree exists.

## 5. Launch

Checkpoint first: `sol-parallel.sh` refuses to launch against a dirty working tree
(every worker worktrees off `HEAD`, so uncommitted changes on the current branch would
leak into all of them). Commit or stash before launching.

One blocking call:

```bash
bash <skill-dir>/scripts/sol-parallel.sh --workers N "$SCRATCHPAD/sol-run"
```

This creates one git worktree and branch (`sol/<slug>`) per brief under
`../.sol-worktrees/<repo-name>/<slug>` (a sibling of the repo, not inside it), runs
`SOL_WORKTREE_SETUP` in each fresh worktree if set, launches one `codex exec` per
worker, and **blocks until every worker finishes or is force-killed** by the
inactivity watchdog — so this single call can run far longer than one tool-call budget.
There is deliberately no default cap on how long a worker may take: a task's duration is
not predictable, so any constant kills productive workers. What is bounded is silence.

If the tool call itself times out before the script returns, the workers are still
running unattended — that is not a failure, just an interrupted wait. Re-attach:

```bash
bash <skill-dir>/scripts/sol-parallel.sh --wait "$SCRATCHPAD/sol-run"
```

`--wait` exits 75 while any worker is still running, 0 once all finished cleanly, 1 if
at least one worker landed in a non-`ok`/`no-changes` state. Keep re-invoking `--wait`
until it stops returning 75. **Never launch `codex` directly in parallel mode** — the
script owns branch, worktree, and session bookkeeping that a hand-rolled command would
break.

(Optional: `sol-parallel.sh --dry-run <run-dir>` creates and bootstraps the worktrees —
including running `SOL_WORKTREE_SETUP` — and stops before launching any codex session,
if you want to sanity-check the setup before spending a run.)

## 6. Read `summary.json`, never the raw logs

Once the launch call (or the last `--wait`) returns 0 or 1, read
`<run-dir>/summary.json` — never `events.jsonl` or `report.md` directly except as
pointed to below. It has one entry per worker in its `workers` array: `slug`, `branch`
(`sol/<slug>`), `worktree` (absolute path), `status`, `exit_code`, `session_id`,
`commit`, `files_changed`, `elapsed_seconds`, `effort_used`, `stall_retries`,
`stall_reason`, and the paths to that worker's `report.md` / `events.jsonl` /
`stderr.txt`.

`status` is one of: `ok` · `no-changes` · `failed-launch` · `failed-run` ·
`failed-setup` · `failed-commit` · `timed-out` · `stalled`.

- `failed-launch` means codex itself never produced an event (bad invocation, binary
  missing) — read the *tail* of that worker's `stderr.txt` (path from `summary.json`),
  not the whole file.
- `failed-setup` means `SOL_WORKTREE_SETUP` failed before codex ever ran — check
  `<run-dir>/workers/<slug>/setup.log`.
- `stalled` means the inactivity watchdog killed the worker: no substantive event
  within `SOL_FIRST_EVENT_TIMEOUT` of launch, or no event at all for
  `SOL_IDLE_TIMEOUT` after work had started (`stall_reason` says which). Silence
  while a command execution or MCP tool call is in flight (an `item.started`
  with no matching `item.completed`) never counts against the idle budget — a
  quiet 15-minute build is not a stall. That exemption is bounded by
  `SOL_COMMAND_TIMEOUT` (default 1800s), so a command that never returns is still
  caught, as a stall rather than a timeout. `timed-out` appears only if you opt
  into an absolute cap with `SOL_WORKER_TIMEOUT`, which is off by default. The
  launcher already retried it `SOL_STALL_RETRIES` times, each attempt a fresh
  session one reasoning-effort step lower — `effort_used` and `stall_retries`
  record what happened, and each prior attempt's logs are archived as
  `events-attempt-N.jsonl` beside the final ones. A worker still `stalled` in the
  summary exhausted its retries: report it to the user rather than relaunching by
  hand, and note that a retried worker's worktree keeps whatever the stalled
  attempt had already edited (briefs describe an end state, so the retry builds on
  it).

`files_changed` is the branch's whole diff since `base_sha` for an `ok` worker. For a
non-`ok` worker it is a snapshot of everything its worktree holds that the base does
not — committed, staged, unstaged, or untracked — because a `failed-commit`,
`failed-run`, `timed-out`, or `stalled` worker can leave real work sitting there uncommitted, and
this is the only place the report tells the user where to find it. An empty list for a
non-`ok` worker means that worker genuinely produced nothing.

Every non-`ok` worker is named in the final report with its status — a task that
produced nothing (`no-changes`) is reported as such, never silently omitted.

## 7. Review in task order

For each `ok` worker, in the order the tasks were assigned:

```bash
git diff --stat <base_sha>..sol/<slug>
```

(`<base_sha>` is `summary.json`'s `base_sha`.) Then read the full branch diff. Hold it
to the same standard as the single-worker flow's phase 3 — correctness against the
brief's acceptance criteria, regressions, edge cases, security, missing tests,
out-of-scope changes.

Additionally check the worker stayed inside the file scope its brief declared —
compare against that worker's `files_changed` in `summary.json`.

## 8. Integrate one branch at a time

From the repo root, on the base branch (`summary.json`'s `base_branch` — check it out
first if you're not already on it), once a worker passes review:

```bash
git cherry-pick sol/<slug>
```

This lands the worker's patch as a *new* commit on the base — not a merge — so each
worker's branch still shows its own full diff independently.

Before moving to the next worker, catch cross-worker conflicts early by rebasing every
not-yet-integrated worker onto the base you just advanced:

```bash
git -C <worktree> rebase <base-branch>
```

(`<worktree>` from that worker's `summary.json` entry.) A conflict here is evidence the
"these tasks are independent" premise was wrong for that pair — report it to the user;
never resolve it yourself by picking one side.

## 9. Correct

Write the delta — same three parts as a single-worker correction (where, what's wrong
and required, what check must pass), not a restatement of the brief — to
`<run-dir>/workers/<slug>/correction.md`, then:

```bash
bash <skill-dir>/scripts/sol-parallel.sh --resume "$SCRATCHPAD/sol-run"
```

This resumes every worker that currently has a `correction.md` waiting, each **by its
own recorded session id** — never `--last`. With N sessions in flight, `--last` resumes
whichever session codex last touched, which may not be the worker you meant to correct,
silently applying your fix to the wrong worker's branch. `codex exec resume` also
accepts no `-C`, `-s`, or `--color` (verified against codex-cli 0.144.6); the script
already handles this. **Never construct a `codex exec resume` command by hand in
parallel mode.**

`--resume` blocks the same way `--workers` does in step 5 — if the tool call times out
before it returns, re-attach with the same `--wait "$SCRATCHPAD/sol-run"` call, then
re-read `summary.json` once it exits 0 or 1.

Each round's `correction.md` is renamed to `correction-1.md`, `correction-2.md`, ... as
it's consumed, so rounds are visible on disk per worker. Track rounds per worker
independently and stop after 2 rounds for that worker, same ceiling as single-worker
mode — after 2 rounds, stop and report the remaining issue to the user instead of
looping. Re-review after each round (back to step 7 for that worker).

## 10. Re-run the checks on the merged branch

Mandatory, once, after every worker in this wave is integrated: run the project's
actual test/lint/typecheck commands yourself, on the now-merged base branch. **This
result leads the report; per-worker results are supporting detail only.** Each worker
was verified green in isolation, against a base that did not contain the other
workers' changes — green × N is not green combined. Reporting N isolated green runs as
if they were a combined green run reintroduces, one level up, exactly the
self-assessment problem this skill exists to eliminate.

## 11. Clean up

```bash
bash <skill-dir>/scripts/sol-parallel.sh --cleanup "$SCRATCHPAD/sol-run"
```

Removes the worktree and branch for every worker whose status was `ok`/`no-changes`,
whose worktree is clean, and whose branch is fully integrated into the base (checked by
patch equivalence via `git cherry`, not ancestry — cherry-pick creates new commits, so
ancestry alone would miss genuinely-integrated work). Everything else is printed as a
`kept: sol/<slug> <path> (<reason>)` line. Name every one of them in your report —
nothing a worker produced is ever silently discarded.

The integration check fails closed: if it cannot get a trustworthy answer — the base
branch was renamed or deleted after the run, or `git cherry` itself errors — nothing is
removed, every worker is printed as `kept:`, and the reason is written to stderr. A
`--cleanup` that removes nothing and warns about the base ref is that guard firing, not
a no-op; re-point or restore the base branch and run it again.

## 12. Report

- **Per task**: title, branch (`sol/<slug>`), verdict, `git diff --stat` output with
  one clause per file, correction rounds used.
- **Combined**: the merged-branch check output from step 10, commits added to the base,
  wall clock for the run.
- **Survivors and risks**: every branch/worktree `--cleanup` kept and why (from its
  `kept:` line), plus anything unusual noticed across workers.

#!/usr/bin/env bash
# Preflight for the /sol skill. Read-only: checks that the Codex CLI is installed,
# authenticated, and pointed at a usable model. Runs no research and edits nothing.
#
# Usage: bash scripts/check-codex.sh [--json] [model]
#   model  defaults to gpt-5.6-sol

set -uo pipefail

json=0
MODEL="gpt-5.6-sol"
model_set=0
for arg in "$@"; do
  if [ "$arg" = "--json" ]; then
    json=1
  elif [ "$model_set" -eq 0 ]; then
    MODEL="$arg"
    model_set=1
  fi
done

CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
fail=0
warn=0
check_names=()
check_statuses=()
check_details=()

record_check() {
  local index="${#check_names[@]}"
  check_names[index]="$1"
  check_statuses[index]="$2"
  check_details[index]="$3"
}

append_hint() {
  local index=$((${#check_details[@]} - 1))
  if [ "$index" -ge 0 ]; then
    check_details[index]+=$'\n'
    check_details[index]+="$1"
  fi
}

json_escape() {
  local LC_ALL=C
  local value="$1"
  local escaped=""
  local character
  local code
  local sequence
  local i

  for ((i = 0; i < ${#value}; i++)); do
    character="${value:i:1}"
    case "$character" in
      '"') escaped+='\"' ;;
      \\)  escaped+='\\' ;;
      *)
        printf -v code '%d' "'$character"
        if [ "$code" -lt 0 ]; then
          code=$((code + 256))
        fi
        if [ "$code" -lt 32 ]; then
          printf -v sequence '\\u%04x' "$code"
          escaped+="$sequence"
        else
          escaped+="$character"
        fi
        ;;
    esac
  done

  JSON_ESCAPED="$escaped"
}

print_json() {
  local ready=true
  local i
  local name
  local status
  local detail

  if [ "$fail" -gt 0 ]; then
    ready=false
  fi

  json_escape "$MODEL"
  printf '{"ready":%s,"failures":%s,"warnings":%s,"model":"%s","checks":[' \
    "$ready" "$fail" "$warn" "$JSON_ESCAPED"

  for ((i = 0; i < ${#check_names[@]}; i++)); do
    if [ "$i" -gt 0 ]; then
      printf ','
    fi
    json_escape "${check_names[i]}"
    name="$JSON_ESCAPED"
    json_escape "${check_statuses[i]}"
    status="$JSON_ESCAPED"
    json_escape "${check_details[i]}"
    detail="$JSON_ESCAPED"
    printf '{"name":"%s","status":"%s","detail":"%s"}' "$name" "$status" "$detail"
  done

  printf ']}\n'
}

ok() {
  if [ "$json" -eq 1 ]; then
    record_check "$2" "ok" "$1"
  else
    printf '  \033[32mok\033[0m    %s\n' "$1"
  fi
}

bad() {
  if [ "$json" -eq 1 ]; then
    record_check "$2" "fail" "$1"
  else
    printf '  \033[31mFAIL\033[0m  %s\n' "$1"
  fi
  fail=$((fail + 1))
}

note() {
  if [ "$json" -eq 1 ]; then
    record_check "$2" "warn" "$1"
  else
    printf '  \033[33mwarn\033[0m  %s\n' "$1"
  fi
  warn=$((warn + 1))
}

hint() {
  if [ "$json" -eq 1 ]; then
    append_hint "$1"
  else
    printf '        %s\n' "$1"
  fi
}

if [ "$json" -eq 0 ]; then
  printf '\n/sol preflight\n\n'
fi

# 1. Codex CLI on PATH
if command -v codex >/dev/null 2>&1; then
  ok "codex on PATH — $(command -v codex)" "codex_on_path"
  if version=$(codex --version 2>/dev/null | head -1); then
    ok "version — ${version}" "codex_version"
  else
    note "could not read 'codex --version'" "codex_version"
  fi
else
  bad "codex not found on PATH" "codex_on_path"
  hint "install: npm i -g @openai/codex   (see https://github.com/openai/codex)"
  hint "then re-run this script"
  if [ "$json" -eq 1 ]; then
    print_json
  else
    printf '\n%s check(s) failed.\n\n' "$fail"
  fi
  exit 1
fi

# 2. Non-interactive exec subcommand exists
if codex exec --help >/dev/null 2>&1; then
  ok "'codex exec' available (non-interactive mode)" "codex_exec"
else
  bad "'codex exec' not available — CLI too old for this skill" "codex_exec"
  hint "update: npm i -g @openai/codex@latest"
fi

# 3. Authentication
if [ -f "${CODEX_HOME}/auth.json" ]; then
  ok "authenticated — ${CODEX_HOME}/auth.json present" "authentication"
else
  bad "not authenticated — no ${CODEX_HOME}/auth.json" "authentication"
  hint "run: codex login"
fi

# 4. Model reachability. 'codex exec' is the only honest probe, and it costs a
#    request, so this only reports what is configured and whether the slug is
#    known to the local model cache.
config="${CODEX_HOME}/config.toml"
if [ -f "$config" ]; then
  configured=$(grep -E '^[[:space:]]*model[[:space:]]*=' "$config" 2>/dev/null | head -1 | sed 's/.*=[[:space:]]*//; s/"//g')
  effort=$(grep -E '^[[:space:]]*model_reasoning_effort[[:space:]]*=' "$config" 2>/dev/null | head -1 | sed 's/.*=[[:space:]]*//; s/"//g')
  if [ -n "${configured:-}" ]; then
    ok "config default model — ${configured}" "config_model"
  else
    note "no default model in config.toml" "config_model"
  fi
  if [ -n "${effort:-}" ]; then
    ok "config default effort — ${effort}" "config_effort"
  fi
else
  note "no ${config} — the skill passes -m/-c explicitly, so this is not fatal" "config_model"
fi

cache="${CODEX_HOME}/models_cache.json"
if [ -f "$cache" ]; then
  if grep -q "$MODEL" "$cache" 2>/dev/null; then
    ok "target model '${MODEL}' present in local model cache" "model_cache"
  else
    note "target model '${MODEL}' not in local model cache" "model_cache"
    hint "the cache may just be stale; the skill will surface a real error if the slug is wrong"
    hint "override the model per-run: /sol uses -m, so edit the SKILL.md command or pass your own"
  fi
else
  note "no model cache yet — run codex once interactively to populate it" "model_cache"
fi

# 5. Git repo. Not required by Codex, but the skill's review step diffs the tree.
if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  ok "inside a git work tree — diff-based review will work" "git_work_tree"
  if [ -n "$(git status --porcelain 2>/dev/null)" ]; then
    note "working tree is dirty" "git_tree_state"
    hint "commit or stash first, so 'git diff' isolates exactly Sol's changes"
  else
    ok "working tree clean — Sol's diff will be isolated" "git_tree_state"
  fi
else
  note "not a git repo — the review phase cannot diff; commit history won't isolate changes" "git_work_tree"
fi

# 6. Parallel mode: the sibling worktree root must be creatable
if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  repo_root="$(git rev-parse --show-toplevel)"
  wt_root="$(cd "$repo_root/.." && pwd)/.sol-worktrees/$(basename "$repo_root")"
  if mkdir -p "$wt_root" 2>/dev/null; then
    ok "parallel worktree root writable — $wt_root" "worktree_root"
    rmdir "$wt_root" 2>/dev/null
    rmdir "$(dirname "$wt_root")" 2>/dev/null
  else
    note "cannot create $wt_root — parallel mode (--workers) will not run" "worktree_root"
    hint "single-worker /sol is unaffected"
  fi
fi

if [ "$json" -eq 1 ]; then
  print_json
  if [ "$fail" -gt 0 ]; then
    exit 1
  fi
  exit 0
fi

printf '\n'
if [ "$fail" -gt 0 ]; then
  printf '%s check(s) failed, %s warning(s). Fix the failures before running /sol.\n\n' "$fail" "$warn"
  exit 1
fi
printf 'Ready. %s warning(s).\n\n' "$warn"

#!/usr/bin/env bash
# Fan-out launcher for the /sol skill's parallel mode. Runs one `codex exec`
# worker per brief, each in its own git worktree and branch.
#
# Usage:
#   sol-parallel.sh [--workers N] <run-dir>     launch and wait
#   sol-parallel.sh --dry-run   <run-dir>       create and bootstrap worktrees, then
#                                                stop before launching any Codex session
#                                                — inspect the setup before spending runs
#   sol-parallel.sh --wait      <run-dir>       re-attach to a running batch
#   sol-parallel.sh --resume    <run-dir>       send correction briefs
#   sol-parallel.sh --cleanup   <run-dir>       remove merged worktrees/branches
#
# --in-place runs a single brief in the repo itself: no worktree, no branch, no
# commit, changes left in the working tree exactly as a plain single-worker run
# leaves them -- but with the inactivity watchdog and stall retries that a bare
# `codex exec` has no way to get.
#
# A brief may name reference images in a sidecar next to it:
#   <run-dir>/tasks/01-<slug>.images   one path per line, '#' comments ignored
# Each is passed to that worker as `codex exec -i`. Missing paths fail the run
# at preflight, before any worktree is created.
#
# Exit: 0 all ok · 1 a worker failed · 2 precondition/usage · 75 still running
#
# Env: SOL_MAX_WORKERS (default 3)   ceiling on --workers
#      SOL_WORKTREE_SETUP            command run in each fresh worktree
#      SOL_WORKER_TIMEOUT (0)        absolute per-worker wall-clock cap, seconds.
#                                    0 = none, the default: a whole task's
#                                    duration is not predictable, so any
#                                    constant kills productive workers. The
#                                    budgets below bound silence instead.
#      SOL_EFFORT (high)             reasoning effort every worker launches at.
#                                    `high` rather than `xhigh`: stalls are
#                                    effort-correlated (openai/codex#24260,
#                                    #23807), and with a compiler and tests in
#                                    the loop `high` verifies its own work. An
#                                    xhigh stall costs the whole first-event
#                                    budget before anything happens at all —
#                                    observed: an xhigh worker sat 900s with no
#                                    tool call, the same brief at high made its
#                                    first call in 36s. Set xhigh explicitly for
#                                    algorithmically hard briefs.
#      SOL_FIRST_EVENT_TIMEOUT (300) kill a worker whose event log still holds
#                                    nothing but thread/turn bookkeeping after
#                                    this many seconds (codex hangs at high
#                                    reasoning effort emit exactly that shape).
#                                    A real think before the first file read
#                                    rarely runs past five minutes; the old 900
#                                    turned one stall into fifteen lost minutes.
#      SOL_IDLE_TIMEOUT (600)        kill a worker whose event log has gone
#                                    this many seconds without a new event
#      SOL_COMMAND_TIMEOUT (1800)    kill a worker whose in-flight command has
#                                    produced nothing for this long. Bounds the
#                                    exemption that lets long builds run silent.
#      SOL_STALL_RETRIES (1)         automatic relaunches of a stalled worker,
#                                    each one reasoning-effort step lower
#      SOL_SANDBOX (workspace-write) sandbox policy passed as `-s`. Set to
#                                    danger-full-access for a toolchain the
#                                    sandbox cannot reach at all — Docker, whose
#                                    daemon socket is a unix socket connect that
#                                    neither network_access nor writable_roots
#                                    unblocks. Removes ALL confinement, so a
#                                    worker can write anywhere on disk; the run
#                                    warns on stderr whenever it is not default.
#                                    Cannot be set via SOL_CODEX_CONFIG: an
#                                    explicit -s beats -c sandbox_mode=.
#      SOL_CODEX_CONFIG              extra `-c key=value` overrides, space
#                                    separated, applied to every codex
#                                    invocation this script makes. The escape
#                                    hatch for toolchains the default sandbox
#                                    cannot run: a worker that cannot compile
#                                    falls back to grep, and grep-shaped
#                                    verification misses what a compiler
#                                    catches. Values must not contain spaces.
#                                    Deliberately unset by default — relaxing
#                                    the sandbox is the caller's call to make,
#                                    per repo, never this script's.

set -uo pipefail

MODEL="${SOL_MODEL:-gpt-5.6-sol}"
EFFORT="${SOL_EFFORT:-high}"
WORKER_TIMEOUT="${SOL_WORKER_TIMEOUT:-0}"        # 0 = no absolute cap
FIRST_EVENT_TIMEOUT="${SOL_FIRST_EVENT_TIMEOUT:-300}"
IDLE_TIMEOUT="${SOL_IDLE_TIMEOUT:-600}"
COMMAND_TIMEOUT="${SOL_COMMAND_TIMEOUT:-1800}"
STALL_RETRIES="${SOL_STALL_RETRIES:-1}"

# Exported rather than passed positionally: all three codex invocations below
# run inside `nohup bash -c '...'`, and the environment is the one channel that
# reaches every one of them without renumbering their positional arguments.
export SOL_CODEX_CONFIG="${SOL_CODEX_CONFIG:-}"

# The sandbox policy passed as `-s`. Separate from SOL_CODEX_CONFIG because an
# explicit `-s` flag beats `-c sandbox_mode=...`: setting the mode through the
# config channel is silently ignored, which reads as "the escape hatch does not
# work" rather than "wrong channel". Exported for the same reason as above.
export SOL_SANDBOX="${SOL_SANDBOX:-workspace-write}"
if [ "$SOL_SANDBOX" != "workspace-write" ]; then
  printf 'sol-parallel: sandbox is %s, not workspace-write — workers can write outside the workspace\n' \
    "$SOL_SANDBOX" >&2
fi

die() { printf 'sol-parallel: %s\n' "$1" >&2; exit "${2:-2}"; }

# BSD stat (macOS) then GNU stat; 0 for a missing file so age math never
# explodes — callers treat 0 as "no heartbeat yet".
# GNU stat first, and validate the result is numeric. BSD `stat -f %m` gives the
# mtime, but GNU's `-f` means --file-system: it prints a filesystem dump and
# EXITS 0, so a BSD-first `||` chain never falls through on Linux. The garbage
# then reached `[ "$last" -gt 0 ]` as a syntax error and silently disabled the
# entire inactivity watchdog on every Linux host.
mtime_of() {
  local m
  m="$(stat -c %Y "$1" 2>/dev/null)"
  case "$m" in ''|*[!0-9]*) m="$(stat -f %m "$1" 2>/dev/null)" ;; esac
  case "$m" in ''|*[!0-9]*) m=0 ;; esac
  printf '%s' "$m"
}

# One reasoning-effort step down. Stalls are empirically effort-correlated
# (openai/codex#24260, #23807), so a stalled worker retries lower, never equal.
next_effort() {
  case "$1" in
    xhigh)  echo high ;;
    high)   echo medium ;;
    medium) echo low ;;
    *)      echo "$1" ;;
  esac
}

# thread.*/turn.* (and session bookkeeping) arrive before codex does any real
# work; a log holding only those is a worker that has not started. Anything
# else — item events, commands, even errors — is evidence of life.
has_substantive_event() {
  [ -s "$1" ] || return 1
  grep -qvE '"type"[[:space:]]*:[[:space:]]*"(thread\.|turn\.|session)' "$1"
}

# True when the log's last state includes a command execution or MCP tool call
# that started and has not completed: codex emits nothing while a command runs,
# so this silence is a build in progress, not a hang. The idle budget must not
# apply — a command that never returns is SOL_COMMAND_TIMEOUT's job, which
# bounds this exemption. Matching is by item id, so interleaved items resolve
# correctly.
in_flight_item() {
  [ -s "$1" ] || return 1
  python3 - "$1" <<'PY'
import json, sys
started, done = set(), set()
with open(sys.argv[1]) as fh:
    for line in fh:
        try:
            e = json.loads(line)
        except Exception:
            continue
        item = e.get("item") or {}
        if item.get("type") not in ("command_execution", "mcp_tool_call"):
            continue
        t = e.get("type")
        if t == "item.started":
            started.add(item.get("id"))
        elif t == "item.completed":
            done.add(item.get("id"))
sys.exit(0 if started - done else 1)
PY
}

# Signal the whole process group: the wrapper forked `codex`, so killing the
# wrapper alone leaves the real worker running. Single-pid fallback for shells
# that reject the group form.
kill_worker_group() { kill -9 -- -"$1" 2>/dev/null || kill -9 "$1" 2>/dev/null; }

MODE="launch"
WORKERS=""
RUN_DIR=""
DRY_RUN=0
IN_PLACE=0
while [ $# -gt 0 ]; do
  case "$1" in
    --workers) [ $# -ge 2 ] || die "--workers requires a value"
               WORKERS="$2"; shift 2 ;;
    --dry-run) DRY_RUN=1; shift ;;
    --in-place) IN_PLACE=1; shift ;;
    --wait)    MODE="wait";    shift ;;
    --resume)  MODE="resume";  shift ;;
    --cleanup) MODE="cleanup"; shift ;;
    -h|--help) sed -n '2,17p' "$0"; exit 0 ;;
    -*)        die "unknown option: $1" ;;
    *)         RUN_DIR="$1"; shift ;;
  esac
done

[ -n "$RUN_DIR" ] || die "usage: sol-parallel.sh [--workers N] <run-dir>"
git rev-parse --is-inside-work-tree >/dev/null 2>&1 \
  || die "not inside a git work tree"

REPO_ROOT="$(git rev-parse --show-toplevel)"
REPO_NAME="$(basename "$REPO_ROOT")"
WORKTREE_ROOT="$(cd "$REPO_ROOT/.." && pwd)/.sol-worktrees/$REPO_NAME"
TASKS_DIR="$RUN_DIR/tasks"
OUT_DIR="$RUN_DIR/workers"

# --in-place is a property of the RUN, not of the invocation that re-attaches to
# it. Every path below that needs a workspace used to recompute
# "$WORKTREE_ROOT/$slug", which does not exist for an in-place run: the stall
# relaunch ran `codex exec -C <nonexistent>` and died in two seconds, --resume
# exited 2 instantly, and --cleanup pointed at a worktree that was never
# created. The marker means `--wait`, `--resume` and `--cleanup` behave without
# the caller retyping the flag -- which nothing in the output ever prompted for.
[ -f "$RUN_DIR/in-place" ] && IN_PLACE=1

# The recorded workspace of one worker. Recorded beats recomputed for the same
# reason `brief` is a file rather than a glob (see create_worktrees): the run
# knows where it put the worker, and later invocations can only guess. The
# fallbacks cover a run directory written before `worktree` existed.
worktree_of() {
  local path
  path="$(cat "$OUT_DIR/$1/worktree" 2>/dev/null)"
  if [ -n "$path" ]; then
    printf '%s' "$path"
  elif [ "$IN_PLACE" -eq 1 ]; then
    printf '%s' "$REPO_ROOT"
  else
    printf '%s' "$WORKTREE_ROOT/$1"
  fi
}

preflight_launch() {
  local ceiling="${SOL_MAX_WORKERS:-3}"
  [ -d "$TASKS_DIR" ] || die "no briefs: $TASKS_DIR does not exist"

  local briefs=()
  local f
  for f in "$TASKS_DIR"/*.md; do
    [ -e "$f" ] && briefs+=("$f")
  done
  [ "${#briefs[@]}" -gt 0 ] || die "no briefs: $TASKS_DIR/*.md matched nothing"

  [ -n "$WORKERS" ] || WORKERS="$ceiling"
  case "$WORKERS" in ''|*[!0-9]*) die "--workers must be a positive integer" ;; esac
  [ "$WORKERS" -ge 1 ] || die "--workers must be at least 1"

  if [ "$WORKERS" -gt "$ceiling" ]; then
    die "requested $WORKERS workers but the ceiling is $ceiling; raise it with SOL_MAX_WORKERS"
  fi
  if [ "${#briefs[@]}" -gt "$WORKERS" ]; then
    die "${#briefs[@]} briefs exceed $WORKERS workers; run one wave per batch, merging between waves"
  fi

  if [ "$IN_PLACE" -eq 1 ]; then
    # No worktree means no isolation: two workers editing one tree corrupt each
    # other. In-place exists to give one ordinary run the watchdog, not to
    # parallelise.
    [ "$WORKERS" -eq 1 ] || die "--in-place runs in the repo itself and takes exactly one worker"
    [ "${#briefs[@]}" -eq 1 ] || die "--in-place takes exactly one brief, found ${#briefs[@]}"
  fi

  command -v codex >/dev/null 2>&1 || die "codex not found on PATH"
  [ -z "$(git status --porcelain)" ] \
    || die "working tree is dirty; commit or stash so each worker branches from a clean HEAD"

  # Each brief may carry a sidecar naming reference images: `NN-<slug>.images`,
  # one path per line, `#` comments and blanks ignored. Resolved and validated
  # here so a typo fails the run before any worktree exists, rather than after
  # a worker has already burned minutes on a brief referring to a missing file.
  IMAGES_OF=()
  local sidecar resolved line abs
  for f in "${briefs[@]}"; do
    sidecar="${f%.md}.images"
    resolved=""
    if [ -f "$sidecar" ]; then
      while IFS= read -r line || [ -n "$line" ]; do
        case "$line" in ''|'#'*) continue ;; esac
        abs="$line"
        case "$abs" in /*) ;; *) abs="$REPO_ROOT/$abs" ;; esac
        [ -f "$abs" ] || die "$(basename "$sidecar"): image not found: $line"
        resolved+="$abs"$'\n'
      done < "$sidecar"
    fi
    IMAGES_OF+=("$resolved")
  done

  BRIEFS=("${briefs[@]}")
}

slug_for() {
  local base
  base="$(basename "$1" .md)"
  base="${base#[0-9][0-9]-}"
  base="$(printf '%s' "$base" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9' '-')"
  base="${base#-}"; base="${base%-}"
  base="${base:0:32}"
  # A brief name that reduces to nothing must not yield the branch `sol/`.
  [ -n "$base" ] || base="task"
  printf '%s' "$base"
}

bootstrap_worktree() {
  local wt="$1" slug="$2" f
  for f in "$REPO_ROOT"/.env "$REPO_ROOT"/.env.*; do
    [ -f "$f" ] || continue
    case "$(basename "$f")" in .env.example|.env.sample) continue ;; esac
    git -C "$REPO_ROOT" check-ignore -q "$f" || continue
    cp "$f" "$wt/$(basename "$f")" || return 1
  done
  if [ -n "${SOL_WORKTREE_SETUP:-}" ]; then
    ( cd "$wt" && eval "$SOL_WORKTREE_SETUP" ) \
      >"$OUT_DIR/$slug/setup.log" 2>&1 || return 1
  fi
  return 0
}

create_worktrees() {
  local branch sha brief slug wt
  branch="$(git rev-parse --abbrev-ref HEAD)"
  sha="$(git rev-parse HEAD)"
  mkdir -p "$OUT_DIR"
  [ "$IN_PLACE" -eq 1 ] || mkdir -p "$WORKTREE_ROOT"
  printf '%s\t%s\n' "$branch" "$sha" > "$RUN_DIR/base"
  if [ "$IN_PLACE" -eq 1 ]; then
    : > "$RUN_DIR/in-place"
  else
    rm -f "$RUN_DIR/in-place"
  fi

  SLUGS=(); WORKTREES=(); BRIEF_OF=()
  for brief in "${BRIEFS[@]}"; do
    local stem candidate n=2
    stem="$(slug_for "$brief")"
    candidate="$stem"
    # "${SLUGS[@]:-}" on an empty array substitutes one empty word, which would
    # match an empty candidate; guard on length instead.
    while [ "${#SLUGS[@]}" -gt 0 ] && printf '%s\n' "${SLUGS[@]}" | grep -qx "$candidate"; do
      candidate="$stem-$n"; n=$((n + 1))
    done
    slug="$candidate"
    if [ "$IN_PLACE" -eq 0 ] \
       && git show-ref --verify --quiet "refs/heads/sol/$slug"; then
      die "branch sol/$slug already exists; delete it or rename the brief"
    fi
    SLUGS+=("$slug"); BRIEF_OF+=("$brief")
    if [ "$IN_PLACE" -eq 1 ]; then
      WORKTREES+=("$REPO_ROOT")
    else
      WORKTREES+=("$WORKTREE_ROOT/$slug")
    fi
  done

  # The authoritative roster of the run, in task order, written before anything
  # can fail. `pids.all` is not a substitute: launch_workers only records
  # workers it actually launched, so a failed-setup worker never appears there
  # and every re-attach path that rehydrated from it dropped the worker --
  # and with it the run's failure -- entirely.
  printf '%s\n' "${SLUGS[@]}" > "$RUN_DIR/roster"

  local i status=0
  for i in "${!SLUGS[@]}"; do
    slug="${SLUGS[i]}"; wt="${WORKTREES[i]}"
    mkdir -p "$OUT_DIR/$slug"
    # Record the brief now. Recovering it later by globbing `*-<slug>.md` is
    # ambiguous: brief `01-add-auth.md` also matches slug `auth`.
    printf '%s\n' "${BRIEF_OF[i]}" > "$OUT_DIR/$slug/brief"
    # And the workspace, for the same reason: --wait, --resume, --cleanup and
    # the stall relaunch all need to know where this worker runs, and only the
    # launch knows whether that is a worktree or the repo itself.
    printf '%s\n' "$wt" > "$OUT_DIR/$slug/worktree"
    printf '%s' "${IMAGES_OF[i]:-}" > "$OUT_DIR/$slug/images"
    if [ "$IN_PLACE" -eq 1 ]; then
      # The repo is the workspace. Nothing to create, nothing to bootstrap:
      # gitignored files and dependencies are already here, which is the whole
      # reason this mode is cheaper than a worktree for a single run.
      continue
    fi
    git worktree add -q -b "sol/$slug" "$wt" HEAD \
      || die "could not create worktree for $slug"
    if ! bootstrap_worktree "$wt" "$slug"; then
      printf 'failed-setup\n' > "$OUT_DIR/$slug/status"
      printf 'sol-parallel: %s: failed-setup (see %s)\n' \
        "$slug" "$OUT_DIR/$slug/setup.log" >&2
      status=1
    fi
  done
  return "$status"
}

launch_workers() {
  local i slug wt brief w pid
  # Job control puts each background job in its own process group, so the
  # timeout backstop can signal the whole group. Without it, killing the
  # wrapper orphans the `codex` process it forked and the backstop is a no-op.
  set -m
  : > "$RUN_DIR/pids"
  for i in "${!SLUGS[@]}"; do
    slug="${SLUGS[i]}"; wt="${WORKTREES[i]}"; brief="${BRIEF_OF[i]}"
    w="$OUT_DIR/$slug"
    [ -f "$w/status" ] && continue        # failed-setup: never launched
    date +%s > "$w/started-at"
    nohup bash -c '
      img=()
      if [ -s "$4/images" ]; then
        while IFS= read -r p; do [ -n "$p" ] && img+=(-i "$p"); done < "$4/images"
      fi
      cfg=(); for kv in $SOL_CODEX_CONFIG; do cfg+=(-c "$kv"); done
      codex exec --json -m "$1" -c model_reasoning_effort="$2" \
        ${cfg[@]+"${cfg[@]}"} \
        -s "$SOL_SANDBOX" --color never -C "$3" \
        ${img[@]+"${img[@]}"} \
        -o "$4/report.md" - < "$5" \
        > "$4/events.jsonl" 2> "$4/stderr.txt"
      printf "%s\n" "$?" > "$4/exit-code"
    ' _ "$MODEL" "$EFFORT" "$wt" "$w" "$brief" >/dev/null 2>&1 &
    pid=$!
    disown "$pid" 2>/dev/null
    printf '%s\t%s\n' "$slug" "$pid" >> "$RUN_DIR/pids"
  done
  cp "$RUN_DIR/pids" "$RUN_DIR/pids.all"
}

# A stalled worker (exit-code 125, no status yet) is relaunched with the same
# brief in the same worktree — fresh session, one reasoning-effort step lower.
# Fresh session, not `resume`: the stalled session's transport state is exactly
# what cannot be trusted. The worktree is left as the stalled attempt left it;
# briefs describe an end state, so a partial attempt is a head start, not a
# hazard. Prior attempt logs are kept as *-attempt-N files.
#
# Returns 0 if anything was relaunched.
relaunch_stalled() {
  local round="$1" slug w wt brief pid n eff relaunched=0 stalled=()
  # Scan before touching anything: truncating `pids` on a run with no stalls
  # would erase the just-finished workers' records, which --wait re-attach and
  # the pid-per-worker bookkeeping still depend on.
  while read -r slug; do
    [ -n "$slug" ] || continue
    w="$OUT_DIR/$slug"
    [ -f "$w/status" ] && continue                       # already classified
    [ "$(cat "$w/exit-code" 2>/dev/null)" = "125" ] || continue
    stalled+=("$slug")
  done < <(roster)
  [ "${#stalled[@]}" -gt 0 ] || return 1

  : > "$RUN_DIR/pids"
  for slug in "${stalled[@]}"; do
    w="$OUT_DIR/$slug"
    wt="$(worktree_of "$slug")"
    brief="$(cat "$w/brief" 2>/dev/null)"
    [ -f "$brief" ] || { printf 'sol-parallel: %s: brief missing, cannot relaunch\n' "$slug" >&2; continue; }
    # Say which directory and why. Launching anyway hands codex a `-C` it cannot
    # chdir into: it exits in about two seconds with a bare "No such file or
    # directory", the retry lands in summary.json as failed-launch with an
    # elapsed of 2, and the reviewer goes looking for a broken invocation.
    [ -d "$wt" ] || { printf 'sol-parallel: %s: workspace %s missing, cannot relaunch\n' "$slug" "$wt" >&2; continue; }

    n=1
    while [ -e "$w/events-attempt-$n.jsonl" ]; do n=$((n + 1)); done
    mv "$w/events.jsonl" "$w/events-attempt-$n.jsonl" 2>/dev/null
    mv "$w/stderr.txt"   "$w/stderr-attempt-$n.txt"   2>/dev/null
    mv "$w/report.md"    "$w/report-attempt-$n.md"    2>/dev/null
    mv "$w/stall-reason" "$w/stall-reason-attempt-$n" 2>/dev/null

    eff="$(next_effort "$(cat "$w/effort" 2>/dev/null || echo "$EFFORT")")"
    printf '%s\n' "$eff" > "$w/effort"
    printf '%s\n' "$round" > "$w/stall-retries"
    printf 'sol-parallel: %s: relaunching after stall (attempt %d, effort %s)\n' \
      "$slug" $((n + 1)) "$eff" >&2

    rm -f "$w/exit-code"
    date +%s > "$w/started-at"
    nohup bash -c '
      cfg=(); for kv in $SOL_CODEX_CONFIG; do cfg+=(-c "$kv"); done
      codex exec --json -m "$1" -c model_reasoning_effort="$2" \
        ${cfg[@]+"${cfg[@]}"} \
        -s "$SOL_SANDBOX" --color never -C "$3" \
        -o "$4/report.md" - < "$5" \
        > "$4/events.jsonl" 2> "$4/stderr.txt"
      printf "%s\n" "$?" > "$4/exit-code"
    ' _ "$MODEL" "$eff" "$wt" "$w" "$brief" >/dev/null 2>&1 &
    pid=$!
    disown "$pid" 2>/dev/null
    printf '%s\t%s\n' "$slug" "$pid" >> "$RUN_DIR/pids"
    relaunched=1
  done
  [ "$relaunched" -eq 1 ]
}

resume_workers() {
  local slug w wt pid n pending=0
  : > "$RUN_DIR/pids"
  while read -r slug; do
    [ -n "$slug" ] || continue
    w="$OUT_DIR/$slug"
    [ -f "$w/correction.md" ] || continue
    wt="$(worktree_of "$slug")"
    if [ ! -d "$wt" ]; then
      # The resume wrapper's `cd "$3" || exit 2` would otherwise fail silently
      # into an exit 2 with nothing written anywhere explaining it.
      printf 'sol-parallel: %s: workspace %s missing, cannot resume\n' "$slug" "$wt" >&2
      continue
    fi
    if [ ! -s "$w/session-id" ]; then
      # Never launched (failed-setup) or its event log was empty, so there is
      # no session to resume. `codex exec resume ""` would be nonsense.
      printf 'sol-parallel: %s: no session id, cannot resume\n' "$slug" >&2
      continue
    fi
    pending=$((pending + 1))
    n=1
    while [ -f "$w/correction-$n.md" ]; do n=$((n + 1)); done
    mv "$w/correction.md" "$w/correction-$n.md"
    date +%s > "$w/started-at"
    # Clear the terminal state. `post_process` skips any worker that already
    # has a `status` file — the guard that stops a re-attach from downgrading a
    # committed worker to `no-changes` — so a resumed worker that kept its old
    # status would be skipped forever and never reclassified.
    rm -f "$w/exit-code" "$w/status" "$w/files-changed" "$w/commit"
    nohup bash -c '
      cd "$3" || exit 2
      # </dev/null: codex exec hangs forever on an open pipe stdin with no
      # writer (openai/codex#20919); the launch path is safe because it reads
      # the brief from stdin, but resume passes the prompt as an argument.
      cfg=(); for kv in $SOL_CODEX_CONFIG; do cfg+=(-c "$kv"); done
      codex exec resume "$6" --json -m "$1" -c model_reasoning_effort="$2" \
        ${cfg[@]+"${cfg[@]}"} \
        -o "$4/report.md" "$(cat "$5")" \
        > "$4/events.jsonl" 2> "$4/stderr.txt" < /dev/null
      printf "%s\n" "$?" > "$4/exit-code"
    ' _ "$MODEL" "$EFFORT" "$wt" "$w" "$w/correction-$n.md" \
        "$(cat "$w/session-id")" >/dev/null 2>&1 &
    pid=$!
    disown "$pid" 2>/dev/null
    printf '%s\t%s\n' "$slug" "$pid" >> "$RUN_DIR/pids"
  done < <(roster)
  [ "$pending" -gt 0 ] || die "no correction.md found in $OUT_DIR/*/"
}

worker_slugs() { cut -f1 "$RUN_DIR/pids"; }
pid_of() { awk -F'\t' -v s="$1" '$1 == s { print $2 }' "$RUN_DIR/pids"; }

# The full roster of the run, as opposed to worker_slugs() which is only the
# batch currently being waited on. These differ under --resume, where `pids` is
# rewritten to just the corrected workers; rehydrating from `pids` there would
# silently drop every other worker from summary.json.
#
# Prefer `roster`, written by create_worktrees. `pids.all` holds only the
# workers that were actually LAUNCHED, so a failed-setup worker is missing from
# it: --wait and --resume rehydrated a short roster, wrote a summary.json with
# the failed worker erased, and exited 0 after a launch that had correctly
# exited 1. Per parallel-flow.md §5 the re-attach path is the normal one, so
# that was the common case, not an edge case. Fall back to `pids.all` and then
# `pids` for a run directory created before `roster` existed.
#
# Deliberately NOT cleanup_slugs(): that walks OUT_DIR and sorts, which loses
# task order. summary.json's worker array is contractually in task order.
roster() {
  if   [ -s "$RUN_DIR/roster" ];   then cat "$RUN_DIR/roster"
  elif [ -f "$RUN_DIR/pids.all" ]; then cut -f1 "$RUN_DIR/pids.all"
  else worker_slugs; fi
}

rehydrate() {
  local slug
  SLUGS=(); WORKTREES=(); BRIEF_OF=()
  while read -r slug; do
    [ -n "$slug" ] || continue
    SLUGS+=("$slug")
    WORKTREES+=("$(worktree_of "$slug")")
    BRIEF_OF+=("$(cat "$OUT_DIR/$slug/brief" 2>/dev/null)")
  done < <(roster)
}

post_process() {
  local i slug wt w code status session commit files
  for i in "${!SLUGS[@]}"; do
    slug="${SLUGS[i]}"; wt="${WORKTREES[i]}"; w="$OUT_DIR/$slug"

    # Idempotent: a worker already carries a status either from failed-setup
    # (create_worktrees, before any launch) or from a prior post_process pass.
    # The latter matters for --wait: if the original launcher wasn't actually
    # killed by its caller's timeout and ran to completion on its own, it
    # already classified and committed this worker. Re-running the commit
    # logic here would find a clean worktree (already committed) and
    # downgrade a real "ok" to "no-changes".
    if [ -f "$w/status" ]; then
      continue
    fi

    code="$(cat "$w/exit-code" 2>/dev/null || echo 1)"
    session=""; commit=""; files=""

    if [ "$code" = "125" ] || [ "$code" = "124" ]; then
      # Killed by a watchdog (125) or the optional absolute cap (124). Both are
      # classified before the empty-events check: a worker killed before its
      # very first event (a stall on the codex stdin hang, openai/codex#20919,
      # or a cap that expired mid-think) has an empty log but was not a failed
      # launch. Reporting it as one sends the reviewer hunting a bad invocation
      # or a missing binary, and for 125 it also decides the retry ladder.
      if [ "$code" = "125" ]; then status="stalled"; else status="timed-out"; fi
      if [ -s "$w/events.jsonl" ]; then
        session="$(head -1 "$w/events.jsonl" \
          | python3 -c 'import json,sys; print(json.loads(sys.stdin.readline() or "{}").get("thread_id",""))' \
          2>/dev/null)"
        printf '%s\n' "$session" > "$w/session-id"
      fi
    elif [ ! -s "$w/events.jsonl" ]; then
      status="failed-launch"
    else
      session="$(head -1 "$w/events.jsonl" \
        | python3 -c 'import json,sys; print(json.loads(sys.stdin.readline() or "{}").get("thread_id",""))' \
        2>/dev/null)"
      printf '%s\n' "$session" > "$w/session-id"
      if [ "$code" = "124" ]; then
        status="timed-out"
      elif [ "$code" != "0" ]; then
        status="failed-run"
      elif [ -z "$(git -C "$wt" status --porcelain)" ] \
        && { [ "$IN_PLACE" -eq 1 ] \
             || [ "$(git -C "$wt" rev-parse HEAD)" = "$(cut -f2 "$RUN_DIR/base")" ]; }; then
        status="no-changes"
      elif [ -z "$(git -C "$wt" status --porcelain)" ]; then
        # Clean tree but the branch has already moved past base: a resumed
        # worker that committed earlier and made no further edits this round.
        # There is nothing new to add/commit -- doing so anyway would find
        # "nothing to commit" and misreport a real "ok" as failed-commit.
        status="ok"
        commit="$(git -C "$wt" rev-parse HEAD)"
      else
        status="ok"
      fi
    fi

    if [ "$IN_PLACE" -eq 1 ] && [ "$status" = "ok" ]; then
      # Deliberately no commit. In-place mirrors the single-worker flow: the
      # changes stay in the working tree for review, and the reviewer decides
      # what to do with them. Committing here would silently change what a
      # plain /sol leaves behind.
      files="$(git -C "$wt" status --porcelain | sed 's/^...//')"
    elif [ "$status" = "ok" ] && [ -z "$commit" ]; then
      git -C "$wt" add -A >/dev/null 2>&1
      if git -C "$wt" commit -q -m "sol: $slug" >/dev/null 2>&1; then
        commit="$(git -C "$wt" rev-parse HEAD)"
      else
        # `rev-parse HEAD` still succeeds when the commit was rejected (a hook,
        # a bad identity), returning the BASE sha — which looks like a real
        # commit and reported the worker as ok with its work stranded uncommitted.
        status="failed-commit"
        commit=""
      fi
    fi

    # Cumulative since base, computed from the commit rather than from the
    # pre-commit porcelain: the resumed-no-op rung never touches the worktree,
    # and porcelain quotes "unusual" filenames -- notably one containing a
    # double quote, which `-c core.quotePath=false` alone does NOT unquote;
    # that setting only stops quoting for non-ASCII bytes, while git always
    # backslash-escapes literal double quotes in its default text output. `-z`
    # gives NUL-delimited, unquoted names regardless of content; translating
    # NUL to newline matches how files-changed is already stored and parsed.
    # This is the branch's whole diff, which is what a reviewer of it wants.
    if [ "$status" = "ok" ] && [ "$IN_PLACE" -eq 0 ]; then
      files="$(git -C "$wt" diff --name-only -z \
        "$(cut -f2 "$RUN_DIR/base")" HEAD | tr '\0' '\n')"
    else
      # A non-ok worker can still have produced real work, and reporting nothing
      # for it is how that work went missing: failed-commit stages everything and
      # then has its commit rejected, failed-run and timed-out leave it in the
      # worktree untouched. With files_changed empty, summary.json gave the user
      # no way to find any of it -- detected, then never reported. Snapshot
      # everything the worktree holds that base does not: committed since base,
      # staged or unstaged against HEAD, and untracked. `sort -u` because the
      # three sources overlap; `-z | tr` for the same bare-filename reasons as
      # the ok path above.
      files="$( { git -C "$wt" diff --name-only -z \
                    "$(cut -f2 "$RUN_DIR/base")" HEAD 2>/dev/null
                  git -C "$wt" diff --name-only -z HEAD 2>/dev/null
                  git -C "$wt" ls-files -o --exclude-standard -z 2>/dev/null
                } | tr '\0' '\n' | LC_ALL=C sort -u)"
    fi

    if [ -f "$w/started-at" ]; then
      printf '%s\n' "$(( $(date +%s) - $(cat "$w/started-at") ))" > "$w/elapsed"
    fi

    # `status` last: the re-attach idempotency guard treats its presence as
    # "already processed", so an interruption after it but before these two
    # would strand the worker with a correct status and empty metadata.
    printf '%s\n' "$files" > "$w/files-changed"
    printf '%s\n' "$commit" > "$w/commit"
    printf '%s\n' "$status" > "$w/status"
  done
}

write_summary() {
  RUN_DIR="$RUN_DIR" OUT_DIR="$OUT_DIR" \
  SLUG_LIST="$(printf '%s\n' "${SLUGS[@]}")" \
  WT_LIST="$(printf '%s\n' "${WORKTREES[@]}")" \
  BRIEF_LIST="$(printf '%s\n' "${BRIEF_OF[@]}")" \
  EFFORT_DEFAULT="$EFFORT" \
  python3 - <<'PY'
import json, os, pathlib

run_dir = pathlib.Path(os.environ["RUN_DIR"])
out_dir = pathlib.Path(os.environ["OUT_DIR"])
slugs = os.environ["SLUG_LIST"].split("\n")
wts = os.environ["WT_LIST"].split("\n")
briefs = os.environ["BRIEF_LIST"].split("\n")

def read(p, default=""):
    try:
        return p.read_text().strip()
    except OSError:
        return default

base_branch, _, base_sha = read(run_dir / "base").partition("\t")
workers = []
for slug, wt, brief in zip(slugs, wts, briefs):
    w = out_dir / slug
    started = read(w / "started-at")
    files = [f for f in read(w / "files-changed").split("\n") if f]
    workers.append({
        "slug": slug,
        "branch": f"sol/{slug}",
        "worktree": wt,
        "brief": brief,
        "status": read(w / "status", "failed-launch") or "failed-launch",
        "exit_code": int(read(w / "exit-code") or 1),
        "session_id": read(w / "session-id"),
        "commit": read(w / "commit"),
        "files_changed": files,
        "elapsed_seconds": int(read(w / "elapsed") or 0),
        "effort_used": read(w / "effort") or os.environ.get("EFFORT_DEFAULT", ""),
        "stall_retries": int(read(w / "stall-retries") or 0),
        "stall_reason": read(w / "stall-reason"),
        "events_path": str(w / "events.jsonl"),
        "report_path": str(w / "report.md"),
        "stderr_path": str(w / "stderr.txt"),
    })

(run_dir / "summary.json").write_text(json.dumps(
    {"base_branch": base_branch, "base_sha": base_sha, "workers": workers},
    indent=2) + "\n")
PY
}

wait_for_workers() {
  local block="$1" slug pid started now live stall_reason ev last
  while :; do
    live=0
    while read -r slug; do
      [ -n "$slug" ] || continue
      # `-s`, not `-f`: the wrapper creates this file by redirection and fills it
      # an instant later, so an empty one means "not finished", not "exit 0".
      [ -s "$OUT_DIR/$slug/exit-code" ] && continue
      pid="$(pid_of "$slug")"
      if kill -0 "$pid" 2>/dev/null; then
        started="$(cat "$OUT_DIR/$slug/started-at" 2>/dev/null || echo 0)"
        now="$(date +%s)"
        if [ "$WORKER_TIMEOUT" -gt 0 ] && [ "$started" -gt 0 ] \
           && [ $((now - started)) -gt "$WORKER_TIMEOUT" ]; then
          kill_worker_group "$pid"
          # Same shape as the re-stat below: a worker that finished in the
          # instant between the liveness check and this kill has already
          # recorded its real result, and 124 would overwrite it with a
          # fabricated timeout.
          [ -s "$OUT_DIR/$slug/exit-code" ] \
            || printf '124\n' > "$OUT_DIR/$slug/exit-code"
          continue
        fi
        # Stall watchdog. The absolute cap above cannot tell a worker deep in
        # productive silence from one that hung after `turn.started` and will
        # never speak again (a known codex failure shape at high reasoning
        # effort: openai/codex#24260, #23807 — its internal stream retries can
        # sit silent for many minutes). The event log is the heartbeat:
        #   - nothing substantive yet → allow FIRST_EVENT_TIMEOUT from launch
        #     (a real think before the first item is silent, but minutes, not
        #     the quarter hour the old 900s budget waited out)
        #   - substantive events exist → allow IDLE_TIMEOUT since the last
        #     write of any kind (mtime advances with every event)
        # Exit code 125 marks the kill as a stall so post_process can class it
        # `stalled` and the retry ladder can tell it apart from `timed-out`.
        stall_reason=""
        ev="$OUT_DIR/$slug/events.jsonl"
        if has_substantive_event "$ev"; then
          last="$(mtime_of "$ev")"
          if [ "$last" -gt 0 ] && [ $((now - last)) -gt "$IDLE_TIMEOUT" ]; then
            if ! in_flight_item "$ev"; then
              stall_reason="no events for $((now - last))s (idle budget ${IDLE_TIMEOUT}s)"
            elif [ "$COMMAND_TIMEOUT" -gt 0 ] \
                 && [ $((now - last)) -gt "$COMMAND_TIMEOUT" ]; then
              # The in-flight exemption below is what lets a long build run in
              # silence. Unbounded, it also lets a command that will never
              # return run forever. A single command's runtime is predictable
              # in a way a whole task's is not, so this budget bounds the
              # exemption without capping the task.
              stall_reason="command in flight $((now - last))s with no output (command budget ${COMMAND_TIMEOUT}s)"
            fi
          fi
        elif [ "$started" -gt 0 ] && [ $((now - started)) -gt "$FIRST_EVENT_TIMEOUT" ]; then
          stall_reason="no substantive event $((now - started))s after launch (budget ${FIRST_EVENT_TIMEOUT}s)"
        fi
        if [ -n "$stall_reason" ]; then
          kill_worker_group "$pid"
          if [ ! -s "$OUT_DIR/$slug/exit-code" ]; then
            printf '%s\n' "$stall_reason" > "$OUT_DIR/$slug/stall-reason"
            printf '125\n' > "$OUT_DIR/$slug/exit-code"
            printf 'sol-parallel: %s: stalled — %s\n' "$slug" "$stall_reason" >&2
          fi
          continue
        fi
        live=$((live + 1))
      else
        # Re-stat before concluding the worker vanished. The wrapper writes
        # `exit-code` and only THEN exits, so a worker that finished between the
        # check at the top of this iteration and the `kill -0` just above is not
        # gone-without-a-result — it is done. Forking `pid_of` in between widens
        # that window enough to land in it routinely under load. Treating it as
        # a kill overwrote a real exit code with 137, which post_process then
        # classified `failed-run`: a fully successful worker reported as failed,
        # its work never committed and left stranded in the worktree, and the
        # run exiting 1. Cheap stat, and the only thing standing between a
        # finished worker and a fabricated failure.
        if [ -s "$OUT_DIR/$slug/exit-code" ]; then
          continue
        fi
        # Genuinely gone without recording an exit code: killed or interrupted.
        # The wrapper may have been killed alone (an operator's `kill -9`, an OOM
        # kill), leaving `codex` orphaned — reap the group for the same reason
        # the timeout branch does, with the same single-pid fallback for a shell
        # that rejects the group form. Residual risk: if the pid has been
        # recycled since we recorded it, this signals an unrelated group; the
        # window is small and the alternative is a worker that runs unobserved
        # forever.
        kill -9 -- -"$pid" 2>/dev/null || kill -9 "$pid" 2>/dev/null
        printf '137\n' > "$OUT_DIR/$slug/exit-code"
      fi
    done < <(worker_slugs)
    [ "$live" -eq 0 ] && return 0
    [ "$block" -eq 1 ] || return 75
    sleep 2
  done
}

# Integration here means cherry-pick, which produces a commit with a NEW sha and
# an identical patch. Ancestry therefore reports genuinely integrated work as
# unmerged, and only coincides when the cherry-pick lands in the same second as
# the original commit — which is why a same-second test passed and real use would
# not have. `git cherry` marks '+' any commit whose patch is not upstream.
#
# Fails CLOSED. Every "I don't know" answer must come back as "not integrated",
# because the only caller uses a true answer to delete a branch and force-remove
# a worktree. `grep -c` always prints a number, so gating on the output alone
# turned a `git cherry` that never ran (base branch renamed or deleted after the
# run — an ordinary merged-and-pruned action) into "0 unmerged commits" and thus
# into a silent, unrecoverable removal of un-integrated work.
branch_integrated() {
  local base="$1" branch="$2" unmerged cherry rc
  git show-ref --verify --quiet "refs/heads/$branch" || return 1
  # If base no longer resolves, nothing below can judge anything.
  git rev-parse --verify --quiet "$base^{commit}" >/dev/null 2>&1 || return 1
  git merge-base --is-ancestor "$branch" "$base" 2>/dev/null && return 0
  # Capture `git cherry`'s own exit status, not just its (possibly empty) output.
  # It has to be read before anything else runs: reading PIPESTATUS after
  # `unmerged="$(... | grep -c ...)"` would report grep's status, not git's.
  cherry="$(git cherry "$base" "$branch" 2>/dev/null)"; rc=$?
  [ "$rc" -eq 0 ] || return 1
  unmerged="$(printf '%s\n' "$cherry" | grep -c '^+')"
  [ "$unmerged" = "0" ]
}

# Every worker directory the run ever created, not just roster()'s
# pids.all-derived list. create_worktrees creates the worktree and branch for
# a failed-setup worker before bootstrap runs, but launch_workers never adds
# a failed-setup slug to pids/pids.all -- it was never launched. roster()
# alone would make that worker's real branch and worktree permanently
# invisible to --cleanup: not reported as kept, not removed, just silently
# unreachable forever. Every worker directory under OUT_DIR is a strict
# superset of roster(), so walking it covers both without touching roster()'s
# own contract (used elsewhere for --resume/--wait bookkeeping).
cleanup_slugs() {
  local d
  for d in "$OUT_DIR"/*/; do
    [ -d "$d" ] && basename "$d"
  done | sort -u
}

# Removes the worktree and branch for every worker whose branch is fully
# merged into base; prints one `kept:` line per survivor so nothing a worker
# produced is ever stranded without the caller being told it exists. When in
# doubt (branch missing, removal partially failing) this errs toward keeping
# and reporting rather than silently discarding.
cleanup_run() {
  local base slug wt rm_ok br_ok d wt_dirty removable
  base="$(cut -f1 "$RUN_DIR/base")"
  # Say so once, out loud. branch_integrated fails closed on an unresolvable
  # base, so everything is about to be kept — without this line the operator
  # sees a --cleanup that cleans nothing up and is told nothing about why.
  if ! git rev-parse --verify --quiet "$base^{commit}" >/dev/null 2>&1; then
    printf 'sol-parallel: base ref %s no longer resolves (renamed or deleted?); integration cannot be verified, so nothing will be removed\n' \
      "$base" >&2
  fi
  while read -r slug; do
    [ -n "$slug" ] || continue
    wt="$(worktree_of "$slug")"

    if ! git show-ref --verify --quiet "refs/heads/sol/$slug"; then
      # No branch to check merge-base against: already cleaned up by a prior
      # --cleanup run, or removed by hand. `git merge-base --is-ancestor` on a
      # missing branch also returns non-zero -- indistinguishable from "not
      # merged" -- which would otherwise fall into the "kept" branch below and
      # print a worktree path for a branch that no longer exists. Report only
      # if a worktree is still orphaned there; otherwise there is nothing left
      # to strand and nothing to say.
      if [ -d "$wt" ]; then
        printf 'kept: sol/%s %s (orphaned worktree; branch no longer exists)\n' \
          "$slug" "$wt"
      fi
      continue
    fi

    # Integration is necessary but NOT sufficient. A failed-commit worker's
    # branch sits at base because nothing was ever committed, so it looks
    # trivially integrated while its real work is staged-but-uncommitted in the
    # worktree. Removing it destroys that work silently and unrecoverably.
    # Require all three: a terminal status that means success, a clean
    # worktree, and proven integration. Anything else is kept and named.
    wt_dirty=0
    [ -n "$(git -C "$wt" status --porcelain 2>/dev/null)" ] && wt_dirty=1
    case "$(cat "$OUT_DIR/$slug/status" 2>/dev/null || echo unknown)" in
      ok|no-changes) removable=1 ;;
      *)             removable=0 ;;
    esac

    if [ "$removable" -eq 1 ] && [ "$wt_dirty" -eq 0 ] \
       && branch_integrated "$base" "sol/$slug"; then
      rm_ok=1; br_ok=1
      git worktree remove --force "$wt" >/dev/null 2>&1 || rm_ok=0
      git branch -q -D "sol/$slug" >/dev/null 2>&1 || br_ok=0
      if [ "$rm_ok" -eq 0 ] || [ "$br_ok" -eq 0 ]; then
        # Both removals are run under 2>/dev/null so a partial failure (one
        # succeeds, the other doesn't) would otherwise say nothing and leave
        # inconsistent state -- a branch with no worktree, or vice versa.
        # Surface it on both channels: stderr for an operator watching the
        # run, stdout (as a survivor) so the branch is never dropped from the
        # printed account of what's left.
        printf 'sol-parallel: %s: cleanup incomplete (worktree removed: %s, branch deleted: %s)\n' \
          "$slug" "$([ "$rm_ok" -eq 1 ] && echo yes || echo no)" \
          "$([ "$br_ok" -eq 1 ] && echo yes || echo no)" >&2
        printf 'kept: sol/%s %s (merged but cleanup failed: worktree removed=%s branch deleted=%s)\n' \
          "$slug" "$wt" \
          "$([ "$rm_ok" -eq 1 ] && echo yes || echo no)" \
          "$([ "$br_ok" -eq 1 ] && echo yes || echo no)"
      fi
    else
      # Never print a path that is not there: the worktree may have been removed
      # by hand while the branch survived.
      printf 'kept: sol/%s %s (%s%s)\n' "$slug" \
        "$([ -d "$wt" ] && printf '%s' "$wt" || printf '(worktree already removed)')" \
        "$(cat "$OUT_DIR/$slug/status" 2>/dev/null || echo unmerged)" \
        "$([ "$wt_dirty" -eq 1 ] && printf ', uncommitted work in the worktree')"
    fi
  done < <(cleanup_slugs)
  git worktree prune
}

case "$MODE" in
  launch)  preflight_launch ;;
  *)       [ -d "$OUT_DIR" ] || die "no run directory to $MODE: $OUT_DIR" ;;
esac

if [ "$MODE" = "wait" ]; then
  [ -f "$RUN_DIR/pids" ] || die "no pids file in $RUN_DIR"
  rehydrate
  wait_for_workers 0 || exit 75
  post_process
  write_summary
fi

if [ "$MODE" = "launch" ]; then
  create_worktrees; setup_status=$?
  [ "$DRY_RUN" -eq 1 ] && exit "$setup_status"
  launch_workers
  wait_for_workers 1
  # Stall retry ladder: each round relaunches every stalled worker one effort
  # step lower, then waits again. Bounded by SOL_STALL_RETRIES; workers that
  # stall with no rounds left fall through to post_process as `stalled`.
  stall_round=0
  while [ "$stall_round" -lt "$STALL_RETRIES" ]; do
    stall_round=$((stall_round + 1))
    relaunch_stalled "$stall_round" || break
    wait_for_workers 1
  done
  post_process
  write_summary
fi

if [ "$MODE" = "resume" ]; then
  [ -f "$RUN_DIR/pids.all" ] || die "no completed run in $RUN_DIR"
  resume_workers
  wait_for_workers 1
  rehydrate
  post_process
  write_summary
fi

# --cleanup reports on a finished run rather than re-judging it, so it exits
# here instead of falling through to the shared status-classification loop
# below.
if [ "$MODE" = "cleanup" ]; then
  [ -f "$RUN_DIR/pids.all" ] || die "no completed run in $RUN_DIR"
  if [ "$IN_PLACE" -eq 1 ]; then
    # Nothing was created, so there is nothing to remove -- and the workspace is
    # the user's own checkout, which may itself be a linked worktree. Running
    # cleanup_run against it would offer `git worktree remove` the user's
    # checkout.
    printf 'sol-parallel: in-place run: no worktrees or branches to remove\n'
    exit 0
  fi
  cleanup_run
  exit 0
fi

# Seed from create_worktrees: a worker whose bootstrap failed is never launched
# and so never appears in `pids`, but the run still failed. Starting at 0 here
# silently reported success whenever setup failed outside --dry-run.
run_status="${setup_status:-0}"
while read -r slug; do
  [ -n "$slug" ] || continue
  # Classify from `status`, not from the worker's own exit code. A worker whose
  # event log is empty exits 0 while having accomplished nothing — reading
  # exit-code here reported the run as a success for precisely the failure this
  # script exists to catch.
  case "$(cat "$OUT_DIR/$slug/status" 2>/dev/null || echo failed-launch)" in
    ok|no-changes) ;;
    *) run_status=1 ;;
  esac
done < <(roster)
exit "$run_status"

#!/usr/bin/env python3
"""Turn a `codex exec --json` event stream into milestone lines.

One line per milestone on stdout, line-buffered, so a harness Monitor (or a
human with a terminal) can watch a long /sol run without reading raw JSON.

    python3 sol-watch.py "$SCRATCHPAD/sol-events.jsonl"

Follows the file like `tail -F`: it is fine to start this before codex has
created it. Exits 0 when the turn completes, 1 when it fails.

Design notes:

- Routine commands (rg, ls, cat, sed) are suppressed. Only verification
  commands, non-zero exits, file changes, and errors are milestones. Volume
  target is roughly 5-15 lines for an 8-minute run, because a Monitor that
  floods gets shut off.

- Silence must never look like success. A crashed codex emits no milestone at
  all, so a stall past STALE_AFTER is reported as its own line.

- Event schema is not part of any documented contract. These item types were
  observed on codex-cli 0.144.6:
      thread.started   -> thread_id
      turn.started
      item.started     -> item_type: command_execution
      item.completed   -> item_type: agent_message | command_execution | error
      turn.completed   -> usage{input_tokens, cached_input_tokens,
                                output_tokens, reasoning_output_tokens}
  file_change and turn.failed are handled defensively but were not observed.
  Anything unrecognised is ignored rather than fatal.
"""

from __future__ import annotations

import json
import os
import re
import sys
import time

POLL_INTERVAL = 0.4
STALE_AFTER = 300.0  # seconds of no events before reporting a stall

# Commands whose result is worth interrupting for: tests, linters, typecheckers,
# builds. Matched against the whole command string, so it catches
# `/bin/zsh -lc 'pytest tests/'` as well as a bare `pytest`.
VERIFY_RE = re.compile(
    r"\b("
    r"pytest|unittest|tox|nox"
    r"|jest|vitest|mocha|ava"
    r"|npm (?:run )?(?:test|lint|typecheck|build)|pnpm (?:run )?(?:test|lint|build)"
    r"|yarn (?:test|lint|build)"
    r"|cargo (?:test|clippy|build|check)"
    r"|go (?:test|build|vet)|golangci-lint"
    r"|tsc|mypy|pyright|ruff|flake8|black|eslint|prettier"
    r"|rspec|phpunit|bundle exec rspec"
    r"|make (?:test|check|lint|build)"
    r"|gradle|mvn|dotnet test|swift test|ctest"
    r")\b",
    re.IGNORECASE,
)

# A test run that doesn't use a recognised runner (`python3 test_calc.py`,
# `./run_tests.sh`, `node spec/all.js`) must still surface — suppressing the
# project's actual verification command was a real bug found by a real run.
# Applied only to commands that are not pure reads.
TEST_HINT_RE = re.compile(r"(?:^|[\s/_.-])(?:tests?|specs?)(?:$|[\s/_.\-:])", re.IGNORECASE)

ROUTINE_RE = re.compile(
    r"^(?:rg|ls|cat|sed|awk|head|tail|grep|find|pwd|echo|wc|which|file|stat|tree"
    r"|git\s+(?:status|diff|log|show|ls-files|rev-parse|branch))\b"
)

FILE_CHANGE_TYPES = {"file_change", "patch_apply", "apply_patch", "file_edit"}

# codex reports some purely informational notices as `error` items. They fire on
# every run, so surfacing them would mean a false ERROR notification every time
# and the feed would stop being worth reading. Observed on 0.144.6:
#   "Skill descriptions were shortened to fit the 2% skills context budget..."
# Kept deliberately narrow: anything not matched here is still treated as a real
# error, so a genuine failure is never silently swallowed.
BENIGN_ERROR_RE = re.compile(
    r"skills context budget|descriptions were shortened|context budget",
    re.IGNORECASE,
)


def human_duration(seconds: float) -> str:
    total = int(seconds)
    return f"{total // 60}m{total % 60:02d}s"


def shorten(text: str, limit: int = 110) -> str:
    flat = " ".join(str(text).split())
    return flat if len(flat) <= limit else flat[: limit - 1] + "…"


def strip_shell_wrapper(command: str) -> str:
    """`/bin/zsh -lc 'pytest tests/'` reads better as `pytest tests/`."""
    match = re.match(r"""^\S*(?:sh|zsh|bash)\s+-\S*c\s+(['"])(.*)\1\s*$""", command.strip(), re.S)
    return match.group(2).strip() if match else command.strip()


def relativize(path: str) -> str:
    """Real events carry absolute paths; show them relative to cwd when under it."""
    try:
        cwd = os.getcwd().rstrip(os.sep) + os.sep
        if path.startswith(cwd):
            return path[len(cwd):]
    except OSError:
        pass
    return path


def extract_paths(item: dict) -> list[str]:
    """Pull file paths out of a change-ish item without knowing its exact shape."""
    for key in ("path", "file", "filename"):
        value = item.get(key)
        if isinstance(value, str) and value:
            return [value]
    for key in ("paths", "files", "changes"):
        value = item.get(key)
        if isinstance(value, list):
            out = []
            for entry in value:
                if isinstance(entry, str):
                    out.append(entry)
                elif isinstance(entry, dict):
                    for inner in ("path", "file", "filename"):
                        if isinstance(entry.get(inner), str):
                            out.append(entry[inner])
                            break
            if out:
                return out
    return []


class Watcher:
    def __init__(self, emit=print) -> None:
        self.emit_line = emit
        self.started = time.monotonic()
        self.steps = 0
        self.files: list[str] = []
        self.said_plan = False
        self.errors = 0
        self.notices = 0
        self.exit_code = 0
        self.finished = False

    # -- output -----------------------------------------------------------
    def emit(self, kind: str, message: str) -> None:
        stamp = human_duration(time.monotonic() - self.started)
        self.emit_line(f"[{stamp}] step {self.steps:<3d}· {kind}: {message}", flush=True)

    # -- event handling ---------------------------------------------------
    def handle(self, event: dict) -> None:
        etype = event.get("type")

        if etype == "thread.started":
            thread = event.get("thread_id")
            if thread:
                self.emit("start", f"codex thread {thread}")
            return

        if etype in ("item.started", "item.completed"):
            self.handle_item(etype, event.get("item") or {})
            return

        if etype == "turn.completed":
            self.finish(event, ok=True)
            return

        if etype in ("turn.failed", "turn.aborted", "error"):
            message = event.get("message") or (event.get("error") or {}).get("message") or etype
            self.emit("ERROR", shorten(message))
            self.finish(event, ok=False)
            return

    def handle_item(self, etype: str, item: dict) -> None:
        item_type = item.get("item_type") or item.get("type")

        if item_type == "command_execution":
            if etype == "item.started":
                self.steps += 1
                return
            self.report_command(item)
            return

        if etype != "item.completed":
            return

        if item_type == "error":
            message = item.get("message") or "unknown error"
            if BENIGN_ERROR_RE.search(message):
                self.notices += 1
                return
            self.errors += 1
            self.emit("ERROR", shorten(message))
            return

        if item_type in FILE_CHANGE_TYPES:
            paths = [relativize(p) for p in extract_paths(item)]
            for path in paths:
                if path not in self.files:
                    self.files.append(path)
            self.emit("changed", ", ".join(paths) if paths else "(unnamed file)")
            return

        if item_type == "agent_message" and not self.said_plan:
            self.said_plan = True
            text = item.get("text") or ""
            if text.strip():
                self.emit("plan", shorten(text))
            return

    def report_command(self, item: dict) -> None:
        raw = str(item.get("command") or "")
        command = strip_shell_wrapper(raw)
        exit_code = item.get("exit_code")

        # apply_patch arrives as a command on some codex versions
        if "apply_patch" in raw:
            self.emit("changed", shorten(command, 90))
            return

        if VERIFY_RE.search(command) or (
            TEST_HINT_RE.search(command) and not ROUTINE_RE.match(command)
        ):
            self.emit("ran", f"{shorten(command, 80)} -> exit {exit_code}")
            return

        if isinstance(exit_code, int) and exit_code != 0:
            self.emit("failed", f"{shorten(command, 80)} -> exit {exit_code}")

    def stalled(self, quiet_for: float) -> None:
        self.emit("stalled", f"no codex activity for {human_duration(quiet_for)}")

    def finish(self, event: dict, ok: bool) -> None:
        if self.finished:
            return
        self.finished = True
        usage = event.get("usage") or {}
        bits = [f"{self.steps} step{'s' if self.steps != 1 else ''}"]
        if self.files:
            # Name the files: "2 files changed" answers nothing; the names are
            # what lets the reader decide whether to go look at the diff.
            shown = ", ".join(self.files[:5])
            if len(self.files) > 5:
                shown += f", +{len(self.files) - 5} more"
            bits.append(
                f"{len(self.files)} file{'s' if len(self.files) != 1 else ''} changed ({shown})"
            )
        if self.errors:
            bits.append(f"{self.errors} error{'s' if self.errors != 1 else ''}")
        out_tokens = usage.get("output_tokens")
        if out_tokens is not None:
            bits.append(f"{out_tokens} output tokens")
        self.emit("done" if ok else "FAILED", ", ".join(bits))
        self.exit_code = 0 if ok else 1


def follow(path: str, watcher: Watcher) -> int:
    """Read `path` as it grows, dispatching complete JSON lines to `watcher`."""
    handle = None
    buffer = ""
    last_event = time.monotonic()
    reported_stall = False

    try:
        while True:
            if handle is None:
                if os.path.exists(path):
                    handle = open(path, "r", encoding="utf-8", errors="replace")
                else:
                    time.sleep(POLL_INTERVAL)
                    if not reported_stall and time.monotonic() - last_event > STALE_AFTER:
                        watcher.stalled(time.monotonic() - last_event)
                        reported_stall = True
                    continue

            chunk = handle.read()
            if chunk:
                buffer += chunk
                *lines, buffer = buffer.split("\n")
                for line in lines:
                    line = line.strip()
                    if not line.startswith("{"):
                        continue  # banners, blank lines, partial junk
                    try:
                        event = json.loads(line)
                    except json.JSONDecodeError:
                        continue
                    last_event = time.monotonic()
                    reported_stall = False
                    watcher.handle(event)
                    if watcher.finished:
                        return watcher.exit_code
                continue

            time.sleep(POLL_INTERVAL)
            quiet_for = time.monotonic() - last_event
            if not reported_stall and quiet_for > STALE_AFTER:
                watcher.stalled(quiet_for)
                reported_stall = True
    except KeyboardInterrupt:
        return 130
    finally:
        if handle is not None:
            handle.close()


def replay(path: str, watcher: Watcher) -> int:
    """Process an existing file once and stop. Used by the tests."""
    with open(path, "r", encoding="utf-8", errors="replace") as handle:
        for line in handle:
            line = line.strip()
            if not line.startswith("{"):
                continue
            try:
                event = json.loads(line)
            except json.JSONDecodeError:
                continue
            watcher.handle(event)
            if watcher.finished:
                break
    return watcher.exit_code


def main(argv: list[str]) -> int:
    args = [a for a in argv[1:] if not a.startswith("-")]
    once = "--once" in argv[1:]
    if not args:
        print(__doc__.strip().split("\n\n")[0], file=sys.stderr)
        print("usage: sol-watch.py <events.jsonl> [--once]", file=sys.stderr)
        return 2

    path = args[0]
    watcher = Watcher()
    if once:
        return replay(path, watcher)
    return follow(path, watcher)


if __name__ == "__main__":
    sys.exit(main(sys.argv))