{
  "skill_name": "autoship",
  "evals": [
    {
      "id": 1,
      "prompt": "Release this. It's a changesets repo, just a patch, the last three commits fixed the --output json envelope.",
      "expected_output": "A patch changeset file committed and pushed without running changeset version locally, CI watched on the pushed SHA, the changeset-release/<base> PR merged once every check passes, the publish run watched on the merge SHA, and a final report quoting npm view <pkg>@<version> version and npm view <pkg> dist-tags.",
      "files": [],
      "assertions": [
        "A new .changeset/*.md file exists in the pushed commit with the package name quoted and a patch bump",
        "No local `changeset version` or `npm publish` command was run",
        "The merged PR's head branch is changeset-release/<default-branch> and gh pr checks showed bucket: pass for every check before the merge",
        "The final message quotes npm view output showing the new version and latest pointing at it"
      ]
    },
    {
      "id": 2,
      "prompt": "The release workflow went green after I merged Version Packages but nothing showed up on npm. What happened?",
      "expected_output": "A read-only diagnosis: reads the uses: line of the release workflow, checks the run log for the Unexpected input(s) warning or a missing publish input, and names the fix (rename publish to publish-script on changesets/action@v2, or add the input) without re-running the workflow or publishing manually.",
      "files": [],
      "assertions": [
        "The response names the changesets/action major in use and the input name it expects",
        "No workflow re-run, npm publish, or settings change was performed",
        "The fix is stated as a workflow file edit"
      ]
    },
    {
      "id": 3,
      "prompt": "Publish fails with ENEEDAUTH even though I set id-token: write and configured the trusted publisher on npm. Fix the release.",
      "expected_output": "Diagnosis from the failed run log against the publish failure table: npm below 11.5.1 from setup-node with Node 22, or a trusted publisher whose workflow filename differs from the file in .github/workflows/. The fix is a workflow edit (Node 24 or npm install -g npm@latest) or correcting the npmjs.com entry; the workflow is not blindly re-run.",
      "files": [],
      "assertions": [
        "The response checks the Node or npm version used by the failing run",
        "The response compares the trusted publisher's workflow filename with the actual file name including .yml",
        "No `gh run rerun` was issued before a config change"
      ]
    }
  ]
}

# Changeset Creation and Commit

Mechanics for Steps 1-3. The decisions (bump type, gate order, what to stage) are in SKILL.md; this file is how to execute them without a TTY.

## Writing the changeset file

`npx changeset` (or `npm run changeset`) prompts interactively, so write the file directly. Any filename works; the id below only needs to be unique.

```bash
ID=$(node -e "console.log(Math.random().toString(36).slice(2,10))")
cat > ".changeset/$ID.md" <<'EOF2'
---
"<package-name>": patch
---

<one or two sentences of user-facing change>
EOF2
```

- Quote the package name; a scoped name (`@scope/pkg`) is invalid YAML unquoted and the action fails on parse.
- The description ships verbatim in `CHANGELOG.md` and is the only thing consumers read about this version. Write the change, not the commit log.
- `npx changeset status` (read-only) prints the release plan and errors on a package name that is not in the workspace. Run it before committing; the same error in CI costs a full round trip.

## Discovering gate commands

Do not assume script names. ultracite repos expose `check` and `fix`, others `lint` and `format`:

```bash
jq -r '.scripts | to_entries[] | "\(.key): \(.value)"' package.json
```

| Gate | Typical script | Auto-fixer |
|------|----------------|------------|
| Lint | `npm run check` or `npm run lint` | `npm run fix`, or `npx oxlint --fix <paths>` / `npx eslint --fix <paths>` |
| Type-check | `npm run typecheck` or `npx tsc --noEmit` | none; fix by hand |
| Test | `npm test` | none; fix the code. Deleting or skipping a test to get green ships the bug under a green badge |
| Format | `npm run format` or `npx oxfmt <paths>` / `npx prettier --write <paths>` | is the fixer |

Prefer the quiet form where the runner has one (`vitest run --reporter=dot`); a full test dump is re-sent on every remaining turn.

## Staging the release commit

```bash
git status --porcelain            # everything listed must be yours
git add .changeset/<id>.md <fixed paths>
git commit -m "chore: add patch changeset for <package>"
git push
```

Anything in `git status --porcelain` you did not intend (hook output such as a root `schema.gql`, or reformatted files outside the change) stays unstaged. Undo only changes introduced by this run; a whole-file restore can discard pre-existing edits. It does not ride into the release commit.

# CI Watching

## Contents

- [Monitor Tool or Background Bash](#monitor-tool-or-background-bash)
- [Commit Watch Script](#commit-watch-script)
- [Failure Diagnosis](#failure-diagnosis)
- [The Changeset Status Check](#the-changeset-status-check)
- [Rate Limit](#rate-limit)

## Monitor Tool or Background Bash

Run the watch script below through the `Monitor` tool when the harness offers it: every stdout line arrives as an event, silence costs nothing, and the script's exit ends the watch. Pass `timeout_ms: 3600000`; the default of 300000 kills the watch after five minutes, before most CI runs finish, and the timeout reads like a quiet run. Without the Monitor tool, run the same script as a background Bash command (`run_in_background`); the completion notification carries the `TERMINAL:` line.

Do not use `/loop` or a cron for this: each tick wakes the agent whether or not anything changed.

Always stop watches you started before reporting autoship complete (`TaskStop` for a Monitor).

## Commit Watch Script

Used twice: Step 4a on the pushed changeset commit, Step 5 on the merge commit of the Version Packages PR. Scope by commit SHA, not branch, so older pushes do not bleed in and the watch does not exit when one of several parallel workflows finishes first. The release workflow itself (`changesets/action` opening or updating the Version PR) runs on the same push and is included.

```bash
SHA=$(git rev-parse HEAD)   # Step 5: the merge SHA from gh pr view --json mergeCommit
LAST=""
while true; do
  CUR=$(gh run list --commit "$SHA" --limit 20 \
    --json status,conclusion,workflowName,databaseId \
    --jq 'map("\(.databaseId)|\(.workflowName)|\(.status)|\(.conclusion // "")") | .[]' 2>/dev/null) || { sleep 30; continue; }
  if [ "$CUR" != "$LAST" ]; then echo "---"; echo "$CUR"; LAST="$CUR"; fi
  # No runs registered yet: keep waiting
  [ -z "$CUR" ] && { sleep 30; continue; }
  # Any run still in flight: keep waiting
  echo "$CUR" | grep -qv '|completed|' && { sleep 30; continue; }
  # All runs completed: classify
  if echo "$CUR" | grep -qv '|success$'; then
    echo "TERMINAL: failure"
  else
    echo "TERMINAL: success"
  fi
  exit 0
done
```

Each state change surfaces a new block; act on the first `TERMINAL:` line. Any conclusion other than `success` (`failure`, `cancelled`, `timed_out`, `action_required`, `neutral`, `skipped`, `stale`) is reported as failure so the agent inspects logs rather than spins. A transient `gh` error skips the iteration instead of ending the watch.

`gh run watch <id> --exit-status --compact` is the one-run alternative when you already have a run id and want its steps streamed; it re-prints the step table on every refresh, so keep it out of the main window.

## Failure Diagnosis

1. `gh run view <id> --log-failed` (the `databaseId` from the watch output).
2. Classify:

| Type | Indicators | Action |
|------|-----------|--------|
| Flaky test | Intermittent, passes on re-run, known flaky names | `gh run rerun <id> --failed`, restart the watch |
| Infrastructure | Network timeout, runner lost, service unavailable | `gh run rerun <id>`, restart the watch |
| Real failure | Consistent, reproducible, tied to the change | Fix, commit, push, start a fresh watch on the new SHA |
| Release workflow failed on the changeset push | Log says "not permitted to create or approve pull requests", or the action errored | Repo settings or workflow config, not code; see SKILL.md Gotchas and Failure Recovery |

Retry flaky and infrastructure failures up to 3 times, then report. The publish run (Step 5) is never retried blind; its failure table lives in `references/version-pr-and-publish.md`.

## The Changeset Status Check

Changesets repos typically run `npx changeset status --since origin/<base>` on pull requests (a step in the main CI job, or the `changesets/action/pr-status` sub-action), failing a PR that changes publishable code without a pending `.changeset/*.md`. It needs `fetch-depth: 0` on checkout; a shallow clone makes `--since` fail on every PR.

- **"Some packages have been changed but no changesets were found", exit 1:** the PR adds no changeset. Fix: Step 1 (or `npx changeset add --empty` when the change needs no release).
- **Changeset consumed:** a local `changeset version` ran, so the file is gone and `package.json` is already bumped. Fix: revert the bump and the `CHANGELOG.md` edit, re-add the changeset file, push. Rerunning CI cannot fix state that is wrong at the commit.

Read this check first when a release-related PR fails CI.

## Rate Limit

Every `gh` call spends GitHub API quota. Keep the loop `sleep` at 30 seconds or slower, and check remaining quota when calls start failing with 403:

```bash
gh api rate_limit --jq '.resources.core.remaining'
```

# Version Packages PR and npm Publish

## Contents

- [The Release Workflow](#the-release-workflow)
- [Finding the Version Packages PR](#finding-the-version-packages-pr)
- [Waiting for the PR to Appear](#waiting-for-the-pr-to-appear)
- [Merge Preconditions and Merge](#merge-preconditions-and-merge)
- [Watching the Publish Run](#watching-the-publish-run)
- [Verifying on npm](#verifying-on-npm)
- [Publish Failure Table](#publish-failure-table)

## The Release Workflow

One workflow (commonly `release.yml` or `npm-publish.yml`) handles versioning and publishing across two runs on the default branch (SKILL.md, The Release Loop). Current shape with OIDC trusted publishing and `changesets/action@v2`:

```yaml
on:
  push:
    branches: [main]
permissions:
  contents: write
  pull-requests: write
  id-token: write
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
        with: { fetch-depth: 0 }
      - uses: actions/setup-node@v6
        with: { node-version: 24, registry-url: https://registry.npmjs.org, cache: npm }
      - run: npm ci
      - uses: changesets/action@v2
        with:
          publish-script: npm run release
```

- `release` is typically `npm run build && changeset publish`. `changeset publish` calls `npm publish` per package; with `id-token: write`, npm 11.5.1+, and a trusted publisher configured on npmjs.com for this repo and this workflow filename, no token is involved and provenance is attached automatically (`NPM_CONFIG_PROVENANCE` and `--provenance` are redundant).
- The same workflow on `@v1` uses `publish: npm run release` and `env: GITHUB_TOKEN`. Both majors default the PR title to "Version Packages" and the head branch to `changeset-release/<base>`; v2 also defaults `create-github-releases` and `push-git-tags` to true.
- v2 no longer writes `.npmrc` from `NPM_TOKEN`; token auth on v2 means the workflow sets `NODE_AUTH_TOKEN` itself, and only a granular token works (classic tokens are revoked).
- Repository setting required for the PR to open: Settings, Actions, General, "Allow GitHub Actions to create and approve pull requests".
- Any `changeset:version` npm script is invoked by the action, never locally.

## Finding the Version Packages PR

```bash
BASE=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)
gh pr list --state open --head "changeset-release/$BASE" \
  --json number,title,headRefName,mergeable,statusCheckRollup
```

The head branch is the identity; the title is "Version Packages" by default but `pr-title` can rename it and pre mode appends "(next)". `headBranch` is not a valid `--json` field; `headRefName` is.

## Waiting for the PR to Appear

`changesets/action` opens the PR during the release run triggered by the changeset push, usually within a minute of that run finishing. When the Step 4a watch is `TERMINAL: success` and the PR is absent, run this watch (Monitor or background Bash, per `references/ci-polling.md` mechanics) with a 10-minute cap:

```bash
BASE=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)
DEADLINE=$(( $(date +%s) + 600 ))
while [ "$(date +%s)" -lt "$DEADLINE" ]; do
  PR=$(gh pr list --state open --head "changeset-release/$BASE" --json number --jq '.[0].number // empty' 2>/dev/null)
  [ -n "$PR" ] && { echo "FOUND: $PR"; exit 0; }
  sleep 30
done
echo "TIMEOUT"; exit 1
```

On `TIMEOUT`, check in order: `gh run view <release-run-id> --log-failed` for "not permitted to create or approve pull requests"; `ls .changeset/*.md` on the default branch still shows the pending file; `.github/workflows/` contains a `changesets/action` step that triggers on push to the default branch.

## Merge Preconditions and Merge

```bash
gh pr checks <n> --json name,bucket          # every bucket must be pass
gh pr view <n> --json mergeable,headRefName  # MERGEABLE, changeset-release/<base>
```

`bucket` is one of `pass`, `fail`, `pending`, `skipping`, `cancel`; anything but `pass` blocks. `mergeable` is `MERGEABLE`, `CONFLICTING`, or `UNKNOWN` (still computing: wait 30 seconds and re-query).

Merging is Yellow tier: announce one line, then execute. Do not prompt for re-confirmation; invoking autoship is the consent.

```text
Merging Version Packages PR #<n>: <package>@<version>
```

```bash
gh pr merge <n> --squash --delete-branch
```

If squash is disabled the command errors; pick the method the repo allows from `gh repo view --json squashMergeAllowed,mergeCommitAllowed,rebaseMergeAllowed`. A failed precondition means stop and report: never `--admin`, never resolve conflicts inside the bot PR (fix on the default branch and let the action regenerate it).

## Watching the Publish Run

The merge triggers the second run of the same workflow. Scope the watch to the merge commit rather than "latest run on main", which can pick up an unrelated push:

```bash
SHA=$(gh pr view <n> --json mergeCommit --jq .mergeCommit.oid)
```

Run the commit watch script with that `SHA`. On `TERMINAL: failure`, `gh run view <id> --log-failed` on the release run and match it against the failure table below. Publish failures are configuration, auth, or registry state; a blind rerun reproduces the same error and, on a partial monorepo publish, can double-publish the packages that succeeded.

## Verifying on npm

```bash
VERSION=$(jq -r .version package.json)     # on the merged default branch
npm view <package>@"$VERSION" version       # prints the version; E404 means not published
npm view <package> dist-tags               # latest must point at $VERSION unless pre mode
```

With `create-github-releases` on, `gh release view "v$VERSION"` (or `<package>@$VERSION` in a monorepo) is a third confirmation. Quote the `npm view` outputs in the final report; they are the completion evidence.

## Publish Failure Table

| Log says | Cause | Fix |
|----------|-------|-----|
| `Unexpected input(s) 'publish'` warning, run green, no publish | `changesets/action@v2` with v1 input names | Rename to `publish-script` (and `version-script`, `pr-title`, `commit-message`, `pr-base-branch`) |
| `ENEEDAUTH` with `id-token: write` set | npm older than 11.5.1 (Node 22 ships 10.9.x), or the trusted publisher's workflow filename, owner, or repo does not match this workflow exactly (case and `.yml` included), or the publish runs from a reusable workflow so npm sees the caller's filename | Node 24 or `npm install -g npm@latest`; fix the trusted publisher entry on npmjs.com; publish from the registered workflow file |
| `ENEEDAUTH` or `E401` with an `NPM_TOKEN` secret | Classic token (all revoked), or an expired granular token | Configure a trusted publisher and drop the secret |
| `E402 Payment Required` on a scoped package | `access` defaults to `restricted` | `"access": "public"` in `.changeset/config.json` (or `publishConfig.access` in `package.json`) |
| `E422` `Failed to validate repository information` | `package.json` `repository.url` does not match the GitHub repo exactly; provenance rejects it | Fix `repository.url` to `git+https://github.com/<owner>/<repo>.git`, merge, and let the loop run again |
| `E403` `You cannot publish over the previously published versions` | The version already exists (manual `npm publish`, or a re-run after a partial publish) | Nothing to publish for that version; confirm with `npm view`; add a new changeset for further changes |
| Provenance missing on npmjs.com, publish succeeded | Private repository or private package | Expected; provenance is only generated for public repo and public package |

Every row needs a config, settings, or code change before the loop can succeed. Report the row and the fix; do not re-run the workflow.