ci-fix
Use when "CI is red", "fix the checks", or "make CI green", one check needs classifying, or a bounded sweep runs. Not for deploys, credentials, or rerun-as-fix. Non-CI bugs: use strike-the-root.
Install
npx skills add https://github.com/OutlineDriven/outline-driven-development/tree/main/.devin/skills/ci-fix
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install outlinedriven-outline-driven-development@llmmart
git clone https://github.com/OutlineDriven/outline-driven-development.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole outlinedriven/outline-driven-development collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
CI fix
Repair, classify, or sweep failing CI on a PR or branch. Three modes: fix (default; interactive or autonomous), classify (read-only diagnosis), and sweep (bounded background monitor). Never widen workflow permissions, never add pull_request_target, never offer a rerun in place of a code fix. Merge conflicts route to resolve-merge-conflicts; non-CI bugs route to strike-the-root.
Contract
| Field | Bound contract |
|---|---|
| Trigger | Fix: user asks to diagnose or repair failing CI on a PR or branch; autonomous fix only when the user explicitly asks for the fix to be pushed. Classify: a check is failed, absent, pending too long, unstable, or reported unexpectedly and no patch is requested. Sweep: a recurring or requested sweep monitors CI failures over a bounded attempt window. |
| Authority | Human-gated: classify is read-only and writes nothing. Fix interactive writes only named source files after plan approval; rollback is version control. Fix autonomous pushes a fix branch, never opens a PR, and is a remote mutation that requires explicit human invocation. Sweep writes one minimal isolated repair in a worktree; rollback is discarding the worktree; it never pushes, merges, or publishes. |
| Side effect | Classify: chat output only. Fix interactive: named local source files edited and a local-equivalent check passes; remote CI unchanged. Fix autonomous: a fix branch pushed with the approved change set and a new CI run observed. Sweep: one worktree repair plus one independent verifier run, returned as a proposal. |
| Done | Classify: one deterministic failure class supported by observed evidence, a next owner, a one-sentence next action, and a fix plan when a PR and gh are available; nothing mutated. Fix interactive: failing checks identified, the approved local fix applied, and every local equivalent passes; return local-fix-ready because remote CI is unchanged. Fix autonomous: a new run on ci-fix/<original-branch> is green and the ordered root-cause record is delivered; checks-pass is claimed only after a green remote run is observed. Sweep: root cause reproduced or classified non-actionable; any patch is minimal, independently verified, and returned as a proposal; retries stop at the configured cap without symptom patching. |
Inputs
mode:fix(default),classify, orsweep.repo: path inside the repo (default.). Must be a Git repository.- Target: PR number or URL, original branch name, or run ID; defaults to the current branch's PR. A working tree already carrying the in-progress fix is fine.
ghauthenticated for the repo host with workflow/repo scopes. Classify stands alone withoutgh; in fix and sweep an unauthenticated CLI stops the skill before any change.- Optional: a specific job ID for full-log inspection, or an artifact download when logs alone do not identify the cause.
- Classify: the check name, its status (failed, absent, pending, unstable, or unexpected), and the run identifier or URL that produced the report; optional: the failing job log, the changed files or commit range under test, and any prior classification of the same check.
- Sweep: the base commit the CI run used, the verifier command that independently confirms a repair, and the attempt cap; optional: a flake-detection window and a circuit-breaker threshold.
Procedure
- Select the mode:
classifywhen the user asks for a diagnosis or plan without a patch,sweepwhen a bounded monitor over an attempt window is requested,fixotherwise. Done when: the mode is stated and its inputs above are present, or the missing input is reported. - Set
GH_PAGER=caton everyghinvocation; the CLI has no global--no-pageroption and blocks on a pager in non-interactive contexts without it. Verify authentication withgh auth status. In fix and sweep, stop and ask the user to rungh auth loginwhen unauthenticated; no mutation before this. In classify, continue withoutgh; the classification stands alone. Done when: an authenticated identity is confirmed bygh auth statusoutput, the handoff is asked and no mutation has occurred, or classify proceeds withoutghand says so. - Record the failing identity. Classify: record the check name, status, and run identifier before reading anything else; an absent check or a check that never ran is a distinct input, not a missing one. Fix and sweep: resolve the failing run; PR:
gh pr view --json number,url; non-PR branch or supplied run ID:gh run list --branch <branch> --status failure --limit 5, thengh run view <run-id>. Done when: classify has the check name, status, and run identifier recorded, or fix and sweep have one failing run identified by its run ID and branch name from command output. - Gather failure evidence (GitHub Actions only). Prefer the bundled script, which handles gh field drift, run-id/job-id extraction, failure-snippet extraction, and exits non-zero while failures remain:
python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "<number-or-url>"(--jsonfor machine output).- Manual fallback:
GH_PAGER=cat gh pr checks <pr> --json name,state,bucket,link,startedAt,completedAt,workflow; rerun with the fieldsghaccepts if one is rejected. Extract the run id fromdetailsUrl, thenGH_PAGER=cat gh run view <run_id> --json name,workflowName,conclusion,status,url,event,headBranch,headSha,GH_PAGER=cat gh run view <run_id> --log-failedfor the failed jobs only or--logfor all,GH_PAGER=cat gh run view <run-id> --log --job <job-id>for one failing job; for evidence logs cannot supply,gh run download <run-id> -D .artifacts/<run-id>. If a run log is still in progress, fetch job logs viagh api "/repos/<owner>/<repo>/actions/jobs/<job_id>/logs". - A check whose
detailsUrlis not a GitHub Actions run is external: report the URL only; do not attempt Buildkite or other providers. Done when: every failing check has a name, URL, and a log snippet from command output, or an explicitlog_pending/log_unavailablemarker; never a fabricated snippet. In classify, preserve the exact failure line, signal, or message.
- Mode classify (read-only; stop before any write):
- Compare the failure against the commit range and changed files under test; decide whether the failing code path was touched by the change or predates it. Before flagging a test failure as a regression, check whether the same test previously passed in CI and note environment-specific or flaky cases rather than treating them as regressions. Done when: the failing path is determined as in-diff or pre-existing.
- Classify into exactly one class:
regression(the change introduced or exposed the failure; the failing path is in the diff),flake/watch(timing-, order-, or environment-dependent; passes on retry or across runs without a code change),infrastructure(runner, network, quota, service outage, or resource exhaustion, not the code under test),configuration(build, config, dependency, or environment setup, not product logic),policy/absent-CI(absent, skipped, not configured, or blocked by a branch-protection or policy rule), orhuman escalation(evidence insufficient, contradictory, or outside the five classes; a human must decide). If a prior classification of the same check matches the new evidence, note the repeat; if it contradicts, re-classify from the new evidence. Done when: the failure is classified into exactly one class from observed evidence. - Assign the next owner from the class: regression and configuration to the change author, flake/watch to the test or platform owner, infrastructure to the platform or runner owner, policy/absent-CI to the repository or CI-config owner, human escalation to a human reviewer. State the next action concretely in one sentence. Done when: the next owner is assigned from the class and the one-sentence next action is recorded.
- Emit a reviewable fix plan when a PR and authenticated
ghare available: view the PR withGH_PAGER=cat gh pr view <branch> --json number,title,url,stateand fetch the check rollup with--json statusCheckRollup; extract each failed check's logs withGH_PAGER=cat gh run view <run-id> --log-failed, collecting error messages with file paths and line numbers, compilation errors, lint names, test failure messages and stack traces, and build root causes. If any check is still in progress, report the partial state and stop; do not diagnose incomplete checks. Then write the plan: a problem statement summarizing the failing checks, the current state listing each error and its location, proposed changes grouped by category (formatting, linting, compilation, test failures, platform-specific), and validation steps (fmt, lint, tests, presubmit). Fix one category at a time. Apply no code change. Mark any unread run as untriaged. Done when: the plan carries the problem statement, current state, per-category proposed changes, and validation steps, or the plan is skipped and reported because no PR orghis available. Then stop; if any further step would require a write, returnhuman escalation.
- Mode fix:
- Triage every failing check: classify each as a root cause or a downstream symptom of another failure. A single root cause may surface as several failing checks. Order the work so root causes are fixed before the symptoms they produce. Done when: every failing check is classified as root cause or symptom in a written list, and the work order places root causes before their symptoms.
- For each root-cause failure, read the failure log and the relevant source to determine the smallest deterministic fix for that cause; prefer a code change over workflow plumbing; for a flaky test, fix the nondeterminism instead of rerunning. Request approval before editing any file. Done when: the plan is approved in writing with the file paths and change described, or refused; then report the plan and wait without editing.
- Implement the approved plan and summarize diffs and tests. Keep the change set inside the failing job/step where possible. Done when:
git diff --name-onlylists only the approved file paths and the diff summary matches the approved plan. - Run the local equivalent of each failing CI check to confirm the fix before any push. Done when: the local check command exits 0 with its output captured, confirming the fix reproduces green locally.
- Capture the current remote status with
gh pr checks <pr>. Interactive mode has not pushed, so reportlocal-fix-readywith the local proof and the observed red or pending remote checks; never imply that local edits changed CI. If the remote checks are already green because another actor updated the branch, report the observed green state without attributing it to the local patch. Done when: the currentgh pr checksoutput is captured and the result islocal-fix-readywith remote status quoted, or an independently updated remote run is observed green with its output quoted. Autonomous branch (explicit user ask only): enter only when the user asks for the fix to be pushed (for example "push the fix" or "land it on a fix branch"); never self-initiated. Before any mutation, preview the failing job, the exact change set, and the target branchci-fix/<original-branch>; the plan approval from fix step 2 doubles as the push approval once the preview is acknowledged. - If workflow files must change: keep existing
permissions:minimal, never broaden token access, nopull_request_targetunless the user explicitly requested it. Done when:git diffon workflow files touches no permission key outside the previewed set. - Create or update the branch. New:
git checkout -b ci-fix/<original-branch>. Existing: show its current commits, confirm reuse with the user,git checkout ci-fix/<original-branch>, thengit pull origin <original-branch>. Done when: the branch is checked out andgit log --oneline -1shows it carries only the previewed change. - Stage only the exact paths in the approved preview with
git add -- <paths>, then inspect the staged path list and patch. Stop if the staged set contains any extra path. Commitfix: resolve CI failure in <job-name>, then pushgit push -u origin ci-fix/<original-branch>(plaingit pushonce upstream is set). Done when:git diff --cached --name-onlylists only the approved paths andgit pushsucceeds with the remote branch at the pushed SHA confirmed bygit rev-parse origin/ci-fix/<original-branch>. - Watch:
gh run list --branch ci-fix/<original-branch> --limit 1for the new run id, thengh run watch <new-run-id> --exit-status. Re-trigger only failed jobs withgh run rerun <new-run-id> --failedwhen the evidence is infrastructure or a confirmed flake and the user approves; a rerun never substitutes for a deterministic fix. Done when:gh run view <new-run-id> --json conclusionreportssuccess, or the failure and recovery section applies. - If new failures appear, repeat from fix step 1 treating the new failures as the current set. Stop only when every check in the set is green. Done when: every check in the set is green, or a failure class in the failure and recovery section applies.
- Mode sweep:
- On each tick, fetch the current set of failing CI checks for the target run and record the attempt number against the configured cap. If the attempt cap is reached, stop and hand off; do not start a new repair. Done when: the failing checks are fetched and the attempt number is recorded, or the sweep stops at the cap with a handoff.
- Pick one failing check and reproduce the failure locally in an isolated worktree created from the base commit the CI run used. Classify the failure: reproduce the root cause, or classify it non-actionable (flake, environment, upstream). If the failure is a flake or the cause is ambiguous, hand off and do not patch. Done when: the failure is reproduced locally or confirmed non-reproducible, and classified as root cause or non-actionable, or handed off.
- For a reproducible root cause, implement one minimal isolated repair in the worktree: the smallest change that fixes the reproduced cause and nothing else. Run the verifier in the worktree independently of the repair; confirm the failing check passes and no other check regresses. If the verifier fails or regresses, do not widen the patch; increment the attempt counter and either retry within budget or hand off. Done when: the minimal repair is implemented and the verifier confirms the fix with no regression, or the verifier failure is recorded and the retry-or-handoff decision is made.
- Return the verified patch as a proposal (diff or branch) with the reproduced root cause, the verifier evidence, and the attempt count. Do not push, merge, or publish; a worktree repair is never merged. If the circuit breaker trips (repeated non-convergence, repeated flakes, or budget exhaustion), stop the sweep and hand off with the accumulated evidence. Done when: the verified patch is returned as a proposal with evidence, or the sweep stops with a handoff and accumulated evidence.
Failure and recovery
- gh unauthenticated, or no failing run found for the supplied PR/branch/run ID: stop and report; nothing committed or pushed. In classify, skip the fix plan, report the missing prerequisite, and never attempt credential setup.
- Classify evidence insufficient or contradictory: do not guess a class. Return
human escalationnaming the missing evidence, or stating both signals when they imply different classes; never average or pick arbitrarily. - Classify no-CI state: a check that is absent or never ran is
policy/absent-CI; surface it, never treat it as passing or skip it. A check pending past the expected window or blocked by policy is surfaced with its next owner, never marked done. No PR for the branch: classification stands alone, skip the fix plan, and report that no PR exists; do not create one. - Classify partial evidence: classify each inspectable check and explicitly mark the rest as unevaluated.
gh run view --log-failedempty or erroring: report the unread run, continue triaging the remaining failed checks, and mark the unread run untriaged in the plan. CI still running: report the partial state and stop; do not diagnose incomplete checks. - Logs do not identify a root cause (insufficient logs, unreproducible secret or environment failure): report the evidence found; no guessed fix; in autonomous mode push nothing.
- The smallest fix requires widening scope beyond the failing job/step or broadening workflow permissions: stop and report why; never expand permissions to make tests pass.
- Local check does not reproduce the CI failure: do not push. Report the reproduction gap and attach the CI log as the blocked state.
- A fix resolves one check but breaks another: if not yet pushed, amend or discard the fix; if pushed, revert it. Re-triage treating the new break as a root cause.
- Interactive non-convergence: report remaining failures and the diff applied; roll back edited files via VCS when the user requests it.
- Autonomous run still red: diagnose the new failure and apply one further in-scope fix; still failing with no new actionable cause → blocked, with the failing-run link, analysis so far, and the exact failing job. Leave the branch and runs in place; never delete or force-push to hide a failure unless the user asks.
- Push or watch interrupted: the remote branch state is authoritative: re-run
gh run list --branch ci-fix/<original-branch> --limit 1and resume watching; never re-push blindly. - Push rejected (protected branch, conflict, permission): do not force-push. Report the rejection reason as the blocked state.
- Non-convergent: after repeated fix cycles, new failures keep appearing in the fix's own code. Stop. Report the non-convergent state with the failure chain; do not claim green or that the done predicate holds.
- Sweep flake or ambiguity: the failure does not reproduce locally, or the root cause cannot be isolated to one minimal change: classify non-actionable or hand off with evidence; do not patch.
- Sweep budget exhaustion or breaker trip: the attempt cap is reached, or non-convergence and flakes repeat: stop, hand off with the accumulated evidence and the attempt count, and do not start a new repair or swallow the error.
- Sweep partial result: a worktree repair is never merged; a partial patch is returned as a proposal with its verification state, never as a completed fix. The worktree is the only writable sweep surface and may be discarded.
Output
Classify: one chat record with the check name, run identifier, observed failure signal, the single deterministic class, the next owner, and the one-sentence next action; no-CI and blocked states are included as their own classes, never omitted; the reviewable fix plan follows when a PR and gh are available, with untriaged checks explicitly marked and no code changes applied. Fix interactive: failing-check report (check name, run URL, log snippet, missing-log notes), then the ordered root-cause record (per root cause: check name, root cause, changed paths), applied diff, local-check results, and the observed current gh pr checks status; terminal local-fix-ready, blocked with the exact reason, or non-converged. Fix autonomous: the ordered root-cause record plus a summary in fixed order: failing run (link or ID), root cause, fix (what changed), verification (new run link showing green); terminal checks-pass, blocked (exact reason), or non-passing (link and analysis). Sweep: a terminal classification per swept check, reproduced (with the minimal verified patch as a proposal, verifier evidence, and the attempt count), non-actionable (handoff record with the accumulated evidence), or blocked (stopping reason and accumulated evidence).
Files (outline-driven-development)
-
agents
-
openai.yaml 179 B
interface: display_name: "CI Fix" short_description: "Use when \"CI is red\", \"fix the checks\", or \"make CI green\", one check needs classifying, or a bounded sweep runs."
-
-
scripts
-
inspect_pr_checks.py 14.9 KB
#!/usr/bin/env python3 from __future__ import annotations import argparse import json import re import subprocess import sys from pathlib import Path from typing import Any, Iterable, Sequence FAILURE_CONCLUSIONS = { "failure", "cancelled", "timed_out", "action_required", } FAILURE_STATES = { "failure", "error", "cancelled", "timed_out", "action_required", } FAILURE_BUCKETS = {"fail"} FAILURE_MARKERS = ( "error", "fail", "failed", "traceback", "exception", "assert", "panic", "fatal", "timeout", "segmentation fault", ) DEFAULT_MAX_LINES = 160 DEFAULT_CONTEXT_LINES = 30 PENDING_LOG_MARKERS = ( "still in progress", "log will be available when it is complete", ) class GhResult: def __init__(self, returncode: int, stdout: str, stderr: str): self.returncode = returncode self.stdout = stdout self.stderr = stderr def run_gh_command(args: Sequence[str], cwd: Path) -> GhResult: process = subprocess.run( ["gh", *args], cwd=cwd, text=True, capture_output=True, ) return GhResult(process.returncode, process.stdout, process.stderr) def run_gh_command_raw(args: Sequence[str], cwd: Path) -> tuple[int, bytes, str]: process = subprocess.run( ["gh", *args], cwd=cwd, capture_output=True, ) stderr = process.stderr.decode(errors="replace") return process.returncode, process.stdout, stderr def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser( description=( "Inspect failing GitHub PR checks, fetch GitHub Actions logs, and extract a " "failure snippet." ), formatter_class=argparse.ArgumentDefaultsHelpFormatter, ) parser.add_argument( "--repo", default=".", help="Path inside the target Git repository." ) parser.add_argument( "--pr", default=None, help="PR number or URL (defaults to current branch PR)." ) parser.add_argument("--max-lines", type=int, default=DEFAULT_MAX_LINES) parser.add_argument("--context", type=int, default=DEFAULT_CONTEXT_LINES) parser.add_argument( "--json", action="store_true", help="Emit JSON instead of text output." ) return parser.parse_args() def main() -> int: args = parse_args() repo_root = find_git_root(Path(args.repo)) if repo_root is None: print("Error: not inside a Git repository.", file=sys.stderr) return 1 if not ensure_gh_available(repo_root): return 1 pr_value = resolve_pr(args.pr, repo_root) if pr_value is None: return 1 checks = fetch_checks(pr_value, repo_root) if checks is None: return 1 failing = [c for c in checks if is_failing(c)] if not failing: print(f"PR #{pr_value}: no failing checks detected.") return 0 results = [] for check in failing: results.append( analyze_check( check, repo_root=repo_root, max_lines=max(1, args.max_lines), context=max(1, args.context), ) ) if args.json: print(json.dumps({"pr": pr_value, "results": results}, indent=2)) else: render_results(pr_value, results) return 1 def find_git_root(start: Path) -> Path | None: result = subprocess.run( ["git", "rev-parse", "--show-toplevel"], cwd=start, text=True, capture_output=True, ) if result.returncode != 0: return None return Path(result.stdout.strip()) def ensure_gh_available(repo_root: Path) -> bool: result = run_gh_command(["auth", "status"], cwd=repo_root) if result.returncode == 0: return True message = (result.stderr or result.stdout or "").strip() print(message or "Error: gh not authenticated.", file=sys.stderr) return False def resolve_pr(pr_value: str | None, repo_root: Path) -> str | None: if pr_value: return pr_value result = run_gh_command(["pr", "view", "--json", "number"], cwd=repo_root) if result.returncode != 0: message = (result.stderr or result.stdout or "").strip() print(message or "Error: unable to resolve PR.", file=sys.stderr) return None try: data = json.loads(result.stdout or "{}") except json.JSONDecodeError: print("Error: unable to parse PR JSON.", file=sys.stderr) return None number = data.get("number") if not number: print("Error: no PR number found.", file=sys.stderr) return None return str(number) def fetch_checks(pr_value: str, repo_root: Path) -> list[dict[str, Any]] | None: primary_fields = [ "name", "state", "conclusion", "detailsUrl", "startedAt", "completedAt", ] result = run_gh_command( ["pr", "checks", pr_value, "--json", ",".join(primary_fields)], cwd=repo_root, ) if result.returncode != 0: message = "\n".join(filter(None, [result.stderr, result.stdout])).strip() available_fields = parse_available_fields(message) if available_fields: fallback_fields = [ "name", "state", "bucket", "link", "startedAt", "completedAt", "workflow", ] selected_fields = [ field for field in fallback_fields if field in available_fields ] if not selected_fields: print( "Error: no usable fields available for gh pr checks.", file=sys.stderr, ) return None result = run_gh_command( ["pr", "checks", pr_value, "--json", ",".join(selected_fields)], cwd=repo_root, ) if result.returncode != 0: message = (result.stderr or result.stdout or "").strip() print(message or "Error: gh pr checks failed.", file=sys.stderr) return None else: print(message or "Error: gh pr checks failed.", file=sys.stderr) return None try: data = json.loads(result.stdout or "[]") except json.JSONDecodeError: print("Error: unable to parse checks JSON.", file=sys.stderr) return None if not isinstance(data, list): print("Error: unexpected checks JSON shape.", file=sys.stderr) return None return data def is_failing(check: dict[str, Any]) -> bool: conclusion = normalize_field(check.get("conclusion")) if conclusion in FAILURE_CONCLUSIONS: return True state = normalize_field(check.get("state") or check.get("status")) if state in FAILURE_STATES: return True bucket = normalize_field(check.get("bucket")) return bucket in FAILURE_BUCKETS def analyze_check( check: dict[str, Any], repo_root: Path, max_lines: int, context: int, ) -> dict[str, Any]: url = check.get("detailsUrl") or check.get("link") or "" run_id = extract_run_id(url) job_id = extract_job_id(url) base: dict[str, Any] = { "name": check.get("name", ""), "detailsUrl": url, "runId": run_id, "jobId": job_id, } if run_id is None: base["status"] = "external" base["note"] = "No GitHub Actions run id detected in detailsUrl." return base metadata = fetch_run_metadata(run_id, repo_root) log_text, log_error, log_status = fetch_check_log( run_id=run_id, job_id=job_id, repo_root=repo_root, ) if log_status == "pending": base["status"] = "log_pending" base["note"] = log_error or "Logs are not available yet." if metadata: base["run"] = metadata return base if log_error: base["status"] = "log_unavailable" base["error"] = log_error if metadata: base["run"] = metadata return base snippet = extract_failure_snippet(log_text, max_lines=max_lines, context=context) base["status"] = "ok" base["run"] = metadata or {} base["logSnippet"] = snippet base["logTail"] = tail_lines(log_text, max_lines) return base def extract_run_id(url: str) -> str | None: if not url: return None for pattern in (r"/actions/runs/(\d+)", r"/runs/(\d+)"): match = re.search(pattern, url) if match: return match.group(1) return None def extract_job_id(url: str) -> str | None: if not url: return None match = re.search(r"/actions/runs/\d+/job/(\d+)", url) if match: return match.group(1) match = re.search(r"/job/(\d+)", url) if match: return match.group(1) return None def fetch_run_metadata(run_id: str, repo_root: Path) -> dict[str, Any] | None: fields = [ "conclusion", "status", "workflowName", "name", "event", "headBranch", "headSha", "url", ] result = run_gh_command( ["run", "view", run_id, "--json", ",".join(fields)], cwd=repo_root ) if result.returncode != 0: return None try: data = json.loads(result.stdout or "{}") except json.JSONDecodeError: return None if not isinstance(data, dict): return None return data def fetch_check_log( run_id: str, job_id: str | None, repo_root: Path, ) -> tuple[str, str, str]: log_text, log_error = fetch_run_log(run_id, repo_root) if not log_error: return log_text, "", "ok" if is_log_pending_message(log_error) and job_id: job_log, job_error = fetch_job_log(job_id, repo_root) if job_log: return job_log, "", "ok" if job_error and is_log_pending_message(job_error): return "", job_error, "pending" if job_error: return "", job_error, "error" return "", log_error, "pending" if is_log_pending_message(log_error): return "", log_error, "pending" return "", log_error, "error" def fetch_run_log(run_id: str, repo_root: Path) -> tuple[str, str]: result = run_gh_command(["run", "view", run_id, "--log"], cwd=repo_root) if result.returncode != 0: error = (result.stderr or result.stdout or "").strip() return "", error or "gh run view failed" return result.stdout, "" def fetch_job_log(job_id: str, repo_root: Path) -> tuple[str, str]: repo_slug = fetch_repo_slug(repo_root) if not repo_slug: return "", "Error: unable to resolve repository name for job logs." endpoint = f"/repos/{repo_slug}/actions/jobs/{job_id}/logs" returncode, stdout_bytes, stderr = run_gh_command_raw( ["api", endpoint], cwd=repo_root ) if returncode != 0: message = (stderr or stdout_bytes.decode(errors="replace")).strip() return "", message or "gh api job logs failed" if is_zip_payload(stdout_bytes): return "", "Job logs returned a zip archive; unable to parse." return stdout_bytes.decode(errors="replace"), "" def fetch_repo_slug(repo_root: Path) -> str | None: result = run_gh_command(["repo", "view", "--json", "nameWithOwner"], cwd=repo_root) if result.returncode != 0: return None try: data = json.loads(result.stdout or "{}") except json.JSONDecodeError: return None name_with_owner = data.get("nameWithOwner") if not name_with_owner: return None return str(name_with_owner) def normalize_field(value: Any) -> str: if value is None: return "" return str(value).strip().lower() def parse_available_fields(message: str) -> list[str]: if "Available fields:" not in message: return [] fields: list[str] = [] collecting = False for line in message.splitlines(): if "Available fields:" in line: collecting = True continue if not collecting: continue field = line.strip() if not field: continue fields.append(field) return fields def is_log_pending_message(message: str) -> bool: lowered = message.lower() return any(marker in lowered for marker in PENDING_LOG_MARKERS) def is_zip_payload(payload: bytes) -> bool: return payload.startswith(b"PK") def extract_failure_snippet(log_text: str, max_lines: int, context: int) -> str: lines = log_text.splitlines() if not lines: return "" marker_index = find_failure_index(lines) if marker_index is None: return "\n".join(lines[-max_lines:]) start = max(0, marker_index - context) end = min(len(lines), marker_index + context) window = lines[start:end] if len(window) > max_lines: window = window[-max_lines:] return "\n".join(window) def find_failure_index(lines: Sequence[str]) -> int | None: for idx in range(len(lines) - 1, -1, -1): lowered = lines[idx].lower() if any(marker in lowered for marker in FAILURE_MARKERS): return idx return None def tail_lines(text: str, max_lines: int) -> str: if max_lines <= 0: return "" lines = text.splitlines() return "\n".join(lines[-max_lines:]) def render_results(pr_number: str, results: Iterable[dict[str, Any]]) -> None: results_list = list(results) print(f"PR #{pr_number}: {len(results_list)} failing checks analyzed.") for result in results_list: print("-" * 60) print(f"Check: {result.get('name', '')}") if result.get("detailsUrl"): print(f"Details: {result['detailsUrl']}") run_id = result.get("runId") if run_id: print(f"Run ID: {run_id}") job_id = result.get("jobId") if job_id: print(f"Job ID: {job_id}") status = result.get("status", "unknown") print(f"Status: {status}") run_meta = result.get("run", {}) if run_meta: branch = run_meta.get("headBranch", "") sha = (run_meta.get("headSha") or "")[:12] workflow = run_meta.get("workflowName") or run_meta.get("name") or "" conclusion = run_meta.get("conclusion") or run_meta.get("status") or "" print(f"Workflow: {workflow} ({conclusion})") if branch or sha: print(f"Branch/SHA: {branch} {sha}") if run_meta.get("url"): print(f"Run URL: {run_meta['url']}") if result.get("note"): print(f"Note: {result['note']}") if result.get("error"): print(f"Error fetching logs: {result['error']}") continue snippet = result.get("logSnippet") or "" if snippet: print("Failure snippet:") print(indent_block(snippet, prefix=" ")) else: print("No snippet available.") print("-" * 60) def indent_block(text: str, prefix: str = " ") -> str: return "\n".join(f"{prefix}{line}" for line in text.splitlines()) if __name__ == "__main__": raise SystemExit(main())
-
-
SKILL.md 19.7 KB
--- name: ci-fix description: 'Use when "CI is red", "fix the checks", or "make CI green", one check needs classifying, or a bounded sweep runs. Not for deploys, credentials, or rerun-as-fix. Non-CI bugs: use strike-the-root.' --- # CI fix Repair, classify, or sweep failing CI on a PR or branch. Three modes: `fix` (default; interactive or autonomous), `classify` (read-only diagnosis), and `sweep` (bounded background monitor). Never widen workflow permissions, never add `pull_request_target`, never offer a rerun in place of a code fix. Merge conflicts route to `resolve-merge-conflicts`; non-CI bugs route to `strike-the-root`. ## Contract | Field | Bound contract | |---|---| | Trigger | Fix: user asks to diagnose or repair failing CI on a PR or branch; autonomous fix only when the user explicitly asks for the fix to be pushed. Classify: a check is failed, absent, pending too long, unstable, or reported unexpectedly and no patch is requested. Sweep: a recurring or requested sweep monitors CI failures over a bounded attempt window. | | Authority | Human-gated: classify is read-only and writes nothing. Fix interactive writes only named source files after plan approval; rollback is version control. Fix autonomous pushes a fix branch, never opens a PR, and is a remote mutation that requires explicit human invocation. Sweep writes one minimal isolated repair in a worktree; rollback is discarding the worktree; it never pushes, merges, or publishes. | | Side effect | Classify: chat output only. Fix interactive: named local source files edited and a local-equivalent check passes; remote CI unchanged. Fix autonomous: a fix branch pushed with the approved change set and a new CI run observed. Sweep: one worktree repair plus one independent verifier run, returned as a proposal. | | Done | Classify: one deterministic failure class supported by observed evidence, a next owner, a one-sentence next action, and a fix plan when a PR and `gh` are available; nothing mutated. Fix interactive: failing checks identified, the approved local fix applied, and every local equivalent passes; return `local-fix-ready` because remote CI is unchanged. Fix autonomous: a new run on `ci-fix/<original-branch>` is green and the ordered root-cause record is delivered; `checks-pass` is claimed only after a green remote run is observed. Sweep: root cause reproduced or classified non-actionable; any patch is minimal, independently verified, and returned as a proposal; retries stop at the configured cap without symptom patching. | ## Inputs - `mode`: `fix` (default), `classify`, or `sweep`. - `repo`: path inside the repo (default `.`). Must be a Git repository. - Target: PR number or URL, original branch name, or run ID; defaults to the current branch's PR. A working tree already carrying the in-progress fix is fine. - `gh` authenticated for the repo host with workflow/repo scopes. Classify stands alone without `gh`; in fix and sweep an unauthenticated CLI stops the skill before any change. - Optional: a specific job ID for full-log inspection, or an artifact download when logs alone do not identify the cause. - Classify: the check name, its status (failed, absent, pending, unstable, or unexpected), and the run identifier or URL that produced the report; optional: the failing job log, the changed files or commit range under test, and any prior classification of the same check. - Sweep: the base commit the CI run used, the verifier command that independently confirms a repair, and the attempt cap; optional: a flake-detection window and a circuit-breaker threshold. ## Procedure 1. Select the mode: `classify` when the user asks for a diagnosis or plan without a patch, `sweep` when a bounded monitor over an attempt window is requested, `fix` otherwise. Done when: the mode is stated and its inputs above are present, or the missing input is reported. 2. Set `GH_PAGER=cat` on every `gh` invocation; the CLI has no global `--no-pager` option and blocks on a pager in non-interactive contexts without it. Verify authentication with `gh auth status`. In fix and sweep, stop and ask the user to run `gh auth login` when unauthenticated; no mutation before this. In classify, continue without `gh`; the classification stands alone. Done when: an authenticated identity is confirmed by `gh auth status` output, the handoff is asked and no mutation has occurred, or classify proceeds without `gh` and says so. 3. Record the failing identity. Classify: record the check name, status, and run identifier before reading anything else; an absent check or a check that never ran is a distinct input, not a missing one. Fix and sweep: resolve the failing run; PR: `gh pr view --json number,url`; non-PR branch or supplied run ID: `gh run list --branch <branch> --status failure --limit 5`, then `gh run view <run-id>`. Done when: classify has the check name, status, and run identifier recorded, or fix and sweep have one failing run identified by its run ID and branch name from command output. 4. Gather failure evidence (GitHub Actions only). Prefer the bundled script, which handles gh field drift, run-id/job-id extraction, failure-snippet extraction, and exits non-zero while failures remain: - `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "<number-or-url>"` (`--json` for machine output). - Manual fallback: `GH_PAGER=cat gh pr checks <pr> --json name,state,bucket,link,startedAt,completedAt,workflow`; rerun with the fields `gh` accepts if one is rejected. Extract the run id from `detailsUrl`, then `GH_PAGER=cat gh run view <run_id> --json name,workflowName,conclusion,status,url,event,headBranch,headSha`, `GH_PAGER=cat gh run view <run_id> --log-failed` for the failed jobs only or `--log` for all, `GH_PAGER=cat gh run view <run-id> --log --job <job-id>` for one failing job; for evidence logs cannot supply, `gh run download <run-id> -D .artifacts/<run-id>`. If a run log is still in progress, fetch job logs via `gh api "/repos/<owner>/<repo>/actions/jobs/<job_id>/logs"`. - A check whose `detailsUrl` is not a GitHub Actions run is external: report the URL only; do not attempt Buildkite or other providers. Done when: every failing check has a name, URL, and a log snippet from command output, or an explicit `log_pending`/`log_unavailable` marker; never a fabricated snippet. In classify, preserve the exact failure line, signal, or message. 5. Mode classify (read-only; stop before any write): 1. Compare the failure against the commit range and changed files under test; decide whether the failing code path was touched by the change or predates it. Before flagging a test failure as a regression, check whether the same test previously passed in CI and note environment-specific or flaky cases rather than treating them as regressions. Done when: the failing path is determined as in-diff or pre-existing. 2. Classify into exactly one class: `regression` (the change introduced or exposed the failure; the failing path is in the diff), `flake/watch` (timing-, order-, or environment-dependent; passes on retry or across runs without a code change), `infrastructure` (runner, network, quota, service outage, or resource exhaustion, not the code under test), `configuration` (build, config, dependency, or environment setup, not product logic), `policy/absent-CI` (absent, skipped, not configured, or blocked by a branch-protection or policy rule), or `human escalation` (evidence insufficient, contradictory, or outside the five classes; a human must decide). If a prior classification of the same check matches the new evidence, note the repeat; if it contradicts, re-classify from the new evidence. Done when: the failure is classified into exactly one class from observed evidence. 3. Assign the next owner from the class: regression and configuration to the change author, flake/watch to the test or platform owner, infrastructure to the platform or runner owner, policy/absent-CI to the repository or CI-config owner, human escalation to a human reviewer. State the next action concretely in one sentence. Done when: the next owner is assigned from the class and the one-sentence next action is recorded. 4. Emit a reviewable fix plan when a PR and authenticated `gh` are available: view the PR with `GH_PAGER=cat gh pr view <branch> --json number,title,url,state` and fetch the check rollup with `--json statusCheckRollup`; extract each failed check's logs with `GH_PAGER=cat gh run view <run-id> --log-failed`, collecting error messages with file paths and line numbers, compilation errors, lint names, test failure messages and stack traces, and build root causes. If any check is still in progress, report the partial state and stop; do not diagnose incomplete checks. Then write the plan: a problem statement summarizing the failing checks, the current state listing each error and its location, proposed changes grouped by category (formatting, linting, compilation, test failures, platform-specific), and validation steps (fmt, lint, tests, presubmit). Fix one category at a time. Apply no code change. Mark any unread run as untriaged. Done when: the plan carries the problem statement, current state, per-category proposed changes, and validation steps, or the plan is skipped and reported because no PR or `gh` is available. Then stop; if any further step would require a write, return `human escalation`. 6. Mode fix: 1. Triage every failing check: classify each as a root cause or a downstream symptom of another failure. A single root cause may surface as several failing checks. Order the work so root causes are fixed before the symptoms they produce. Done when: every failing check is classified as root cause or symptom in a written list, and the work order places root causes before their symptoms. 2. For each root-cause failure, read the failure log and the relevant source to determine the smallest deterministic fix for that cause; prefer a code change over workflow plumbing; for a flaky test, fix the nondeterminism instead of rerunning. Request approval before editing any file. Done when: the plan is approved in writing with the file paths and change described, or refused; then report the plan and wait without editing. 3. Implement the approved plan and summarize diffs and tests. Keep the change set inside the failing job/step where possible. Done when: `git diff --name-only` lists only the approved file paths and the diff summary matches the approved plan. 4. Run the local equivalent of each failing CI check to confirm the fix before any push. Done when: the local check command exits 0 with its output captured, confirming the fix reproduces green locally. 5. Capture the current remote status with `gh pr checks <pr>`. Interactive mode has not pushed, so report `local-fix-ready` with the local proof and the observed red or pending remote checks; never imply that local edits changed CI. If the remote checks are already green because another actor updated the branch, report the observed green state without attributing it to the local patch. Done when: the current `gh pr checks` output is captured and the result is `local-fix-ready` with remote status quoted, or an independently updated remote run is observed green with its output quoted. Autonomous branch (explicit user ask only): enter only when the user asks for the fix to be pushed (for example "push the fix" or "land it on a fix branch"); never self-initiated. Before any mutation, preview the failing job, the exact change set, and the target branch `ci-fix/<original-branch>`; the plan approval from fix step 2 doubles as the push approval once the preview is acknowledged. 1. If workflow files must change: keep existing `permissions:` minimal, never broaden token access, no `pull_request_target` unless the user explicitly requested it. Done when: `git diff` on workflow files touches no permission key outside the previewed set. 2. Create or update the branch. New: `git checkout -b ci-fix/<original-branch>`. Existing: show its current commits, confirm reuse with the user, `git checkout ci-fix/<original-branch>`, then `git pull origin <original-branch>`. Done when: the branch is checked out and `git log --oneline -1` shows it carries only the previewed change. 3. Stage only the exact paths in the approved preview with `git add -- <paths>`, then inspect the staged path list and patch. Stop if the staged set contains any extra path. Commit `fix: resolve CI failure in <job-name>`, then push `git push -u origin ci-fix/<original-branch>` (plain `git push` once upstream is set). Done when: `git diff --cached --name-only` lists only the approved paths and `git push` succeeds with the remote branch at the pushed SHA confirmed by `git rev-parse origin/ci-fix/<original-branch>`. 4. Watch: `gh run list --branch ci-fix/<original-branch> --limit 1` for the new run id, then `gh run watch <new-run-id> --exit-status`. Re-trigger only failed jobs with `gh run rerun <new-run-id> --failed` when the evidence is infrastructure or a confirmed flake and the user approves; a rerun never substitutes for a deterministic fix. Done when: `gh run view <new-run-id> --json conclusion` reports `success`, or the failure and recovery section applies. 5. If new failures appear, repeat from fix step 1 treating the new failures as the current set. Stop only when every check in the set is green. Done when: every check in the set is green, or a failure class in the failure and recovery section applies. 7. Mode sweep: 1. On each tick, fetch the current set of failing CI checks for the target run and record the attempt number against the configured cap. If the attempt cap is reached, stop and hand off; do not start a new repair. Done when: the failing checks are fetched and the attempt number is recorded, or the sweep stops at the cap with a handoff. 2. Pick one failing check and reproduce the failure locally in an isolated worktree created from the base commit the CI run used. Classify the failure: reproduce the root cause, or classify it non-actionable (flake, environment, upstream). If the failure is a flake or the cause is ambiguous, hand off and do not patch. Done when: the failure is reproduced locally or confirmed non-reproducible, and classified as root cause or non-actionable, or handed off. 3. For a reproducible root cause, implement one minimal isolated repair in the worktree: the smallest change that fixes the reproduced cause and nothing else. Run the verifier in the worktree independently of the repair; confirm the failing check passes and no other check regresses. If the verifier fails or regresses, do not widen the patch; increment the attempt counter and either retry within budget or hand off. Done when: the minimal repair is implemented and the verifier confirms the fix with no regression, or the verifier failure is recorded and the retry-or-handoff decision is made. 4. Return the verified patch as a proposal (diff or branch) with the reproduced root cause, the verifier evidence, and the attempt count. Do not push, merge, or publish; a worktree repair is never merged. If the circuit breaker trips (repeated non-convergence, repeated flakes, or budget exhaustion), stop the sweep and hand off with the accumulated evidence. Done when: the verified patch is returned as a proposal with evidence, or the sweep stops with a handoff and accumulated evidence. ## Failure and recovery - gh unauthenticated, or no failing run found for the supplied PR/branch/run ID: stop and report; nothing committed or pushed. In classify, skip the fix plan, report the missing prerequisite, and never attempt credential setup. - Classify evidence insufficient or contradictory: do not guess a class. Return `human escalation` naming the missing evidence, or stating both signals when they imply different classes; never average or pick arbitrarily. - Classify no-CI state: a check that is absent or never ran is `policy/absent-CI`; surface it, never treat it as passing or skip it. A check pending past the expected window or blocked by policy is surfaced with its next owner, never marked done. No PR for the branch: classification stands alone, skip the fix plan, and report that no PR exists; do not create one. - Classify partial evidence: classify each inspectable check and explicitly mark the rest as unevaluated. `gh run view --log-failed` empty or erroring: report the unread run, continue triaging the remaining failed checks, and mark the unread run untriaged in the plan. CI still running: report the partial state and stop; do not diagnose incomplete checks. - Logs do not identify a root cause (insufficient logs, unreproducible secret or environment failure): report the evidence found; no guessed fix; in autonomous mode push nothing. - The smallest fix requires widening scope beyond the failing job/step or broadening workflow permissions: stop and report why; never expand permissions to make tests pass. - Local check does not reproduce the CI failure: do not push. Report the reproduction gap and attach the CI log as the blocked state. - A fix resolves one check but breaks another: if not yet pushed, amend or discard the fix; if pushed, revert it. Re-triage treating the new break as a root cause. - Interactive non-convergence: report remaining failures and the diff applied; roll back edited files via VCS when the user requests it. - Autonomous run still red: diagnose the new failure and apply one further in-scope fix; still failing with no new actionable cause → blocked, with the failing-run link, analysis so far, and the exact failing job. Leave the branch and runs in place; never delete or force-push to hide a failure unless the user asks. - Push or watch interrupted: the remote branch state is authoritative: re-run `gh run list --branch ci-fix/<original-branch> --limit 1` and resume watching; never re-push blindly. - Push rejected (protected branch, conflict, permission): do not force-push. Report the rejection reason as the blocked state. - Non-convergent: after repeated fix cycles, new failures keep appearing in the fix's own code. Stop. Report the non-convergent state with the failure chain; do not claim green or that the done predicate holds. - Sweep flake or ambiguity: the failure does not reproduce locally, or the root cause cannot be isolated to one minimal change: classify non-actionable or hand off with evidence; do not patch. - Sweep budget exhaustion or breaker trip: the attempt cap is reached, or non-convergence and flakes repeat: stop, hand off with the accumulated evidence and the attempt count, and do not start a new repair or swallow the error. - Sweep partial result: a worktree repair is never merged; a partial patch is returned as a proposal with its verification state, never as a completed fix. The worktree is the only writable sweep surface and may be discarded. ## Output Classify: one chat record with the check name, run identifier, observed failure signal, the single deterministic class, the next owner, and the one-sentence next action; no-CI and blocked states are included as their own classes, never omitted; the reviewable fix plan follows when a PR and `gh` are available, with untriaged checks explicitly marked and no code changes applied. Fix interactive: failing-check report (check name, run URL, log snippet, missing-log notes), then the ordered root-cause record (per root cause: check name, root cause, changed paths), applied diff, local-check results, and the observed current `gh pr checks` status; terminal `local-fix-ready`, `blocked` with the exact reason, or `non-converged`. Fix autonomous: the ordered root-cause record plus a summary in fixed order: failing run (link or ID), root cause, fix (what changed), verification (new run link showing green); terminal `checks-pass`, `blocked` (exact reason), or `non-passing` (link and analysis). Sweep: a terminal classification per swept check, `reproduced` (with the minimal verified patch as a proposal, verifier evidence, and the attempt count), `non-actionable` (handoff record with the accumulated evidence), or `blocked` (stopping reason and accumulated evidence).
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.