ci-design
Vocabulary and principles for well-designed CI. Use when the user wants to design, review, or audit CI, says CI is noisy, slow, or expensive, or is designing a workflow yml.
Install
npx skills add https://github.com/ConnorGriffin/skills/tree/main/skills/tools/ci-design
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install connorgriffin-skills@llmmart
git clone https://github.com/ConnorGriffin/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole connorgriffin/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
CI Design
Design CI so cost tracks the surface area actually touched, not the number of pushes. Use this language wherever CI is being designed, reviewed, or audited.
Priorities
Fix in this order — each tier assumes the ones above it are already sound:
- Minutes and billing waste. Runner tier, caching, unconditional jobs. Money leaks here even when every run is green.
- PR feedback latency. How long a contributor waits to learn a push is good or bad. Concurrency and trigger surface live here.
- Run and notification volume. Duplicate or invisible checks, noisy scheduling. Annoying, but cheaper than the first two.
Baseline failure noise (a flaky test, a known-red job) matters less than any of these — it's visible and locally fixable. Structural waste isn't; it compounds silently across every run.
Vocabulary
Use these terms exactly.
Trigger surface — the product of events, branches, and paths that fire a
workflow (on: push/pull_request × branch filters × path filters). The
trigger surface is the first lever: a workflow that fires on every push to
every branch has a trigger surface many times larger than the work it
actually needs to validate.
Path filtering — restricting a job to run only when files it cares about
changed. The required-checks-safe pattern: never put paths: at the
workflow level on a job that's a required status check — GitHub can leave a
required check permanently pending if its workflow never triggers. Instead,
filter inside the job with a paths-filter step that gates the real work,
so the workflow still runs and reports green (or explicitly skipped) on
every PR.
Concurrency group — a concurrency: key that cancels superseded runs on
the same branch/PR (cancel-in-progress: true), so a burst of pushes
collapses to one live run instead of a growing queue. Release and deploy
paths are the deliberate exception: don't cancel a run that's mid-deploy
just because a new commit landed.
Caching — persisting dependencies or toolchains across runs, keyed on something that changes only when the cache should invalidate (a lockfile hash, a pinned tool version). The sin isn't the absence of a cache line — it's reinstalling a toolchain, browser binary, or dependency tree from scratch on every single run when the inputs didn't change.
Runner cost tiers — macOS runners cost several times what Linux runners cost per minute; Windows sits in between. A job earns a pricier runner only by a real platform dependency (building a macOS binary, testing an Xcode-only path) — never by inertia or a single assumption (like a temp path) that's trivially fixable on Linux.
Scheduled scans — expensive or slow analysis (security scanning, full matrix builds) that doesn't need to gate every PR. When merge velocity is high, put it on a schedule plus main-branch pushes instead of every pull request — it still runs regularly, just not once per push.
Iteration burst — a string of pushes to the same branch in quick succession, each firing its own run. Expected during active development, not itself a problem. The concurrency group is what determines whether a burst turns into cancellations (cheap, correct) or a queue of runs that finish stale and red (expensive, misleading).
Invisible checks — status checks that show up on a PR with no corresponding yml in the repo, most commonly GitHub's CodeQL default setup configured through repo settings rather than a workflow file. They can't be inventoried, diffed, or reviewed by reading the repo. Export them to a checked-in workflow so every check the repo runs is visible from its files.
Principles
- Every job answers "which changed files require me?" If a job can't name the files that would make it necessary, its trigger surface is too wide.
- Every workflow has exactly one visible definition on disk. No check should exist that isn't traceable to a yml file in the repo.
- Cost scales with touched surface area, not PR volume. Ten pushes that touch nothing relevant should cost less than one push that touches everything.
- CI is a signal. A check that's red mid-iteration by design — because nothing cancelled it, because it always runs even on draft churn — trains people to ignore red, which erodes the signal for the run that matters.
Going deeper
- Running an audit — see references/AUDIT-PLAYBOOK.md: the repeatable procedure for inventorying, measuring, and ranking CI findings across one or more repos.
Files (skills)
-
agents
-
openai.yaml 196 B
interface: display_name: "CI Design" short_description: "Vocabulary and principles for designing well-scoped, low-noise CI" default_prompt: "Use $ci-design to shape or audit this CI setup."
-
-
references
-
AUDIT-PLAYBOOK.md 2.9 KB
# CI Audit Playbook The repeatable procedure for auditing one or more repos' CI, distilled from the 2026-08-04 fleet audit. Assumes the vocabulary in `../SKILL.md`. ## Procedure 1. **Inventory the workflows.** Read every `.github/workflows/*.yml` in the repo. For each job, record: trigger surface (events, branches, paths), runner, matrix dimensions, concurrency group, caching, and any path filtering (workflow-level or in-job). 2. **Pull run stats.** ``` gh run list --limit 50 --json name,conclusion,event,startedAt,updatedAt,displayTitle ``` Compute failure rate, cancellation rate, and median duration per workflow. Group by day to tell a burst (many runs clustered in a short window, one branch) from chronic failure (spread evenly, many branches). 3. **Check for invisible checks.** Compare what's on disk against what actually runs: ``` gh api repos/<owner>/<repo>/code-scanning/default-setup ``` and the check-suites on a recent commit (`gh api repos/<owner>/<repo>/commits/<sha>/check-suites`) against the workflow files in the repo. Anything present in check-suites without a matching yml is invisible. 4. **Rank findings** by the priority order in `../SKILL.md`: minutes/billing waste first, then PR feedback latency, then run/notification volume. 5. **Ship fixes as one self-contained issue per repo.** Each issue states the finding, the evidence (numbers from steps 2-3), and the fix — scoped so it can be picked up and built without re-running the audit. ## Known failure modes Each of these was observed at least once in the 2026-08-04 audit. Check for all of them, not just the first one found. - **No path filters anywhere** — every job runs on every push, regardless of what changed. - **Missing concurrency groups** — stale runs queue up and finish instead of cancelling; `gh run list` shows 0 cancellations despite obvious push churn on the same branch. - **Uncached expensive installs** — Playwright + Chromium (or an equivalent heavy toolchain) reinstalled from scratch every run instead of keyed on a lockfile. - **Wrong runner tier** — e.g., Ansible tests pinned to `macos-latest` for a single temp-path assumption that's fixable on Linux. - **Unconditional publish jobs** — an image or package pushed on every main-branch push regardless of whether anything relevant changed. - **Per-PR multi-language CodeQL matrices** — full security-scan matrices running on every pull request instead of on a schedule plus main pushes. - **Duplicate check sets from settings-configured default setup** — GitHub's UI-configured CodeQL running alongside a checked-in workflow that does the same thing, doubling the check count. - **Permanent no-op jobs** — a job kept alive indefinitely only to detect-and-skip on every run, rather than being removed or replaced with in-job path filtering.
-
-
SKILL.md 4.7 KB
--- name: ci-design description: Vocabulary and principles for well-designed CI. Use when the user wants to design, review, or audit CI, says CI is noisy, slow, or expensive, or is designing a workflow yml. --- # CI Design Design CI so cost tracks the surface area actually touched, not the number of pushes. Use this language wherever CI is being designed, reviewed, or audited. ## Priorities Fix in this order — each tier assumes the ones above it are already sound: 1. **Minutes and billing waste.** Runner tier, caching, unconditional jobs. Money leaks here even when every run is green. 2. **PR feedback latency.** How long a contributor waits to learn a push is good or bad. Concurrency and trigger surface live here. 3. **Run and notification volume.** Duplicate or invisible checks, noisy scheduling. Annoying, but cheaper than the first two. Baseline failure noise (a flaky test, a known-red job) matters less than any of these — it's visible and locally fixable. Structural waste isn't; it compounds silently across every run. ## Vocabulary Use these terms exactly. **Trigger surface** — the product of events, branches, and paths that fire a workflow (`on: push/pull_request` × branch filters × path filters). The trigger surface is the first lever: a workflow that fires on every push to every branch has a trigger surface many times larger than the work it actually needs to validate. **Path filtering** — restricting a job to run only when files it cares about changed. The required-checks-safe pattern: never put `paths:` at the workflow level on a job that's a required status check — GitHub can leave a required check permanently pending if its workflow never triggers. Instead, filter *inside* the job with a paths-filter step that gates the real work, so the workflow still runs and reports green (or explicitly skipped) on every PR. **Concurrency group** — a `concurrency:` key that cancels superseded runs on the same branch/PR (`cancel-in-progress: true`), so a burst of pushes collapses to one live run instead of a growing queue. Release and deploy paths are the deliberate exception: don't cancel a run that's mid-deploy just because a new commit landed. **Caching** — persisting dependencies or toolchains across runs, keyed on something that changes only when the cache should invalidate (a lockfile hash, a pinned tool version). The sin isn't the absence of a cache line — it's reinstalling a toolchain, browser binary, or dependency tree from scratch on every single run when the inputs didn't change. **Runner cost tiers** — macOS runners cost several times what Linux runners cost per minute; Windows sits in between. A job earns a pricier runner only by a real platform dependency (building a macOS binary, testing an Xcode-only path) — never by inertia or a single assumption (like a temp path) that's trivially fixable on Linux. **Scheduled scans** — expensive or slow analysis (security scanning, full matrix builds) that doesn't need to gate every PR. When merge velocity is high, put it on a schedule plus main-branch pushes instead of every pull request — it still runs regularly, just not once per push. **Iteration burst** — a string of pushes to the same branch in quick succession, each firing its own run. Expected during active development, not itself a problem. The concurrency group is what determines whether a burst turns into cancellations (cheap, correct) or a queue of runs that finish stale and red (expensive, misleading). **Invisible checks** — status checks that show up on a PR with no corresponding yml in the repo, most commonly GitHub's CodeQL default setup configured through repo settings rather than a workflow file. They can't be inventoried, diffed, or reviewed by reading the repo. Export them to a checked-in workflow so every check the repo runs is visible from its files. ## Principles - **Every job answers "which changed files require me?"** If a job can't name the files that would make it necessary, its trigger surface is too wide. - **Every workflow has exactly one visible definition on disk.** No check should exist that isn't traceable to a yml file in the repo. - **Cost scales with touched surface area, not PR volume.** Ten pushes that touch nothing relevant should cost less than one push that touches everything. - **CI is a signal.** A check that's red mid-iteration by design — because nothing cancelled it, because it always runs even on draft churn — trains people to ignore red, which erodes the signal for the run that matters. ## Going deeper - **Running an audit** — see [references/AUDIT-PLAYBOOK.md](references/AUDIT-PLAYBOOK.md): the repeatable procedure for inventorying, measuring, and ranking CI findings across one or more repos.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.