sota-docs-workflow
State-of-the-art documentation and engineering-collaboration guidance (2026) covering documentation architecture (Diátaxis, docs-as-code, READMEs, runbooks, onboarding, AI-readable docs), API/reference docs and changelogs, and the team workflow around code: pull requests, code re
Install
npx skills add https://github.com/martinholovsky/SOTA-skills/tree/main/skills/sota-docs-workflow
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install martinholovsky-sota-skills@llmmart
git clone https://github.com/martinholovsky/SOTA-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole martinholovsky/sota-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
SOTA Docs & Engineering Workflow
Purpose
Expert-level rules for the artifacts around the code: documentation (structure, reference, freshness, runbooks) and the collaboration workflow (PRs, review, commits, releases). These are the highest-leverage, lowest-glamour practices — review latency and doc decay quietly dominate team throughput. Rules are imperative with rationale and good/bad examples; every rules file ends with an audit checklist. Load only the files relevant to the task via the index below.
Boundaries: ADR practice lives in sota-architecture; API contract design in
sota-api-design; tag signing/provenance and CI supply chain in
sota-devsecops. This skill references them rather than repeating them.
BUILD mode
When creating docs or setting up workflow:
- Classify before writing. Every doc is exactly one Diátaxis mode
(tutorial / how-to / reference / explanation) and is titled accordingly;
mixed-mode pages are the defect to design out (
rules/01§1). - Docs live with the code: in-repo, PR-reviewed, CI link-checked, examples
executed. If a doc can't change in the same PR as the code, it will decay
(
rules/01§2, §4). - README = what / why / 5-minute quickstart / honest status, then links
out (
rules/01§3). Runbooks are alert-linked and command-exact (rules/01§5). Ship the documentation baseline — README + LICENSE + CHANGELOG always; SECURITY/CONTRIBUTING/CODE_OF_CONDUCT once public — each with one canonical home, not scattered copies (rules/01§8). - Reference is generated from OpenAPI/docstrings/rustdoc/godoc with
warnings-as-errors; doc comments carry the why, contract, and failure
modes; examples run in CI (
rules/02§1–4). - Changelog from day one: Keep a Changelog format,
Unreleasedsection updated in the PR that makes the change, user-impact language (rules/02§6). - Workflow defaults: small single-purpose PRs with what/why/how-tested
descriptions (
rules/03§1–2); trunk-based short-lived branches; atomic commits with imperative ≤72-char subjects; conventional commits only if automation consumes them (rules/04§1–3). - Agent docs: one short, human-curated AGENTS.md/CLAUDE.md with exact
commands and repo-specific traps — never auto-generated bloat, never a
restatement of ambient/global rules (
rules/01§7, §10). On a new repo,.gitignore+ secret scanning and LICENSE land before the first commit (rules/01§10). - Solved failures accrue in a symptom-keyed troubleshooting playbook,
written in the PR that fixed them (
rules/01§9). - Before declaring done, self-review against the relevant files' Audit checklists.
AUDIT mode
When auditing docs or workflow:
- Scope the surfaces: docs tree + README + runbooks (
rules/01), generated reference + changelog + migration guides (rules/02), recent PRs and review threads (rules/03), git history, branches, and tags (rules/04). - Audit reality, not policy. Sample artifacts: run the quickstart on a clean environment, follow a runbook's commands, read 10 docstrings, read the last 20 merged PRs and 50 commits, diff a recent minor release for breaking changes. A CONTRIBUTING.md full of rules nobody follows is itself a finding.
- Work through each loaded file's Audit checklist; probe the classic gaps:
tutorial that fails partway, stale docs contradicting code, README
quickstart requiring tribal knowledge, alert with no runbook,
default: Erroras the only documented failure, changelog that's a commit dump, 2,000-line rubber-stamped PRs, force-push during review, broken commits on main, moved release tags, gitflow on a continuous-deploy service.
Severity conventions
- Critical — actively dangerous artifacts: runbook whose commands are wrong or destructive without warning; docs instructing insecure practice (secrets in config examples, auth bypass); moved/deleted published release tag; breaking change shipped in a minor/patch with no notice; merge to default branch with no review or required checks at all.
- High — reliably costs incidents or releases: page-able alerts without runbooks; quickstart/tutorial that fails; published docs contradicting current released behavior; no changelog or migration guide across breaking releases; review rubber-stamping (large PRs, instant LGTMs, AI volume merged unread); non-bisectable main (broken commits); releases built outside CI.
- Medium — erodes trust and throughput: mixed Diátaxis modes; undocumented public symbols or name-restating docstrings; unexecuted doc examples; no link-checking; PR descriptions missing why/how-tested; review SLA routinely blown; long-lived feature branches; commit-dump changelog; conventional commits adopted without enforcement or automation.
- Low — polish: missing freshness dates; vanity/stale badges; unlabeled
nit comments; subject lines over 72 chars; missing
.git-blame-ignore-revsfor reformat commits; docs index drift.
Finding format
[SEVERITY] <one-line title>
Where: <file:line | doc URL | PR/commit ref | branch/tag>
Rule: <rules-file §section>
Issue: <what is wrong, with observed evidence (quote the doc/PR/commit)>
Impact: <concrete consequence — who is misled, what breaks, what it costs>
Fix: <specific change; corrected text/command/process where load-bearing>
Order by severity; one finding per root cause; every finding cites sampled evidence (a doc you executed, a PR you read) — no findings from vibes.
Rules index
| File | Read this when... |
|---|---|
rules/01-documentation-architecture.md |
Writing/structuring/auditing any docs: Diátaxis modes, docs-as-code CI (link checks, doc tests), README front-door, decay control (ownership, freshness, aggressive deletion), runbooks, onboarding docs, discoverability, AGENTS.md/CLAUDE.md and llms.txt, and the repo documentation baseline (must-have docs + community-health files: LICENSE/SECURITY/CONTRIBUTING/CODE_OF_CONDUCT, and where GitHub looks for them). |
rules/02-api-reference-changelogs.md |
API/library reference docs: generation from source (OpenAPI/docstrings/rustdoc/godoc), docstring content (why/contract/failures), runnable examples and doctests, error documentation, versioned docs, Keep a Changelog discipline, migration guides. |
rules/03-code-review-pr-workflow.md |
PR and review process: PR sizing and slicing, description discipline (what/why/how-tested), review SLAs and WIP limits, reviewer/author conduct, blocking vs non-blocking comments, draft and stacked PRs, automation boundaries, reviewing AI-generated code. |
rules/04-commits-branches-releases.md |
Git history and shipping: atomic/bisectable commits, message discipline, Conventional Commits and when they pay, trunk-based vs gitflow honesty, SemVer semantics, breaking-change pipeline, tag immutability, release notes vs changelog, release automation. |
rules/05-spec-driven-development.md |
Driving non-trivial/agent-built work from a living spec: the intent→plan→tasks→implement→verify loop, separating what from how, testable acceptance criteria, [NEEDS CLARIFICATION] markers, specs-in-repo and spec-drift control, steering/constitution vs per-feature specs, linking ADRs/OpenAPI instead of copying, and when SDD is overhead. Pairs with sota-testing rules/08 (executable acceptance). |
Top-10 non-negotiables
- One Diátaxis mode per document — tutorials teach, how-tos accomplish, reference informs, explanation contextualizes; titles declare which. (rules/01 §1)
- Docs are code: in-repo, PR-reviewed, link-checked in CI; behavior changes update docs in the same PR. (rules/01 §2)
- Wrong docs are worse than none — own every doc, date-review the operational ones, delete stale pages instead of archiving them. (rules/01 §4)
- Every page-able alert links to a command-exact, incident-tested runbook. (rules/01 §5)
- Reference generated from source with undocumented-public-symbol as a build failure; doc examples compile and run in CI. (rules/02 §1–3)
- Document failure modes: every operation states what can fail and what the caller should do; error messages point toward the fix. (rules/02 §4)
- Changelog in Keep a Changelog form, written in user-impact language, updated in the PR — and breaking changes ship with a migration guide after a deprecation period. (rules/02 §6–7)
- Small, single-purpose PRs with what/why/how-tested descriptions; first review response within one business day. (rules/03 §1–3)
- The review bar is provenance-blind: AI-generated code gets the same scrutiny, and style/lint/type findings are CI's job, never a human's. (rules/03 §6–7)
- Main is always green and bisectable; published tags are immutable; version numbers keep SemVer's promises with breaking changes announced before they're shipped. (rules/04 §1, §4–5)
Files (sota-skills)
-
rules
-
01-documentation-architecture.md 31.3 KB
# 01 — Documentation Architecture Structure, placement, and lifecycle of documentation. Covers Diátaxis, docs-as-code, READMEs, decay control, runbooks, onboarding, and AI-readable docs. ## §1 Diátaxis: four modes, never mixed Classify every document as exactly one of four types (diataxis.fr). Each serves a different user need and is written differently; mixing modes is the single most common structural defect in documentation. | Mode | User need | Form | Cardinal sin | |---|---|---|---| | **Tutorial** | Learning (a lesson) | Guided, guaranteed-success path for a beginner | Offering choices, explaining theory mid-lesson | | **How-to guide** | Doing (a task) | Steps for a competent user with a real goal | Teaching basics, exhaustive option coverage | | **Reference** | Information (facts) | Complete, accurate, neutral description | Instructions, opinions, persuasion | | **Explanation** | Understanding (context) | Discussion of why, trade-offs, history | Step-by-step instructions | - **Name documents by mode**: "Tutorial: your first deployment", "How to rotate credentials", "CLI reference", "Why we shard by tenant". Users self-select correctly when the title declares the contract. - **Tutorials must be reliably repeatable.** Test them end-to-end on a clean environment; a tutorial that fails at step 4 burns more goodwill than no tutorial. Pin versions inside tutorials. - **How-to guides assume competence.** Don't re-explain what an environment variable is. Link to the tutorial for beginners instead of inlining basics. - **Reference is generated or mechanically maintained wherever possible** (see rules/02). Hand-written reference drifts. - **Explanation is where opinions live** — design rationale, trade-offs, "why not X". For architecture decisions specifically, use ADRs — see `sota-architecture` (do not duplicate ADR guidance here). - You don't need all four for every project. A small library needs README + reference. Apply the taxonomy when a doc set grows past one page, not before. **Bad** (mode soup, common in wikis): ```markdown ## Deploying the service Deployment uses Kubernetes, which is a container orchestrator that... ← explanation First, let's learn about our Helm chart structure... ← tutorial To deploy: `helm upgrade --install svc ./chart` ← how-to Supported values: replicas (int, default 3), image.tag (string)... ← reference ``` **Good**: four short linked pages, each one mode, each titled by mode. ## §2 Docs-as-code - **Docs live in the repo, versioned with the code they describe.** A doc that can't be updated in the same PR as the code change will not be updated. - **Docs changes go through PR review** with the same rigor as code: a wrong doc merged is a bug shipped. - **CI gates on docs**: markdown lint, broken-link checking (lychee is the current standard — fast, Rust, checks anchors, runs as `lycheeverse/lychee-action` in GitHub Actions), spell check on prose, and doc tests (rules/02 §3). Internal links checked on every PR; external links on a schedule (they break without your involvement — don't fail PRs on the internet's health). - **Code examples in docs are executed in CI** or extracted from tested code. Untested examples are reference-grade claims with tutorial-grade trust. - **Definition of done includes docs.** A feature PR that changes behavior and touches zero docs files should trigger a reviewer question, and ideally a CI nudge (e.g., a check that flags `src/` changes with no `docs/` or README diff — advisory, not blocking). - Prefer plain Markdown in-repo over wiki/Confluence for anything tied to code. Wikis are where docs go to decay: no review, no versioning, no proximity. ## §3 README as front door The README answers, in order, within one screen: **what is this, why would I use it, how do I try it in 5 minutes, what state is it in.** Required sections for a project README: 1. **One-sentence what + one-paragraph why** (the problem it solves, not the implementation). 2. **Quickstart** — copy-pasteable, from clean checkout to observable result in ≤5 minutes. If your quickstart can't fit that, that's a product defect worth knowing; don't paper over it with prose. 3. **Status honesty** — badges that mean something (CI on default branch, coverage, latest release). Delete badges that are red, stale, or vanity (e.g., "downloads" on an internal repo). A green badge pointing at a skipped pipeline is worse than no badge. 4. **Pointers, not content** — link to docs site/dirs for everything beyond quickstart. READMEs that try to be the whole manual go stale fastest. 5. **Support/ownership** — who owns this, where to file issues/ask questions. **Bad**: README opens with badges wall, install section assumes three undocumented prerequisites, "Documentation coming soon", architecture essay before anyone knows what the project does. **Good** opening: ```markdown # payout-svc Computes and schedules creator payouts from settled transactions. Replaces the legacy cron in `billing/jobs/payouts.py` (removed 2025-11). ## Quickstart ```sh docker compose up -d # postgres + localstack make seed run # service on :8080 curl localhost:8080/v1/payouts/preview?creator=demo ``` Expected: JSON payout preview. Full docs: ./docs. Owner: #team-payments. ``` ## §4 The decay problem Wrong documentation is worse than no documentation: it asserts authority while lying. Engineer for decay from day one. - **Proximity**: put docs as close to the code as possible — docstrings > package README > repo /docs > separate docs repo > wiki. Each step away halves update probability. - **Ownership**: every doc/dir has an owner (CODEOWNERS on `docs/` works). Unowned docs are pre-decayed. - **Freshness signals**: last-reviewed date (or rely on git metadata surfaced in the docs site) on operational docs; a periodic (quarterly) review sweep for high-traffic pages. "Reviewed: 2026-05" tells the reader how much to trust it. - **Delete aggressively.** Stale docs are deleted, not archived into an "old-docs" graveyard that search keeps surfacing. Git history is the archive. When deleting a page that had inbound links, leave a redirect or tombstone one-liner pointing to the replacement. - **Don't document what the code/tooling can assert**: link to the schema, the config struct, the generated reference instead of restating values that will drift. Docs should carry intent and context; machines carry facts. - **Duplicate nothing.** Every fact has one home; everything else links to it. The second copy is the one that will be wrong. ## §5 Runbooks Runbooks are read at 3 a.m. by someone with elevated cortisol and possibly no context. Optimize for that reader. - **Alert-linked**: every page-able alert links directly to its runbook; every runbook states which alert(s) fire it. An alert without a runbook link is an audit finding (also see `sota-observability` if present in this library). - **Command-exact**: real commands with real flags, environment names, and expected output — not "check the logs" but the exact query. Placeholders clearly marked (`<pod-name>` with the command to find it). - **Structure**: (1) symptom + alert, (2) impact/severity guidance, (3) triage steps in decision-tree order — most likely/cheapest checks first, (4) mitigation actions with their blast radius stated, (5) escalation path with names/rotations, (6) links to dashboards and recent incidents. - **Tested**: exercised in game days/incident drills, and updated in the incident-review PR when they were wrong during a real incident. A runbook that failed during an incident and wasn't fixed is a repeat incident scheduled. - **State the dangerous steps**: anything destructive (failover, cache flush, restart) carries an explicit "this will cause X" warning and rollback note. **Bad fragment**: "If the queue is backed up, restart the consumers." **Good fragment**: ```markdown ### Queue depth > 100k (alert: payouts-queue-depth-critical) Impact: payouts delayed; no data loss (queue is durable). 1. Check consumer lag: `kubectl -n payments logs deploy/payout-consumer --tail=50` — look for `DeserializationError` (known issue, see INC-2041). 2. If deserialization errors: bad message poisoning the partition. Skip it: `make skip-poison-msg ENV=prod` (safe: dead-letters the message). 3. If consumers healthy but slow: scale `kubectl scale deploy/payout-consumer --replicas=8` (max 12 — DB connection limit, see docs/capacity.md). 4. Not resolved in 15 min → escalate: #team-payments-oncall (secondary: @payments-lead). ``` ## §6 Onboarding docs and discoverability - **Onboarding docs are tested by every new joiner**: their first-week task includes following the setup guide and submitting a PR fixing everything that was wrong or unclear. This is the cheapest doc-testing loop that exists; if the guide survived three joiners unchanged, either it's excellent or they weren't told to fix it. - **Day-one doc** answers: how to get the code running locally, how to run tests, where the architecture overview is, who to ask what, what the team's workflow is (PR/review norms — rules/03). Target: first PR merged in week one. - **Discoverability beats organization.** People find docs via search and via links from where they already are (code, alerts, error messages, PR templates). Invest in: one search surface over all internal docs, links from error messages to docs, links from code to design docs — more than in perfect taxonomy. A perfectly organized doc tree nobody can search loses to a flat searched pile. - Keep an entry-point index per repo/team (`docs/README.md`): what docs exist, one line each, by Diátaxis mode. Indexes decay too — keep them short. **Where the canonical dev loop doesn't run everywhere, ship a capability report.** Plenty of repos have one documented path — a container, a specific OS, a licensed toolchain — and a reality where a given machine runs only part of it. Prose ("requires Docker") doesn't help someone who *has* a runtime that isn't the named one, or who can't tell which of twenty `make` targets are reachable. A small executable report does: it probes the host and prints, per target, **works here / doesn't, and what each gap blocks**. - **Probe capabilities, not one implementation's name.** A check for `docker` reports "no container runtime" on a machine running podman; a check for `LICENSE` misses `LICENSE-MPL`. A single-name probe returns a false absence that then gets acted on — the audit-side statement of the same rule is `sota/rules/03-audit-findings.md` ("a narrow search and a true absence produce identical output"). - **Every "unavailable" line names what it blocks**, so the reader learns the consequence without discovering it through a ten-minute failed build. - **Report, never gate.** It exits 0 by design; it describes the host, it doesn't judge it. Keep the gate a separate command. - Its audience is now agents as much as humans: without one, an agent proposes the documented command, fails, and retries it — the failure mode this artifact exists to prevent. ## §7 AI-era documentation Docs are now read by agents as well as humans. Same content, two consumers. - **AGENTS.md** is the open, Markdown-only convention for repo-level agent instructions (agents.md; 60k+ open-source projects; stewarded by the Agentic AI Foundation under the Linux Foundation; read by Codex CLI, Cursor, Copilot, **Antigravity CLI**, and 20+ other tools as of late 2026 — Antigravity, which **replaces Gemini CLI** (retired for individuals 2026-06-18), discovers `GEMINI.md`/`AGENTS.md` per directory, so `AGENTS.md` alone is enough for it). **CLAUDE.md** is Claude Code's native equivalent — Claude Code reads CLAUDE.md, **not** AGENTS.md, so it is the one mainstream tool still needing a pointer. Maintain one canonical file; if a tool needs the other name, symlink or include rather than fork the content. - **Keep agent docs minimal and high-signal.** Evidence as of 2026: bloated or auto-generated context files often *reduce* agent performance and raise cost; short, human-curated files with genuinely non-obvious repo knowledge help. Content that earns its place: exact build/test commands with flags, deviations from language defaults, files/dirs the agent must not touch, commit/PR conventions, known traps. Content that doesn't: anything the agent can read from code, generic best practices, restated style guides. - **The minimal shape.** Four blocks is enough for most repos; anything past them has to earn its line against the test above. ```text # <repo> — one line: what it is, who runs it ## Tech stack table, only where a wrong guess sends the agent down a wrong path ## Dev commands the exact build / test / lint / run lines, with the flags you use ## Conventions 2–5 repo rules that cannot be inferred from the code ## Traps the things that look fine and aren't ``` A stack table an agent could rebuild by opening `package.json` is padding. The row that earns its place is the one **contradicting** the default: the test runner that isn't the framework's, the port that isn't the documented one. - **Agent docs decay like all docs** — review them when commands change; a wrong test command in AGENTS.md silently corrupts every agent run. - **Check the file's *claims*, not just that its commands exist.** The two rot at different rates and only the first is ever noticed. A verification that confirms every named target resolves passes happily while the file asserts a pinned toolchain seven months stale, an image tag long superseded, or that a given target runs a check it doesn't. Observed exactly that on a 4300-commit project: all seven `make` targets existed; the stated toolchain and the stated contents of `make check` were both wrong. So verify in two passes — every command resolves to a real target, **and** every factual assertion still matches the file it describes. Better still, delete the assertion and link the source of truth: a sentence naming the pin will drift, a pointer to `rust-toolchain.toml` cannot. - **Automation that fires on an agent's edits must check, not rewrite.** A format-on-write hook is fine for a human editor and wrong for an agent: it changes the file *after* the agent wrote it, so the agent's view of that file is now stale and its next edit either fails or clobbers the reformatting. Have such hooks **report** the problem (non-zero, with the file and the fix) and let the agent apply it — the same content, in the one order that keeps both sides consistent. Reserve rewriting for hooks that run at commit or in CI, where no agent holds a live view. - **llms.txt** (llmstxt.org): root-level Markdown index of a site's docs for LLM consumption. Status as of mid-2026: community convention, ~10% site adoption, not an IETF standard, major crawlers don't commit to fetching it — but coding agents and IDE tools do fetch `/llms.txt` and `/llms-full.txt` from docs sites routinely. Verdict: cheap to publish for a public docs site (generate it from the nav tree in CI); don't hand-maintain it; don't expect SEO/answer-engine effects. - **Structure helps both audiences**: stable heading hierarchies, one topic per page, self-contained pages (agents retrieve pages out of context), exact command blocks, tables over prose for facts. These were good practices for humans already; agents just raised the price of ignoring them. ## §8 The documentation baseline — what every repo carries Docs aren't just "whatever got written." A small **baseline set** should exist, each with **one home** (§4 — the baseline is a floor, never a license to duplicate). Create each when its trigger fires, not preemptively: an unmaintained `CONTRIBUTING` or `CODE_OF_CONDUCT` full of rules nobody follows is itself a finding, not a checkbox win. **Always (any repo shared with anyone):** - **README** — the front door (§3): what, why, ≤5-minute quickstart, honest status. - **LICENSE** — without one, default copyright applies and **no one may legally reuse the code** (choosealicense.com). Choose deliberately; it **cannot** be inherited from an org default — GitHub requires it in each repo. - **CHANGELOG** — user-facing change history, updated in the PR that makes the change (rules/02 §6); absent it, users reverse-engineer releases from commits. **When the trigger fires (conditionally required):** | Trigger | Doc | |---|---| | Public, or accepts outside contributions | `CONTRIBUTING.md` (how to propose changes) + `CODE_OF_CONDUCT.md` (behavior + an enforcement contact) | | Handles anything security-relevant | `SECURITY.md` — how to report a vulnerability **privately** (a channel or advisory, never "open an issue" — unless the repo is *private*, see below) + supported-version policy | | Users need a place to ask / triage routing | `SUPPORT.md` — where questions go, so issues stay for bugs | | On-call / production service | runbooks (§5) + an incident/postmortem template | | AI-assisted repo | `AGENTS.md`/`CLAUDE.md` (§7) | | Architectural decisions accrue | an ADR log (`sota-architecture`) | | Multi-maintainer / org project | `CODEOWNERS`, plus `GOVERNANCE.md`/`MAINTAINERS` when decision rights aren't obvious | **Private-repo carve-out.** While a repository is still private the default advice has no reachable channel: GitHub's private vulnerability reporting is **public-repo only** — *"Owners and administrators of public repositories can enable private vulnerability reporting on their repositories"* (GitHub docs, checked 2026-08-16; the API returns `404` against a private repo). Meanwhile a private repo's issue tracker is visible only to collaborators, so for that audience it **is** a private channel. So while private, point reporters at the tracker or a named address, and do not promise a feature that cannot be enabled. **Name the switch to make on going public inside the file itself** — that is the moment the advice inverts, and the moment it is most likely to be forgotten. **Placement.** GitHub recognizes the community-health files (`CODE_OF_CONDUCT`, `CONTRIBUTING`, `SECURITY`, `SUPPORT`, `GOVERNANCE`, `FUNDING`) from **`.github/`, then the repo root, then `docs/`** (that precedence); an org-level **public** `.github` repo supplies defaults for repos lacking their own (GitHub docs). Keep exactly one canonical copy — per repo or via the org default, not both. README and LICENSE live at the repo root (GitHub surfaces them in the repo header) and are **not** inheritable community-health files. ## §9 The troubleshooting playbook — where solved failures accrue §5's runbooks serve the on-call reader mid-incident. A **troubleshooting playbook** serves a different reader: a contributor or an agent who just hit an error on their own machine, in a test run, or in CI. It is not alert-linked and not severity-ranked — it is a growing index of **failures already solved once**, so the second encounter costs a lookup instead of a re-debug. One entry per failure, three parts, nothing else: ```text ### Symptom: <the literal error text or observable behaviour> **Diagnosis:** <the actual cause, stated as a fact about this repo> **Fix:** <exact commands, or the file to change> ``` - **Key on what the reader sees, not on what you now know.** The heading is the string they will paste into a search box — the exception name, the status code, the log line. "Environment misconfiguration" is unfindable by someone staring at `ModuleNotFoundError`. - **Write it in the PR that fixes the bug**, while the cause is still known. A playbook backfilled later doesn't get written. - **The diagnosis must be repo-specific.** "Check your dependencies" is not a diagnosis; "the dev server resolves imports from `src/` only, so a module added outside it resolves after a reinstall and not before" is. - **Delete entries the fix made impossible.** Once a class of failure is closed at the root — a validation, a default, a guard — its entry is a false lead and goes (§4). - **It is a repo artifact, not a chat log**: in-repo, linked from both the README's contributing path and the agent file (§7), or neither audience finds it. A symptom reported a third time is a signal to **stop writing entries** — that fix belongs in the code or the setup script, not the playbook. ## §10 Day zero — a new repo inherits no context, however good your tooling An installed agent skills library, a personal `~/.claude/CLAUDE.md`, a house style guide: all of it is **ambient** — attached to a machine or an account. A freshly initialised repo inherits none of it, and neither does the teammate, the CI runner, or the agent that clones it tomorrow. Whatever the repo's correctness depends on has to live **in the repo**. Two questions, in this order. **1. What must exist before the first commit?** Only two things, because both are expensive to add later: - **`.gitignore` + secret scanning.** A credential committed in commit 1 is not fixed by deleting it in commit 2 — it is in the history, and removing it costs a rewrite *plus* rotating the credential (`sota-secrets-management` rules/04). The pre-commit hook is cheapest while the history is one commit long. - **`LICENSE`.** Its absence is not neutral (§8): all rights reserved, and not inheritable from an org default. Adding one later needs sign-off from everyone who has contributed by then. Everything else in the §8 baseline still follows §8's rule — created when its trigger fires, not pre-seeded. An empty `CONTRIBUTING.md` on day zero is the decay problem (§4) with a head start. **2. What can only the repo tell an agent?** A general skills library knows how to write a Go service; it cannot know that *this* one is tested behind a flag, deploys from a branch that isn't the default one, or has a directory nothing may touch. That gap is the agent file (§7), held to §7's test. Two failure modes to design out on day zero: - **Restating ambient rules in the repo file.** If a personal or org-level agent file already says "use conventional commits", repeating it in-repo creates two copies that drift and spends context twice. Repo file = repo-specific only. The inverse is worse: repo-specific facts stranded in a personal file no teammate has. - **Forking the file per tool.** Keep one canonical `AGENTS.md` (§7) and pick how the other names reach it, knowing the trade of each: 1. **A native import**, where the tool has one — Claude Code expands `@AGENTS.md` in `CLAUDE.md` into context at launch, so it costs no hop and leaves room for tool-specific rules *below* the import. Its own docs recommend this over a symlink. Verified limits: max **four** hops, relative paths resolve against the importing file, and parsing skips code spans, so a backticked `` `@path` `` stays literal. **The trap is an import that resolves outside the repo** (e.g. `@~/.claude/…`): the first session asks to approve it, and **declining disables it permanently without asking again** — the file is present, the content never loads. Gemini CLI has the same mechanism (`@./AGENTS.md`; its own examples are all `./`-prefixed) and also takes a **list** of context filenames in settings, which needs no pointer file at all. **Check each tool's own docs for the form** — assuming one tool's syntax works in another is how you ship a file that loads nothing. 2. **A symlink** — exact, and two failure modes. Git records a symlink as such, and where `core.symlinks` is false (set automatically at clone time on filesystems that can't represent one) symlinks are "checked out as small plain files that contain the link text" (`git help config`): the agent then reads the bare string `AGENTS.md` as the whole file and follows nothing. And on **Windows, creating one needs Administrator or Developer Mode**, so a repo that depends on it is a repo some contributors cannot set up. 3. **A one-line pointer file** — `CLAUDE.md` containing `See [AGENTS.md](AGENTS.md).` Platform-independent and it cannot silently degrade, at the cost of a hop the agent must choose to follow. 4. **CI-generated duplicates** from the canonical file, failing the build on drift. Exact and platform-independent, at the cost of a job to maintain. Anything else — hand-maintained copies — drifts, and the copy that goes stale is the one your teammate's tool reads. A one-time importer that *copies* instructions between tools (Claude Code's `/import`) is a migration, not a link: it does not re-sync when the original changes. - **Whichever you pick, confirm it LOADED — presence is not loading.** Ask the tool what it actually read (Claude Code: `/context`, under *Memory files*), and where a hook can log it, log it (`InstructionsLoaded`). This is the §7 agent-doc case of the rule the rest of this library keeps hitting: a file in the repo is not a file in the context. **Bootstrap the invariants as checks, not prose.** Anything the repo must never regress — no secret in a commit, every internal link resolving, a required file present — is worth a hook or CI job on day zero, when it costs one file and passes trivially. The same check proposed at ten thousand commits arrives red and gets disabled. **An ambient install doesn't travel — decide per repo.** Rule libraries, agent skills and linters installed at user scope resolve against *your* home directory. A teammate's clone and a CI runner resolve nothing, so a repo that depends on them is one that only works on one machine. Two honest options, and the choice belongs in the repo's contributing docs either way: - **Solo or private-to-you** — the ambient install is enough; write down nothing, and don't pretend the repo is self-contained. - **Shared, public, or CI-checked** — make it repo-resident: vendor or project-scope the tooling (for this library, `install.sh --project .`, or `--copy` to pin a snapshot rather than link to a path only you have), *or* state the install step in `CONTRIBUTING.md` so a contributor can reproduce it. Whichever you choose, the **gates** must be repo-resident regardless — a secret scan that exists only in your shell is not a control on anyone else's commit (`sota-code-security` rules/10: a control nobody else runs is a no-op everywhere but your machine). A pointer file that references tooling the reader doesn't have is worse than no pointer: it reads as a satisfied requirement. ## §11 A durable note records an invariant, not an observation Anything written to outlive the session — a handover note, an `AGENTS.md` line, an agent's memory file — should record the **invariant**, not the **observation** it produced. Before writing one, ask: *what would have to change for this to become false, and is that likely?* If the answer is "an ordinary refactor", write the invariant instead. Field-reported: a note said an agent could neither sign *nor anchor* a ledger, because both needed a hardware token. True when written. An hour later the design split timestamping from signing and timestamping needed no hardware — so the note was wrong in the direction that makes an agent ask for help it does not need. The invariant was stable and would have survived: *"signing requires the hardware token; nothing else does."* **A wrong note is worse than a missing one, in both directions**: it can let an agent proceed where it should stop, and stop it where it could proceed. The second is quieter and therefore lives longer — and the notes that most need updating are the ones nobody re-reads. ## Audit checklist - [ ] **Durable notes state invariants, not observations** (§11). For each line meant to outlive the session, name what would falsify it; if an ordinary refactor would, it records an observation and should be rewritten as the invariant beneath it. - [ ] Baseline present for the repo's stage: README + LICENSE + CHANGELOG always; CONTRIBUTING/CODE_OF_CONDUCT/SECURITY/SUPPORT once public or contribution-accepting; runbooks once on-call — each with one canonical home (no duplicate copies across root/`.github`/`docs`). - [ ] `SECURITY.md` gives a private vulnerability-reporting channel (not "open an issue" — except on a *private* repo, where the collaborator-only tracker is acceptable and GitHub's private reporting cannot be enabled at all; the file must then name the switch to make on going public) and a supported-version policy; `LICENSE` is present and intentional (its absence = all-rights-reserved, blocking reuse). - [ ] Docs classified by Diátaxis mode; no page mixes tutorial/how-to/reference/explanation; titles declare the mode. - [ ] Tutorials run end-to-end on a clean environment (verified recently, versions pinned). - [ ] Docs live in-repo, change via reviewed PRs, and behavior-changing code PRs touch docs. - [ ] CI checks links (lychee or equivalent) and lints docs; internal links gate PRs. - [ ] The agent file carries only what the agent would otherwise get wrong (exact commands, deviations, traps) — not a restatement of ambient/global rules or anything readable from the code; one canonical file, with per-tool names symlinked or CI-generated rather than forked. - [ ] Solved failures land in a troubleshooting playbook keyed on the literal symptom, written in the PR that fixed them; entries invalidated by a root-cause fix are deleted, and a thrice-reported symptom was fixed in code rather than re-documented. - [ ] The repo got `.gitignore` + secret scanning and a LICENSE before its first commit, and its must-never-regress invariants are enforced by a hook or CI job rather than described in prose. - [ ] Nothing the repo depends on resolves only against one contributor's home directory: gates are repo-resident, and any user-scoped tooling a shared repo assumes is either vendored/project-scoped or documented as an install step (§10). - [ ] README: one-sentence what, why, ≤5-minute copy-pasteable quickstart, honest badges, ownership/support pointer. - [ ] Every doc/dir has an owner (CODEOWNERS or equivalent); high-traffic pages have a freshness/review signal. - [ ] No known-stale pages kept "for reference"; deletions leave redirects/tombstones; no duplicated facts across pages. - [ ] Every page-able alert links to a runbook; runbooks are command-exact, decision-tree ordered, flag destructive steps, and were updated after the last incident that used them. - [ ] Onboarding guide exists, and the newest joiner actually filed fixes against it. - [ ] Where the canonical dev loop doesn't run on every supported host, a capability report says per target what works here and what each gap blocks — probing capabilities rather than one implementation's name, and reporting rather than gating (§6). - [ ] Agent-file verification covers claims as well as commands: every named target resolves **and** every factual assertion (pins, tags, what a target does) still matches its source — with drift-prone assertions replaced by a link to the file that owns the fact (§7). - [ ] No automation rewrites files in response to an agent's edits (format-on-write hooks report instead); rewriting is confined to commit-time or CI, where nothing holds a live view of the file (§7). - [ ] One search surface covers internal docs; error messages/alerts/code link into docs. - [ ] AGENTS.md/CLAUDE.md exists, is short and human-curated, has exact build/test commands, and matches current reality; no forked divergent copies. - [ ] Public docs site: llms.txt generated (not hand-written) if published; pages are self-contained with stable headings. -
02-api-reference-changelogs.md 9.7 KB
# 02 — API Docs, Reference & Changelogs Generated reference documentation, docstring discipline, runnable examples, error documentation, versioned docs, and changelog/migration-guide practice. ## §1 Generate reference from source Hand-written reference documentation drifts from the code within weeks. The baseline is generation from the artifact of record: - **HTTP APIs**: OpenAPI document is the source of truth; rendered docs are generated from it (Redoc/Scalar/Swagger UI class of renderers — Scalar adds try-it + snippets, Redoc is read-optimized; the renderer matters less than the pipeline). Spec-first or code-first is a per-team choice, but the spec must be CI-validated against the implementation either way (contract tests, or generated-from-code with lint gates — see `sota-api-design` for spec design itself). - **Libraries**: rustdoc, godoc, javadoc, Sphinx/mkdocstrings from docstrings, TypeDoc from TSDoc. Doc generation runs in CI; warnings are errors (`#![deny(missing_docs)]`, `RUSTDOCFLAGS="-D warnings"`, Sphinx `-W`). - **Lint the spec/docstrings**: OpenAPI linting (Spectral/Redocly CLI class) and docstring-coverage checks in CI, so "undocumented public symbol" is a build failure, not a review nitpick. - Generated reference is necessary, not sufficient: it gives you *what*; §2 makes it useful. ## §2 Document the why, not just the what A signature already says what. The doc comment earns its existence by adding what the signature can't: - **Every public symbol documented** — function, type, field, error variant, config knob, CLI flag, endpoint, header. "Public" means "someone outside this module can depend on it," which means someone will. - Content that belongs in a doc comment, in priority order: 1. **Behavioral contract**: invariants, units, ranges, nullability, ordering guarantees, idempotency, thread-safety, blocking behavior. 2. **Failure modes**: what errors/exceptions, under what conditions (§4). 3. **Why / when to use**: what problem it solves, when to prefer the alternative (`see also` links are load-bearing). 4. **Surprises**: anything a reasonable caller would guess wrong — costs (O(n²), network call, allocation), side effects, caching. - **Bad**: `// GetUser gets a user.` Restating the name is negative-value: it occupies the slot where real information should be and passes docstring- coverage checks while adding nothing. - **Good**: ```go // GetUser returns the user by ID from the primary store. // It does NOT consult the cache — use CachedUser for read paths; // GetUser exists for read-after-write consistency in signup flows. // Returns ErrNotFound for unknown or soft-deleted IDs (callers must // not distinguish the two; deletion state is not part of the contract). // Makes one DB round trip; safe for concurrent use. ``` - Module/package-level docs explain the *shape* of the package: what it's for, the two or three entry-point types, what it deliberately doesn't do. ## §3 Examples that execute An example that doesn't run in CI is a claim, not an example. - **Use the language's doctest facility**: Rust doc-examples (compiled and run by `cargo test` by default), Python `doctest` via pytest (`--doctest-modules`), Go `Example*` functions (output-checked by `go test`), Documenter.jl doctests. If the language lacks one, extract fenced code blocks from docs and compile/run them in CI (e.g., mdBook test, custom extractor). - **Every non-trivial public entry point gets at least one example** showing the intended call pattern, not the minimal one. Examples are the most-read part of any reference page; most users copy the first example and edit. - Examples show **realistic values and error handling**, not `foo`/`bar` and swallowed errors — copied example code becomes production code. - For HTTP APIs: request/response examples in the OpenAPI doc, validated against the schemas in CI (example-schema mismatch is a classic silent lie). - Quickstarts and tutorials count as examples: run them in CI on a schedule if full per-PR execution is too slow (rules/01 §2). ## §4 Error documentation What can fail, and what the caller should do about it, is the half of the contract most docs omit. - Per operation, document: **which errors/status codes occur, what condition triggers each, whether retrying helps, and what the caller should do** (retry with backoff / fix the request / escalate / impossible-by-construction). - HTTP: enumerate non-2xx responses in the OpenAPI doc with the error body schema and machine-readable codes; "default: Error" as the only error response is an audit finding (error shape design: `sota-api-design`). - Libraries: document error variants where they're defined *and* which operations raise them; distinguish programmer errors (panic/assert — don't catch) from operational errors (handle). - Error *messages* are documentation too: include what was expected, what was seen, and ideally a link/pointer to the fix. The error message is the doc page with a 100% read rate at exactly the right moment. ## §5 Versioned docs - **Docs versions match released versions.** A user on v2 reading v3 docs gets lies with confidence. Docs sites carry a version switcher (mike for MkDocs, Sphinx multiversion, docusaurus versions, docs.rs does it for free); the default view is the latest *stable* release, not the dev branch. - Docs-as-code makes this nearly free: docs are in the repo, so the tag that cut the release pinned the docs (rules/04 §5). - Mark removed/changed behavior inline with the version it changed (“Since v2.3”, “Removed in v3.0 — use X”), because users land on pages from search without knowing which version they're reading. - Pre-1.0 or internal-only projects can skip multi-version publishing, but the docs must still state which version they describe. ## §6 Changelog discipline - **Format: Keep a Changelog** (keepachangelog.com, v2.0.0 — 2026-06; the format itself is unchanged from 1.1.x, so existing changelogs stay valid): `## [version] - date` sections with `Added / Changed / Deprecated / Removed / Fixed / Security` subsections, plus an `Unreleased` section at top so changes are recorded in the PR that makes them, not reconstructed at release time. - **Write user-impact language, not commit dumps.** The changelog audience is users deciding whether and how to upgrade. `git log` is not a changelog; auto-generated commit lists are an input, not the product. - **Bad**: `refactor: extract PayoutScheduler; bump deps; fix flaky test` - **Good**: `Fixed: payouts scheduled across a DST boundary no longer run an hour early (#412).` - Every entry names the observable change, and links the issue/PR for the archaeology. Internal refactors with zero user-visible effect don't appear. - Automation (conventional commits → changelog via semantic-release / release-please / git-cliff — rules/04 §2) generates the *draft*; a human edits it into user language before release. Automation removes the blank page, not the editorial pass (Keep a Changelog 2.0.0 codifies the same stance for LLM-assisted drafting: machines draft, humans curate). - **Changelog vs release notes**: the changelog is the complete, append-only, per-version record in the repo; release notes are the curated announcement (highlights, upgrade guidance, thanks) per release. Generate release notes *from* the changelog, never maintain two divergent histories. ## §7 Migration guides for breaking changes A breaking change without a migration guide is a breaking change with the cost shipped to every user individually. - Required for any major version / breaking release: a guide with **every breaking change listed**, each with: what broke, why (one line), the exact before→after diff, and mechanical migration steps (codemod/script/sed where possible). - Order by "what you'll hit first," not by module. - **Deprecate before removing**: ship the new path + deprecation warnings pointing at the guide for ≥1 minor release before removal, so users migrate on warnings, not on compile errors (HTTP deprecation signaling: `sota-api-design`). **Good fragment**: ```markdown ### `client.fetch()` removed → `client.get()` Why: fetch() silently retried non-idempotent requests (#388). Before: `resp = client.fetch(url, retries=3)` After: `resp = client.get(url, retry=Retry(total=3, idempotent_only=True))` Mechanical: `npx @ourlib/codemod v3-fetch-to-get src/` Behavior change: POSTs are no longer retried by default — opt in per call. ``` ## Audit checklist - [ ] Reference docs generated from source (OpenAPI/docstrings/rustdoc/godoc); generation runs in CI with warnings-as-errors. - [ ] Docstring/spec lint gates exist: undocumented public symbols fail the build, OpenAPI is linted and contract-checked against the implementation. - [ ] Doc comments carry contract/failure/why/surprises — sample 10 public symbols; name-restating docstrings are a finding. - [ ] Doc examples compile/run in CI (doctests/Example funcs/extracted blocks); first example per page shows realistic use with error handling. - [ ] OpenAPI examples validate against their schemas. - [ ] Every operation documents its failure modes and the caller's correct reaction; HTTP error responses enumerated with body schema. - [ ] Published docs are versioned to match releases; default view is latest stable; pages state the version they describe. - [ ] CHANGELOG follows Keep a Changelog with an Unreleased section maintained in PRs; entries are user-impact language with issue/PR links, not commit subjects. - [ ] Release notes derive from the changelog (no divergent second history). - [ ] Breaking releases ship a migration guide: every break listed with before→after and mechanical steps; deprecation warnings preceded removal. -
03-code-review-pr-workflow.md 18.9 KB
# 03 — Code Review & PR Workflow Pull request sizing and description, reviewer and author conduct, review SLAs, stacked/draft PRs, automation boundaries, and reviewing AI-generated code. ## §1 Small PRs are the highest-leverage practice Review effectiveness degrades sharply with size: defect-detection rate falls as diff size grows, and large PRs converge on "LGTM" because nobody can hold 2,000 lines in their head. Everything else in this file is downstream of size. - **Target ≤ ~400 changed lines of substantive diff; treat ~1,000 as a hard ceiling** requiring justification (generated code, lockfiles, and mechanical renames don't count — but call them out so reviewers can skip them). - **One logical change per PR.** "Implements X *and* refactors Y" is two PRs. The refactor goes first, separately — it's the one that can be approved in minutes because behavior is unchanged and tests prove it. - **Slicing strategies** for big features: vertical slices behind a feature flag; preparatory refactors first; interface/contract PR before implementation; data-model migration separate from the code that uses it; stacked PRs (§5). - A PR that *can't* be made small (large migration, vendored code) gets a reviewing map in the description: read order, where the decisions are, what's mechanical. ## §2 PR description discipline The description is documentation with a deadline: it's how the reviewer loads context, and how `git log` archaeology works in two years. Required content — **what / why / how-tested**: - **What**: the observable change, one or two sentences. - **Why**: the problem/motivation and the linked issue (`Fixes #123` to auto-close). The diff shows what changed; only the description can say why. - **How tested**: specific — which tests added/updated, what was exercised manually, what wasn't and why that's acceptable. - **UI changes**: before/after screenshots or a clip. Reviewers can't render JSX in their heads. - **Risk & rollout** when relevant: feature flag, migration ordering, rollback plan, blast radius. - State what's deliberately out of scope to pre-empt scope-creep review. Use a PR template (`.github/pull_request_template.md`) to make the structure the default. Keep it short enough that people fill it instead of deleting it. **Bad**: title `fix bug`, body empty, 23 files changed. **Good**: ```markdown ## What Reject payout previews for creators with frozen accounts (422 + code `account_frozen`). ## Why Frozen accounts could see previews that would never execute, generating support tickets (#892). Execution was already blocked; preview wasn't. ## How tested - New: `test_preview_frozen_account_422` - Manual: froze demo creator in staging, verified 422 body + UI message - Not covered: bulk-preview path — frozen filter happens upstream (see `BulkPreviewService:88`), existing tests cover it. Out of scope: unfreezing flow cleanup → #901. ``` ## §3 Reviewer behavior - **Review SLA: first response within one business day** (same-day for small PRs as an aspiration). Review latency is the dominant term in cycle time, and slow review is what teaches people to make giant batched PRs. Reviewing others' code outranks writing your own in the daily priority order. - **WIP limits**: when your review queue is full, finish reviews before starting new work. Ten open PRs awaiting review is a team-level incident, not ten individual delays. - **Label every comment as blocking or non-blocking.** Conventional prefixes (`blocking:`, `nit:`, `question:`, `suggestion:`, `praise:` — the "conventional comments" style) remove the guess about what must be resolved before merge. - **Suggest, don't command; ask, don't assert.** "What happens if `items` is empty here?" beats "this is broken" — it's both kinder and more often correct, because sometimes the answer is "it can't be, see the validator." Comment on the code, never the author. - **Approve with nits.** If everything remaining is non-blocking, approve and trust the author to address nits before merge. Holding approval hostage to trivia trains people to argue instead of fix. - **Review for what automation can't catch**: design fit, correctness under concurrency/failure, missing tests, naming, security, API contract, "should this exist." If you're commenting on formatting, the CI config is the bug (§6). - **Know when to take it offline.** Three back-and-forth rounds on one thread means the medium failed: call/pair, then record the conclusion in the thread for the archaeologists. - An approval means "I understood this and stake my name on it," not "the author seems confident." If you didn't understand it, say so — that's a finding about the PR, usually. ## §3a `git diff main..pr` is not what the PR changes Two-dot **diff** compares two *tips*. If the branch forked before `main` gained a commit, everything `main` gained since renders as a **deletion on the branch's side** — so the form nearly everyone types systematically *invents* regressions. It never hides one: the error is directional, always in the alarming direction. In a lockfile or a dependency manifest an invented regression reads as a supply-chain attack, which is exactly the finding a reviewer will escalate fastest and check least. **Why the habit survives: the same token means different things to different subcommands.** This is the trap, not carelessness — ``` git log main..feature # commits in feature, not in main <- CORRECT git diff main..feature # compare the two tips <- NOT the PR's changes ``` `A..B` is right where people learn it (`log`, `rev-list`, and as a `--log-opts` range for a history scanner) and wrong in `diff`. Both succeed, both print well-formed output, and nothing distinguishes *"B removed this"* from *"B never had it."* **Use the merge base**: `git diff $(git merge-base main pr)..pr`, or the three-dot `git diff main...pr`, which for `diff` is defined as exactly that. Forge UIs ("Files changed") and `gh pr diff` show the merge-base diff, so a local two-dot result that disagrees with the web view is *your* query, not a stale page — verified 2026-09-15 on a public PR **42 commits behind its base**, where `gh pr diff`, the `pulls/:n/files` endpoint behind "Files changed", and a three-dot compare all returned the same **11** files while the two-dot direction returned **33**. Measured the same day on a branch four commits behind its base, which is the trap at full strength: two-dot reported **15 files and 742 deletions**, three-dot and the forge's compare API both reported **zero files changed**. The branch had changed nothing. **Before reporting that a PR removes, downgrades or reverts anything, re-run it against the merge base.** The tell is that the "removed" content is something *you recently added to `main`* — which is also why this fires hardest right after a security bump, when the stakes of the false claim are highest. Field- reported 2026-09-15: a reviewer read a two-dot lockfile diff as a TLS library being downgraded past the previous day's advisory fix, wrote that into a commit message and drafted a PR comment saying so. The PR did not touch that dependency at all. Nothing caught it; a merge check surfaced the real diff by accident. Under §8 that claim was one step from being published under the maintainer's name. ## §4 Author behavior - **Self-review first.** Read your own diff in the review UI before requesting review; you'll catch the debug print, the stray file, the TODO. Annotate the diff with PR comments where the reviewer will need context ("this rename is mechanical, the real change is in `scheduler.py`"). - **Respond to every comment** — fix, push back with reasoning, or file a follow-up issue. Silently ignoring a comment, or marking it resolved without action, destroys reviewer trust permanently. - **Don't force-push during active review**: it orphans comment anchors and destroys the reviewer's "what changed since my last pass" diff. Push fixup/appended commits during review; clean up history (squash/autosquash) at merge time. (Pre-review and stacked-PR rebases are fine.) - The author merges (where the platform allows) — they own the timing against deploys and freezes. - Don't request review on red CI. Reviewer attention is the scarce resource; spend it on code that at least compiles and passes tests. ## §5 Draft and stacked PRs - **Draft PRs for direction checks**: open as draft with a specific question ("is this the right seam?") before investing in polish. Cheap course correction beats a finished PR built on the wrong design. Mark ready only when it meets the full bar (§2, green CI). - **Stacked PRs for large changes**: a sequence of dependent, individually reviewable PRs (each targeting the previous branch), reviewed and merged in order. This is how you keep §1's size discipline on multi-thousand-line features. GitHub now ships native stacked PRs (private preview as of mid-2026: `gh stack` CLI, a stack map in the PR UI, branch protection and CI evaluated against the final target branch, auto-rebase of the remaining stack after each merge) — prefer it where enabled, since reviewers need no third-party account. Until then, tooling (Graphite, `gh`/`git` stacking workflows, `git-spice`, jj-based flows) automates the rebase cascade; without tooling, keep stacks ≤3 deep or the rebase tax exceeds the review benefit. - Each PR in a stack must stand alone: green CI, coherent description, no forward references that make it unreviewable without reading the whole stack. ## §5a The change surface is a design decision, not a side effect §1 says keep PRs small. This is the harder half: **what belongs in the diff at all.** Agents are unusually good at producing code and unusually bad at bounding what they touched, so this is where an otherwise correct change becomes unreviewable. **Do not change unrelated files.** A drive-by reformat, an import reorder, a "while I was here" rename — each is defensible alone and collectively they destroy the diff. The reviewer can no longer see the change, only the noise around it, and `git blame` on those lines now points at your unrelated PR forever. If a cleanup is worth doing, it is worth its own PR; if it is not worth its own PR, it is not worth burying in this one. **Generated files have a policy, and the policy is stated.** Lockfiles, generated clients, compiled assets, snapshots: either they are committed (and regenerated in a separate commit, so a review can skip them) or they are not committed at all. What must not happen is a diff where hand-written and generated lines are interleaved and the reviewer cannot tell which is which. Say which in `CONTRIBUTING.md`, and keep the regeneration in its own commit even inside one PR. **Decompose by reviewability, not by size.** A 600-line PR that is one mechanical rename plus one 20-line behaviour change is *two* PRs — and the behaviour change is the one that needs eyes. The test is not "how many lines" but "can a reviewer hold the intent of this diff in their head at once". Common seams: mechanical vs semantic; refactor vs behaviour; interface vs implementation; migration vs the code that uses it. **Sequence migrations so each commit is deployable.** Expand, migrate, contract — a PR that adds a column and starts writing to it is safe; one that also drops the old column is not, because a rollback of the code without a rollback of the data leaves production reading a field that no longer exists. Same rule for API deprecations (`rules/02` §2) and for anything with a consumer you do not control. **Revert or fix forward — decide by blast radius, not by pride.** If the change is live and wrong, revert first and diagnose after: a revert is a known-good state and a fix-forward is a hypothesis. Fix forward when the revert is *itself* risky (a migration has run, a revert would re-break a dependent change). Either way the decision is recorded — a revert with no explanation reads as a mistake by whoever reverted. **Preserve archaeology.** Keep mechanical moves separate from edits so `git log --follow` and `git blame` still work; where a tool supports it, record the move-only commit so future bisects step over it rather than into it. A `git bisect` that lands on a 4,000-line reformat has cost more than the reformat saved. ## §6 Automation does the robot work - **Lint, formatting, type errors, import order, coverage thresholds, secret scanning, license checks are CI's job.** A human pointing out a formatting issue is a process failure: add the rule to CI and it never recurs. Reviewer attention is for judgment (§3). - Format-on-save + pre-commit hooks catch locally; CI enforces. Nobody debates style in review because style isn't an opinion anymore — it's a config file. - Bot-noise budget: auto-comments (coverage deltas, preview links, size labels) must collapse/update in place. A PR where human comments drown in bot spam gets worse review. - CI status gates merge: required checks, no `--no-verify` culture, branch protection on the default branch (supply-chain side: `sota-devsecops`). ## §7 Reviewing AI-generated code AI assistance raises PR volume; the review bar does not move. - **Same bar, same process.** "An agent wrote it" is not a provenance excuse — the human who opens the PR owns every line of it, including understanding it. If the author can't explain a hunk, it isn't ready for review. - **No rubber-stamping volume.** The failure mode of 2025–26 is plausible, confident, subtly-wrong code reviewed at "looks idiomatic" depth. Spot-check the parts AI gets wrong most: edge cases, error paths, concurrency, off-by-one boundaries, invented APIs, tests that assert the implementation rather than the requirement. - **AI-generated tests deserve the most suspicion**: verify they fail without the change (mutation thinking), not just that they pass with it. - Disclose substantial AI generation in the PR when team policy asks; either way, size limits (§1) apply with extra force — generated code is cheap to produce and expensive to review, so the queue saturates from the author side. - AI *reviewers* (CI-integrated review bots) are a pre-filter on the author's side, like a linter with opinions — they reduce trivial findings reaching humans, they don't replace the human approval (§3's "stake my name on it"). ## §8 Claims you publish under someone else's name A finding you hand to the person who asked costs one reader's trust and is cheap to withdraw. The same finding posted to a **PR review comment, an issue, a commit message, or a mailing list** is public, attributed to whoever's account sent it, indexed, and quoted back years later. Upstream, it is also read by maintainers who have never met you and will calibrate every later report against this one. Treat the audience change as an evidence-bar change, not a tone change. Before anything leaves for a third party: - **Verify by execution, not inference.** Reading a source file and concluding how a flag behaves is a hypothesis; running `--help`, the test, or the actual code path is evidence. Most wrong public claims are correct reasoning from an unchecked premise. - **Say what you did not test.** "Reproduced on x86-64 with `make run`; not tried under TDX" is a stronger comment than one that quietly implies full coverage. - **Check whether it is already known** — read the whole thread, the linked issues, and the commit that introduced the line. A duplicate report costs the maintainer more than silence would have. - **Attack your own claim first** (`sota` principle 7): restate it from the tool output rather than from your own earlier summary. A false report in a maintainer's own subsystem is the most expensive kind. - **Never publish on someone's behalf without their approval of the final text.** Draft it, show it, let them send it. This is not a formality — their name is on it and the reputational cost lands on them, not on the drafter. The `§3` conduct rules still apply on top: ask rather than assert ("what happens if `items` is empty here?"), label blocking vs non-blocking, comment on the code and never the author. Conduct and evidence are separate axes — a politely worded claim that turns out to be false still burns the credibility. ## Audit checklist - [ ] **Does the diff contain anything the PR title does not describe?** Drive-by reformats, import reorders, opportunistic renames — each defensible alone, together they make the change invisible and misattribute `git blame` (§5a). - [ ] **Is there a stated policy for generated files**, and are generated and hand-written lines in separate commits rather than interleaved (§5a)? - [ ] **Was the PR decomposed by reviewability** — mechanical split from semantic, migration from the code that uses it — rather than by line count (§5a)? - [ ] **Is every commit in a migration sequence independently deployable and rollback-safe** (expand → migrate → contract), so a code rollback does not strand the data (§5a)? - [ ] Anything published to a **third party under someone else's name** (PR comment, issue, commit message, upstream post) verified by **execution not inference**, with untested parts named, the thread checked for a duplicate, the claim restated from tool output rather than your own summary, and the final text approved by the person whose account sends it (§8)? - [ ] **Any claim that a PR removes, downgrades or reverts something — re-run against the merge base before it is reported** (§3a). The audit question is not "is the finding alarming" but "which range produced it": a two-dot `git diff main..pr` renders everything `main` gained since the fork as a deletion on the PR's side. Check whether the "removed" content is something recently added to `main`, and whether the local result matches the forge's "Files changed" — a disagreement there means the query is wrong, not the page. - [ ] Median merged-PR size is small (≲400 substantive lines); large PRs are exceptions with stated justification or a reviewing map. - [ ] PRs are one logical change; refactors land separately from behavior changes. - [ ] PR template exists; sampled recent PRs have what/why/how-tested, linked issues, screenshots for UI changes. - [ ] Review first-response time is ~1 business day or better; no PRs silently aging past the SLA. - [ ] Blocking vs non-blocking comments are distinguishable (prefixes/labels); approve-with-nits happens in practice. - [ ] Authors self-review (look for author-annotated diffs), respond to all comments, and don't force-push mid-review. - [ ] Draft PRs used for early direction; oversized features arrive as stacks of independently green PRs. - [ ] CI owns lint/format/type/coverage; sampled reviews contain zero human style comments; required checks gate merge. - [ ] Review depth on AI-heavy PRs matches human-written ones: comments engage with logic, tests proven to fail without the change, no volume rubber-stamps. - [ ] Merged PR descriptions are useful in `git log` archaeology (pick 5 from six months ago and try to reconstruct the why). -
04-commits-branches-releases.md 8.5 KB
# 04 — Commits, Branches & Releases Commit message and atomicity discipline, conventional commits and when they pay, branch strategy, versioning, and release/changelog mechanics. ## §1 Atomic commits A commit is the unit of `bisect`, `revert`, `cherry-pick`, and `blame`. Optimize for those four operations. - **One logical change per commit**: the change and its tests together, nothing unrelated. "Fix X and also format the file" poisons blame; "half of feature X" breaks bisect. - **Every commit on the main branch builds and passes tests.** A bisect that lands on "WIP, doesn't compile" is a bisect wasted. With squash-merge this is automatic (each merged PR = one green commit); with merge/rebase flows, clean the branch (autosquash fixups) before merge — rules/03 §4. - Mechanical changes (rename, format, codemod) get their own commit, labeled as such, so reviewers and blame can skip them. Consider `.git-blame-ignore-revs` for repo-wide reformat commits. - **Don't commit what the build produces or the developer machine leaks**: artifacts, `.env`, editor droppings — enforce via `.gitignore` + pre-commit secret scanning (deeper treatment: `sota-secrets-management`). ## §2 Commit messages and Conventional Commits - **Subject**: imperative mood ("Add", not "Added"/"Adds"), ≤72 characters (50 is the classic ideal), no trailing period, capable of completing "if applied, this commit will ___". - **Body**: explains *why* and what the alternatives were — the diff already shows the what. Wrap at 72. Link issues/incidents. A one-line `fix typo` body is fine; a one-line body on a 300-line behavioral change is a finding. - **Conventional Commits** (conventionalcommits.org, spec v1.0.0 — still the current version as of 2026): `type(scope)!: description` with `feat`, `fix`, and friends; `!` or a `BREAKING CHANGE:` footer marks breaking changes. - **When it pays**: you automate something with it — changelog drafts and version bumps via semantic-release / release-please / git-cliff, monorepo per-scope releases, commit-lint gates. `feat` → minor, `fix` → patch, breaking → major gives you releases as a by-product of merging. - **When it doesn't**: no automation consuming the types. Then it's ceremony; plain well-written messages (above) are the actual requirement. Don't adopt the syntax without the pipeline. - If adopted: enforce mechanically (commitlint/PR-title check on squash merges), keep the type honest (`feat` that's really a `fix` corrupts the generated changelog), and remember the generated changelog still gets a human edit (rules/02 §6). **Bad**: `fixed stuff`, `WIP`, `address review comments`, `Update scheduler.py`. **Good**: ``` fix(payouts): skip frozen accounts in preview Preview ran the eligibility check but not the account-status check, so frozen creators saw payouts that execution would reject (#892). Reuse StatusGate from the execution path instead of duplicating the check, so the two paths cannot drift again. ``` ## §3 Branch strategy honesty Pick the simplest strategy your release reality allows, and admit which one you're actually running. - **Default: trunk-based with short-lived branches.** Branch from main, PR within a day or two, merge, delete. Long-lived feature branches are merge debt with interest; integrate continuously behind feature flags instead. Branch age is a metric worth alerting on (>3 days: ask why). - **Gitflow only with real release trains**: multiple maintained versions in the field (on-prem, mobile, embedded), genuine hardening windows. If you deploy from main continuously, `develop` + `release/*` + `hotfix/*` is process cosplay — every extra long-lived branch is a divergence to reconcile. - **Release branches without gitflow** are a legitimate middle: cut `release/1.4` from main at code freeze, cherry-pick fixes, tag from it — while feature work continues on main. Use when you ship versioned artifacts but develop trunk-based. - Hotfix path is defined *before* the incident: where it branches from (the release tag), where it must land back (main, always — a fix that exists only on a release branch regresses in the next release). - Default branch is protected: required reviews + checks, no direct pushes, no force push (rules/03 §6). ## §4 Versioning discipline - **SemVer means the numbers are promises**: major = breaking, minor = additive, patch = fixes. Breaking in a minor is a contract violation no matter what the release notes say. If you can't honor that, use explicit CalVer or 0.x rather than fake SemVer — consumers' tooling (`^`, `~`, dependabot) acts on the numbers, not your intentions. - **What counts as breaking is broader than compilation**: removed/renamed symbols and endpoints, tightened validation, changed defaults, changed error codes/types callers branch on, changed wire formats, raised minimum runtime/OS versions. Behavioral contracts documented in rules/02 §2 are part of the API. - 0.x is an explicit "anything may change" signal — fine for pre-stable, but don't live there for years while telling users you're production-ready. - **Breaking-change communication is a pipeline, not a number**: deprecation warnings in code → changelog `Deprecated` section → migration guide (rules/02 §7) → major release → removal. The version bump is the last step, never the first notice. - API surface evolution rules (additive change, deprecation headers, sunset) are covered in `sota-api-design`; this file covers the release mechanics. ## §5 Releases, tags, and release notes - **Releases are tagged, and tags are immutable**: never delete or re-point a published tag — downstream caches, lockfiles, and humans have already resolved it; a moved tag is a supply-chain incident (signed tags, provenance, artifact immutability: `sota-devsecops`). A bad release gets a *new* patch version, not a recycled tag. - **Releases are reproducible from the tag**: version is derived from the tag (not hand-edited in three files — or if files must contain it, the bump is automated and the tag is the source of truth), built by CI, never from a laptop. - **Automate the release train**: tag (or merged release PR) triggers build, changelog finalization (move `Unreleased` → version section), artifact publish, docs version publish (rules/02 §5). release-please / semantic-release style tooling does this off conventional commits (§2); the more humans in the loop, the rarer and scarier releases become. - **Release notes are the curated story** (highlights, upgrade notes, breaking changes up top, who should care); the changelog is the complete record (rules/02 §6). Generate the skeleton, edit for humans; leading with "Bump deps (#1042)" tells users you didn't. - Pre-releases use ordered, machine-readable identifiers (`2.0.0-rc.1`), published to pre-release channels (`next` dist-tag, pre-release flag) so they never resolve as `latest`. - Ship breaking changes in a release that contains *only* the breaking changes and their migration affordances where feasible — mixing "must-have fix" with "must-rewrite-callers" forces users to take the break to get the fix. ## Audit checklist - [ ] History bisects: sampled main-branch commits build and pass tests; no WIP/broken commits on main. - [ ] Commits are atomic — one logical change with its tests; mechanical changes isolated (and in `.git-blame-ignore-revs` where repo-wide). - [ ] Subjects imperative and ≤72 chars; bodies on non-trivial commits explain why; sampled `git log` is readable as a narrative. - [ ] If Conventional Commits: enforced by commitlint/PR-title check, types honest, and an automation pipeline actually consumes them; if not adopted, no half-applied `feat:` cargo-culting. - [ ] Branches are short-lived (check age of open branches); no zombie long-lived feature branches; flags used for incomplete work. - [ ] Branch model matches release reality (no gitflow on a continuous-deploy service); hotfix path documented and lands back on main. - [ ] Version numbers honor SemVer (diff a recent minor for breaking changes) or the project explicitly uses CalVer/0.x. - [ ] Breaking changes followed the pipeline: deprecation → changelog → migration guide → major; never a surprise minor. - [ ] Published tags never moved/deleted (compare tag dates vs. registry artifacts if suspicious); releases built by CI from the tag. - [ ] Release notes curated with breaking changes first; changelog `Unreleased` section flows into versioned sections at release; pre-releases can't resolve as latest. -
05-spec-driven-development.md 6.5 KB
# Spec-driven development Spec-driven development (SDD) makes a written, living **specification the source of truth** that a human or agent builds from, instead of prompting code into existence ad hoc. The spec is durable; the code is regenerable. Reserve it for non-trivial, multi-step, or multi-session work — it is overhead for a one-file change (§7). By 2026 the pattern has a common shape across tools — GitHub Spec Kit (`specify` CLI; github.com/github/spec-kit), AWS Kiro (kiro.dev/docs/specs), OpenSpec, BMAD — and is tool-agnostic at its core: plain Markdown in the repo is enough. The tool is interchangeable; the discipline is not. ## 1. The loop: intent → plan → tasks → implement → verify The canonical flow (Spec Kit's *Spec → Plan → Tasks → Implement*; Kiro's `requirements.md → design.md → tasks.md`): 1. **Intent / requirements** — *what and why*: user stories and **testable acceptance criteria**. No implementation detail. 2. **Design / plan** — *how*: architecture, interfaces, data shapes, sequencing. Links out to ADRs and API contracts; does not copy them (§5). 3. **Tasks** — an ordered list of **PR-sized, independently verifiable** units. 4. **Implement** task-by-task, each as a reviewable change. 5. **Verify** against the acceptance criteria — that list *is* the done-gate. Keeping these as separate artifacts is the point: requirements survive a rewrite of the design, and the design survives a re-implementation of the tasks. ## 2. Writing a spec an agent can build from - **Separate *what* from *how*.** The requirements doc states behavior and outcomes; the design doc holds mechanism. Implementation detail leaking into requirements is the most common smell. - **Every requirement is testable** — a user story plus acceptance criteria in concrete, checkable terms. Vague criteria ("works correctly") generate vague code. Concrete criteria become scenarios/tests (`sota-testing` rules/08). **EARS notation** ("WHEN *trigger* THE SYSTEM SHALL *response*") is a proven concrete syntax — unambiguous to humans and agents, and machine-checkable: tooling can now verify implementations against EARS-style requirements (e.g. Kiro writes `requirements.md` in EARS and property-tests code against it). - **Mark unknowns explicitly.** Use an open-questions list or a `[NEEDS CLARIFICATION]` marker (Spec Kit convention) rather than letting the author — or the agent — silently assume. Unresolved markers **block** implementation; they are a stop-and-ask trigger, not a guess. - **State scope and non-goals.** Ambiguity, not wrong syntax, is the number-one cause of generated code that misses the intent. ## 3. Living artifacts, in the repo, with the code - Specs live **in-repo** (e.g. `specs/<feature>/`), versioned and PR-reviewed exactly like docs and tests. The spec changes **in the same PR** as the behavior — the same rule as docs-in-the-same-PR (`rules/01` §2) and tests-in-the-same-PR (`sota-testing` rules/01 §1.8). - **Spec drift is the failure mode.** A spec nobody updates is worse than none — it actively misleads the next reader or agent (mirror `rules/01` §4). On a behavior change, update the spec or delete it; never let it rot in place. ## 4. Steering vs per-feature specs A per-feature spec captures *this feature's* intended behavior. **Durable house rules** — stack, conventions, security baseline, constraints — belong in a persistent steering file the agent always reads, not repeated in every spec: Spec Kit's *constitution*, Kiro's *steering files*, and in this ecosystem `AGENTS.md` / `CLAUDE.md` / `profiles/` (cross-ref `rules/01` §7). Per-feature spec = changeable intent; steering = stable rules. Don't conflate them. ## 5. Don't double-maintain — link, don't copy Each concern has one home; the spec **references** it rather than duplicating: - Architectural **decisions** → ADRs (`sota-architecture`). - API **contract** → OpenAPI / GraphQL SDL, spec-first (`sota-api-design` rules/01 §10, rules/03). - **Acceptance criteria** → executable scenarios (`sota-testing` rules/08). Copying a contract or decision into the feature spec guarantees the two drift. ## 6. Agent-execution discipline The library's audience runs these specs through coding agents, so: - **Each task lands as a small, reviewed PR** with tests and the spec update together — treat agent output like any contribution (`rules/03`, reviewing AI-generated code). - **Acceptance criteria are the verification gate.** "Done" means the criteria pass, not that the agent reported success — the claim-validation principle applies verbatim. - **No silent scope expansion.** If implementation needs a decision the spec doesn't cover, that is a `[NEEDS CLARIFICATION]` → stop and ask, not improvise. ## 7. When SDD pays / when it's overhead - **Pays:** non-trivial features, multi-session or multi-agent work, cross-team handoffs, and cutting the "regenerate from scratch" churn that ad-hoc prompting produces (GitHub's reported Spec Kit benefit). - **Overhead:** one-file fixes, exploratory spikes (spike → throw away → spec the real thing, mirroring the TDD stance), and thin glue. Don't ceremonialize trivial work — the spec must earn its maintenance cost. ## Audit checklist - [ ] Do non-trivial/multi-step features carry a written, in-repo spec (intent + acceptance criteria), not just chat history or a PR description? Missing on a multi-step feature → Medium. - [ ] Do specs separate *what* (requirements) from *how* (design)? Implementation detail in the requirements doc → Low. - [ ] Are acceptance criteria concrete and testable (each maps to a check)? Vague "works correctly" criteria → Medium. - [ ] Are unknowns marked explicitly (open-questions / `[NEEDS CLARIFICATION]`) rather than silently assumed? `grep -rin "NEEDS CLARIFICATION" specs/` surfacing markers shipped into implementation → Medium. - [ ] Is the spec updated in the same PR as the behavior (no drift)? Sample a recent feature; a spec older than the code it describes → High (it misleads). - [ ] Are durable house rules in a steering/constitution/AGENTS file, not copy-pasted into each spec? Duplication → Low. - [ ] Are decisions/contracts **linked** (ADRs, OpenAPI/SDL) rather than copied into the spec? A copied contract drifting from its source → Medium. - [ ] Do agent-built tasks land as small reviewed PRs with tests, verified against the acceptance criteria? "Done" with no criteria check on a money/auth/critical path → High.
-
-
SKILL.md 9.9 KB
--- name: sota-docs-workflow description: >- State-of-the-art documentation and engineering-collaboration guidance (2026) covering documentation architecture (Diátaxis, docs-as-code, READMEs, runbooks, onboarding, AI-readable docs), API/reference docs and changelogs, and the team workflow around code: pull requests, code review conduct, commit discipline, branching, and releases. Use when writing or structuring any documentation AND when auditing docs quality, and when designing or auditing collaboration workflow. Not for in-product UI microcopy (use sota-ux-writing) or marketing/landing-page copy (use sota-copywriting). Trigger keywords: documentation, docs, README, API docs, docstring, changelog, release notes, migration guide, code review, pull request, PR description, commit messages, conventional commits, branching, semver, onboarding, runbook, AGENTS.md, llms.txt, spec-driven development, SDD, Spec Kit, Kiro, specification, acceptance criteria, steering files. --- # SOTA Docs & Engineering Workflow ## Purpose Expert-level rules for the artifacts around the code: documentation (structure, reference, freshness, runbooks) and the collaboration workflow (PRs, review, commits, releases). These are the highest-leverage, lowest-glamour practices — review latency and doc decay quietly dominate team throughput. Rules are imperative with rationale and good/bad examples; every rules file ends with an audit checklist. Load only the files relevant to the task via the index below. Boundaries: ADR practice lives in `sota-architecture`; API contract design in `sota-api-design`; tag signing/provenance and CI supply chain in `sota-devsecops`. This skill references them rather than repeating them. ## BUILD mode When creating docs or setting up workflow: 1. **Classify before writing.** Every doc is exactly one Diátaxis mode (tutorial / how-to / reference / explanation) and is titled accordingly; mixed-mode pages are the defect to design out (`rules/01` §1). 2. **Docs live with the code**: in-repo, PR-reviewed, CI link-checked, examples executed. If a doc can't change in the same PR as the code, it will decay (`rules/01` §2, §4). 3. **README = what / why / 5-minute quickstart / honest status**, then links out (`rules/01` §3). Runbooks are alert-linked and command-exact (`rules/01` §5). Ship the **documentation baseline** — README + LICENSE + CHANGELOG always; SECURITY/CONTRIBUTING/CODE_OF_CONDUCT once public — each with one canonical home, not scattered copies (`rules/01` §8). 4. **Reference is generated** from OpenAPI/docstrings/rustdoc/godoc with warnings-as-errors; doc comments carry the why, contract, and failure modes; examples run in CI (`rules/02` §1–4). 5. **Changelog from day one**: Keep a Changelog format, `Unreleased` section updated in the PR that makes the change, user-impact language (`rules/02` §6). 6. **Workflow defaults**: small single-purpose PRs with what/why/how-tested descriptions (`rules/03` §1–2); trunk-based short-lived branches; atomic commits with imperative ≤72-char subjects; conventional commits only if automation consumes them (`rules/04` §1–3). 7. **Agent docs**: one short, human-curated AGENTS.md/CLAUDE.md with exact commands and repo-specific traps — never auto-generated bloat, never a restatement of ambient/global rules (`rules/01` §7, §10). On a **new repo**, `.gitignore` + secret scanning and LICENSE land before the first commit (`rules/01` §10). 8. **Solved failures accrue** in a symptom-keyed troubleshooting playbook, written in the PR that fixed them (`rules/01` §9). 9. Before declaring done, self-review against the relevant files' **Audit checklists**. ## AUDIT mode When auditing docs or workflow: 1. Scope the surfaces: docs tree + README + runbooks (`rules/01`), generated reference + changelog + migration guides (`rules/02`), recent PRs and review threads (`rules/03`), git history, branches, and tags (`rules/04`). 2. **Audit reality, not policy.** Sample artifacts: run the quickstart on a clean environment, follow a runbook's commands, read 10 docstrings, read the last 20 merged PRs and 50 commits, diff a recent minor release for breaking changes. A CONTRIBUTING.md full of rules nobody follows is itself a finding. 3. Work through each loaded file's **Audit checklist**; probe the classic gaps: tutorial that fails partway, stale docs contradicting code, README quickstart requiring tribal knowledge, alert with no runbook, `default: Error` as the only documented failure, changelog that's a commit dump, 2,000-line rubber-stamped PRs, force-push during review, broken commits on main, moved release tags, gitflow on a continuous-deploy service. ### Severity conventions - **Critical** — actively dangerous artifacts: runbook whose commands are wrong or destructive without warning; docs instructing insecure practice (secrets in config examples, auth bypass); moved/deleted published release tag; breaking change shipped in a minor/patch with no notice; merge to default branch with no review or required checks at all. - **High** — reliably costs incidents or releases: page-able alerts without runbooks; quickstart/tutorial that fails; published docs contradicting current released behavior; no changelog or migration guide across breaking releases; review rubber-stamping (large PRs, instant LGTMs, AI volume merged unread); non-bisectable main (broken commits); releases built outside CI. - **Medium** — erodes trust and throughput: mixed Diátaxis modes; undocumented public symbols or name-restating docstrings; unexecuted doc examples; no link-checking; PR descriptions missing why/how-tested; review SLA routinely blown; long-lived feature branches; commit-dump changelog; conventional commits adopted without enforcement or automation. - **Low** — polish: missing freshness dates; vanity/stale badges; unlabeled nit comments; subject lines over 72 chars; missing `.git-blame-ignore-revs` for reformat commits; docs index drift. ### Finding format ``` [SEVERITY] <one-line title> Where: <file:line | doc URL | PR/commit ref | branch/tag> Rule: <rules-file §section> Issue: <what is wrong, with observed evidence (quote the doc/PR/commit)> Impact: <concrete consequence — who is misled, what breaks, what it costs> Fix: <specific change; corrected text/command/process where load-bearing> ``` Order by severity; one finding per root cause; every finding cites sampled evidence (a doc you executed, a PR you read) — no findings from vibes. ## Rules index | File | Read this when... | |---|---| | `rules/01-documentation-architecture.md` | Writing/structuring/auditing any docs: Diátaxis modes, docs-as-code CI (link checks, doc tests), README front-door, decay control (ownership, freshness, aggressive deletion), runbooks, onboarding docs, discoverability, AGENTS.md/CLAUDE.md and llms.txt, and the repo **documentation baseline** (must-have docs + community-health files: LICENSE/SECURITY/CONTRIBUTING/CODE_OF_CONDUCT, and where GitHub looks for them). | | `rules/02-api-reference-changelogs.md` | API/library reference docs: generation from source (OpenAPI/docstrings/rustdoc/godoc), docstring content (why/contract/failures), runnable examples and doctests, error documentation, versioned docs, Keep a Changelog discipline, migration guides. | | `rules/03-code-review-pr-workflow.md` | PR and review process: PR sizing and slicing, description discipline (what/why/how-tested), review SLAs and WIP limits, reviewer/author conduct, blocking vs non-blocking comments, draft and stacked PRs, automation boundaries, reviewing AI-generated code. | | `rules/04-commits-branches-releases.md` | Git history and shipping: atomic/bisectable commits, message discipline, Conventional Commits and when they pay, trunk-based vs gitflow honesty, SemVer semantics, breaking-change pipeline, tag immutability, release notes vs changelog, release automation. | | `rules/05-spec-driven-development.md` | Driving non-trivial/agent-built work from a living spec: the intent→plan→tasks→implement→verify loop, separating what from how, testable acceptance criteria, `[NEEDS CLARIFICATION]` markers, specs-in-repo and spec-drift control, steering/constitution vs per-feature specs, linking ADRs/OpenAPI instead of copying, and when SDD is overhead. Pairs with `sota-testing` rules/08 (executable acceptance). | ## Top-10 non-negotiables 1. **One Diátaxis mode per document** — tutorials teach, how-tos accomplish, reference informs, explanation contextualizes; titles declare which. (rules/01 §1) 2. **Docs are code**: in-repo, PR-reviewed, link-checked in CI; behavior changes update docs in the same PR. (rules/01 §2) 3. **Wrong docs are worse than none** — own every doc, date-review the operational ones, delete stale pages instead of archiving them. (rules/01 §4) 4. **Every page-able alert links to a command-exact, incident-tested runbook.** (rules/01 §5) 5. **Reference generated from source with undocumented-public-symbol as a build failure; doc examples compile and run in CI.** (rules/02 §1–3) 6. **Document failure modes**: every operation states what can fail and what the caller should do; error messages point toward the fix. (rules/02 §4) 7. **Changelog in Keep a Changelog form, written in user-impact language, updated in the PR — and breaking changes ship with a migration guide after a deprecation period.** (rules/02 §6–7) 8. **Small, single-purpose PRs with what/why/how-tested descriptions; first review response within one business day.** (rules/03 §1–3) 9. **The review bar is provenance-blind**: AI-generated code gets the same scrutiny, and style/lint/type findings are CI's job, never a human's. (rules/03 §6–7) 10. **Main is always green and bisectable; published tags are immutable; version numbers keep SemVer's promises with breaking changes announced before they're shipped.** (rules/04 §1, §4–5)
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.