pr-body
Write a pull-request body and score it before the PR is opened, or audit an existing PR's body against the same rubric. Use when about to run gh pr create, when a PR body needs writing or rewriting, or when asked whether a PR description is any good. Invoked as /pr-body write <bo
Install
npx skills add https://github.com/ConnorGriffin/skills/tree/main/skills/tools/pr-body
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install connorgriffin-skills@llmmart
git clone https://github.com/ConnorGriffin/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole connorgriffin/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
PR body
The body is for whoever merges the change. It carries facts about the change in the system's own terms. The diff already carries the files.
The gold standard
This is the target. Copy its shape, not its subject.
The whole thing is what you would say to the team in one sentence, plus the handful of facts a merger cannot get from the title. The decisions, the alternatives, and the operator log are in the diff and the spec, and the body says where they are rather than repeating them.
## Motivation and Context
ghes-config now sets the memory ceilings and rate-limit posture that production already runs: 17 keys per ring for prd and stg, read off the production primary, rate limiting off. Staging had sandbox-sized ceilings on production-sized hardware.
* The 14 rate-limit keys are new to the reconciler's allowlist, on the `ghe-config` transport. dev.yaml is unchanged.
* Staging's primary was brought to those values by hand. Three of the four keys written did not exist there before, so the comparison below shows staging carrying production's values, not the appliance honouring the new ceilings.
* No reconciler instance exists in any ring, so nothing reads these files today.
Decisions and the operator log are in the change folder under `openspec/changes/archive/`.
Fixes PROJ-7351
## How Has This Been Tested?
* Production primary against staging primary after the apply: 17 lines per side, diff exit 0. Replication in sync on both replicas.
* `parity_test.sh` 37 checks pass, `python3 -m unittest discover -s scripts` 36 OK.
* `pulumi preview` against `origin/main`: this branch moves no resource.
## Checklist before requesting a review
- [x] I have performed a self-review of my code.
- [x] I have stepped through the README as though I was a new user to ensure clarity.
- [ ] I have added new or changed keys, tokens, other secrets to the DevOps 1Pass.
- [ ] I have labeled all "TO DO" items with associated Jira ticket number.
- [x] I have removed commented code from PRs to `main`.
- [x] I have followed conventional-commits standards when making this PR.
> Written by an AI agent operating for <operator>. Verify before relying on it.
What that body does
- Sections come from the team, never from you. Fill
.github/pull_request_template.mdwhen the repo ships one, then the organization default in the organization's.githubrepo. When neither exists, read the last two merged pull requests written by someone else and use their headings; a team convention lives in the pull requests even when it lives in no file. Only a repo with no template and no history gets a body with no headings. - The opening states what the system now does, present tense, then the state it replaced. One sentence each. It is the line you would say to the team, and a reader who stops there has the change.
- Three or four bullets, not twelve. A bullet earns its place by carrying a fact the opening does not imply. Never a bullet per file, and two bullets that restate each other are one bullet.
- Point at the spec and the diff instead of summarizing them. Why this value, what else was considered, which command ran in what order: one line saying where that lives. A reader who wants the decision opens the spec, and a body that retells it goes stale against it.
- Evidence is the raw number. "17 lines per side, diff exit 0", "37 checks pass", "moves no resource". Not the method, not the command line, not a sentence about having been careful.
- The caveat states what the evidence does not show. "The comparison shows staging carrying production's values, not the appliance honouring the new ceilings." That clause is the most valuable one in the body, and it is the first one a weaker author drops. Fold the deviation that caused it into the same bullet.
- Blast radius as mechanism. "No reconciler instance exists in any ring, so nothing reads these files today." Never a rating of the risk.
Fixes <KEY>on its own line, and the ticket appears nowhere else.- Tick only what is true. An unticked box for a check that does not apply is correct, and it is more credible than a body where every box is ticked.
- Length follows the facts, and most facts are not the body's job. The gold standard is about 1.5 KB. An earlier draft of the same change ran 2.4 KB by carrying decisions the spec already held, and prose ran 5.8 KB and said no more.
Never
- Motivation nobody told you. "Ahead of the next release cycle" is a guess wearing a fact's clothes.
- A clause that rates the change: "low risk", "no impact expected", "improves maintainability", "lands cleanly". The reader can check a mechanism and cannot check a rating.
- Anything addressed to the reviewer: "worth checking", "please look at the IAM change". Telling a reviewer where to look tells them where not to.
- Method narration: "I ran preview against dev and prd", "verified with".
- A retelling of the spec or the diff: the reasoning behind a value, the alternatives weighed, the order the commands ran in. Name where it lives.
- An empty template section. Cut the heading only if the template does not ship it; otherwise fill it.
- An em dash (use parens or two sentences), emoji,
-bullets, capitalized host or account names, and a bare file path outside code formatting. - Three parentheses in a sentence. One in a body is fine.
Verbs
write
Write the body to a file. The gate only reads
--body-file, so the file has to exist anyway. Pass an absolute path with no~: the gate reads the command text, and it cannot resolve a tilde or a path that does not exist yet. Create the file in one call and runghin the next, or the gate sees a file that is not there.Score it, with the target repo so the scorer can read the repo's template:
python3 <pr-body-skill-directory>/scripts/pr_body_lint.py \ --body-file <path> --repo <repo-root> --jsonFix every blocking finding. Each carries fix text written as an instruction; apply it rather than arguing with it. A warning is a judgment call you own.
Run the voice judge: references/judge.md. Give it the body and
git diff <base>...HEAD. The judge exists for one thing the scorer cannot see: a fact the diff makes load-bearing that the body never mentions. Apply the rewrites you accept, then re-score, because a rewrite can reintroduce a rule finding.Record the receipt, then open or edit the pull request with
--body-filepointing at the same file:python3 <pr-body-skill-directory>/scripts/pr_body_receipt.py write <path>
Editing the file after step 5 changes its hash and voids the receipt. Run the write verb again.
audit
Score an existing body and report. Never modify it.
gh pr view <pr> --json body -q .body > <path>.- Score it with
--repopointed at a checkout of the target repo. - Run the judge over the same body, with
gh pr diff <pr>as the diff. - Report the scorer findings by rule with their line numbers, then the judge's rewrites. No receipt: the audit did not author the body.
The gate
A separately installed PreToolUse hook on Bash matches gh pr create and gh pr edit, hashes the
--body-file it was handed, and denies when no receipt matches that hash. It
also denies the forms it cannot read: inline --body, heredocs, a tilde path, a
file that does not exist yet.
When that hook is installed, there is no bypass and it fails closed. The escape that exists is
uninstalling the hook from ~/.claude/hooks/, which a human can do and an agent
must not.
Rules the scorer implements
scripts/pr_body_lint.py is the only rule engine. A rule named here and absent
there is decorative. Grounds and fix text for each:
references/rubric.md.
| Rule | Fires on | Severity |
|---|---|---|
empty-body |
a body with no non-whitespace content | blocks |
ai-disclosure-missing |
no line disclosing AI assistance | blocks |
empty-template-section |
a heading with nothing under it | blocks |
vacuous-opener |
a first line like "Fix bug", "Phase 1", "Cleanup" | blocks |
oversized-input |
input past the scorer's robustness cap | blocks |
em-dash |
an em dash in prose | blocks |
emoji |
an emoji codepoint in prose | blocks |
path-in-prose |
a file path outside code formatting | blocks |
method-narration |
"verified with", "I ran", "tested by running" | blocks |
verdict-clause |
a clause rating the change instead of describing it | blocks |
reviewer-instruction |
"please review", "worth checking", "take a look at" | blocks |
bullet-per-file |
most bullets in a list of 3 or more lead with a filename | blocks |
symbol-in-prose |
an identifier or function name outside code formatting | warns |
Lines matching the repo's template, and the AI disclosure blockquote, are scaffolding and exempt from the prose rules. Below 40 characters of prose the density rules do not run at all; the structural rules run at every length.
A scorer pass means the countable defects are absent. On the labeled set it caught 4 of 12 rejected bodies, and all 4 were empty or near-empty. Everything else was voice, which is what the judge and the gold standard above are for.
Files (skills)
-
agents
-
openai.yaml 256 B
interface: display_name: "PR Body" short_description: "Write a PR body and score it before the PR opens, or audit an existing one" default_prompt: "Use $pr-body to write this pull-request body, score it, and write the receipt before opening the PR."
-
-
references
-
judge.md 10.8 KB
# The voice judge The linter already ran. It checked countable defects and found none, which on the operator's labeled set means very little: it caught 4 of 12 rejected bodies and all 4 were empty or near-empty. Everything that decided the other 8 was phrasing. That is what this pass is for. The judge reads voice. It does not re-check rules, re-count characters, or verify that the change is correct. It reads the body **and the diff**. The diff is not there to be reviewed. It is there for one question the body cannot answer on its own: does the change do something the body never mentions? That is the eighth texture below, and it is the only one the diff is used for. ## Output A verdict and line-level rewrites. Never a score on its own, and never a note that the body "could be tightened" without saying which line and to what. ```json { "verdict": "rewrite", "rewrites": [ { "line": 1, "was": "This PR undertakes a comprehensive overhaul of the volume sizing story.", "now": "Grows the appliance data volume from 2 TB to 4 TB." }, { "line": 3, "was": "", "now": "* Capacity alarm threshold follows the new size, 1600 to 3200 GiB." } ] } ``` An entry with an empty `was` is an added line, and `line` is the line it follows (0 for the top of the body). Every other entry replaces the line it names. `pass` carries an empty `rewrites` list. A `rewrite` verdict with no rewrites is malformed output, not a soft fail. ## Bias toward passing Pass unless a specific line is wrong and you can write the line that replaces it. If the honest reaction is "this reads a bit long", that is a pass. The tradeoff is deliberate and it is asymmetric. A false deny costs the operator a rewrite cycle on a body that was already fine, on a change that is otherwise ready to merge. A false pass costs one mediocre PR description. The second is cheaper, so the judge takes that side of the error. Two consequences worth stating outright. One genuine parenthesis is not a finding, and one long-ish paragraph is not a finding. A finding needs a pattern the reader would notice, or a single sentence that is plainly wrong. ## What to look for Eight textures, each drawn from the operator's own annotations on real bodies. Examples first; the rules underneath them are short on purpose, because the quiz that produced these preferences showed examples carry voice where abstract rules do not. ### Invented motivation The author is claiming to know something they do not. The most reliable single finding in the set, and the one worth being least lenient about. > Grows the data volume ahead of the next release cycle, so capacity is in place > before traffic picks up. Becomes: > Grows the data volume from 2 TB to 4 TB. It has been running near capacity. If the ticket said why, that reason is a fact and belongs. If nobody said, the body says what changed and stops. Tells: "in preparation for", "ahead of", "so that the team can", "as part of our ongoing". ### Verdict clauses that rate rather than describe The linter catches the stock phrasings. The class is wider than the list, and new phrasings are exactly what the judge is for. > The blast radius here is quite contained and reviewers can be confident this > lands cleanly. Becomes: > One resource updates. EBS expands online. Test: could the reader check this sentence against the system? A mechanism can be checked. A rating cannot. ### Parenthetical density > Grows the volume (currently 2 TB, provisioned in 2023) to 4 TB (the next size > that clears the projected growth curve), with no change to the snapshot > schedule (which already runs nightly). Becomes: > Grows the volume from 2 TB to 4 TB. The nightly snapshot schedule is > unchanged. One parenthesis in a body is fine. Three in a sentence is an author negotiating with themselves in public. ### Inline-code and symbol density > Updates `volumeSize` in `NewApplianceStack()` so `pulumi up` produces the > larger `aws.ebs.Volume`. Becomes: > Grows the appliance data volume to 4 TB. Every backticked symbol sends the reader to the diff to find out what the sentence meant. Name the behavior instead. A symbol earns its place when the symbol itself is the fact the reader needs (a config key they will set, a flag they will pass). ### Paragraphs where bullets belong > This change updates the volume size, and it also raises the alarm threshold so > the new capacity is reflected, and the runbook has been updated to match, plus > the dashboard query needed a small change to pick up the new dimension. Becomes: > * Volume grows from 2 TB to 4 TB. > * Capacity alarm threshold follows the new size. > * Runbook and dashboard query updated to match. A list of facts is a list. Prose is for the one place it earns its keep, which is the risk mechanism. ### Prose volume with no action behind it The most common annotation on the labeled set: "too much prose for not enough action". Weigh the word count against the number of distinct facts, not against the diff. > This pull request represents an incremental step in our capacity management > approach for the appliance fleet. Storage pressure has been an ongoing area of > attention, and this change addresses it directly by adjusting the provisioned > capacity of the data volume upward, bringing it into line with observed usage > patterns and giving headroom for the foreseeable future. Becomes: > Grows the appliance data volume from 2 TB to 4 TB. It has been running near > capacity. Three sentences carrying one fact is a rewrite. So is a five-bullet list where two bullets restate the other three. ### Pretentious or juvenile phrasing Both directions of the same failure, and both appear in the labels. Pretentious: > This change harmonizes the storage posture of the appliance tier with the > realities of its consumption profile. Juvenile: > Turns out the disk was basically full! Bumped it up so we should be good now. Both become: > Grows the appliance data volume from 2 TB to 4 TB. It has been running near > capacity. ### A load-bearing fact the body omits The only texture that needs the diff, and the only one that catches underfill. The labeled set rejects short bodies as well as long ones (a 77-char body annotated "way too thin", among others), and no length threshold separates them: an 80-char body passed and an 82-char one failed. The defect is never the size. It is that the change did a second thing and the body mentions only the first. Body: > Grows the appliance data volume from 2 TB to 4 TB. It has been running near > capacity. Diff: the volume grows, **and** the capacity alarm threshold moves from 1600 to 3200 GiB. Add: > * Capacity alarm threshold follows the new size, 1600 to 3200 GiB. Fires on exactly two shapes: * The change does something a reader of the body would not expect from the body. * A second distinct thing changed, and only the first is mentioned. Hard boundary, because this is the one rule that could turn the judge into a code reviewer. It does not fire on style, correctness, scope, test coverage, or whether the change is a good idea. Those stay on the list below, diff or no diff. The rewrite is always the missing fact, written out as the line to add. Never "add more detail", never "expand on the rationale". If you cannot state the missing fact in one line, there is no finding. The fact has to be load-bearing, not merely present in the diff. A renamed local variable is in the diff and belongs nowhere near the body. The test is whether a reviewer who read only the body would be surprised by what they find. ## Not the judge's business Do not raise any of these. The rest of the system owns them, and a judge that wanders into them produces findings the author cannot act on. * Length, in either direction. There is no target and no ceiling, and the judge never asks for more words. It asks for one named missing fact, or for nothing. A body of one true line that omits nothing is a pass. * Markdown headers, and any section the repo's PR template supplies. * Checkboxes. Legitimate when the template ships them, when they record testing done, and when they list post-merge steps to verify. * The AI disclosure line. Boilerplate, not authored prose. * Anything the linter already reported. It ran first. * Whether the change itself is correct, well-scoped, or a good idea. Having the diff does not change this. The diff answers one question only: did the change do something the body never mentions. ## The prompt Paste the body under this with line numbers, then the diff. > You are judging the voice of a pull-request body. A deterministic linter has > already checked it for countable defects and found none, so do not look for > file paths, em dashes, emoji, or stock verdict phrasings. Judge only how it > reads, plus one question about coverage (rule 8). > > You are given the body with line numbers, and the diff of the change. The diff > is not for review. Use it only for rule 8. > > Return JSON: a `verdict` of `pass` or `rewrite`, and a `rewrites` list, each > entry carrying `line`, `was`, and `now`. `now` is the replacement text, > written out in full. An entry with an empty `was` is a line to add after > `line` (0 for the top). Never return a rewrite you cannot write the > replacement for, and never return a score or a general comment in place of a > rewrite. > > Be biased toward passing. A false deny costs a rewrite cycle on a body that > was fine, which is worse than letting one mediocre body through. One > parenthesis is not a finding. One long paragraph is not a finding. Pass unless > a specific line is wrong, or a specific fact is missing. > > Raise a rewrite for any of these, and nothing else: > > 1. Motivation the author could not know. "Ahead of the next release cycle" > when nobody said that. > 2. A clause that rates the change instead of describing it. The reader can > check a mechanism, not a rating. > 3. Parenthetical asides stacked up, several in a sentence or throughout. > 4. Inline code spans and symbol names where the behavior would say it better. > 5. A paragraph carrying a list of facts that should be bullets. > 6. Word count out of proportion to the number of distinct facts. > 7. Phrasing that reads as pretentious, or as juvenile. > 8. A fact the diff makes load-bearing that the body does not mention: the > change does something a reader of the body would not expect, or a second > distinct thing changed and only the first is named. The rewrite is the > missing fact written out as a line to add, never a request for more detail. > If you cannot state it in one line, there is no finding. A fact that is > merely present in the diff (a renamed local, a reformatted block) is not > load-bearing. > > Say nothing about length, markdown headers, template sections, checkboxes, the > AI disclosure line, or whether the change is correct, well-scoped, or a good > idea. Never ask for more words. Ask for one named missing fact, or for nothing. -
rubric.md 13.6 KB
# pr-body rubric One entry per rule `../scripts/pr_body_lint.py` implements. That file is the only rule engine; this one records what each rule fires on, what its fix says, and what grounds it. Corpus rates are AI-era (n=100) against human-era (n=109) merged PRs from the operator's own history. Labels are the hand-labeled set of 25 bodies the operator judged blind. The corpus is one author's, and the labels are one author's. A rule grounded only in "elicited" is a preference this pack encodes on purpose, not an industry standard. Where a standard says otherwise, see *Known disagreements*. ## Rules ### `empty-body` (block) **Fires on** a body with no non-whitespace content at all. **Fix** State what changed and why; an empty body tells the reviewer nothing. **Grounds** Corpus: 33 empty bodies, all human-era. Labels: every empty or stub body in the sample was rejected. Shopify's review guide treats an inadequate description as grounds to send the PR back before judging the code at all (https://shopify.engineering/great-code-reviews). ### `ai-disclosure-missing` (block) **Fires on** a body with no line naming an AI tool alongside a preparation or assistance verb. The match is deliberately loose so any phrasing of the disclosure counts, not one fixed sentence. **Fix** Add a disclosure line: '> Written by an AI agent operating for <operator>. Verify before relying on it.' **Grounds** The Kubernetes contributor guide is the only verified institutional policy requiring disclosure in the description (https://www.kubernetes.dev/docs/guide/pull-requests/). This pack's canonical wording is the blockquote form already used elsewhere for agent-authored content, so the disclosure reads the same way across surfaces. The older plain sentence still satisfies the check: it matches the same loose pattern, and retro-failing bodies written before this change would be pure audit noise with no behavior fixed. Scoping is free here: the hook fires on an agent's Bash tool, so every body it scores is agent-mediated by construction. The rule needs no heuristic for whether a body was AI-written, because the trigger already answers that. Kubernetes' companion ban on AI co-author commit trailers is left alone; this pack's commit convention already keeps attribution out of trailers. ### `empty-template-section` (block) **Fires on** a heading with nothing under it, excluding a blank final section when the body has more than one heading (a trailing blank section is almost always optional metadata, not an abandoned one). **Fix** Fill in the named section. Delete a heading only when it is a non-shipped optional heading; retain every heading supplied by the repository or organization template and fill it with applicable substance. **Grounds** Corpus: the human-era failure mode is underfill, typically a 52-char body that is the repo template with every section left empty. Labels confirm it. GitHub's own docs frame templates as a delivery mechanism and make no claim that they improve description quality (https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/about-issue-and-pull-request-templates), so an unfilled template earns nothing. ### `vacuous-opener` (block) **Fires on** a first prose line matching a named-bad pattern: "Fix bug", "Fix build", "Add patch", "Add convenience functions", "Moving code from A to B", "Phase 1", "WIP", "Minor fixes", "Update", "Cleanup", "kill weird URLs". Only lines under 200 characters are tested, so a real opening sentence is never matched. **Fix** Open with the behavior that changed, not a generic label like this line. **Grounds** Google names these first lines outright as failing to provide useful information (https://google.github.io/eng-practices/review/developer/cl-descriptions.html). Zulip names the same failure from the other side: a summary that does not say what part of the codebase changed (https://zulip.readthedocs.io/en/stable/contributing/commit-discipline.html). ### `oversized-input` (block) **Fires on** input past the scorer's robustness cap of 20,000 characters. The body is refused outright rather than run through every regex. **Fix** Pass the body as a file with `--body-file` instead of inline; this input is too large to lint directly. **Grounds** Not a judgment about length. This is the cap that keeps a pathological input from spending the hook's timeout budget. See *Known disagreements* for why there is no length rule underneath it. ### `em-dash` (block) **Fires on** an em dash in prose (code fences, inline code, link targets, and bare URLs are masked out first). **Fix** Rewrite without an em dash; use a period or parentheses instead. **Grounds** Corpus 26% / 0%, the cleanest discriminator measured. Already banned by the operator's own style rules, which makes the corpus rate a confirmation rather than the reason. Freeburg's suppression study across 12 models and 5 providers found em-dash frequency survives explicit instruction to drop markdown and reads as a fingerprint of fine-tuning methodology (https://arxiv.org/abs/2603.27006). That study is a single preprint about general prose, not PR bodies, and the counter-position is on record: a single em dash cannot convict an individual document, and em dashes are also a marker of skilled human writers (https://www.howtogeek.com/no-an-em-dash-cant-help-you-detect-ai-text/). The rule survives that objection because it is a house style ban, not a detector. ### `emoji` (block) **Fires on** an emoji codepoint in prose. **Fix** Remove the emoji; state the fact in words. **Grounds** Corpus 29% / 0%. No institutional source addresses emoji directly. The plain declarative register of every source that does address tone (Google, Zulip, the kernel) leaves no room for one, and the operator's style rules ban emoji in technical content outright. ### `path-in-prose` (block) **Fires on** a path-shaped token (a slash, and a short alphanumeric extension on the last segment) in prose, outside code formatting and URLs. **Fix** Name the behavior, not the file; the diff already lists files. **Grounds** Corpus 24% / 2%. This is the countable proxy for the most corroborated rule in the research pass, which no regex can test directly: the body explains what and why, because the diff already shows how. Five independent sources converge on it (https://google.github.io/eng-practices/review/developer/cl-descriptions.html, https://cbea.ms/git-commit/, https://zulip.readthedocs.io/en/stable/contributing/commit-discipline.html, https://github.com/sourcegraph/handbook/blob/main/content/departments/engineering/dev/process/pull-requests.md, https://docs.kernel.org/process/submitting-patches.html). A path is a "how". ### `method-narration` (block) **Fires on** "verified with", "tested by running", "I ran", "ran the". **Fix** State the result of verification, not how you performed it. **Grounds** Corpus 7% / 0%. Elicited: the operator chose `Resources: 1 to update` over a sentence describing running preview against dev and prd. Zulip states the same rule for commit bodies, that unnecessary personal narrative about the process does not belong (https://zulip.readthedocs.io/en/stable/contributing/commit-discipline.html). ### `verdict-clause` (block) **Fires on** clauses that rate the change rather than describe it: "this is a low-risk / simple / straightforward / minor / trivial change", "no downtime is expected", "should have no impact", "improves maintainability", "makes the code cleaner", "straightforward", "should be safe". **Fix** Replace the rating with the concrete fact that supports it. **Grounds** Corpus 2% / 0%, the lowest rate of any shipped rule, so the corpus is not what carries it. Elicited, and broadened from the elicited example to the whole class: the operator accepted "EBS expands online, no downtime" (mechanism) and rejected "no downtime is expected" (verdict). The distinction is that a verdict is a claim about the change that the reader cannot check, where a mechanism is a fact about the system that they can. ### `reviewer-instruction` (block) **Fires on** "please review", "worth checking", "reviewers should", "take a look at", "let me know if". **Fix** Delete the instruction to the reviewer; let the diff and description stand on their own. **Grounds** Corpus 13% / 0%. Elicited: offered a targeted pointer, a generic ask, and nothing, the operator chose nothing. This one contradicts a published standard. See *Known disagreements*. ### `bullet-per-file` (block) **Fires on** a list of 3 or more bullets where at least 60% lead with a filename or path. **Fix** Describe the behavior the files implement together, not a bullet per file. **Grounds** Elicited: never a bullet per file. No corpus rate was measured for this shape specifically. The thresholds exist so a legitimately multi-part change that names two files among six bullets does not trip it. Practitioner sources name the anti-pattern directly (line-by-line diff restatement), but at blog tier only; the institutional backing is the same what-and-why-not-how convergence cited under `path-in-prose`. ### `symbol-in-prose` (warn) **Fires on** an identifier-shaped token in prose: snake_case, camelCase, or a bare `name()` call, outside code formatting. **Fix** Wrap the identifier in backticks, or name the behavior instead of the symbol. **Grounds** Elicited, from the operator's own annotations on the labeled set: "too many code references" is one of the recurring notes on bodies that were passed but marked down. Warn rather than block, because the pattern also matches legitimate prose words and product names, and a false block costs more than a false warning. The judge picks up the density judgment this rule cannot make. ## Documented non-signals Three candidate rules were considered against the same evidence and refused. They are recorded here with their evidence so a future agent reading the corpus does not re-add them. ### Markdown headers and section structure Corpus 79% AI-era against 83% human-era. Headers are not an AI tell here because the repos ship `.github/pull_request_template.md` supplying `## What does this PR do?` and friends, so a rule against headers would fire hardest on the era it is not meant to catch. Independently corroborated. Freeburg's suppression experiment found that overt markdown features (headers, bullets, bold) are eliminated or nearly eliminated the moment a model is told to drop markdown, which makes them near-pure instruction-following and close to worthless as a fingerprint (https://arxiv.org/abs/2603.27006). The rule is about what sits under a heading, not the heading. That is `empty-template-section`, and it is the only structure rule that ships. ### AI-vocabulary wordlists Corpus 1% in both eras. "comprehensive", "leverage", "robust", "seamlessly": a rule built on that list fires on nothing and catches nothing. The detection literature is against the approach as a class, not just against this list. A comparison of 14 commercial AI-text detectors found none reaching 80% accuracy and only five above 70%. That figure reached the research pass at search-summary level and was not traced to a single fetched paper, so treat it as corroborating direction rather than a hard number. No institutional or academic source was found endorsing wordlist detection at all. The one credible adjacent finding (Freeburg, above) is deliberately narrow: one punctuation mark, mechanistically explained and empirically tested, which is the opposite of a broad vocabulary list. ### Checkboxes Corpus 18% AI-era against 0% human-era, a clean discriminator, and still not a defect. Operator ruling: checkboxes are legitimate whenever a template ships them, whenever they record what testing was done, and whenever they list post-merge steps someone has to verify. This is the entry that shows corpus rates alone cannot decide a rule. A clean discriminator can be a good practice the author only recently adopted. Every shipped rule above was re-checked against that standard. ## Known disagreements ### `reviewer-instruction` runs against GitHub's own advice GitHub's engineering blog recommends the opposite of this rule: be explicit about the kind of feedback you want, a quick look against a design critique (https://github.blog/developer-skills/github/how-to-write-the-perfect-pull-request/). The corpus rate (13% / 0%) measures a real era split, but the era split is not the argument; the elicited preference is. This rule is encoded as the operator wants it, and the counter-citation is recorded here so the preference is not laundered into a standard. ### A length ceiling was built, measured, and removed The Linux kernel supplies the best argument for one: a description getting long is a signal the patch needs splitting, which frames a ceiling as a scope diagnostic rather than a prose-economy rule (https://docs.kernel.org/process/submitting-patches.html). No institutional source states a target length for a description; the kernel's is the closest thing, and it is diagnostic, not prescriptive. The operator's labels refuted it anyway. Passing bodies span 80 to 2242 characters and failing bodies span 0 to 1703; the ranges nest, the longest body in the sample passed, and a 1703-character body failed. The best single length threshold reaches 72% against a 50% baseline and earns all of it at the bottom end, separating empty stubs from everything else. That is `empty-body`, not a ceiling. `oversized-input` is a robustness cap on the scorer, not a judgment about the body. One calibration note belongs with this. Era is barred as a tuning target: the labels put AI-era bodies at 69% pass and human-era at 33%, so a linter tuned to flag the AI era would have been tuned toward the labels the operator rejects more often, while reporting green.
-
-
scripts
-
pr_body_lint.py 18.5 KB
#!/usr/bin/env python3 """Lint a PR body against the pr-body skill's rubric.""" from __future__ import annotations import argparse import json import re import sys from pathlib import Path from typing import Optional THRESHOLDS = { # Robustness cap only, not a judgment about the body: hand-labeled data # showed body length does not separate pass from fail (a 2242-char body # passed, a 1703-char body failed), so length is not a scored rule here. # Past this, the input is refused outright rather than run through every # regex below. "max_input_chars": 20_000, # Below this many prose chars, a body is too short for the density rules # (em-dash, emoji, paths, narration, verdicts, reviewer asks, # bullet-per-file, symbol-in-prose) to have signal. "density_floor_chars": 40, "bullet_per_file_min_bullets": 3, "bullet_per_file_ratio": 0.6, "excerpt_max_chars": 120, "vacuous_opener_max_chars": 200, } CODE_FENCE_RE = re.compile(r"```.*?```", re.DOTALL) INLINE_CODE_RE = re.compile(r"`[^`]*`") LINK_TARGET_RE = re.compile(r"(?<=\])\([^)]*\)") BARE_URL_RE = re.compile(r"https?://\S+") HEADING_RE = re.compile(r"^(#{1,6})\s+(.*)$") BULLET_RE = re.compile(r"^\s*(?:[-*+]|\d+[.)])\s+(.*)$") EMOJI_RE = re.compile( "[" "\U0001F1E6-\U0001F1FF" "\U0001F300-\U0001FAFF" "\U00002600-\U000027BF" "\U00002B00-\U00002BFF" "]" ) VACUOUS_OPENER_PATTERNS = [ re.compile(r"^fix(es|ed)?\.?$", re.I), re.compile(r"^fix(es|ed)?\s+(the\s+)?(bug|build|issue|error)s?\.?$", re.I), re.compile(r"^add(s|ed)?\s+(a\s+)?patch(es)?\.?$", re.I), re.compile(r"^add(s|ed)?\s+convenience\s+functions?\.?$", re.I), re.compile(r"^mov(e|ing)\s+code\s+from\s+.+\s+to\s+.+\.?$", re.I), re.compile(r"^phase\s*\d+\.?$", re.I), re.compile(r"^(wip|work[\s-]in[\s-]progress)\.?$", re.I), re.compile( r"^(minor|small|misc(ellaneous)?)\s+(fix(es)?|change(s)?|update(s)?)\.?$", re.I, ), re.compile(r"^update(s|d)?\.?$", re.I), re.compile(r"^clean\s*up\.?$", re.I), re.compile(r"^kill\s+weird\s+urls\.?$", re.I), ] METHOD_NARRATION_PATTERNS = [ re.compile(r"\bverified\s+with\b", re.I), re.compile(r"\btested\s+by\s+running\b", re.I), re.compile(r"\bi\s+ran\b", re.I), re.compile(r"\bran\s+the\b", re.I), ] VERDICT_CLAUSE_PATTERNS = [ re.compile(r"\bthis\s+is\s+a\s+(low[\s-]risk|simple|straightforward|minor|trivial)\s+change\b", re.I), re.compile(r"\bno\s+downtime\s+is\s+expected\b", re.I), re.compile(r"\bshould\s+have\s+(no|little|minimal)\s+impact\b", re.I), re.compile(r"\bimproves?\s+maintainability\b", re.I), re.compile(r"\bmakes?\s+the\s+code\s+(cleaner|more\s+readable)\b", re.I), re.compile(r"\bstraightforward\b", re.I), re.compile(r"\bshould\s+be\s+safe\b", re.I), ] REVIEWER_INSTRUCTION_PATTERNS = [ re.compile(r"\bplease\s+review\b", re.I), re.compile(r"\bworth\s+checking\b", re.I), re.compile(r"\breviewers?\s+should\b", re.I), re.compile(r"\btake\s+a\s+look\s+at\b", re.I), re.compile(r"\blet\s+me\s+know\s+if\b", re.I), ] # Identifier-shaped tokens: snake_case, camelCase, or a bare function call. SYMBOL_PATTERNS = [ re.compile(r"\b[a-z][a-z0-9]*_[a-z0-9_]+\b"), re.compile(r"\b[a-z]+[A-Z][a-zA-Z0-9]*\b"), re.compile(r"\b[A-Za-z_][A-Za-z0-9_]*\(\)"), ] FILENAME_RE = re.compile(r"^[\w.-]+\.[A-Za-z0-9]{1,8}$") # Flexible AI-disclosure match: an AI-related term and a preparation/assist # verb on the same line, so any wording of the Kubernetes-style disclosure # sentence is accepted rather than one fixed string. AI_TERM_RE = re.compile( r"\b(ai|artificial intelligence|generative ai|genai|an? assistant|" r"copilot|claude(?:\s+code)?|chatgpt|gpt-?\d*|llm)\b", re.I, ) ASSIST_VERB_RE = re.compile( r"\b(assist(ed|ance)?|help(ed)?|prepar(ed|ing)?|wrote|written|writing|" r"generat(ed|ing)?|draft(ed|ing)?)\b", re.I, ) FIXES = { "empty-body": "State what changed and why; an empty body tells the reviewer nothing.", "ai-disclosure-missing": ( "Add a disclosure line: '> Written by an AI agent operating for " "<operator>. Verify before relying on it.'" ), "empty-template-section": "Fill in the {heading!r} section, or delete it if it does not apply.", "vacuous-opener": "Open with the behavior that changed, not a generic label like this line.", "oversized-input": "Pass the body as a file with --body-file instead of inline; this input is too large to lint directly.", "em-dash": "Rewrite without an em dash; use a period or parentheses instead.", "emoji": "Remove the emoji; state the fact in words.", "path-in-prose": "Name the behavior, not the file; the diff already lists files.", "method-narration": "State the result of verification, not how you performed it.", "verdict-clause": "Replace the rating with the concrete fact that supports it.", "reviewer-instruction": "Delete the instruction to the reviewer; let the diff and description stand on their own.", "bullet-per-file": "Describe the behavior the files implement together, not a bullet per file.", "symbol-in-prose": "Wrap the identifier in backticks, or name the behavior instead of the symbol.", } SEVERITY = { "empty-body": "block", "ai-disclosure-missing": "block", "empty-template-section": "block", "vacuous-opener": "block", "oversized-input": "block", "em-dash": "block", "emoji": "block", "path-in-prose": "block", "method-narration": "block", "verdict-clause": "block", "reviewer-instruction": "block", "bullet-per-file": "block", "symbol-in-prose": "warn", } class LintError(RuntimeError): """An internal failure that should surface as a JSON error object.""" def _excerpt(text: str) -> str: limit = THRESHOLDS["excerpt_max_chars"] text = text.strip() return text if len(text) <= limit else text[: limit - 1] + "…" def _blank_spans(pattern: re.Pattern, text: str) -> str: """Replace every match with just its own newlines, preserving line numbers.""" return pattern.sub(lambda m: "\n" * m.group(0).count("\n"), text) def _prose_only(text: str) -> str: text = _blank_spans(CODE_FENCE_RE, text) text = _blank_spans(INLINE_CODE_RE, text) text = _blank_spans(LINK_TARGET_RE, text) text = _blank_spans(BARE_URL_RE, text) return text def _normalize_line(line: str) -> str: return re.sub(r"\s+", " ", line.strip()) def _load_template_lines(repo: Optional[Path]) -> set[str]: if repo is None: return set() template = repo / ".github" / "pull_request_template.md" if not template.is_file(): return set() try: raw = template.read_text(encoding="utf-8", errors="replace") except OSError: return set() return { _normalize_line(line) for line in raw.splitlines() if _normalize_line(line) } def _scaffolding_mask(lines: list[str], template_lines: set[str]) -> list[bool]: # A disclosure line (the Claude Code footer, e.g.) is tool-appended # boilerplate, not authored prose, so it is exempt the same way a # template heading is. return [ (bool(template_lines) and _normalize_line(line) in template_lines) or (AI_TERM_RE.search(line) is not None and ASSIST_VERB_RE.search(line) is not None) for line in lines ] def _first_prose_line( lines: list[str], scaffolding: list[bool] ) -> Optional[tuple[int, str]]: for index, raw in enumerate(lines): if scaffolding[index]: continue text = raw.strip() if not text or HEADING_RE.match(text): continue bullet_match = BULLET_RE.match(raw) content = bullet_match.group(1).strip() if bullet_match else text if content: return index, content return None def _check_vacuous_opener( lines: list[str], scaffolding: list[bool] ) -> list[dict]: found = _first_prose_line(lines, scaffolding) if found is None: return [] line_index, content = found if len(content) > THRESHOLDS["vacuous_opener_max_chars"]: return [] stripped = content.strip().rstrip(".") for pattern in VACUOUS_OPENER_PATTERNS: if pattern.match(stripped) or pattern.match(content): return [ { "rule": "vacuous-opener", "line": line_index + 1, "excerpt": _excerpt(content), } ] return [] def _check_empty_template_sections(lines: list[str]) -> list[dict]: # A heading with nothing under it is the underfill failure mode whether # or not the heading happens to also live in a repo's PR template. headings: list[tuple[int, str]] = [] for index, line in enumerate(lines): match = HEADING_RE.match(line.strip()) if match: headings.append((index, match.group(2).strip())) findings = [] for position, (index, heading_text) in enumerate(headings): # A blank final section is almost always optional trailing metadata # (a ticket link, a screenshot placeholder); only a blank section # with more document after it is the abandoned-template failure. # A single lone heading has nowhere else to carry content, so it # stays in scope even though it is technically "last". if position == len(headings) - 1 and len(headings) > 1: continue is_last = position + 1 >= len(headings) end = len(lines) if is_last else headings[position + 1][0] section = lines[index + 1 : end] if not any(line.strip() for line in section): findings.append( { "rule": "empty-template-section", "line": index + 1, "excerpt": _excerpt(heading_text or lines[index].strip()), "heading": heading_text or lines[index].strip(), } ) return findings def _check_pattern_rule( rule: str, patterns: list[re.Pattern], lines: list[str], scaffolding: list[bool], ) -> list[dict]: for index, line in enumerate(lines): if scaffolding[index]: continue for pattern in patterns: match = pattern.search(line) if match: return [ { "rule": rule, "line": index + 1, "excerpt": _excerpt(line), } ] return [] def _check_em_dash(lines: list[str], scaffolding: list[bool]) -> list[dict]: for index, line in enumerate(lines): if scaffolding[index]: continue if "—" in line: return [ { "rule": "em-dash", "line": index + 1, "excerpt": _excerpt(line), } ] return [] def _check_emoji(lines: list[str], scaffolding: list[bool]) -> list[dict]: for index, line in enumerate(lines): if scaffolding[index]: continue if EMOJI_RE.search(line): return [ { "rule": "emoji", "line": index + 1, "excerpt": _excerpt(line), } ] return [] def _looks_like_path(token: str) -> bool: candidate = token.strip(".,;:()[]{}\"'") if not candidate or "/" not in candidate: return False if candidate.startswith(("http://", "https://")): return False tail = candidate.rsplit("/", 1)[-1] if "." not in tail: return False ext = tail.rsplit(".", 1)[-1] return 1 <= len(ext) <= 8 and ext.isalnum() def _check_path_in_prose( prose_lines: list[str], scaffolding: list[bool] ) -> list[dict]: for index, line in enumerate(prose_lines): if scaffolding[index]: continue for token in line.split(): if _looks_like_path(token): return [ { "rule": "path-in-prose", "line": index + 1, "excerpt": _excerpt(token), } ] return [] def _check_symbol_in_prose( prose_lines: list[str], scaffolding: list[bool] ) -> list[dict]: for index, line in enumerate(prose_lines): if scaffolding[index]: continue for pattern in SYMBOL_PATTERNS: match = pattern.search(line) if match: return [ { "rule": "symbol-in-prose", "line": index + 1, "excerpt": _excerpt(line), } ] return [] def _leads_with_file(bullet_text: str) -> bool: stripped = bullet_text.strip().lstrip("`*_\"'") if not stripped: return False first_token = stripped.split()[0].rstrip(":,.") if "/" in first_token: return _looks_like_path(first_token) return bool(FILENAME_RE.match(first_token)) def _check_bullet_per_file( lines: list[str], scaffolding: list[bool] ) -> list[dict]: bullets = [] for index, line in enumerate(lines): if scaffolding[index]: continue match = BULLET_RE.match(line) if match: bullets.append((index, match.group(1))) if len(bullets) < THRESHOLDS["bullet_per_file_min_bullets"]: return [] file_led = [item for item in bullets if _leads_with_file(item[1])] ratio = len(file_led) / len(bullets) if ratio >= THRESHOLDS["bullet_per_file_ratio"]: first_index, first_text = file_led[0] return [ { "rule": "bullet-per-file", "line": first_index + 1, "excerpt": _excerpt(first_text), } ] return [] def _check_ai_disclosure(body: str) -> list[dict]: for line in body.splitlines(): if AI_TERM_RE.search(line) and ASSIST_VERB_RE.search(line): return [] return [{"rule": "ai-disclosure-missing", "line": 1, "excerpt": ""}] def _finding(raw: dict) -> dict: rule = raw["rule"] fix = FIXES[rule] if "{heading!r}" in fix: fix = fix.format(heading=raw.get("heading", raw["excerpt"])) return { "rule": rule, "severity": SEVERITY[rule], "line": raw["line"], "excerpt": raw["excerpt"], "fix": fix, } def lint(body: str, *, repo: Optional[Path] = None) -> dict: chars = len(body) if chars > THRESHOLDS["max_input_chars"]: finding = _finding( { "rule": "oversized-input", "line": 1, "excerpt": _excerpt(body[: THRESHOLDS["excerpt_max_chars"]]), } ) return {"verdict": "fail", "chars": chars, "findings": [finding]} if not body.strip(): findings = [ _finding({"rule": "empty-body", "line": 1, "excerpt": ""}), _finding({"rule": "ai-disclosure-missing", "line": 1, "excerpt": ""}), ] return {"verdict": "fail", "chars": chars, "findings": findings} template_lines = _load_template_lines(repo) lines = body.splitlines() scaffolding = _scaffolding_mask(lines, template_lines) raw_findings: list[dict] = [] raw_findings.extend(_check_empty_template_sections(lines)) raw_findings.extend(_check_vacuous_opener(lines, scaffolding)) raw_findings.extend(_check_ai_disclosure(body)) prose_text = _prose_only(body) prose_lines = prose_text.splitlines() prose_chars = len(prose_text) if prose_chars >= THRESHOLDS["density_floor_chars"]: raw_findings.extend(_check_em_dash(prose_lines, scaffolding)) raw_findings.extend(_check_emoji(prose_lines, scaffolding)) raw_findings.extend(_check_path_in_prose(prose_lines, scaffolding)) raw_findings.extend( _check_pattern_rule( "method-narration", METHOD_NARRATION_PATTERNS, prose_lines, scaffolding ) ) raw_findings.extend( _check_pattern_rule( "verdict-clause", VERDICT_CLAUSE_PATTERNS, prose_lines, scaffolding ) ) raw_findings.extend( _check_pattern_rule( "reviewer-instruction", REVIEWER_INSTRUCTION_PATTERNS, prose_lines, scaffolding, ) ) raw_findings.extend(_check_bullet_per_file(lines, scaffolding)) raw_findings.extend(_check_symbol_in_prose(prose_lines, scaffolding)) findings = [_finding(item) for item in raw_findings] verdict = "fail" if any(f["severity"] == "block" for f in findings) else "pass" return {"verdict": verdict, "chars": chars, "findings": findings} def _extract_body(arguments: argparse.Namespace) -> str: if arguments.body_file: path = Path(arguments.body_file) try: return path.read_text(encoding="utf-8", errors="replace") except OSError as error: raise LintError(f"cannot read --body-file: {error}") from error return sys.stdin.read() def _render_human(result: dict) -> str: lines = [f"verdict: {result['verdict']} ({result['chars']} chars)"] for finding in result["findings"]: lines.append( f"[{finding['severity']}] {finding['rule']} (line {finding['line']}): " f"{finding['excerpt']!r}" ) lines.append(f" fix: {finding['fix']}") return "\n".join(lines) def parse_arguments(argv: Optional[list[str]] = None) -> argparse.Namespace: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--body-file", help="path to the PR body; defaults to stdin") parser.add_argument("--repo", help="repo root, used to find the PR template") parser.add_argument("--json", action="store_true", help="emit machine-readable JSON") return parser.parse_args(argv) def main(argv: Optional[list[str]] = None) -> int: try: arguments = parse_arguments(argv) body = _extract_body(arguments) repo = Path(arguments.repo).expanduser() if arguments.repo else None result = lint(body, repo=repo) except LintError as error: print(json.dumps({"verdict": "error", "error": str(error)})) return 2 except Exception as error: # noqa: BLE001 - must never crash the caller print(json.dumps({"verdict": "error", "error": f"internal error: {error}"})) return 2 if arguments.json: print(json.dumps(result)) else: print(_render_human(result)) return 0 if result["verdict"] == "pass" else 1 if __name__ == "__main__": raise SystemExit(main()) -
pr_body_receipt.py 5.1 KB
#!/usr/bin/env python3 """Manage receipts proving a PR body was scored and passed. A receipt is keyed by the sha256 of the body file's exact bytes, so editing the file after scoring invalidates it; that is the property pr-body-gate relies on. `write` is called by the skill only after both the linter and the judge pass. `check` is the same lookup as a standalone exit code. `prune` clears receipts older than a week so the state directory does not grow without bound. """ from __future__ import annotations import argparse import hashlib import json import os import sys from datetime import datetime, timedelta, timezone from pathlib import Path from typing import Optional STATE_ROOT = Path.home() / ".local" / "state" / "pr-body" RECEIPTS_DIR = STATE_ROOT / "receipts" BY_PATH_DIR = RECEIPTS_DIR / "by-path" PRUNE_AGE = timedelta(days=7) def _sha256_bytes(data: bytes) -> str: return hashlib.sha256(data).hexdigest() def _receipt_path(digest: str) -> Path: return RECEIPTS_DIR / f"{digest}.json" def _path_pointer_path(abs_path: str) -> Path: # Keyed by a hash of the path, not the path itself, so the receipt tree # never has to worry about characters a filesystem rejects in a name. return BY_PATH_DIR / f"{_sha256_bytes(abs_path.encode('utf-8'))}.json" def _now_iso() -> str: return datetime.now(timezone.utc).isoformat() def _ensure_dir(path: Path) -> None: path.mkdir(parents=True, exist_ok=True) os.chmod(path, 0o700) def _read_json(path: Path) -> Optional[dict]: try: return json.loads(path.read_text(encoding="utf-8")) except (OSError, ValueError): return None def _read_body_bytes(body_file: str) -> Optional[bytes]: try: return Path(body_file).read_bytes() except OSError as error: print(f"pr_body_receipt: cannot read {body_file}: {error}", file=sys.stderr) return None def cmd_write(body_file: str) -> int: data = _read_body_bytes(body_file) if data is None: return 1 digest = _sha256_bytes(data) _ensure_dir(RECEIPTS_DIR) _ensure_dir(BY_PATH_DIR) timestamp = _now_iso() receipt = {"sha256": digest, "timestamp": timestamp, "verdict": "pass"} _receipt_path(digest).write_text(json.dumps(receipt), encoding="utf-8") # The by-path pointer lets a later `check` on the same path tell "never # scored" apart from "scored, then edited" instead of just failing both # the same way. abs_path = str(Path(body_file).resolve()) pointer = {"path": abs_path, "sha256": digest, "timestamp": timestamp} _path_pointer_path(abs_path).write_text(json.dumps(pointer), encoding="utf-8") return 0 def cmd_check(body_file: str) -> int: data = _read_body_bytes(body_file) if data is None: return 1 digest = _sha256_bytes(data) receipt = _read_json(_receipt_path(digest)) if receipt and receipt.get("sha256") == digest and receipt.get("verdict") == "pass": print("valid") return 0 abs_path = str(Path(body_file).resolve()) pointer = _read_json(_path_pointer_path(abs_path)) if pointer and pointer.get("sha256") and pointer.get("sha256") != digest: print("stale: body edited after it was scored", file=sys.stderr) else: print("no receipt for this body", file=sys.stderr) return 1 def cmd_prune() -> int: cutoff = datetime.now(timezone.utc) - PRUNE_AGE removed = 0 for directory in (RECEIPTS_DIR, BY_PATH_DIR): if not directory.is_dir(): continue for entry in directory.glob("*.json"): record = _read_json(entry) timestamp = record.get("timestamp") if record else None when = None if timestamp: try: when = datetime.fromisoformat(timestamp) except ValueError: when = None if when is None or when < cutoff: entry.unlink(missing_ok=True) removed += 1 print(f"pruned {removed} receipt(s)") return 0 def parse_arguments(argv: Optional[list[str]] = None) -> argparse.Namespace: parser = argparse.ArgumentParser(description=__doc__) subparsers = parser.add_subparsers(dest="action", required=True) write_parser = subparsers.add_parser("write", help="record a passing receipt") write_parser.add_argument("body_file") check_parser = subparsers.add_parser("check", help="exit 0 if a valid receipt exists") check_parser.add_argument("body_file") subparsers.add_parser("prune", help="delete receipts older than 7 days") return parser.parse_args(argv) def main(argv: Optional[list[str]] = None) -> int: try: arguments = parse_arguments(argv) if arguments.action == "write": return cmd_write(arguments.body_file) if arguments.action == "check": return cmd_check(arguments.body_file) if arguments.action == "prune": return cmd_prune() except Exception as error: # noqa: BLE001 - must never crash the caller print(f"pr_body_receipt: internal error: {error}", file=sys.stderr) return 2 return 2 if __name__ == "__main__": raise SystemExit(main())
-
-
SKILL.md 9.6 KB
--- name: pr-body description: Write a pull-request body and score it before the PR is opened, or audit an existing PR's body against the same rubric. Use when about to run gh pr create, when a PR body needs writing or rewriting, or when asked whether a PR description is any good. Invoked as /pr-body write <body-file> or /pr-body audit <pr>. --- # PR body The body is for whoever merges the change. It carries facts about the change in the system's own terms. The diff already carries the files. ## The gold standard This is the target. Copy its shape, not its subject. The whole thing is what you would say to the team in one sentence, plus the handful of facts a merger cannot get from the title. The decisions, the alternatives, and the operator log are in the diff and the spec, and the body says where they are rather than repeating them. ```markdown ## Motivation and Context ghes-config now sets the memory ceilings and rate-limit posture that production already runs: 17 keys per ring for prd and stg, read off the production primary, rate limiting off. Staging had sandbox-sized ceilings on production-sized hardware. * The 14 rate-limit keys are new to the reconciler's allowlist, on the `ghe-config` transport. dev.yaml is unchanged. * Staging's primary was brought to those values by hand. Three of the four keys written did not exist there before, so the comparison below shows staging carrying production's values, not the appliance honouring the new ceilings. * No reconciler instance exists in any ring, so nothing reads these files today. Decisions and the operator log are in the change folder under `openspec/changes/archive/`. Fixes PROJ-7351 ## How Has This Been Tested? * Production primary against staging primary after the apply: 17 lines per side, diff exit 0. Replication in sync on both replicas. * `parity_test.sh` 37 checks pass, `python3 -m unittest discover -s scripts` 36 OK. * `pulumi preview` against `origin/main`: this branch moves no resource. ## Checklist before requesting a review - [x] I have performed a self-review of my code. - [x] I have stepped through the README as though I was a new user to ensure clarity. - [ ] I have added new or changed keys, tokens, other secrets to the DevOps 1Pass. - [ ] I have labeled all "TO DO" items with associated Jira ticket number. - [x] I have removed commented code from PRs to `main`. - [x] I have followed conventional-commits standards when making this PR. > Written by an AI agent operating for <operator>. Verify before relying on it. ``` ## What that body does 1. **Sections come from the team, never from you.** Fill `.github/pull_request_template.md` when the repo ships one, then the organization default in the organization's `.github` repo. When neither exists, read the last two merged pull requests written by someone else and use their headings; a team convention lives in the pull requests even when it lives in no file. Only a repo with no template and no history gets a body with no headings. 2. **The opening states what the system now does**, present tense, then the state it replaced. One sentence each. It is the line you would say to the team, and a reader who stops there has the change. 3. **Three or four bullets, not twelve.** A bullet earns its place by carrying a fact the opening does not imply. Never a bullet per file, and two bullets that restate each other are one bullet. 4. **Point at the spec and the diff instead of summarizing them.** Why this value, what else was considered, which command ran in what order: one line saying where that lives. A reader who wants the decision opens the spec, and a body that retells it goes stale against it. 5. **Evidence is the raw number.** "17 lines per side, diff exit 0", "37 checks pass", "moves no resource". Not the method, not the command line, not a sentence about having been careful. 6. **The caveat states what the evidence does not show.** "The comparison shows staging carrying production's values, not the appliance honouring the new ceilings." That clause is the most valuable one in the body, and it is the first one a weaker author drops. Fold the deviation that caused it into the same bullet. 7. **Blast radius as mechanism.** "No reconciler instance exists in any ring, so nothing reads these files today." Never a rating of the risk. 8. **`Fixes <KEY>` on its own line**, and the ticket appears nowhere else. 9. **Tick only what is true.** An unticked box for a check that does not apply is correct, and it is more credible than a body where every box is ticked. 10. **Length follows the facts, and most facts are not the body's job.** The gold standard is about 1.5 KB. An earlier draft of the same change ran 2.4 KB by carrying decisions the spec already held, and prose ran 5.8 KB and said no more. ## Never * Motivation nobody told you. "Ahead of the next release cycle" is a guess wearing a fact's clothes. * A clause that rates the change: "low risk", "no impact expected", "improves maintainability", "lands cleanly". The reader can check a mechanism and cannot check a rating. * Anything addressed to the reviewer: "worth checking", "please look at the IAM change". Telling a reviewer where to look tells them where not to. * Method narration: "I ran preview against dev and prd", "verified with". * A retelling of the spec or the diff: the reasoning behind a value, the alternatives weighed, the order the commands ran in. Name where it lives. * An empty template section. Cut the heading only if the template does not ship it; otherwise fill it. * An em dash (use parens or two sentences), emoji, `-` bullets, capitalized host or account names, and a bare file path outside code formatting. * Three parentheses in a sentence. One in a body is fine. ## Verbs ### write 1. Write the body to a file. The gate only reads `--body-file`, so the file has to exist anyway. Pass an absolute path with no `~`: the gate reads the command text, and it cannot resolve a tilde or a path that does not exist yet. Create the file in one call and run `gh` in the next, or the gate sees a file that is not there. 2. Score it, with the target repo so the scorer can read the repo's template: ```sh python3 <pr-body-skill-directory>/scripts/pr_body_lint.py \ --body-file <path> --repo <repo-root> --json ``` 3. Fix every blocking finding. Each carries fix text written as an instruction; apply it rather than arguing with it. A warning is a judgment call you own. 4. Run the voice judge: [references/judge.md](references/judge.md). Give it the body and `git diff <base>...HEAD`. The judge exists for one thing the scorer cannot see: a fact the diff makes load-bearing that the body never mentions. Apply the rewrites you accept, then re-score, because a rewrite can reintroduce a rule finding. 5. Record the receipt, then open or edit the pull request with `--body-file` pointing at the same file: ```sh python3 <pr-body-skill-directory>/scripts/pr_body_receipt.py write <path> ``` Editing the file after step 5 changes its hash and voids the receipt. Run the write verb again. ### audit Score an existing body and report. Never modify it. 1. `gh pr view <pr> --json body -q .body > <path>`. 2. Score it with `--repo` pointed at a checkout of the target repo. 3. Run the judge over the same body, with `gh pr diff <pr>` as the diff. 4. Report the scorer findings by rule with their line numbers, then the judge's rewrites. No receipt: the audit did not author the body. ## The gate A separately installed PreToolUse hook on `Bash` matches `gh pr create` and `gh pr edit`, hashes the `--body-file` it was handed, and denies when no receipt matches that hash. It also denies the forms it cannot read: inline `--body`, heredocs, a tilde path, a file that does not exist yet. When that hook is installed, there is no bypass and it fails closed. The escape that exists is uninstalling the hook from `~/.claude/hooks/`, which a human can do and an agent must not. ## Rules the scorer implements `scripts/pr_body_lint.py` is the only rule engine. A rule named here and absent there is decorative. Grounds and fix text for each: [references/rubric.md](references/rubric.md). | Rule | Fires on | Severity | | --- | --- | --- | | `empty-body` | a body with no non-whitespace content | blocks | | `ai-disclosure-missing` | no line disclosing AI assistance | blocks | | `empty-template-section` | a heading with nothing under it | blocks | | `vacuous-opener` | a first line like "Fix bug", "Phase 1", "Cleanup" | blocks | | `oversized-input` | input past the scorer's robustness cap | blocks | | `em-dash` | an em dash in prose | blocks | | `emoji` | an emoji codepoint in prose | blocks | | `path-in-prose` | a file path outside code formatting | blocks | | `method-narration` | "verified with", "I ran", "tested by running" | blocks | | `verdict-clause` | a clause rating the change instead of describing it | blocks | | `reviewer-instruction` | "please review", "worth checking", "take a look at" | blocks | | `bullet-per-file` | most bullets in a list of 3 or more lead with a filename | blocks | | `symbol-in-prose` | an identifier or function name outside code formatting | warns | Lines matching the repo's template, and the AI disclosure blockquote, are scaffolding and exempt from the prose rules. Below 40 characters of prose the density rules do not run at all; the structural rules run at every length. A scorer pass means the countable defects are absent. On the labeled set it caught 4 of 12 rejected bodies, and all 4 were empty or near-empty. Everything else was voice, which is what the judge and the gold standard above are for.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.