Codex CLI Skill Verified

Review-before-apply skill for Codex CLI

Forces Codex CLI to show the diff, review its own changes, run focused validation, and list uncertainty before you keep the patch.

LLM Mart · 0 points · 25 views 222 listing impressions

#code-review #agents #coding

What vetted this — trust report


When to use it

On any non-trivial edit where Codex changed logic, touched multiple files, or made an assumption you didn't verify yourself.

The failure this prevents

Codex is strongest when it can edit and validate. The easy failure mode is stopping one step too early: the edit is made, the summary says "fixed the issue", and nothing ever ran. The summary is not lying — the model genuinely believes it, because from inside the loop a plausible edit and a verified edit look the same.

The fix is procedural, not motivational. Make proof a required output, and "done" stops being something the model can assert without evidence.

Skill instructions

After you make code changes, do not move straight to "done". Run this loop in order.

1. Show the patch

Run /diff and summarize the meaningful behaviour changes in plain language. Ignore pure formatting unless it hides a real change.

Include untracked files — /diff covers them, and a new file is the easiest thing to forget you created. State the file count and total lines changed; a number much larger than expected is itself a finding.

2. Ask for a review

Run /review and capture every finding that names a concrete failure mode, missing test, or regression risk. Discard findings that are style preferences with no stated consequence.

For each surviving finding, either fix it or state explicitly why it's a false positive. "Noted" is not a resolution.

3. Run focused validation

Execute the narrowest relevant test, build, or lint check for the changed slice — not the full suite first. A targeted command that runs in four seconds is one you'll actually run between edits; a twelve-minute suite is one you'll skip.

Report the exact command and its exit status. If no targeted command exists, say so plainly and explain what you ran instead. Then run the broader check once the narrow one is green.

Never summarize a failing command as a success. If it failed, show the failure output.

4. Resolve uncertainty explicitly

List anything validation did not cover under a heading Unverified assumptions. Typical entries:

  • behaviour under concurrency, which the tests run single-threaded
  • the error path, if only the happy path is covered
  • anything depending on production data shape or volume
  • migrations — did they run, in both directions?
  • performance, if you changed something in a loop
  • other callers of a function whose signature or contract you changed

An empty Unverified assumptions section on a substantive change is a red flag, not a clean bill of health.

5. Only then summarize

Changed
  <files, and the behaviour change in each>
Validated
  <exact commands run, and their results>
Unverified
  <assumptions the validation did not cover>
Risk
  <the single thing most likely to be wrong>

Hard rules

  • Never claim something is fixed unless a concrete check ran, or you explicitly state that none did.
  • Never hide a failing command behind a vague summary.
  • If /review surfaces a plausible bug, address it or explain why it's a false positive before finishing.
  • Never modify a test to make it pass. If a test fails, that's the finding.
  • If you changed a public signature, find the callers before declaring done.

Codex specifics

  • /status before you start: confirm the approval mode and writable roots. A patch that failed because the sandbox blocked a write looks a lot like a patch that failed for a code reason.
  • Approval mode matters here. In read-only mode this loop is advisory. In auto-approve it's the only thing between a bad edit and your working tree — run it on a branch.
  • codex exec can run this loop non-interactively in CI. Give it a definition of done it can actually check, or it'll produce a confident summary in a pipeline where nobody reads it.
  • Put the loop in AGENTS.md so it applies by default rather than depending on you remembering to invoke it.

Adapting it

  • Small edits: steps 1, 3, 5 only. The full loop on a one-line change is ceremony.
  • Multi-file refactors: add a step between 2 and 3 — "list every call site of every signature you changed, and confirm each was updated."
  • Bug fixes: require the red→green evidence explicitly. "Show the test failing before the fix and passing after."

Failure modes

  • /review returns only style comments. Re-prompt for behaviour: "only findings with a concrete failure mode."
  • Validation is claimed but no command appears. The most common and most important failure. Ask for the exact command and its output — every time, until it becomes the default.
  • Unverified assumptions is always empty. It's being treated as a formality. Ask directly: "name three things that could still be wrong."
  • The loop runs but you don't read it. Then it's an expensive way to feel safe. The output is short on purpose — read the Risk line at minimum.

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related