{"slug":"validate-2","title":"validate","summary":"Freshly judge a finished change against its acceptance: PASS, FAIL, or NOT_PROVEN. Not for claim-vs-tree checks; that is reality-check. Triggers: \"validate\", \"is this proven\", \"check this change\", \"cross-model review\".","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-08T21:44:52.820015Z","repo":{"url":"https://github.com/boshu2/agentops","stars":445,"forks":41,"license":"Apache-2.0","updatedAt":"2026-09-24T01:09:16Z"},"bodyHtml":"<hr>\n<p>name: validate\ndescription: 'Freshly judge a finished change against its acceptance: PASS, FAIL, or NOT_PROVEN. Not for claim-vs-tree checks; that is reality-check. Triggers: \"validate\", \"is this proven\", \"check this change\", \"cross-model review\".'\npractices:</p>\n<ul>\n<li>design-by-contract</li>\n<li>llm-eval-harness</li>\n<li>content-addressed-storage\nhexagonal_role: driving-adapter\nconsumes:</li>\n<li>subject-manifest.v1\nproduces:</li>\n<li>subject-manifest.v1</li>\n<li>validation-result</li>\n<li>verdict.v2\ncontext_rel:</li>\n<li>kind: customer-of\nwith: plan</li>\n<li>kind: customer-of\nwith: implement\nskill_api_version: 1\nuser-invocable: true\nmetadata:\ngraph_root: true\ntier: judgment\ndependencies: []\ncapabilities: [compute_subject_identity, judge_acceptance, return_validation_result, persist_verdict]\neffects: [write_verdict_artifact]\ncanonical_status: canonical\ndisposition: keep\noutput_contract: 'PASS | FAIL | NOT_PROVEN with criteria, evidence, checked/not_checked, identity, and freshness; optional schemas/verdict.v2.schema.json persistence'</li>\n</ul>\n<hr>\n<h1>Validate</h1>\n<p>Independently judge one exact subject against the acceptance in its existing\nbead or caller source, return one semantic result, and stop. Validate is the\nsole semantic author of <code>verdict.v2</code> when persistence is requested.\n<code>ao provenance store-verdict</code> supplies structural verification and atomic storage. Before the verdict,\nread <code>boundaries.md</code> in the rpi skill's <code>references</code> directory for the state\nValidate leaves to the caller.</p>\n<h2>Prompt</h2>\n<pre><code>Validate bead ag-1234 in this fresh context. Intent: the bead text and digest.\nSubject: manifest.json from:\nao provenance manifest --root . --include cli/internal/gates\nAuthor context ctx-a1. Re-run `cd cli &amp;&amp; go test ./internal/gates/...`.\nReturn PASS, FAIL, or NOT_PROVEN with evidence; stop.\n</code></pre>\n<h2>Preconditions</h2>\n<ul>\n<li>The subject is a nonempty implementation candidate: the manifest lists at\nleast one entry. Plans, audits, and reviews are subjects only when the\ncaller explicitly requested document review.</li>\n<li>The intent source is a caller-owned artifact or a runtime-owned\ncontent-addressed snapshot; its acceptance digest is derived automatically.</li>\n<li>Author and validator context IDs are explicit, and freshness is attested\nwith <code>source: runtime | caller</code> and an attester identity. Missing,\ncolliding, or unattested identities produce <code>NOT_PROVEN</code>: a declared trust\nfact, not cryptographic proof of isolation.</li>\n</ul>\n<h2>Fresh validator and model selection</h2>\n<p>Default to one fresh, author-distinct validator from the author's model family:\nCodex/OpenAI work uses a fresh Codex/OpenAI reviewer; Claude/Anthropic work uses\na fresh Claude/Anthropic reviewer. Use the runtime's configured capable model\nunless the caller pins one. Fresh context is required even when model weights\nare identical; a new role instruction in the author's session is not fresh.\nRisk determines the depth of evidence inspection, not an automatic second family.</p>\n<p>Acceptance, tests/gates, stopping, allowances, safety, disclosure, hooks and\nexecutable controls warrant deeper checks, including policy written as prose.\nConservative risk cues include <code>cli/internal/gates/**</code>, <code>scripts/check-*.sh</code>,\n<code>tests/**</code>, <code>skills/*/scripts/**</code>, <code>skills/cc-hooks/policies/**</code>, <code>lib/**</code>,\n<code>.github/workflows/**</code> and <code>scripts/security-gate.sh</code>. Unknown risk receives\ndeeper inspection; it does not silently change the selected model families.</p>\n<p>The caller can request <code>--cross-model</code> or say \"cross-model review\" to add one\nfresh validator from a different family. <code>--cross-model &lt;model&gt;</code> pins that\nadditional reviewer, for example <code>--cross-model claude-fable-5-1</code> from Codex\nor <code>--cross-model gpt-6-astra</code> from Claude. These are skill prompt options,\nnot <code>ao</code> CLI flags. RPI forwards them unchanged. Without a pin, use an available,\nauthorized capable model from the other family; never silently substitute for\na pinned model or count two models in one family as cross-family diversity.\nAn explicit caller requirement remains required until the caller changes it;\nchanging the default cannot erase a finding or relabel a missing verdict as PASS.</p>\n<p>Route selection and invocation through\n<a href=\"../agent-native/references/model-dispatch.md\">agent-native model-dispatch</a>;\n<a href=\"references/mechanics.md\">references/mechanics.md</a> owns evidence storage.\nThere is no fixed ten-minute review timeout. Use the caller's selected review\ntimeout or remaining native deadline, respecting any earlier host or goal limit.\nA timeout is missing judgment, not FAIL; never restart to renew an allowance.</p>\n<p>Each selected judge receives the exact subject, unchanged acceptance and\nauthorized evidence independently, without the author's desired verdict or peer\nconclusions. Record actual model/context identities and runtime receipts.\nMissing freshness, an unbound subject, incomplete acceptance evidence, or\nnonempty <code>not_checked</code> prevents PASS regardless of model family.\nWith no authorized adapter for requested diversity, disclose\n<code>diversity_unsatisfied</code>: the required combined result is <code>NOT_PROVEN</code> even if\nthe same-family judge passed. A delivered FAIL stands. Advisory diversity that\nthe caller explicitly made optional may accompany the same-family result with\nthat limitation; it cannot discard an acceptance-relevant finding.</p>\n<p>When selected judges disagree, preserve both verdicts and their evidence.\nRequired diversity converges only when both pass; neither majority vote nor a\npreferred judge settles a split. Repair addresses findings under the existing\nconvergence law. Report unresolved dissent and the caller's decision openly.\nSame-family fresh judgment reduces anchoring, but does not prove independence\nfrom shared training biases; cross-family agreement is corroboration, not truth.</p>\n<h2>Mutating-check quarantine</h2>\n<p>Classify every acceptance-listed command as read-only or subject-mutating\nbefore running it: regen scripts, sync scripts, formatters, and anything with\n<code>--force</code> are mutating until proven otherwise. Run a mutating check only\nagainst a disposable copy or a committed subject, never the judged working\ntree (the boundaries appendix records the regen that overwrote a subject).</p>\n<h2>Scope disclosure</h2>\n<p><code>not_checked</code> has exactly one meaning: <strong>in-scope acceptance surface this\nvalidation did not verify</strong>. PASS asserts the whole declared acceptance\nsurface was verified, so a PASS carries no <code>not_checked</code> entries; every other\nscope limit has a home that survives inside a PASS: a bounded proof in\n<code>criteria[].reason</code>, a declared non-goal in the intent source, residual risk\nin the report (table in the mechanics reference). Emptying <code>not_checked</code> to\nobtain PASS is a contract violation: unverified acceptance makes the honest\nresult <code>NOT_PROVEN</code>, and an entry that was never acceptance moves to its home\nand stays visible. A finding necessary to acceptance cannot be relabeled as\noptional, residual risk, or a non-goal to obtain PASS.</p>\n<h2>Workflow</h2>\n<ol>\n<li>Derive <code>subject-manifest.v1</code> with the helper's <code>manifest</code> command (flags\nin the mechanics reference) at the start and again at the end; any\nmismatch is subject mutation and returns <code>NOT_PROVEN</code>.</li>\n<li>Confirm the intent-source digest is unchanged since implementation, every\ncited evidence digest matches the artifact it names, and complete\nchanged-path coverage can be derived; otherwise <code>NOT_PROVEN</code>.</li>\n<li>Adjudicate the actual diff: runtime-derived changed paths against the\nintent's scope classes. A proven out-of-scope path is <code>FAIL</code>; incomplete\nscope evidence is <code>NOT_PROVEN</code>.</li>\n<li>Inspect the exact subject and evidence. Reported exit codes are claims:\nre-execute the proofs that bear on acceptance. A changed test, gate,\nfixture, golden, tolerance, suppression, or acceptance source must be\nrequired by the original intent, with green coming from implemented\nbehavior; green obtained by weakening acceptance is <code>FAIL</code>. Judge every\nacceptance criterion against its own evidence reference; a criterion with\nno evidence of its own is unverified, not passed.</li>\n<li>Choose exactly one semantic result: <code>PASS</code>, <code>FAIL</code>, or <code>NOT_PROVEN</code>. Return\nit with criterion-level results, findings, evidence references, <code>checked</code>,\n<code>not_checked</code>, both identities, both context IDs, and the freshness\nattestation. Name a <code>class</code> for each finding: one short stable name for the\nkind of defect, one per finding, reused word for word when the same kind\nrecurs, so the orchestrator can see a closed kind come back. Explain in the\nexisting summary and evidence whether a newly exposed defect pre-existed the\nchange, was introduced by it, or has unknown cause. Use before/after proof or\nequivalent causal evidence under unchanged acceptance; counts and timestamps\ndo not establish cause. Recurrence calls for causal examination and does not\nby itself prove a design failure. Name the class or omit it; a <code>class</code> that is\npresent and blank is a finding against this validator,\nand so is a class that does not describe its finding. PASS\nrequires distinct identities, explicit freshness, nonempty checked scope,\nnonempty top-level evidence, evidence for every criterion, and an empty\n<code>not_checked</code>. A documentation sentence claiming something is published,\npinned, or proven is an acceptance criterion like any other: it needs a\ncheck this validator can run, or it is <code>not_checked</code>. The\n<code>docs.claims-tracked</code> gate covers the tracked-file half of that and nothing\nmore.</li>\n<li>Only when the caller requests machine-readable evidence or a declared\ndownstream consumer requires it, persist canonical <code>verdict.v2</code> with the\nhelper's <code>store-verdict</code> (mechanics reference), then return the artifact\npath and digest with the result. Stop.</li>\n</ol>\n<p>Fresh validation is independent judgment over the exact subject, not a replay\nof every author command: rerun the risk-critical, uncertain, or thinly\nevidenced checks; a digest-bound deterministic receipt may prove routine\nfacts; replay an expensive full suite only when acceptance requires it. The\nrepository's full literal CI command set, as quoted in <code>AGENTS.md</code>, runs\nonce, on the final integrated subject.</p>\n<h2>It's working if</h2>\n<p>Observable in the trace, without reading the prose, and the rubric a fresh\nindependent judge scores this skill against:</p>\n<ul>\n<li>A criterion whose evidence is a justification rather than a proof is named,\nand the result is <code>NOT_PROVEN</code> rather than <code>PASS</code>.</li>\n<li>Green obtained by widening a tolerance, skipping a case, or re-baselining a\nbudget is reported as <code>FAIL</code>, never as completion.</li>\n<li>Every scope limit is placed in one of the Scope-disclosure homes; none was\ndeleted to reach <code>PASS</code>.</li>\n<li>The subject manifest is derived twice, at the start and at the end, and the\ntwo are compared.</li>\n</ul>\n<h2>Boundary</h2>\n<p>Validate emits no next action, repair, retry, replan, or delivery state (full\nlist in the boundaries reference); ledger availability cannot change a\nverdict's validity.</p>\n","files":[{"path":"references/mechanics.md","sizeBytes":11632,"isText":true},{"path":"references/validate.feature","sizeBytes":1984,"isText":false},{"path":"scripts/validate.sh","sizeBytes":535,"isText":true},{"path":"SKILL.md","sizeBytes":9175,"isText":true},{"path":"tests/check_contract_corpus.py","sizeBytes":5555,"isText":true},{"path":"tests/test_evidence_cli.py","sizeBytes":14311,"isText":true},{"path":"tests/test_validate.py","sizeBytes":16224,"isText":true},{"path":"tests/validate.py","sizeBytes":29691,"isText":true},{"path":"tests/validate.sh","sizeBytes":1035,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-24T06:49:26.267619Z","sha256":"EB64301C845521A93F496C9094628964E5B9B1686AEB420A880161CE5AD20B0D","sizeBytes":28053},"review":null,"source":{"repositoryUrl":"https://github.com/boshu2/agentops","path":"skills/validate","license":"Apache-2.0","commit":"c3fe161dce0b85d1e0490df757bbb841d22e4ea1","subtreeSha":"99DB4E849A25DF6CDE44A8C5924015A0256EAA7DC642B5A5651ACD24D8B94BC3","lastSyncedAt":"2026-09-24T06:48:55.360254Z"},"reviewedAt":"2026-09-24T06:50:39.56681Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/boshu2/agentops/tree/main/skills/validate"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install boshu2-agentops@llmmart"},{"target":"git","command":"git clone https://github.com/boshu2/agentops.git"}]}