maestro-verify
Verify and close - cross-check coverage, run the VERIFY table, deliver the verdict, harvest durable lessons into decisions, close the bundle, and never claim remote state from local evidence.
Install
npx skills add https://github.com/ReinaMacCredy/maestro/tree/main/src/plugins/skills/maestro-verify
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install reinamaccredy-maestro@llmmart
git clone https://github.com/ReinaMacCredy/maestro.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole reinamaccredy/maestro collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
maestro-verify
Use for verification and close. Read WORKFLOW.md for testing, recovery, authorization, and completion rules. Delivery actions such as commit, install, push, or release remain separate gates.
Precondition: an open bundle with a drafted VERIFY.md. Without a bundle,
verify inline and close a tracked item with maestro work done; an untracked
quickfix needs no record. This skill's table pass is for Full work. The
evidence-layer vocabulary below still applies to any claim at any tier.
Evidence layers
Proof follows five links. Claim only as far as the last proven link.
source- source-level tests, lint, type checks, or direct inspection.artifact- the built or packaged output is present and has been read back.installed- the installed stamp, version, or files match the intended artifact.live- the running process, pid, or active runtime matches the installed layer.journey- the real user path reaches the observable outcome end to end.
"Tests pass" is a source claim. A claim that touches install or runtime must
include a readback at that layer. A suite that is green only on this machine
is not a source claim about the repo: before any commit, release, or handback
gate, re-run the touched suite with the developer environment removed
(HOME=$(mktemp -d), env -u HERDR_ENV). A test that reads the installed
copy, the room, or a home config passes for you and fails in CI. Every proof
and VERIFY result lists untested links explicitly as NOT TESTED, never by
omission:
proof: "suite 135 pass @ a52bd4a7 (source); runtime stamp readback a52bd4a7 (installed); live: NOT TESTED"
Assumptions not verified: None
Residual risks: None
Verify
- Cross-check the evidence plan against acceptance and relevant risks. Map existing tests, necessary new checks, readbacks, or baselines to VERIFY.md. A missing behavior check is a gap; the absence of a newly written test is not.
- Run every VERIFY.md scenario against its work item's acceptance/claims and
fill the Result column; run each anti-goal check (grep, diff, readback).
Stamp the pass with its date and commit. Results hold this run only: a
re-run replaces prior results wholesale, and a failed pass leaves its
one-line
failed:note on the work item, never accumulated rounds in VERIFY.md. Apply Recovery and verification when a scenario cannot run as written: document and execute an equivalent check without changing acceptance, or report the gap if equivalence is unknown. - Run the repo's checks for the touched surface (tests, lint, types, build), then freeze and review the task-owned diff: every changed line traces to the SPEC's scope or a linked work item; nothing unrelated is staged.
- For a concrete assertion-strength concern, inspect whether the existing check distinguishes the approved outcome from the suspected wrong behavior. A focused mutation can establish that; restore it before continuing. Do not expand verification into an unrelated edge-case or coverage campaign.
- Re-read the user's exact delivery authority and target before any gate.
- Select one legal next gate at a time: final verification, independent QA or witness, scoped commit, local install, external delivery, or stop. Do not bundle gates whose authority differs.
- Read back the actual result: test output, commit hash, installed version. A started or interrupted command is not delivery evidence.
For substantial diffs, verify in a fresh context: dispatch a subagent that reads only the bundle and the diff - the implementer verifying their own work invites confirmation bias. The subagent never fixes anything: mutants it flips are reverted before reporting, and on FAIL it records the verdict and stops; routing back to implementation belongs to the parent turn that holds the user's ask. A subagent that fails to start or report is a dispatch failure, not evidence: run the checklist in this session instead of polling for it.
On FAIL, leave maestro work note <id> "failed: <one line>" and return the
evidence to the implementation owner. Use the shared recovery rule to choose
the next action from the cause, not a failure count. Read prior failed notes
so a new session does not repeat the same uninformative attempt.
Read-only review method: references/audit.md. When the failure location is unclear, follow references/triage.md.
Red flags
| The thought | The reality |
|---|---|
| "It obviously passes - running it is a formality" | Scenarios exist because "obviously" has been wrong before. Run every one and record the output. |
| "The scenario command is stale, so I can skip the check" | Repair it or demonstrate an equivalent measurement; preserve acceptance and record the change. |
| "The mutant survived, but the code is clearly fine" | If the mutant violates acceptance, the check is weak; report the gap rather than filling PASS. |
| "I wrote this diff - I know it works" | That is the confirmation bias the fresh-context rule exists for. |
| "I'll just fix this small failure while I'm verifying" | Verify delivers a verdict, never fixes. A FAIL routes back to maestro-work. |
Learn, then close
Before closing, harvest what outlives the bundle (references/learning.md): a verified correction or durable constraint becomes a locked decision or a work note - never only chat.
Use Completion and delivery to decide whether the accepted scope is complete or an authorized transfer is ready. Close procedure:
- Run
maestro handoff <bundle-id>one last time, then add a dated close-out line citing verification evidence and the candidate (base commit plus task-owned diff if uncommitted), or the explicit handoff/cancellation. Record pending delivery actions and retained authority in the handoff. - Harvest: any mid-flight choice that is hard to reverse, surprising without
context, and a real trade-off is a locked decision with its rejected
alternative; a new domain term is
maestro term add. maestro bundle close <id>: snapshots the trio into the store and archives the directory.
The snapshot is the durable memory; after close the directory is disposable
and maestro search still recalls the text.
If acceptance includes delivery not yet authorized or proven, leave the work open with that exact next action and blocker. Otherwise a verified implementation may close without a commit. Do not mark failed acceptance complete; an explicit transfer or cancellation records the unresolved failure rather than calling it PASS. Never stage or commit bundle contents.
Use the shared review routing. A code change after the verdict reruns the affected VERIFY.md scenarios before close; the old verdict does not cover the new diff.
Definition of done
Acceptance met, changed surface verified, available test/lint/type/build checks pass, claims name their falsifier, risky changes carry rollback notes. Never claim push, release, or publish from local state; those gates are the user's.
Files (maestro)
-
references
-
audit.md 1.2 KB
# Audit Use this recipe for a review, risk scan, or verification request. Audit is read-only unless implementation is separately authorized. When the symptom does not yet name an owning layer, run the ordered [triage](triage.md) recipe before selecting probes. ## Loop anatomy ### Perceive Read the requested scope, acceptance, current diff, relevant tests, and source. State the exact boundary and the failure classes worth probing. ### Choose Pick the highest-risk bounded probe that could expose a real defect. Prefer the real consumer path over internal state or mock interactions. ### Act Run the probe and inspect the smallest relevant source slice. Do not silently fix findings during a read-only review. ### Observe For each candidate, prove reproducibility and impact. Discard speculation. Rank surviving findings by severity and name the missing regression check. ### Learn Create or note follow-up work only when the finding is actionable and within the repository's authority model. Tie every durable lesson to evidence. ### Continue Return findings first with concrete locations and reproduction paths, or state that no issues were found. Separate verified findings from residual risk. -
learning.md 1.8 KB
# Learning Use this recipe when a verified correction, repeated failure, or durable decision should guide future work. ## Better loop Move through this chain without skipping a link: 1. Capture the episode and its literal evidence. 2. Identify the common mechanism across episodes. 3. Locate the smallest owning layer. 4. Make one rollback-able correction there. 5. Run a positive canary and a negative canary. 6. Promote only after both canaries and one real use pass. 7. Put a review/delete date on every promoted rule. Useful metrics expose outcomes and correction quality: recurrence after the change, rollback rate, time to locate the owning layer, and canary-to-real-use agreement. Gameable metrics reward activity instead: note count, rule count, raw test count, agent turns, and claims without layer-qualified proof. Use a metric only when gaming it would still improve the intended outcome. ## Loop anatomy ### Perceive Read the source event, evidence, decision, work note, or repeated failed approach. A plausible idea without a durable source is not a lesson yet. ### Choose Choose the narrowest durable home: a work note for local continuity, a locked decision for an accepted fork, repository memory for stable project knowledge, or a proposed workflow change for a repeated procedure. ### Act Write a concise lesson that names its scope and source. Do not paste logs or store secrets. Do not change global workflow rules without accepted scope. ### Observe Read the artifact back, search for duplication, and ensure later agents can tell when the lesson applies. ### Learn Treat the saved artifact, not this conversation, as the durable result. ### Continue Return to the active work or design item. If no reusable lesson exists, say so and leave durable memory unchanged. -
triage.md 1.6 KB
# Triage Diagnose in this order. Stop at the first step whose answer is unknown, gather the smallest evidence that settles it, then continue. 1. **Problem** - What observable outcome differs from the intended one? 2. **Authority** - Who may decide, write, reclaim, install, or publish here? 3. **Topology** - Which sessions, processes, stores, worktrees, and consumers are involved? 4. **Attention** - Who can currently see the evidence and who is silent? 5. **Capability** - Does the responsible actor have the required tool and permission? 6. **State** - What branch, commit, lease, decision, cursor, pid, and dirty path is current? 7. **Evidence** - Which source, artifact, installed, live, or journey link is actually proven? 8. **Owning layer** - What is the smallest layer that can correct the common mechanism? 9. **Learning** - What should be retained, promoted, reviewed later, or deleted? | Symptom | First question | |---|---| | Wrong result | Problem: what exact observable differs? | | Work cannot proceed | Authority: who owns the decision or mutation? | | Two agents conflict | Topology: which leases and paths overlap? | | A holder is silent | Attention: who has seen its latest activity? | | A command cannot run | Capability: is the tool or permission present? | | Behavior changed between sessions | State: which commit, runtime, and store is each using? | | Tests pass but users still fail | Evidence: which downstream layer is NOT TESTED? | | The same defect returns | Owning layer: where does the shared mechanism live? | | A workaround keeps growing | Learning: what correction earned promotion, and until when? |
-
-
SKILL.md 7.4 KB
--- name: maestro-verify description: Verify and close - cross-check coverage, run the VERIFY table, deliver the verdict, harvest durable lessons into decisions, close the bundle, and never claim remote state from local evidence. review-date: 2026-11-28 --- <!-- maestro-skill-version: dev --> # maestro-verify Use for verification and close. Read [WORKFLOW.md](~/maestro/WORKFLOW.md) for testing, recovery, authorization, and completion rules. Delivery actions such as commit, install, push, or release remain separate gates. Precondition: an open bundle with a drafted VERIFY.md. Without a bundle, verify inline and close a tracked item with `maestro work done`; an untracked quickfix needs no record. This skill's table pass is for Full work. The evidence-layer vocabulary below still applies to any claim at any tier. ## Evidence layers Proof follows five links. Claim only as far as the last proven link. - `source` - source-level tests, lint, type checks, or direct inspection. - `artifact` - the built or packaged output is present and has been read back. - `installed` - the installed stamp, version, or files match the intended artifact. - `live` - the running process, pid, or active runtime matches the installed layer. - `journey` - the real user path reaches the observable outcome end to end. "Tests pass" is a source claim. A claim that touches install or runtime must include a readback at that layer. A suite that is green only on this machine is not a `source` claim about the repo: before any commit, release, or handback gate, re-run the touched suite with the developer environment removed (`HOME=$(mktemp -d)`, `env -u HERDR_ENV`). A test that reads the installed copy, the room, or a home config passes for you and fails in CI. Every proof and VERIFY result lists untested links explicitly as `NOT TESTED`, never by omission: ```text proof: "suite 135 pass @ a52bd4a7 (source); runtime stamp readback a52bd4a7 (installed); live: NOT TESTED" Assumptions not verified: None Residual risks: None ``` ## Verify - Cross-check the evidence plan against acceptance and relevant risks. Map existing tests, necessary new checks, readbacks, or baselines to VERIFY.md. A missing behavior check is a gap; the absence of a newly written test is not. - Run every VERIFY.md scenario against its work item's acceptance/claims and fill the Result column; run each anti-goal check (grep, diff, readback). Stamp the pass with its date and commit. Results hold this run only: a re-run replaces prior results wholesale, and a failed pass leaves its one-line `failed:` note on the work item, never accumulated rounds in VERIFY.md. Apply [Recovery and verification](~/maestro/WORKFLOW.md#recovery-and-verification) when a scenario cannot run as written: document and execute an equivalent check without changing acceptance, or report the gap if equivalence is unknown. - Run the repo's checks for the touched surface (tests, lint, types, build), then freeze and review the task-owned diff: every changed line traces to the SPEC's scope or a linked work item; nothing unrelated is staged. - For a concrete assertion-strength concern, inspect whether the existing check distinguishes the approved outcome from the suspected wrong behavior. A focused mutation can establish that; restore it before continuing. Do not expand verification into an unrelated edge-case or coverage campaign. - Re-read the user's exact delivery authority and target before any gate. - Select one legal next gate at a time: final verification, independent QA or witness, scoped commit, local install, external delivery, or stop. Do not bundle gates whose authority differs. - Read back the actual result: test output, commit hash, installed version. A started or interrupted command is not delivery evidence. For substantial diffs, verify in a fresh context: dispatch a subagent that reads only the bundle and the diff - the implementer verifying their own work invites confirmation bias. The subagent never fixes anything: mutants it flips are reverted before reporting, and on FAIL it records the verdict and stops; routing back to implementation belongs to the parent turn that holds the user's ask. A subagent that fails to start or report is a dispatch failure, not evidence: run the checklist in this session instead of polling for it. On FAIL, leave `maestro work note <id> "failed: <one line>"` and return the evidence to the implementation owner. Use the shared recovery rule to choose the next action from the cause, not a failure count. Read prior failed notes so a new session does not repeat the same uninformative attempt. Read-only review method: [references/audit.md](references/audit.md). When the failure location is unclear, follow [references/triage.md](references/triage.md). ## Red flags | The thought | The reality | |---|---| | "It obviously passes - running it is a formality" | Scenarios exist because "obviously" has been wrong before. Run every one and record the output. | | "The scenario command is stale, so I can skip the check" | Repair it or demonstrate an equivalent measurement; preserve acceptance and record the change. | | "The mutant survived, but the code is clearly fine" | If the mutant violates acceptance, the check is weak; report the gap rather than filling PASS. | | "I wrote this diff - I know it works" | That is the confirmation bias the fresh-context rule exists for. | | "I'll just fix this small failure while I'm verifying" | Verify delivers a verdict, never fixes. A FAIL routes back to `maestro-work`. | ## Learn, then close Before closing, harvest what outlives the bundle ([references/learning.md](references/learning.md)): a verified correction or durable constraint becomes a locked decision or a work note - never only chat. Use [Completion and delivery](~/maestro/WORKFLOW.md#completion-and-delivery) to decide whether the accepted scope is complete or an authorized transfer is ready. Close procedure: 1. Run `maestro handoff <bundle-id>` one last time, then add a dated close-out line citing verification evidence and the candidate (base commit plus task-owned diff if uncommitted), or the explicit handoff/cancellation. Record pending delivery actions and retained authority in the handoff. 2. Harvest: any mid-flight choice that is hard to reverse, surprising without context, and a real trade-off is a locked decision with its rejected alternative; a new domain term is `maestro term add`. 3. `maestro bundle close <id>`: snapshots the trio into the store and archives the directory. The snapshot is the durable memory; after close the directory is disposable and `maestro search` still recalls the text. If acceptance includes delivery not yet authorized or proven, leave the work open with that exact next action and blocker. Otherwise a verified implementation may close without a commit. Do not mark failed acceptance complete; an explicit transfer or cancellation records the unresolved failure rather than calling it PASS. Never stage or commit bundle contents. Use the shared [review routing](~/maestro/WORKFLOW.md#per-tool-adapters). A code change after the verdict reruns the affected VERIFY.md scenarios before close; the old verdict does not cover the new diff. ## Definition of done Acceptance met, changed surface verified, available test/lint/type/build checks pass, claims name their falsifier, risky changes carry rollback notes. Never claim push, release, or publish from local state; those gates are the user's.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.