shift-left-testing
Orchestrates pre-sprint Shift-Left QA on a batch of backlog Stories. Use when the user wants to refine acceptance criteria, surface ambiguities + gaps, draft an ATP outline, and hand off to PO/Dev BEFORE the Story enters a sprint — so defects are prevented in the requirements, no
Install
npx skills add https://github.com/upex-galaxy/agentic-qa-boilerplate/tree/main/.agents/skills/shift-left-testing
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install upex-galaxy-agentic-qa-boilerplate@llmmart
git clone https://github.com/upex-galaxy/agentic-qa-boilerplate.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole upex-galaxy/agentic-qa-boilerplate collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Inputs
Read in order; stop earlier when the batch is small enough that later inputs add no signal.
.context/business/business-feature-map.md+.context/business/business-data-map.md+.context/business/business-api-map.md— domain vocabulary, entity model, CRUD matrix, auth model + endpoint contracts. Anchors refined ACs in real entities, flows, and API behavior. (All three are hard-required by the Readiness Preflight Gate + Phase 0.3.).context/master-test-plan.md— regression Epic + in-scope modules. Tells the refinement whether the Story falls inside an already-prioritized area.- The Story's Acceptance Criteria +
**Source spec:**reference on Jira. Detailed read viabun run jira:sync-issues get <STORY_KEY> --include-comments, then read the syncedacceptance-criteria.md(+ description). NEVERacli viewfor custom fields. Canonical input — every refined AC must trace back here. .context/PBI/epics/EPIC-<KEY>-<slug>/stories/STORY-<KEY>-<slug>/if a PBI folder already exists for this Story (created by a prior/sprint-testingcycle). Carries earlier session notes worth honoring..agents/jira-workflows.json— Story workflow + valid transitions (backlog -> shift_left_qa -> estimation). Source of{{jira.transition.story.*}}slugs used in Phase 3. ALSO the resolver for the[QA] Shift-Left Reviewtracking subtask: read it to confirm a subtask work type exists (+ its transitions) before Phase 1 creates any subtask; if the catalog has no subtask work type, the subtask steps are skipped with a warning — never blocked on..agents/jira-required.yaml— canonical slug catalog. Source of{{jira.acceptance_test_plan}}and other Jira field slugs touched in handoff.
Forbidden invocations
NEVER invoke /sdd-* skills from this workflow. SDD is an optional
user-installed ceremony; this skill ships self-contained and does not chain
SDD under any condition. If you need to refactor KATA, fixtures, cli/,
scripts/, or api/schemas/ pipeline, exit this skill first and invoke
/framework-development.
This boundary is mechanical, not advisory: scripts/lint-skills.ts rejects
any /sdd- mention outside this section. See:
.agents/skills/agentic-qa-core/references/skill-composition-strategy.md §4
(governs users who manually install SDD).
Shift-Left Testing — Pre-Sprint AC Refinement on a Backlog Batch
Drive Stage 0 — the pre-sprint Shift-Left loop — on a set of backlog Stories. Three phases, always in this order: Phase 1 Selection -> Phase 2 Refinement -> Phase 3 Handoff. Hand off afterwards to /sprint-testing once each Story reaches Ready For QA.
The skill is batch-by-design: one session refines N Stories from the backlog so PO + Dev lead can run a single grooming pass with the team. There is no single-issue mode — for a one-off urgent refinement, pass a list of length 1; the cadence stays the same.
Why this skill exists (separation from /sprint-testing)
| Eje | /sprint-testing |
/shift-left-testing |
|---|---|---|
| Cadence | In-sprint, ticket-by-ticket loop | Pre-sprint, batch grooming of N Stories |
| Entry status | {{jira.status.story.ready_for_qa}} |
{{jira.status.story.backlog}} / shift_left_qa / estimation / ready_for_dev |
| Exit status | {{jira.status.story.qa_approved}} (full execution) |
{{jira.status.story.estimation}} (refined, awaiting estimate by PO + Dev) |
| Audience | Dev + tester | PO / BA + tester (Dev lead optional) |
| Output | ATP + ATR + bugs + execution evidence | Refined ACs + risk map + pre-sprint ATP in the Story field (outlines only) + [QA] Shift-Left Review subtask + batch report |
| Execution | Smoke + UI / API / DB exploration | NONE — feature does not exist yet |
| Code reads | Deep, targeted (reproduce / verify) | Light (feasibility only — does the codebase support this?) |
| TC creation | Yes (TCs created in Stage 1) | No — Stage 4 (test-documentation) creates TCs after the Story ships |
| Sprint-testing later | Runs full pipeline | Short-circuits Phases 1-3 (label shift-left-reviewed detected) and just validates |
The reuse story is deliberate: ~70% of the refinement logic already lives in sprint-testing/references/acceptance-test-planning.md Phases 1-3. This skill cites that reference instead of duplicating it — see Phase 2 below.
Dependencies
Requires agentic-qa-core. Loads on demand:
agentic-qa-core/references/test-design-doctrine.md— MANDATORY before refining ACs or estimating outline coverage. A refinement that does not surface risk-beyond-AC and 1:N coverage is incomplete.agentic-qa-core/references/defect-management-doctrine.md— MANDATORY for the QA-Assignee hook (Part 2). This skill is the EARLIEST QA pickup of a backlog Story: when a QA takes a Story into Shift-Left refinement, setqa_assigneeto the authenticated session user (self) — read-before-write, NEVER overwrite an existing owner except on explicit, justified handover. This skill still files NO Bug/Defect/Improvement (Phase 1 rejects non-Story types); only the QA-Assignee semantics of Part 2 apply here.agentic-qa-core/references/briefing-template.md,agentic-qa-core/references/dispatch-patterns.md,agentic-qa-core/references/orchestration-doctrine.md,agentic-qa-core/references/session-management.md,agentic-qa-core/references/preflight-gate.md— cited inline by the sections that use them.
Compact Rules
Test-design doctrine (binding — full canon: agentic-qa-core/references/test-design-doctrine.md):
- ACs are the FLOOR. Refinement's job is to push past the happy-path contract: surface the boundaries, exceptions, states, and anomalies the Story is silent on.
- 1:N is the default: a non-trivial AC implies multiple outlines (valid partition + each distinct invalid + boundaries + states). A 1-outline AC requires a written "trivially atomic" justification — never the default.
- Tag each refinement gap to a technique: ranges/limits → BVA; status/lifecycle fields → State-Transition; 2+ interacting conditions → Decision Table; 3+ combinable factors → Pairwise.
- A refined AC (Given/When/Then) is the business assertion; the outline (
Should <behavior> <condition>) is its exploration. Keep them distinct.
Shift-left operational rules:
- Stories ONLY (no bugs — nothing to refine upstream). Entry status Backlog / Shift-Left QA / Estimation / Ready For Dev.
- Output = refined ACs + gap/ambiguity questions + the pre-sprint ATP in the
{{jira.acceptance_test_plan}}field (outline NAMES + coverage estimate, no test code, no execution, NO Test Plan item —/sprint-testingStage 1 creates the item from the field) + the closed[QA] Shift-Left Reviewsubtask + the batch report. - Tracking subtask
[QA] Shift-Left Reviewper accepted Story: find-or-create in Phase 1 (assignee = self; Jira'screatelands it in{{jira.status.subtask.active}}), close in Phase 3 handoff via{{jira.transition.subtask.complete}}(->{{jira.status.subtask.close}}). The subtask workflow's status NAMES areACTIVE/Close, not "In Progress" / "Done". Exhaustive session annotations (long analysis, refinement traces) go on the SUBTASK, keeping the Story clean. Work type + transitions resolved from.agents/jira-workflows.json; no subtask work type in the catalog → skip with a warning, never block. - The heart of the skill (Phase 2) = edge cases not in story + ambiguities + gaps — feed them to PO/Dev as questions AND as derived outlines.
- On taking a Story into refinement (first QA pickup), set
qa_assigneeto self — read-before-write, never overwrite an existing owner (agentic-qa-core/references/defect-management-doctrine.mdPart 2). This skill files NO Bug/Defect/Improvement; only the QA-Assignee hook applies. - On completion: add label
shift-left-reviewed; transition Backlog → Shift-Left QA → Estimation.
Read full SKILL.md when: running the batch grooming pipeline, writing the per-Story shift-left-refinement.md, or handling the PO/Dev handoff.
Subagent Dispatch Strategy
Orchestration & Session contracts: this skill follows
agentic-qa-core/references/orchestration-doctrine.md(mandatory subagent dispatch — main thread is command center) ANDagentic-qa-core/references/session-management.md(Phase 0 resume check, plan-first persistence at.session/<skill-slug>/<scope>/, archive on completion). Phase 0 (resume check) and Phase 1 (plan write) are NOT optional. The orchestrator also applies the per-stage Definition-of-Done gates inagentic-qa-core/references/stage-gates.md: verify a stage's DoD (planning stages include the Test-Design Checklist) BEFORE recording its progress checkpoint and advancing.
This skill is per-batch scope: <scope> = <YYYY-MM-DD>-<descriptor> (e.g. 2026-05-20-payments-area). Session state lives at .session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/{plan.md, progress.md} per agentic-qa-core/references/session-management.md §3 + §9. The per-Story shift-left-refinement.md files stay under each Story's PBI folder ([LOCAL] staging buffer for Phase 2 → Phase 3 — disposable once Phase 3 publishes to Jira; see references/atp-outline-template.md and .context/PBI/README.md).
This skill is compliant with the doctrine in AGENTS.md §"Orchestration Mode (Subagent Strategy)" and the session contract in .agents/skills/agentic-qa-core/references/session-management.md. Every dispatch follows the 7-component briefing format defined in .agents/skills/agentic-qa-core/references/briefing-template.md, and the pattern selected per phase matches the decision guide in .agents/skills/agentic-qa-core/references/dispatch-patterns.md.
| Phase | Pattern | Subagent role |
|---|---|---|
| Phase 1 — Selection | Single | Backlog Selection subagent: pull candidate Stories via [ISSUE_TRACKER_TOOL], apply veto + risk-score triage, return ranked candidate table. After user OK: find-or-create the [QA] Shift-Left Review subtask under each accepted Story with assignee = self, left at {{jira.status.subtask.active}} where create lands it (skip with warning if the catalog has no subtask work type) |
| Phase 2 — Refinement (per Story) | Sequential — looped per Story | Refinement subagent: load acceptance-test-planning.md Phases 1-3 + outline-only Phase 4, write shift-left-refinement.md, append PO/Dev questions, return summary block. ONE subagent per Story. NEVER parallel across Stories (each subagent writes a different PBI file but the orchestrator must present each summary to the user sequentially before the next dispatch) |
| Phase 3 — Handoff (per Story) | Sequential — looped per Story | Handoff subagent: update Jira description + {{jira.acceptance_test_plan}} custom field (both modalities — no Test Plan item pre-sprint) + handoff comment + labels + subtask annotations + subtask transition to Done + Story transition backlog -> shift_left_qa -> estimation. Returns transition log + trace verification |
| Phase 3 — Batch report | Single | Batch Report subagent: aggregate per-Story summaries into .session/shift-left-testing/<batch-id>/batch-report.md + post to parent epic if Stories share one |
Sequential by design. Phase 2 refinement looks parallelizable (each Story is independent in Jira), but the orchestrator must present each Story's refinement summary to the user before moving on. This keeps the user in the loop, lets them veto a Story mid-batch, and matches the team-grooming cadence the skill is designed for. Parallelism would burn the user's attention budget.
On any subagent failure: STOP, report the partial state (which Stories refined, which Jira mutations landed), present retry / skip-story / abort options. Do NOT auto-fix nor auto-rollback. Jira mutations are recorded in the batch report so partial sessions are resumable. See
.agents/skills/agentic-qa-core/references/orchestration-doctrine.md.
Fleet seam (optional)
Refinement is sequential by design (above) and stays that way by default. A fleet — one persistent worker session per Story, a conductor collecting their reports — is available only when the user explicitly asks for it on a large batch. Then:
- Topology: same checkout. This skill writes nothing but Jira: no code, no branch, no git index to contend on. A worktree per Story would be pure cost.
- The conductor writes
launch.txtin.session/shift-left-testing/<batch-id>/— one self-contained line per Story — always, whether or not any orchestration transport exists on the machine. Launching, supervising and closing those sessions isorca-orchestration/SKILL.md([ORCHESTRATION_TOOL]): supervised launch is the native path, andlaunch.txtis the payload for the human-paste fallback when nothing can launch it. - The per-Story user OK does not disappear, it moves: the conductor presents each worker's refinement summary as it lands and waits for the same approve / veto decision before that Story's Phase 3 handoff runs. A fleet that skips this is not this skill running faster, it is this skill not running.
- One writer per Story. Two workers never refine the same Story, and every Jira write for a Story belongs to that Story's worker. The batch report and the epic comment stay with the conductor.
- Silence rule: the absence of an orchestration transport is never named to the user, never appears in the preflight gate, and never appears in the batch report.
Workflow — one pipeline, three phases
Phase 0 — Session resume check + Session Init (always first)
-> Check .session/shift-left-testing/<batch-id>/progress.md → offer resume / restart / abort
-> Resolve TMS modality (A: Xray / B: Jira-native — recorded for Stage 1; the
pre-sprint ATP write is field-first in BOTH modalities)
-> Load /acli (no /xray-cli — this skill creates no TMS items)
-> Verify project-wide context files
-> Resolve candidate list (explicit IDs OR backlog JQL)
-> Create session folder .session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/
(writes plan.md after candidate list confirmed; progress.md appended per phase)
Phase 1 — Selection
-> Detailed-read each candidate via `bun run jira:sync-issues get <STORY> --include-comments`
(batch: `jql "<backlog JQL>"`), then read the synced .md
-> Reject non-Story types (Bug / Spike / Sub-task / Tech-debt)
-> Apply veto + risk-score triage per candidate
-> Present ranked candidate table -> WAIT for user OK
-> Per accepted Story: find-or-create subtask "[QA] Shift-Left Review"
-> assignee = self; stays at {{jira.status.subtask.active}} (skip + warn if no
subtask work type in the catalog)
Phase 2 — Refinement (loop per accepted Story)
-> Dispatch Refinement subagent: produce shift-left-refinement.md
(Critical Analysis + Story Quality Analysis + Refined ACs + ATP outlines)
-> Reuses sprint-testing/references/acceptance-test-planning.md §Phases 1-3
with shift-left-mode delta (no test-data generation, no parametrization tables, outline names only)
-> Present per-Story summary -> WAIT for user OK before next Story
Phase 3 — Handoff
-> Per Story sequentially:
- Update Jira description with "QA Refinements (Shift-Left Analysis)"
- Populate the ATP field {{jira.acceptance_test_plan}} (both modalities;
fallback: "## Acceptance Test Plan (ATP)" comment when the field is absent.
NO Test Plan item — /sprint-testing Stage 1 creates it from the field)
- Labels: shift-left-reviewed + shift-left-{YYYY-MM-DD}
- Transition: backlog -> shift_left_qa (analyze) -> estimation (estimate)
- Subtask "[QA] Shift-Left Review": post session annotations -> complete -> close
-> Light stage verifier (artifact-lifecycle.md §5) closes the stage
- Verify trace
-> Batch report posted to .session/shift-left-testing/<batch-id>/batch-report.md
+ posted as comment on parent epic if Stories share one
-> Archive: orchestrator moves .session/shift-left-testing/<batch-id>/ to
.session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/
per agentic-qa-core/references/session-management.md §8
---> Cross-skill handoff (NOT this skill):
When each Story later reaches Ready For QA:
/sprint-testing reads label `shift-left-reviewed` and short-circuits
Phases 1-3 to validation-only (sprint-testing/references/acceptance-test-planning.md §Phase 0).
Stage 1 ALSO creates the Test Plan item (`ATP: {STORY-KEY}: {title}`)
FROM the {{jira.acceptance_test_plan}} field content — the item is
in-sprint work, never pre-sprint.
Readiness Preflight Gate (MANDATORY — runs before Phase 0)
Full doctrine:
agentic-qa-core/references/preflight-gate.md. Runs FIRST, before the resume check. Two laws: (1) args-as-answers — treat anything the user already stated (the Story IDs, the modality, "groom the backlog") as provided args; ask only real gaps. (2) probe, don't assume. Surface gaps + REDs as ONEAskUserQuestionchecklist; self-fix with approval + explanation; STOP on any blocking RED. This skill does NO live execution (no env/DB/API/browser), so its gate is light — it is mostly a tooling + context readiness check. Generic baseline (env resolution, test-user creds, secret/restart handling, the two laws, output contract) is inherited from the reference §3.1 — not repeated here. Below is only this skill's specific capability delta.
| Capability | Need | Why here |
|---|---|---|
Issue-tracker ([ISSUE_TRACKER_TOOL]) |
REQUIRED | All refinement output lands on Jira (description, ATP field, comment, labels, transitions). Load /acli; validate setup via bun run jira:check. |
| TMS modality resolved | REQUIRED | Recorded in plan.md and carried into the handoff so /sprint-testing Stage 1 knows which engine will materialize the Test Plan item later. The pre-sprint ATP write itself is modality-independent — field-first in both. 4-step probe; ask only if all auto-checks fail. |
/xray-cli + XRAY_* creds |
NOT NEEDED | Shift-Left creates no TMS items in either modality. The pre-sprint ATP lives in the {{jira.acceptance_test_plan}} field (fallback: comment); the Test Plan item is created by /sprint-testing Stage 1 from the field content. |
| Business context files | REQUIRED | .context/business/* + .context/master-test-plan.md — refinement without them produces low-value questions. Missing → hand off to /project-discovery. |
| Candidate Story list | REQUIRED | Explicit IDs (args) or a backlog JQL. Confirm size with the user before Phase 1. |
Env reachability, test-user creds, DBHub, OpenAPI/API_TOKEN, Playwright and resend are N/A here — shift-left never executes against a running system. After the gate clears (all REQUIRED GREEN), continue to Phase 0 below.
Phase 0 — Session resume check + Session Init
0.0 Session resume check (per agentic-qa-core/references/session-management.md §4). Compute <batch-id> = <YYYY-MM-DD>-<descriptor> from the invocation context. Check .session/shift-left-testing/<batch-id>/progress.md. If it exists, read plan.md + the tail of progress.md, surface the last completed phase + next planned phase + any blocking notes, and offer resume / restart / abort. On restart, archive the current directory to .session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>-aborted/ before proceeding. On abort, stop here.
0.1 Resolve TMS modality. Same 4-step probe as sprint-testing Session Start (test-documentation/SKILL.md §Phase 0). Persist the result in .session/shift-left-testing/<batch-id>/plan.md (under the ## Inputs H2 — the plan.md is the canonical record per session-management §6).
0.2 Load required tool skills:
- Always load
/acli(custom-field update, comment, transition, label, subtask create — all writes; plus the trivial key+summary+status candidate search). Story DETAIL reads (description, ACs, scope, comments, parent epic) go throughbun run jira:sync-issues get/jql— NOTacli view. /xray-cliis NOT loaded by this skill. Shift-Left creates no TMS items in either modality: the pre-sprint ATP lives in the{{jira.acceptance_test_plan}}custom field (fallback: the## Acceptance Test Plan (ATP)comment when the field is absent). The Test Plan ITEM is created by/sprint-testingStage 1 from the field content, once PO has estimated and the Story enters the sprint.- Both modalities:
/aclialone covers every write this skill performs.
This step is mandatory before any pseudocode block below executes. The skills carry the concrete syntax, flags, and JSON payloads this skill intentionally omits.
0.3 Verify project-wide context files exist:
.context/business/business-data-map.md.context/business/business-feature-map.md.context/business/business-api-map.md.context/master-test-plan.md
If any of these is missing, STOP and hand off to project-discovery (or the individual /business-*-map and /master-test-plan commands). Shift-left refinement without business context produces low-value PO/Dev questions and bloats the batch report.
0.4 Resolve the candidate Story list. Two modes:
- Explicit IDs — user passes
UPEX-100,101,102,103(or any natural-language list of Story keys). Use these verbatim; no JQL. - Backlog JQL — user says "groom the backlog" with no IDs. Build a JQL via
[ISSUE_TRACKER_TOOL]filtering on:project = {{PROJECT_KEY}}issueType = Storystatus in ({{jira.status.story.backlog}}, {{jira.status.story.shift_left_qa}}, {{jira.status.story.estimation}}, {{jira.status.story.ready_for_dev}})- Optionally
sprint in openSprints()if the user says "next sprint candidates" - Sort by Priority DESC, then Created DESC
- Confirm the resolved list size with the user before Phase 1 starts. A batch of 1-12 Stories is the practical sweet spot; >12 should be split into multiple sessions.
0.5 Create the session folder + write plan.md:
.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/
plan.md # session-management.md §6 schema — Goal, Inputs, Approach,
# Phase breakdown, Risks, Verification checklist, Cross-references.
# Inputs includes TMS modality + candidate list.
progress.md # append-only, one entry per phase (§7 schema)
candidates.md # Phase 1 output (domain artifact)
batch-report.md # Phase 3 final output (domain artifact)
# Per-Story refinement files live under each Story's own PBI folder,
# NOT inside the session folder:
# .context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/shift-left-refinement.md
The <descriptor> is kebab-case (e.g. morning, payments-area) and lets two sessions on the same day stay independent.
After this step, append the first entry to progress.md: ## Phase 0 — Session Init — <ISO-8601 UTC> with status: completed, next: Phase 1 — Selection. Subsequent phases follow the same shape per agentic-qa-core/references/session-management.md §7.
Phase 1 — Selection
Decides which Stories actually enter the refinement loop and at what depth.
- For each candidate ID, detailed-read via
bun run jira:sync-issues get <STORY_KEY> --include-comments(or batchbun run jira:sync-issues jql "<backlog JQL>") and read the synced.md(title, description, ACs, priority, type, labels, sprint, parent epic, comments). NEVERacli view— it returnsnullfor custom fields.acli searchis fine for the trivial key+summary+status candidate list only. - Type filter (hard): reject anything where
issueType != Story. Surface the rejected list to the user with a one-line reason. Do NOT silently drop them. - Label filter: any Story already carrying
shift-left-reviewedAND a dated labelshift-left-{YYYY-MM-DD}less than 30 days old is treated as already refined — surface it separately under "Already Shift-Left Reviewed (skip or refresh?)". The user decides per-Story whether to skip or re-refine. (Freshness comes from the dated label, never from the issue'supdatedtimestamp — any comment or rank change resetsupdated; seereferences/backlog-selection.md§Step 2.) - Triage per accepted candidate (veto + risk score). Read
references/backlog-selection.mdfor the full rubric. Outcomes:- VETO -> SKIP: pure CSS / docs / static copy / tech-debt with no behavior change -> drop from refinement set, log reason.
- REQUIRE FULL: money / data integrity / auth / external integration / state machine / calculation -> force refinement regardless of score.
- Score 0-3 LOW -> SKIP (PO/Dev can write ACs directly without QA refinement).
- Score 4-7 MEDIUM -> Full refinement (standard).
- Score 8+ HIGH -> Full refinement + extended ambiguity / edge-case scan.
- Present the ranked candidate table (see
references/backlog-selection.md§Output format) and WAIT for user OK before Phase 2. Same pattern as sprint-testing's Story Explanation gate. - Tracking subtask per accepted Story (after user OK): find-or-create a subtask titled
[QA] Shift-Left Reviewunder the Story, withassignee= the authenticated session user (agentic-qa-core/references/artifact-lifecycle.md§2 — an unassigned QA artifact is a blocker waiting to happen). Jira'screatetransition lands it in{{jira.status.subtask.active}}; leave it there, Phase 3 closes it. Resolve the subtask work type from.agents/jira-workflows.json; if the catalog has no subtask work type (or the project disallows subtasks), log a warning inprogress.md+ the batch report and SKIP — never block the batch. Find-or-create: match the Story's existing subtasks by exact title before creating; an existing one already at{{jira.status.subtask.close}}is re-opened with{{jira.transition.subtask.reactive}}(refresh run). This makes QA's pre-sprint work visible on the board, and the subtask later receives the exhaustive session annotations in Phase 3.
Persist the accepted list into plan.md §Inputs so a resumed session reads the same canonical decision. After user OK, append a progress entry: ## Phase 1 — Selection — <ts> with status: completed, artifacts_touched: [candidates.md, plan.md], next: Phase 2 — Refinement.
Phase 2 — Refinement (per Story)
For each accepted Story, dispatch ONE Refinement subagent. The subagent loads the existing in-skill reference and applies a shift-left-mode delta.
Reuse contract: the subagent reads .agents/skills/sprint-testing/references/acceptance-test-planning.md §Phases 1-3 + Phase 4 (outline names only). The delta for shift-left mode:
| acceptance-test-planning.md Phase | Shift-Left adaptation |
|---|---|
| Phase 0 — Triage | Already done in this skill's Phase 1. Skip. |
| Phase 1 — Critical Analysis | Run as-is. Light code exploration only (feasibility check, not reproduction). |
| Phase 2 — Story Quality Analysis | Run as-is. This is the heart of shift-left — ambiguities + gaps + edge cases not in story + testability validation. |
| Phase 3 — Refined ACs | Run as-is — Given/When/Then with specific data. Mark inferred scenarios with NEEDS PO/DEV CONFIRMATION. |
| Phase 4 — Test Design (outlines) | OUTLINE NAMES ONLY. No parametrization tables. No exhaustive per-outline test-data JSON. Coverage estimate (Positive / Negative / Boundary / Integration counts) IS included — it informs PO estimation. |
| Phase 5 — Edge case + Test-data summary | Edge-case names + criticality only. No data generation strategy, no Faker recipes — feature does not exist yet. |
| Phase 6 — Traceability + Ticket updates | Phase 3 of THIS skill owns this. Refinement subagent only WRITES the local file; Handoff subagent does Jira mutations. |
| Phase 7 — Final QA Feedback Report | Per-Story summary returned to orchestrator. Aggregated into the batch report in Phase 3. |
| Phase 8 — Commit | SKIPPED. Jira is canonical. No git branch, no commit. |
Staging file: .context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/shift-left-refinement.md (module = Epic, 1:1). Author it locally; it is NOT a Jira mirror, so the hand-write ban does not apply to it.
It is a buffer, not a deliverable. Phase 2 writes it, Phase 3 publishes its full body to the Jira acceptance_test_plan field. After that, Jira holds the canonical copy and the synced acceptance-test-plan.md is the readable one. The staging file lives under .context/PBI/**, which is gitignored, so it exists only on the machine that ran the batch.
Two consequences that are easy to get wrong:
- Nothing downstream may depend on the staging file being on disk.
/sprint-testingStage 1 short-circuits off the SYNCEDacceptance-test-plan.md, never offshift-left-refinement.md— otherwise the short-circuit silently degrades to a full re-run on any other machine. - There is ONE ATP per Story. This skill authors it early into the
{{jira.acceptance_test_plan}}field;/sprint-testingStage 1 creates the Test Plan ITEM from that field content and refines the same ATP into the executable superset. No(Shift-Left DRAFT)variant, no second Test Plan to reconcile, no pre-sprint Test Plan issue at all.
Folder bootstrap: if .context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/ does not exist yet (Story has not been through sprint-testing), the refinement subagent creates it. Jira-mirrored content (story.md, acceptance-criteria.md, parent epic, comments) comes from bun run jira:sync-issues get <STORY_KEY> --include-comments — NEVER hand-write those files. The only hand-authored files here are the NON-Jira working artifacts (shift-left-refinement.md, context.md with local session notes). This mirrors sprint-testing/references/session-entry-points.md §Step 7. The evidence/ subfolder is NOT created — there is nothing to capture yet.
Story Explanation step: replaced by the per-Story summary the orchestrator presents AFTER the subagent returns. The user OKs (or vetoes) each Story before the next refinement dispatch. This matches the "explain story -> WAIT for OK" rhythm in sprint-testing.
Progress checkpoint per Story: after each Refinement subagent returns AND the user OKs the summary, the orchestrator appends a phase entry to .session/shift-left-testing/<batch-id>/progress.md per agentic-qa-core/references/session-management.md §7: ## Phase 2.<n> — Refine <STORY_KEY> — <ts> with status: completed, artifacts_touched: [.context/PBI/.../shift-left-refinement.md], next: Phase 2.<n+1> | Phase 3. This lets a mid-batch resume skip already-refined Stories.
After Phase 2 finishes the full accepted list, the per-Story summaries feed Phase 3.
Phase 3 — Handoff
For each refined Story, dispatch a Handoff subagent. Sequential, one Story at a time, so the user can review the post-handoff Jira state before the next mutation.
Per-Story handoff sequence
Prerequisite: Phase 0.2 already loaded
/acli. Pseudocode below uses[ISSUE_TRACKER_TOOL]only — this skill creates no TMS items, so[TMS_TOOL]never fires pre-sprint.
1. Write the refined ACs to the Jira acceptance_criteria field, then append the
supporting analysis. Jira is source of truth — local Jira-mirrored .md files are
read-only caches generated by the sync, never hand-written.
[ISSUE_TRACKER_TOOL] Update Issue:
issue: {STORY_KEY}
fields:
{{jira.acceptance_criteria}}: <refined ACs from Phase 2>
# FALLBACK (field absent): post as a structured comment headed
# "## Acceptance Criteria" per .agents/jira-required.yaml fallback: key. Never block.
Then append the "QA Refinements (Shift-Left Analysis)" supporting section to the
Story description:
- Edge Cases Identified (from Phase 2)
- Clarified Business Rules (from Phase 2)
- Open Questions for PO / Dev (from Phase 2)
After writing, run `bun run jira:sync-issues get {STORY_KEY} --include-comments`
and read back the synced `acceptance-criteria.md` to confirm the field landed.
2. Populate the ATP — field-first, IDENTICAL in both modalities. ONE ATP per Story,
authored early: there is no separate DRAFT item and no `(Shift-Left DRAFT)` title.
Pre-sprint the ATP's home is the `{{jira.acceptance_test_plan}}` custom field;
the Test Plan ITEM is created by `/sprint-testing` Stage 1 FROM this field content
once the Story enters the sprint. Full rationale + mutation sequence:
`references/handoff-protocol.md` Step 2.
[ISSUE_TRACKER_TOOL] Update Issue:
issue: {STORY_KEY}
fields:
{{jira.acceptance_test_plan}}: <full shift-left-refinement.md body>
# FALLBACK (field absent on this instance): skip this write — step 3's comment
# carries the full body inline per .agents/jira-required.yaml fallback. Never block.
3. Handoff notification on the Story (the ATP lives in {{jira.acceptance_test_plan}} — do NOT mirror it; inline the full body as a `## Acceptance Test Plan (ATP)` comment ONLY if that field is absent — fallback per jira-required.yaml):
[ISSUE_TRACKER_TOOL] Add Comment:
issue: {STORY_KEY}
body: |
## Acceptance Test Plan (ATP) — ready for pre-sprint review
The ATP lives in the {{jira.acceptance_test_plan}} field.
# FALLBACK ONLY (field absent): replace the pointer line above with the full staged refinement body.
4. Labels:
[ISSUE_TRACKER_TOOL] Update Issue:
issue: {STORY_KEY}
labels: +shift-left-reviewed, +shift-left-{{YYYY-MM-DD}}
5. Set QA Assignee + Transition (Story must be currently in backlog / shift_left_qa / estimation):
# First QA pickup of the Story — set qa_assignee to the authenticated session
# user (self), the same moment the QA pulls it into Shift-Left refinement
# (backlog -> shift_left_qa, the earliest pickup). Read-before-write:
# set ONLY when empty; NEVER overwrite an existing QA owner except on explicit,
# justified handover. Per defect-management-doctrine.md Part 2. NOTE: acli
# `workitem edit` CANNOT set customfields → use REST PUT /rest/api/3/issue/{KEY}
# with { fields: { {{jira.qa_assignee}}: { accountId } } } (doctrine Part 6).
[ISSUE_TRACKER_TOOL] Set qa_assignee = <authenticated session user> # read-before-write; skip if already owned
# If currently in backlog:
[ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.analyze}} # backlog -> shift_left_qa
[ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.estimate}} # shift_left_qa -> estimation
# If already in shift_left_qa or estimation, advance only the missing leg.
# NEVER advance beyond estimation — PO/Dev lead estimates and moves to ready_for_dev.
6. Close the tracking subtask (created in Phase 1; skip with warning if it was skipped there):
# Exhaustive session annotations (long analysis, refinement traces — anything too
# verbose for the Story) go on the SUBTASK, keeping the Story clean.
[ISSUE_TRACKER_TOOL] Add Comment / Update description:
issue: {SUBTASK_KEY} # "[QA] Shift-Left Review" under {STORY_KEY}
body: <exhaustive session annotations>
[ISSUE_TRACKER_TOOL] Transition: {{jira.transition.subtask.complete}} # active -> close
# Unmapped slug -> artifact-lifecycle.md §4 fallback (ask, never skip silently)
7. Verify trace (both modalities — field-first, no Test Plan item to trace pre-sprint):
`bun run jira:sync-issues get {STORY_KEY} --include-comments`,
then read back the synced acceptance-test-plan field file + handoff comment;
confirm the field is populated and the comment points to it
(full body in the comment ONLY in fallback mode — field absent);
confirm the subtask (when created) is at {{jira.status.subtask.close}}.
8. Run the **light stage verifier** (`agentic-qa-core/references/artifact-lifecycle.md` §5)
— the stage-specific lines are in `references/handoff-protocol.md` §Step 6b.
The Handoff subagent returns a per-Story log: {story: KEY, atp_container: <field|fallback_comment>, subtask: <done|skipped>, labels_added: [...], transitions: [...], trace_status: ok|warning|fail}.
Batch report + Archive
After all Stories handed off, dispatch ONE Batch Report subagent (Single pattern) to aggregate:
.session/shift-left-testing/<batch-id>/batch-report.md
Contents (see references/handoff-protocol.md §Batch report template):
- Session metadata (date, mode, candidate count, accepted count, rejected count)
- Per-Story line: ID, title, risk level, # gaps, # critical questions, transition status
- Aggregated top PO/Dev open questions (deduped across Stories)
- Risk distribution chart (LOW / MEDIUM / HIGH counts)
- Blockers (Stories that surfaced data feasibility gaps — flagged for PO before sprint planning)
- Recommended sprint-planning order (by risk + dependency)
- Cross-skill pointer: "When each Story reaches Ready For QA, run
/sprint-testing— it will short-circuit Phases 1-3 thanks to theshift-left-reviewedlabel."
If all Stories in the batch share a single parent epic, ALSO post the batch report as a comment on that epic. Otherwise, deliver inline to the user as the session-closing message.
After the batch report lands, append the final progress entry ## Phase 3 — Handoff + Batch report — <ts> with status: completed, next: stop, then run Archive per agentic-qa-core/references/session-management.md §8: move .session/shift-left-testing/<batch-id>/ to .session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/ (two-file dir preserved) and call mem_session_summary with the session template + archive path.
Gotchas — inline rules to apply every invocation
- Credentials always from
.env. Never hardcode. Same as sprint-testing. - Stories only. Bugs / Spikes / Sub-tasks / Tech-debt are rejected in Phase 1. Bugs are reactive — no upstream ACs to refine. If the user really wants to run shift-left on a Tech-debt with behavior changes, ask them to convert it to a Story first.
- Veto beats risk score. Same rule as
acceptance-test-planning.md§Phase 0. Money / data integrity / auth / external integrations / state machines / calculations -> FORCE Full refinement regardless of score. - Already-reviewed Stories (label
shift-left-reviewed<30 days old) are NOT auto-skipped. Surface them and let the user pick: skip / refresh. - Outline names only. Phase 2 produces outline TITLES + brief preconditions per outline. NO parametrization tables, NO per-outline test-data JSON, NO Faker recipes. Those belong to in-sprint planning (
/sprint-testing) or Stage 4 (/test-documentation). - NEEDS PO/DEV CONFIRMATION. Any AC or edge case the refinement infers (not literally in the original story) is flagged with this marker. The flag appears verbatim in the Jira description + comment + custom field, so PO sees it during sprint planning.
- No execution. This skill does not run smoke, does not query DBs for data presence, does not run the app. Feasibility is established by READING code + APIs + DB schema only.
- Sequential Phase 2. One refinement subagent at a time, even if the user is impatient. Parallelism would prevent per-Story user OK and break the grooming cadence.
- Transition guardrail. Never advance beyond
{{jira.status.story.estimation}}. PO/Dev lead ownsestimate -> ready_for_dev. If a Story is already pastestimationwhen the session starts, log a warning and SKIP the transition step — refinement still lands on Jira, but the workflow stays untouched. - Label hygiene. Always add BOTH
shift-left-reviewedANDshift-left-{{YYYY-MM-DD}}. The dated label lets/sprint-testingdecide whether the refinement is still fresh (<30 days) and short-circuit, or whether to redo Phases 1-3. - Jira is canonical. No git commit, no test branch. Local
shift-left-refinement.mdis a working artifact — gitignored under.context/PBI/**. The populated{{jira.acceptance_test_plan}}field (or its## Acceptance Test Plan (ATP)fallback comment when the field is absent) is the contractfix-traceabilitychecks later. - Language: artifacts + Jira content always English. Mirror the user's language only in conversation (per AGENTS.md §1 Rule #14).
- Session-footer contract (mandatory at close). The final phase is not done until the two chat-facing blocks from
../agentic-qa-core/references/session-footer-contract.mdare printed: (1) consolidated screenshot list — repo-relative paths, verified on disk, bug annotations first — plus in-flow surfacing of every capture's path the instant it lands; (2) Session Footer listing skills/MCPs/CLIs actually used + testing levels touched, with explicit "none" entries for expected-but-untouched levels. Framing for this skill: execution. Multi-subagent sessions: each stage report carries the five footer fields (skills_loaded,mcps_used,clis_used,testing_levels_touched,screenshots_captured); the orchestrator compiles the footer ONCE at close. Chat only — never in a Jira comment or ATR body. - Subtask tracking is best-effort. The
[QA] Shift-Left Reviewsubtask makes QA's pre-sprint work visible on the board and holds the exhaustive session annotations that would otherwise clutter the Story. If.agents/jira-workflows.jsonhas no subtask work type (or the project disallows subtasks), warn once in the batch report and proceed — never block a refinement on subtask support.
Anti-patterns — NEVER do these
L1. NEVER force ambiguity questions onto a Story to fill a checklist — raise PO/Dev questions ONLY when a genuine gap, ambiguity, or untestable AC exists. Per AGENTS.md §1 Rule #4: shift-left adds value by surfacing real risk, not by inflating question counts. A clean Story exits with an empty question list and that is a valid outcome.
L2. NEVER skip the shift-left-reviewed label when transitioning a Story out of Phase 3. /sprint-testing Phase 0 inspects that label to short-circuit Phases 1-3 of in-sprint planning; missing the label forces redundant work later and breaks the cadence this skill exists to enable.
L3. NEVER mix Story refinement with bug retest in the same batch. /shift-left-testing accepts Stories only (Phase 1 type filter is a hard reject). Bugs are reactive — they have no upstream ACs to refine and belong to /sprint-testing instead.
L4. NEVER hand-write the ATP body as raw ADF JSON. Author the body in Markdown locally (shift-left-refinement.md) and let [ISSUE_TRACKER_TOOL] convert via its md-to-ADF path on update. Hand-rolled ADF drifts from the field content that fix-traceability later validates.
L5. NEVER transition a Story to estimation without a populated ATP (the {{jira.acceptance_test_plan}} custom field in BOTH modalities; the ## Acceptance Test Plan (ATP) fallback comment when the field is absent). The pre-sprint ATP is what makes the Story estimable — without it, Dev and PO guess scope and the shift-left effort delivers no signal.
L8. NEVER create the Test Plan item pre-sprint. The pre-sprint ATP's only home is the {{jira.acceptance_test_plan}} field (or its fallback comment); /sprint-testing Stage 1 creates the Test Plan issue FROM that field content once the Story enters the sprint. A pre-sprint item wastes an artifact on a Story whose scope may still shrink and creates a second copy Stage 1 must reconcile.
L6. NEVER refine more than ~10-12 Stories in a single batch. Refinement quality degrades past that — user attention budget collapses, summaries blur, the batch report loses signal. Split larger groomings into multiple sessions with distinct <descriptor> values.
L7. NEVER add a PO/Dev question that the AC body already answers in plain text. The reader's bandwidth is the scarcest resource in a grooming session; redundant questions train the team to skim future shift-left output.
Cross-skill handoff — what this skill does NOT do
| After Phase 3 you need... | Load this skill / command | Reason |
|---|---|---|
| Wait for Dev to estimate + commit the Story into a sprint | (manual / PO) | This skill stops at {{jira.status.story.estimation}}. PO + Dev lead drive estimate -> ready_for_dev and sprint commitment. |
In-sprint manual QA once the Story reaches Ready For QA |
/sprint-testing |
Will detect label shift-left-reviewed, validate the refinement is still fresh, short-circuit Phases 1-3, and run Phases 4-8 + Stages 2 + 3 normally. Stage 1 also creates the Test Plan ITEM from the {{jira.acceptance_test_plan}} field this skill populated. |
| Formal TC creation + ROI scoring after Story ships | /test-documentation |
Stage 4 turns the outlines + refined ACs into formal Xray TCs (Modality jira-xray) or Jira Test issues (Modality jira-native) with ROI scoring. |
| Automated test code | /test-automation |
Stage 5. |
| Regression suite execution | /regression-testing |
Stage 6. |
| Generate / refresh business + master test plan context | /project-discovery + /business-*-map + /master-test-plan |
This skill consumes those; it does not create them. |
| Adversarial dual-review of the refinement (optional) | /judgment-day |
Useful when shift-left output goes to a high-risk Story. Not auto-invoked. |
If Phase 0.3 reports any project-wide context file missing, STOP and hand off — refinement without business context produces vague PO questions and dilutes the batch report.
Pseudocode tags used here
| Tag | Resolves to | Defined in |
|---|---|---|
[ISSUE_TRACKER_TOOL] |
acli, Atlassian MCP, or {{ISSUE_TRACKER_CLI}} |
AGENTS.md Tool Resolution |
[TMS_TOOL] |
xray-cli skill (Modality jira-xray) OR acli (Modality jira-native) |
AGENTS.md Tool Resolution |
[ORCHESTRATION_TOOL] |
the multi-session orchestration CLI (fleet seam only) | orca-orchestration/SKILL.md |
Reads vs writes split (per
agentic-qa-core/references/acli-integration.md§"Reads vs writes"): detailed reads (description, ACs, scope, comments, parent epic) →bun run jira:sync-issues get/jql, then read the synced.md. Writes (custom-field update, comment, transition, label, link) + the trivial key+summary+status candidate list →acli. NEVERacli viewfor a custom field. |[DB_TOOL]| DBHub MCP or Supabase MCP |AGENTS.mdTool Resolution | |[API_TOOL]| OpenAPI MCP, Postman, or curl |AGENTS.mdTool Resolution |
Concrete tools (bun, git, gh) used literally. Project variables resolve from .agents/project.yaml (env-scoped vars resolve to the active environment). Jira variables ({{jira.status.story.*}}, {{jira.transition.story.*}}, {{jira.acceptance_test_plan}}) resolve from .agents/jira-workflows.json + .agents/jira-fields.json.
References — read the narrow one for the situation
All references are self-contained. Load one at a time.
| Reference | Read when |
|---|---|
references/backlog-selection.md |
Phase 0.4 + Phase 1 — building the candidate JQL, applying veto + risk-score triage per candidate, formatting the candidate table for user approval. |
references/refinement-playbook.md |
Phase 2 — running the per-Story refinement subagent. Cites acceptance-test-planning.md Phases 1-3 + outline-only Phase 4. Documents the shift-left deltas (no parametrization, no test-data gen, outline names only). |
references/atp-outline-template.md |
Phase 2 — body skeleton for shift-left-refinement.md (the pre-sprint ATP at outline maturity). Different from sprint-testing's full ATP body. |
references/refinement-questions.md |
Phase 2 — catalog of typical PO / Dev / Design gap-spotting questions, grouped by AC archetype (auth, money, search, state machine, etc.). Use as a checklist when the Story is sparse. |
references/handoff-protocol.md |
Phase 3 — exact Jira mutation sequence per Story (field-first ATP write, subtask close), label + transition rules, batch report template + epic-comment posting rules. |
../agentic-qa-core/references/artifact-lifecycle.md |
Before any transition — the Story + subtask lifecycle rows (§1), assignee-at-create (§2), the unmapped-status fallback protocol (§4), the light stage verifier template (§5). |
../agentic-qa-core/references/session-management.md |
Phase 0 + Phase 4 — resume contract, plan.md/progress.md schemas, archive policy, Engram per-phase checkpoint. This skill is a producer of session/shift-left-testing/<batch-id>/... topic keys. |
Pre-flight checklist
- Session resume check ran (Phase 0.0); user chose resume / restart / abort if prior state existed
- TMS modality resolved + persisted to
plan.md§Inputs -
/acliloaded (no/xray-cli— this skill creates no TMS items) - Project-wide context files present (else handed off to
/project-discovery) - Candidate Story list resolved (explicit IDs or backlog JQL) + confirmed with user
- Session folder
.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/created withplan.mdwritten - Phase 1 produced the ranked candidate table, user OK'd the refinement set
- Phase 1 found-or-created the
[QA] Shift-Left Reviewsubtask per accepted Story, assignee = self, at{{jira.status.subtask.active}}(or skipped with warning — no subtask work type) - Phase 2 ran ONE refinement subagent per accepted Story, user OK'd each summary
- Per-Story
shift-left-refinement.mdwritten under each Story's PBI folder - Phase 3 handoff applied per Story: Jira description + ATP field (
{{jira.acceptance_test_plan}}, both modalities) + handoff comment + labels + transition (stops atestimation) — NO Test Plan item created - Subtask closed per Story: session annotations posted on it +
{{jira.transition.subtask.complete}}fired →{{jira.status.subtask.close}}(or skipped with warning) - Light stage verifier run per
agentic-qa-core/references/artifact-lifecycle.md§5 — every line YES or a stated N/A - Trace verified per Story (both modalities: field populated + pointer comment; full body in comment only in fallback mode)
- Batch report written + posted to parent epic (if applicable)
- Archive:
.session/shift-left-testing/<batch-id>/moved to.session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/andmem_session_summarycalled - No git commit (Jira is canonical for this skill)
- User informed: when each Story reaches
Ready For QA, run/sprint-testing(will short-circuit thanks toshift-left-reviewed) - Session footer + consolidated screenshot list printed in chat per session-footer-contract (never in a Jira comment)
Files (agentic-qa-boilerplate)
-
references
-
atp-outline-template.md 11.5 KB
# ATP Outline Template — Skeleton for `shift-left-refinement.md` > **Subagent context**: this file is the "Context docs" template the Phase 2 Refinement subagent writes into the Story's PBI folder (see `shift-left-testing/SKILL.md` §Phase 2 and `refinement-playbook.md`). This is the pre-sprint ATP body at **outline maturity**. It is the Story's single, real ATP — pre-sprint it lives in the `{{jira.acceptance_test_plan}}` field; `/sprint-testing` Stage 1 later creates the Test Plan issue FROM that field content and refines the SAME ATP into the executable superset (see `acceptance-test-planning.md` for that stage's full body). What changes between the two passes is scope (and home: field → item), not identity. The skeleton below is the canonical body for `shift-left-refinement.md`, the `[LOCAL]` staging buffer. Phase 3 publishes it to Jira (custom field `{{jira.acceptance_test_plan}}` + Story description QA Refinements section); the Phase 3 comment is a POINTER to the field — full body in a comment only in fallback mode when the field is absent (see `handoff-protocol.md`). > **ATP item title** — the `# Shift-Left Refinement: …` heading below is only the local file heading. Phase 3's publish target is the `{{jira.acceptance_test_plan}}` field — Shift-Left creates NO Test Plan issue. The real **Test Plan** item titled `ATP: {STORY-KEY}: {story title}`, parented to the **QA Master Test Plan** epic, is created by `/sprint-testing` Stage 1 from the field content — never a title variant, never a second ATP. > **Before Phase 3 publishes the ATP body to Jira rich-text fields**, read `../../agentic-qa-core/references/jira-publishing-gotchas.md` — covers the two ADF conversion gotchas (`md-to-adf` mark collision + MCP batched custom-field rejection) that silently fail HTTP 400. --- ## Skeleton ```markdown # Shift-Left Refinement: {{PROJECT_KEY}}-{n} — {Title} **Status**: Refined — Awaiting PO Estimation **Mode**: Shift-Left (pre-sprint, batch grooming) **Refined on**: {{YYYY-MM-DD}} **Refined by**: QA — Shift-Left batch session **Modality**: {Xray | Jira-native} --- ## Phase 1 — Critical Analysis ### Business context - **Primary persona affected**: ... - **Secondary personas (if any)**: ... - **Business value proposition**: ... - **KPI(s) influenced**: ... - **User journey position**: which step / which flow ### Technical context - **Frontend**: components, pages / routes, state management (if any) - **Backend**: endpoints (cite `business-api-map.md`), services, DB tables - **External services**: ... - **Integration points specific to this Story**: ... ### Story complexity | Axis | Rating | Why | |------|--------|-----| | Business logic | Low / Medium / High | ... | | Integration | Low / Medium / High | ... | | Data validation | Low / Medium / High | ... | | UI | Low / Medium / High | ... | **Estimated test effort**: ... (informs PO estimation) ### Epic-level inheritance (if applicable) - Risks restated at Story level: ... - Integration points inherited: ... - PO/Dev answers already given at epic level: ... (reused, NOT re-asked) - Test strategy inherited: ... --- ## Phase 2 — Story Quality Analysis ### Ambiguities | # | Location in Story | Question for PO/Dev | Impact on testing | Suggested clarification | |---|-------------------|---------------------|-------------------|------------------------| | 1 | AC2 line "user can ..." | What does "successful" mean — HTTP 200 or rendered toast? | Cannot design assertion | Specify expected feedback channel | ### Gaps (missing info) | # | Type | Why critical | What to add | Risk if omitted | |---|------|--------------|-------------|-----------------| | 1 | AC | No error path defined for invalid input | Add Negative AC covering 400 / validation message | False positive in QA + production defect | ### Edge cases not in Story | # | Scenario | Expected behavior (best guess) | Criticality | Action | |---|----------|-------------------------------|-------------|--------| | 1 | User submits OTP after session expires | Show "Session expired — restart" | High | Add to AC (NEEDS PO/DEV CONFIRMATION) | ### Contradictions {Any disagreement between description, ACs, comments, designs — surface explicitly. If none: "No contradictions found."} ### Testability validation **Verdict**: Yes / Partial / No If Partial / No, list issues: - Vague AC ("works fast" — no metric) - Missing error messages - No test-data examples - Missing performance criteria - Cannot isolate (depends on parallel feature) --- ## Phase 3 — Refined Acceptance Criteria ### Original AC1 — {summary} #### Scenario 1.1: Should {behavior} {condition} (Type: Positive, Priority: High) - **Given**: ... - **When**: ... (exact input values) - **Then**: ... - UI: ... - API: status + body shape - DB: table / record / fields changed - System state: ... #### Scenario 1.2: Should {behavior} {condition} (Type: Negative, Priority: High) - **Given**: ... - **When**: ... - **Then**: error message verbatim + status code + no DB change (...repeat per AC...) ### New scenarios surfaced from Phase 2 edge cases — NEEDS PO/DEV CONFIRMATION #### Scenario E1: Should {behavior} {condition} (Type: Edge, Priority: ?) - **NEEDS PO/DEV CONFIRMATION**: behavior inferred — confirm before sprint planning - **Given**: ... - **When**: ... - **Then**: ... --- ## Phase 4 — Test Outlines (outline names only) ### Coverage estimate | Type | Count | Notes | |------|-------|-------| | Positive | X | Happy path variants | | Negative | Y | Invalid inputs, unauthorized, missing fields | | Boundary | Z | Min / max / empty / null / unicode | | Integration | W | Per integration point | | API | V | Per endpoint touched | | **Total** | **N** | (drives PO estimation) | **Rationale**: 2-3 sentences explaining why this count given the complexity axes from Phase 1. ### Outline list (NAMES ONLY — preconditions in 1 line, expected in 1 line) #### Positive - **Should login successfully with valid OTP** — Pre: user requested OTP <5 min ago. Expected: redirect to /dashboard + 200 + session cookie set. - ... #### Negative - **Should reject login when OTP is incorrect** — Pre: user with active OTP. Expected: 401 + "Invalid code" + no session. - ... #### Boundary - **Should reject OTP entered exactly at 5-minute expiry boundary** — Pre: OTP issued 5:00 ago. Expected: 401 + "Code expired". - ... #### Integration - **Should validate Auth0-issued OTP via /verify endpoint** — Pre: Auth0 sandbox configured. Expected: 200 from Auth0 + 200 from app. - ... > **NOT included here** (deferred to in-sprint planning by `/sprint-testing` Stage 1): parametrization tables, per-outline test-data JSON, numbered test steps, Faker generation strategies. Coverage estimate IS included because PO uses it for estimation. --- ## Phase 5 — Edge Cases (outline) | # | Edge case | In original Story? | Criticality | Action | |---|-----------|-------------------|-------------|--------| | 1 | OTP submitted after session expiry | No | High | Add to AC (PO confirm) | | 2 | OTP entered with leading whitespace | No | Medium | Test only — don't add AC | > Test-data generation strategy + Faker recipes are NOT defined here. They land in `/sprint-testing` Stage 1 when the feature exists. --- ## Story Quality Assessment **Verdict**: Good / Needs Improvement / Significant Issues **Key findings** (1-3 bullets): - ... - ... --- ## Critical Questions for PO > These BLOCK sprint planning until answered. 1. **{Question}** - **Context**: ... - **Impact if unanswered**: ... - **Suggested answer (if you have one)**: ... --- ## Technical Questions for Dev > These do not block PO but block implementation. 1. **{Question}** — context + testing impact. --- ## Suggested Story Improvements | # | Current state | Suggested change | Benefit | |---|---------------|------------------|---------| | 1 | "User can ..." | "User can ... within 2 seconds" | Adds performance bar → measurable AC | --- ## Data feasibility flags {If Phase 1 Selection marked DATA-FEASIBILITY-RISK, restate concretely. If none: "No data feasibility risks identified."} - **Entity / fixture missing**: ... - **API contract gap**: ... - **Required pre-work**: ... --- ## Recommended testing strategy ### Pre-implementation - ... ### During implementation - ... ### Post-implementation (in-sprint by /sprint-testing) - ... --- ## Risks & mitigation | # | Risk | Likelihood | Impact | Mitigated by which outlines | |---|------|-----------|--------|-----------------------------| | 1 | ... | Low / Med / High | Low / Med / High | Outlines #X, #Y | --- ## Next steps - [ ] PO answers Critical Questions before sprint planning - [ ] Dev answers Technical Questions before estimation - [ ] Story enters sprint at status `{{jira.status.story.ready_for_dev}}` once estimated - [ ] When Story reaches `{{jira.status.story.ready_for_qa}}`, `/sprint-testing` will short-circuit refinement (label `shift-left-reviewed` detected) ``` --- ## Differences from sprint-testing's full ATP | Section | In-sprint ATP (sprint-testing Stage 1) | Pre-sprint ATP (this skill, Phase 2) | |---------|----------------------------------------------|-------------------------------------------------| | Phase 4 parametrization tables | YES | NO | | Phase 4 per-outline test-data JSON | YES | NO | | Phase 4 numbered test steps | YES | NO (titles + 1-line precondition + 1-line expected only) | | Phase 5 test-data generation strategy | YES | NO | | Phase 5 Faker recipes | YES | NO | | TC creation in Xray (Modality jira-xray) | YES (Phase 6) | NO — TCs created in Stage 4 `/test-documentation` | | Commit on `test/{KEY}/...` branch | YES | NO — Jira is canonical | | Bug variant section | YES | NO — bugs out of scope | | Status flag at top | "Test Analysis — Ready for Execution" | "Refined — Awaiting PO Estimation" | When `/sprint-testing` later runs Stage 1 on a Story with `shift-left-reviewed` label fresh (<30 days), it syncs and reads `acceptance-test-plan.md` (which at that point holds THIS body, published to Jira in Phase 3) and: 1. Validates Refined ACs still match the (possibly updated) Story description. 2. Skips Phases 1-3 of `acceptance-test-planning.md` (already done). 3. Adds the deferred sections: parametrization, test-data JSON, Faker recipes, numbered steps. 4. Creates the Test Plan issue (`ATP: {STORY-KEY}: {story title}`, parented to the QA Master Test Plan epic, linked to the Story) from the field content, then writes the full ATP back to the SAME Jira `acceptance_test_plan` field (or fallback comment) and into that Test Plan issue, as a SUPERSET of the pre-sprint body. **One ATP per Story.** `shift-left-refinement.md` is a local staging buffer for Phase 2 → Phase 3, not a durable artifact: once Phase 3 publishes, Jira holds the canonical copy and the synced `acceptance-test-plan.md` is the readable one. Never make the in-sprint short-circuit depend on the staging file being present on disk — it lives under a gitignored path and will be missing on any other machine. --- ## Gotchas 1. **The skeleton above is canonical.** Do not reorder sections. Section names map 1:1 to the Jira description block PO sees during planning. 2. **NEEDS PO/DEV CONFIRMATION** marker is verbatim — never paraphrased. Tooling later greps for this string. 3. **Empty sections stay**, just with "None identified." — easier for the reviewer to verify completeness than to discover an absent section. 4. **Coverage estimate table** must show 0 for empty Types. Hidden zeros bias PO estimation. 5. **Mirror discipline**: the local file is the source of truth for what gets pushed to Jira in Phase 3. Diff = error. -
backlog-selection.md 10.7 KB
# Backlog Selection — Phase 1 Triage > **Subagent context**: this file is part of the "Context docs" briefing component for the Phase 1 Selection subagent (see `shift-left-testing/SKILL.md` §Subagent Dispatch Strategy). Owns the candidate-resolution + triage rubric only — refinement itself lives in `refinement-playbook.md`. Decides which Backlog Stories deserve a Shift-Left refinement pass and at what depth. The output is a ranked, user-facing candidate table that the orchestrator presents BEFORE Phase 2 dispatches refinement subagents. This reference is for **pre-sprint batch grooming**. It does NOT cover in-sprint triage (that lives in `sprint-testing/references/acceptance-test-planning.md` §Phase 0) nor bug triage (bugs are out of scope for this skill). --- ## Inputs | Input | Source | |-------|--------| | Candidate Story IDs (explicit mode) | User argument: `UPEX-100,101,102` or natural-language list | | Candidate Story IDs (JQL mode) | `[ISSUE_TRACKER_TOOL]` search using the query in §JQL Query below (trivial key+summary+status+type+labels list — `acli search` is fine here) | | Story metadata (detailed: description, ACs, comments) | `bun run jira:sync-issues get <STORY_KEY> --include-comments` (batch: `jql "<query>"`), then read the synced `.md`. NEVER `acli view` — returns `null` for custom fields. For the candidate triage itself (type/labels/priority) the trivial `acli search` list above suffices; sync only when a refined-AC-grade read is needed. | | Project-wide context | `.context/business/business-data-map.md`, `.context/business/business-feature-map.md`, `.context/business/business-api-map.md`, `.context/master-test-plan.md` | | Jira workflow values | `.agents/jira-workflows.json` -> `{{jira.status.story.*}}` | --- ## JQL Query (for "groom the backlog" mode) ```jql project = {{PROJECT_KEY}} AND issueType = Story AND status in ( "{{jira.status.story.backlog}}", "{{jira.status.story.shift_left_qa}}", "{{jira.status.story.estimation}}", "{{jira.status.story.ready_for_dev}}" ) ORDER BY priority DESC, created DESC ``` Optional filters the user may layer on: | Filter | When | |--------|------| | `AND sprint in openSprints()` | "Next sprint candidates only" | | `AND "Epic Link" = <<EPIC_KEY>>` | Grooming one epic at a time (epic key supplied by the user at session start) | | `AND labels != "shift-left-reviewed"` | "Skip already-refined" — but PREFER surfacing those separately and letting the user decide (see §3 below) | | `AND updated >= -14d` | Recent additions only | The Selection subagent proposes the JQL, executes it via `[ISSUE_TRACKER_TOOL]`, and surfaces the result count to the user BEFORE Phase 1 triage starts. If the result set is > 12 Stories, recommend splitting into multiple sessions. --- ## Step 1 — Type filter (hard) For each fetched candidate: | Issue type | Action | |------------|--------| | Story | -> Step 2 | | Bug, Spike, Sub-task, Tech-debt, Task, Epic | REJECT. Surface with one-line reason. Do NOT silently drop. | Reason surfaced to user: ``` Rejected (non-Story): - UPEX-115 [Bug] — Shift-Left is reactive-defect-free; this skill only refines forward-looking ACs. - UPEX-127 [Spike] — Spikes have no implementable ACs; refine the follow-up Story instead. - UPEX-141 [Tech-debt] — No user-facing AC; convert to a Story first if needed. ``` --- ## Step 2 — Label freshness check For each surviving candidate, check labels: | Label state | Bucket | Default action | |-------------|--------|----------------| | No `shift-left-reviewed` | NEW | -> Step 3 (triage) | | Has `shift-left-reviewed`, dated label `shift-left-{date}` < 30 days old | RECENTLY REFINED | Surface separately. Default: skip. Ask user: "Refresh anyway?" | | Has `shift-left-reviewed`, dated label > 30 days OR missing | STALE | Surface separately. Default: re-refine (Story may have drifted). Ask user: "Refresh or skip?" | This step prevents accidental double-work and gives the user explicit control over re-refinement. --- ## Step 3 — Veto table (beats risk score) Apply before scoring. Same rubric as `acceptance-test-planning.md` §0.1 — duplicated here for the subagent's convenience but the source of truth is the sprint-testing reference. ### SKIP REFINEMENT (drop from refinement set, log reason) | Condition | Why | |-----------|-----| | Backend-only with no UI surface AND no API contract change | No user-observable behavior to refine | | Pure CSS / styling | No AC complexity to refine | | Static copy / documentation | No business logic | | Tech-debt refactor with zero behavior change | No AC change | | DB-only setup with no business logic | No external surface | ### FORCE FULL REFINEMENT (override score) | Condition | Why | |-----------|-----| | Money, billing, payment, refund, currency conversion | Calculation correctness + audit trail | | Data integrity on core entities (user, account, order, transaction) | Persistent state corruption risk | | Authentication / authorization / session / RBAC | Security perimeter | | External integrations (Stripe, Auth0, Salesforce, partner APIs) | Contract drift risk | | State machines (order lifecycle, subscription, dispute, kyc) | Transition validity matters | | Calculations / formulas (tax, discount, accrual, interest) | Precision matters | If veto fires, write the reason into the candidate row and skip Step 4. --- ## Step 4 — Risk score (only if no veto) | Factor | Score | Condition | |--------|-------|-----------| | New feature | +3 | New functionality vs modification | | Dynamic data (API / DB) | +3 | Not hardcoded / static | | Explicit ACs present | +2 | Acceptance criteria defined in the Story | | User-facing | +2 | Affects UI or visible behavior | | High effort | +2 | Story-point estimate >= 5 OR description >= 600 words | | High priority | +1 | Priority High or Critical | | Multi-component | +1 | Multiple codebase areas touched (frontend + backend, multiple services) | | External dependency | +1 | Mentions third-party API, library upgrade, partner integration | | AC-light flag | +2 | < 2 ACs present OR ACs are 1-line each (auto-detected) — likely needs the most refinement | | Comment activity | +1 | > 5 substantive comments — usually means team disagreement still unresolved | | Score | Level | Recommendation | |-------|-------|----------------| | 0-3 | LOW | SKIP — PO / Dev can write ACs without QA refinement | | 4-7 | MEDIUM | Standard refinement (`acceptance-test-planning.md` §Phases 1-3 + outline-only Phase 4) | | 8+ | HIGH | Extended refinement — emphasize Phase 2 (ambiguities + edge cases not in story); produce extended PO/Dev question set | --- ## Step 5 — Data feasibility scan (lightweight, optional) For HIGH-risk candidates only, the Selection subagent does a quick feasibility probe: - Read `business-data-map.md` and `business-api-map.md`. Does the data model support the Story's ACs? - Read `master-test-plan.md`. Are there test-data fixtures for this entity / flow? - If neither answers "yes", flag the candidate with a `DATA-FEASIBILITY-RISK` marker. The refinement subagent will surface this as a critical PO question in Phase 2. This is a LIGHT scan — NO DB queries, NO API calls. The skill does not execute. Feasibility is established by reading the context docs only. --- ## Output format — candidate table presented to the user The Selection subagent returns this table verbatim. The orchestrator pastes it into the conversation and waits for the user's OK. ``` # Shift-Left Candidate Triage — {{YYYY-MM-DD}} ## Accepted for refinement ({n}) | # | Story | Title | Priority | Type | Risk | Score | Veto override | Data-feasibility | Notes | |---|-------|-------|----------|------|------|-------|---------------|------------------|-------| | 1 | UPEX-100 | Add OTP login | High | Story | HIGH | 11 | auth perimeter | OK | extended scan | | 2 | UPEX-101 | Refund partial amount | Medium | Story | HIGH | 9 | money | DATA-FEASIBILITY-RISK | flag for PO | | 3 | UPEX-102 | Filter orders by status | Low | Story | MEDIUM | 6 | — | OK | standard | ## Rejected (non-Story) ({n}) | # | ID | Type | Reason | |---|----|------|--------| | 1 | UPEX-115 | Bug | Shift-Left skill scope = Stories only | ## Skipped — LOW risk ({n}) | # | Story | Title | Score | Reason | |---|-------|-------|-------|--------| | 1 | UPEX-117 | Add tooltip to footer | 2 | Pure UI copy — PO can write the AC directly | ## Already Shift-Left Reviewed ({n}) | # | Story | Title | Refined on | Last-updated | Suggested action | |---|-------|-------|------------|--------------|------------------| | 1 | UPEX-098 | Search by phone | 2026-04-30 (12d ago) | 2026-05-15 | Refresh? (post-refinement edits detected) | ## Recommended order for refinement 1. UPEX-100 — HIGH risk, auth perimeter, no dependencies 2. UPEX-101 — HIGH risk + data-feasibility flag, surface to PO first 3. UPEX-102 — MEDIUM, can wait ## Open questions before Phase 2 starts - Confirm rejected set is correct? (1 item) - Confirm refresh decision on "Already Refined" set? (1 item) - Confirm refinement order above? (or override) ``` The user replies with OKs or overrides. The orchestrator persists the FINAL accepted list into the session's `progress.md` (`.session/shift-left-testing/<batch-id>/`) before dispatching Phase 2 refinement subagents. --- ## Gotchas 1. **Non-Story rejection is loud, not silent.** Surfacing them is the value — the user often discovers that what they thought was a Story is actually a Bug or Tech-debt mislabeled. 2. **Refresh detection.** If a Story has `shift-left-reviewed` but its description changed after the dated label, that is a signal the original refinement is stale. Surface it. 3. **JQL result-set sanity.** > 12 Stories in a single session burns user attention. Suggest splitting (e.g. by epic, by priority, by sprint candidate). 1-12 is the practical sweet spot. 4. **Data-feasibility-risk is a SOFT flag**. It does not block — it surfaces a critical PO question in Phase 2. The Story still enters refinement. 5. **Score is advisory, not authoritative.** The user can override LOW skips into the refinement set if they smell hidden complexity. 6. **No execution.** The Selection subagent does NOT run smoke tests, DB queries, or API calls. Feasibility is established from `.context/business/*` reads only. --- ## Checklist before handing the table to the user - [ ] Candidates resolved (explicit IDs or JQL) - [ ] Type filter applied; rejected set logged with reasons - [ ] Label freshness checked; recently-refined and stale-refined surfaced separately - [ ] Veto applied per surviving candidate (table source of truth) - [ ] Risk score computed per non-veto candidate - [ ] Data-feasibility scan run for HIGH-risk candidates (light, doc-only) - [ ] Output table assembled per §Output format - [ ] Open questions enumerated at the bottom of the table -
handoff-protocol.md 22.3 KB
# Handoff Protocol — Phase 3 (Per-Story Jira Mutation + Batch Report) > **Subagent context**: this file is the "Context docs" reference for the Phase 3 Handoff subagent + Batch Report subagent (see `shift-left-testing/SKILL.md` §Subagent Dispatch Strategy and §Phase 3). Owns Jira mutations + final report only — refinement content comes from `atp-outline-template.md` and the per-Story `shift-left-refinement.md`. The Handoff subagent runs ONCE PER REFINED STORY. The Batch Report subagent runs ONCE PER SESSION at the end. Both are dispatched sequentially. This reference defines: 1. Per-Story Jira mutation sequence (description update, ATP field population, handoff comment, labels, transition, subtask close, trace verify). 2. Modality handling — the ATP write is **field-first in BOTH modalities**; modality only matters downstream, when `/sprint-testing` Stage 1 creates the Test Plan item from the field. 3. Transition guardrails (stop at `estimation`). 4. Batch report template + epic-comment posting rules. --- ## Inputs (per Story) | Input | Source | |-------|--------| | Refined refinement file (NON-Jira working file) | `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/shift-left-refinement.md` | | Current Story status | `bun run jira:sync-issues get {STORY_KEY}`, then read synced status (or `acli search` for the trivial status-only lookup) | | Current Story labels | Same synced read — labels list | | Modality | From the session's `progress.md` (`.session/shift-left-testing/<batch-id>/`, resolved in shift-left-testing Phase 0.1). Informational here — the ATP write is field-first in both modalities | | TMS field map | `.agents/jira-fields.json` → `{{jira.acceptance_criteria}}`, `{{jira.acceptance_test_plan}}` | | Workflow transitions | `.agents/jira-workflows.json` → `{{jira.transition.story.analyze}}`, `{{jira.transition.story.estimate}}` | | Tracking subtask | The `[QA] Shift-Left Review` subtask created in Phase 1 (`{{jira.status.subtask.active}}` — the subtask workflow's names are `ACTIVE` / `Close`, NOT "In Progress" / "Done"). Closed at Step 5b via `{{jira.transition.subtask.complete}}`; if the catalog has no subtask work type, Phase 1 skipped it — Step 5b then skips with a warning too | | Artifact lifecycle | `agentic-qa-core/references/artifact-lifecycle.md` — §1 (Story + subtask rows), §2 (assignee = self on the subtask at create), §4 (unmapped-status fallback), §5 (light stage verifier) | --- ## Per-Story handoff sequence > **Prerequisite**: `/acli` skill loaded (Phase 0.2). `/xray-cli` is never needed here — this protocol creates no TMS items. ### Step 1a — Write refined ACs to the Jira `acceptance_criteria` field The refined Acceptance Criteria are CANONICAL and belong in the dedicated Jira field — Jira is source of truth; the local `acceptance-criteria.md` is a read-only cache produced by the sync. NEVER hand-write that file. ``` [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} fields: {{jira.acceptance_criteria}}: <Refined ACs — Phase 3 of shift-left-refinement.md verbatim> ``` FALLBACK (field absent on this instance): post the refined ACs as a structured comment headed `## Acceptance Criteria`, per `.agents/jira-required.yaml` → `acceptance_criteria.fallback` (`{ target: comment, label: "Acceptance Criteria" }`). Never block. After writing, run `bun run jira:sync-issues get {STORY_KEY} --include-comments` and read back the synced `acceptance-criteria.md` to confirm the field (or fallback comment) landed. ### Step 1b — Append supporting analysis to Story description Append a "QA Refinements (Shift-Left Analysis)" section to the Story description. The section body is a CONDENSED extract of the SUPPORTING analysis from `shift-left-refinement.md` (refined ACs themselves live in the `acceptance_criteria` field from Step 1a) — the full body goes into the ATP (Step 2); the Step 3 comment is a pointer to it. Description should be readable in the Jira UI without scrolling forever. Description section template: ```markdown --- ## QA Refinements (Shift-Left Analysis) — Added {{YYYY-MM-DD}} > Refined Acceptance Criteria live in the `acceptance_criteria` field (Step 1a). ### Edge Cases Identified {Copy Phase 5 edge-case table verbatim.} ### Clarified Business Rules {Any rule extracted from Phase 2 Story Quality Analysis that the original Story did not state explicitly.} ### Critical Questions for PO {Numbered list — copy verbatim from shift-left-refinement.md §Critical Questions for PO.} ### Technical Questions for Dev {Numbered list — copy verbatim from §Technical Questions for Dev.} > Full refinement (Phases 1-5, coverage outlines, risk + data feasibility) lives in the ATP — the `{{jira.acceptance_test_plan}}` field (Step 2; `/sprint-testing` Stage 1 later materializes it as the Test Plan issue). ``` ``` [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} description: <existing description + "\n\n" + QA Refinements section> ``` The Handoff subagent must read the current description FIRST (from the synced `.md`), then append. Never overwrite. ### Step 2 — Populate the ATP (field-first — both modalities) > **ONE ATP per Story, authored early — as FIELD content, never as an item.** Shift-Left does not create a separate DRAFT artifact, and it does not create the Test Plan issue either. Pre-sprint, the ATP's home is the `{{jira.acceptance_test_plan}}` custom field on the Story. `/sprint-testing` Stage 1 creates the real **Test Plan** issue (titled `ATP: {STORY-KEY}: {story title}`, parented to the **QA Master Test Plan** epic, linked to the Story) FROM this field content, and refines the same ATP into the executable superset. There is no `(Shift-Left DRAFT)` title variant and no second Test Plan to reconcile later. > > **Why field-first pre-sprint**: PO has not estimated yet and scope may shrink — creating the item this early wastes an artifact and leaves a second copy for Stage 1 to reconcile. The field IS the only pre-sprint ATP write, in BOTH modalities; modality only decides which engine Stage 1 uses later. > > What marks this ATP as pre-sprint is the Story's `shift-left-reviewed` + `shift-left-{YYYY-MM-DD}` labels, not the title. Those labels are what Stage 1 reads to decide whether to short-circuit. The write is identical in Modality jira-xray and Modality jira-native: ``` [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} fields: {{jira.acceptance_test_plan}}: <full shift-left-refinement.md body> ``` FALLBACK (the instance has not provisioned `{{jira.acceptance_test_plan}}`): skip the field write and switch Step 3 to fallback mode — the `## Acceptance Test Plan (ATP)` comment carries the full body inline, per `.agents/jira-required.yaml` → `acceptance_test_plan.fallback` (`{ target: comment, label: "Acceptance Test Plan (ATP)" }`). Warn the user in the per-Story summary. Never block. ### Step 3 — Handoff notification + fallback comment Jira is the source of truth: the ATP lives in the `{{jira.acceptance_test_plan}}` field (Step 2) — do NOT mirror it into a comment when the field exists. Post ONE handoff comment on the Story: - **Field present (default)**: a SHORT notification — the pre-sprint ATP is ready for review in the `{{jira.acceptance_test_plan}}` field. Do NOT paste the full body. - **FALLBACK — only if `{{jira.acceptance_test_plan}}` is absent on this instance** (per `.agents/jira-required.yaml` → `acceptance_test_plan.fallback`, `{ target: comment, label: "Acceptance Test Plan (ATP)" }`): inline the full body under a `## Acceptance Test Plan (ATP)` heading so the content still lands somewhere readable. ``` [ISSUE_TRACKER_TOOL] Add Comment: issue: {STORY_KEY} body: | ## Acceptance Test Plan (ATP) — ready for pre-sprint review {@PO_HANDLE} {@DEV_LEAD_HANDLE} The ATP lives in the {{jira.acceptance_test_plan}} field. # FALLBACK ONLY (field absent): replace the pointer line above with the full staged refinement body. Action Required: review ambiguities, answer critical questions, confirm edge-case behavior, validate parametrization. Refined on: {{YYYY-MM-DD}} — QA Shift-Left batch session This ATP is refined in-sprint by /sprint-testing Stage 1, which creates the Test Plan issue from this field and refines it into the executable superset. ``` `fix-traceability` checks the `{{jira.acceptance_test_plan}}` field, or this `## Acceptance Test Plan (ATP)` fallback comment when the field is absent. Mention rule: include `@PO_HANDLE` and `@DEV_LEAD_HANDLE` in the comment IF those handles are available in `.agents/project.yaml`. Otherwise omit — mention-spam is worse than no mention. ### Step 4 — Labels ``` [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} labels: +shift-left-reviewed, +shift-left-{{YYYY-MM-DD}} ``` - `shift-left-reviewed` is the SOFT MARKER — `/sprint-testing` Stage 1 reads it. - `shift-left-{{YYYY-MM-DD}}` is the FRESHNESS MARKER — `/sprint-testing` uses the date to decide whether refinement is still <30 days old and can be short-circuited. Both labels are appended (never replaced). If the Story already carries an older `shift-left-{date}`, leave it — it documents the refinement timeline. ### Step 5 — Transition > **Guardrail**: NEVER advance beyond `{{jira.status.story.estimation}}`. PO + Dev lead own `estimate -> ready_for_dev`. Read current status, then transition along the shortest valid path to `estimation`: | Current status | Transitions to apply | Resolved IDs | |----------------|----------------------|--------------| | `{{jira.status.story.backlog}}` | `{{jira.transition.story.analyze}}` → `{{jira.transition.story.estimate}}` | id 2 (Analyze), then id 3 (Estimate) | | `{{jira.status.story.shift_left_qa}}` | `{{jira.transition.story.estimate}}` | id 3 (Estimate) | | `{{jira.status.story.estimation}}` | (none — already there) | — | | `{{jira.status.story.ready_for_dev}}`, `{{jira.status.story.in_progress}}`, `{{jira.status.story.in_review}}`, `{{jira.status.story.ready_for_qa}}`, ... | SKIP transition — log warning | refinement still lands; workflow untouched | | `{{jira.status.story.aborted}}`, `{{jira.status.story.deployed_to_production}}` | SKIP transition + WARN user — terminal | refinement is informational only | Pseudocode: ``` [ISSUE_TRACKER_TOOL] read status: {STORY_KEY} # trivial status-only lookup — acli search OK if status == backlog: [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.analyze}} # -> shift_left_qa [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.estimate}} # -> estimation elif status == shift_left_qa: [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.estimate}} # -> estimation elif status == estimation: # noop — already at target elif status in (ready_for_dev, in_progress, in_review, ready_for_qa, qa_approved, in_test, ready_for_release, deployed_to_production, blocked, aborted): log warning "Story past estimation — refinement landed; workflow untouched" else: log warning "Unknown status {status}; SKIP transition" ``` The skill NEVER applies the `back_from_shift_left_qa` transition automatically. That is a PO / Dev decision (Story does not meet readiness for estimation — go back to backlog). ### Step 5b — Close the `[QA] Shift-Left Review` subtask Phase 1 found-or-created this subtask under the Story (assignee = self per `agentic-qa-core/references/artifact-lifecycle.md` §2), where Jira's `create` transition lands it in `{{jira.status.subtask.active}}`. The handoff closes it: 1. Locate the subtask under the Story by exact title `[QA] Shift-Left Review`. 2. **Post the exhaustive session annotations on the SUBTASK first** — the long analysis, refinement traces, and per-phase notes that are too verbose for the Story. The Story keeps only its canonical outputs (refined ACs field, description section, ATP field, pointer comment, labels); the subtask is where the full working trail lives. ``` [ISSUE_TRACKER_TOOL] Add Comment (or Update description): issue: {SUBTASK_KEY} body: <exhaustive session annotations — analysis trail, refinement traces> [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.subtask.complete}} # active -> close ``` > **On an unmapped slug** (this project renamed the subtask statuses, or the catalog is > stale): run the fallback protocol in `agentic-qa-core/references/artifact-lifecycle.md` > §4 — list the LIVE transitions, propose the closest synonym in ONE `AskUserQuestion`, > fire the live id on yes, and recommend `bun run jira:sync-workflows`. Never skip silently. 3. If Phase 1 skipped subtask creation (no subtask work type in `.agents/jira-workflows.json`, or the project disallows subtasks): skip this step with a warning in the per-Story log. Never block the handoff on subtask support. ### Step 6 — Verify trace Both modalities (field-first — there is no Test Plan item to trace pre-sprint): ``` bun run jira:sync-issues get {STORY_KEY} --include-comments # then read the synced field files + comments.md. Verify: # - acceptance_criteria field (or "## Acceptance Criteria" fallback comment) != empty # - field {{jira.acceptance_test_plan}} != empty # (or the "## Acceptance Test Plan (ATP)" fallback comment carries the full body # when the field is absent on this instance) # - handoff comment "## Acceptance Test Plan (ATP)" present and points to the field # (full body inline ONLY in fallback mode — field absent on this instance) # - "[QA] Shift-Left Review" subtask (when created) is at {{jira.status.subtask.close}} ``` ### Step 6b — Light stage verifier (closes the Shift-Left stage) Run the eight-line template in `agentic-qa-core/references/artifact-lifecycle.md` §5. The stage-specific status lines are: ``` [ ] Story at {{jira.status.story.estimation}} — via analyze -> estimate, and STOPPED there (a Story already past estimation keeps the refinement and skips the transition) [ ] `[QA] Shift-Left Review` subtask at {{jira.status.subtask.close}} — via complete (stated N/A when the instance has no subtask work type) [ ] Subtask assignee = self, set at create time [ ] Any unmapped slug went through the §4 fallback (asked), never a silent skip ``` ### Step 7 — Return per-Story log ```json { "story": "UPEX-100", "atp_container": "custom_field|fallback_comment", "subtask_key": "UPEX-205 (null when skipped — no subtask work type)", "subtask_status": "close|skipped", "light_verifier": "8/8 (N/A: <stated reasons>)", "description_appended": true, "comment_posted": true, "labels_added": ["shift-left-reviewed", "shift-left-2026-05-20"], "transitions_applied": ["analyze", "estimate"], "final_status": "Estimation", "trace_status": "ok|warning|fail", "warnings": [], "errors": [] } ``` Warnings DO NOT abort the per-Story handoff — they are surfaced in the batch report. Errors DO abort. Per AGENTS.md §Orchestration Mode, the orchestrator presents retry / skip / abort to the user. Do NOT auto-rollback Jira mutations — they are recorded in the partial log so a future session can resume. --- ## Batch report Dispatched ONCE after every Story has run Phase 3. ### Output path ``` .session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/batch-report.md ``` ### Template ```markdown # Shift-Left Batch Report — {{YYYY-MM-DD}} ## Session metadata - **Session folder**: `.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/` - **Mode**: Modality {A | B} - **Candidate count**: {n} - **Accepted for refinement**: {n} - **Rejected (non-Story)**: {n} - **Skipped (LOW risk)**: {n} - **Already refined (skipped)**: {n} - **Already refined (refreshed)**: {n} - **Refined this session**: {n} ## Per-Story summary | # | Story | Title | Risk | Gaps | Critical PO Qs | Tech Qs | Outlines | Data-feasibility flag | Final status | Trace | |---|-------|-------|------|------|----------------|---------|----------|----------------------|--------------|-------| | 1 | UPEX-100 | Add OTP login | HIGH | 4 | 2 | 3 | 12 | — | Estimation | ok | | 2 | UPEX-101 | Refund partial amount | HIGH | 6 | 3 | 2 | 14 | DATA-FEASIBILITY | Estimation | warning | | 3 | UPEX-102 | Filter orders by status | MEDIUM | 2 | 0 | 1 | 6 | — | Estimation | ok | ## Aggregated Critical Questions for PO (deduped) Top questions across the batch, ranked by how many Stories they block: 1. **Q: What is the refund cap policy — % of original or hard limit?** - Blocks: UPEX-101, UPEX-118 (2 Stories) - Suggested action: PO decision required before sprint planning. 2. **Q: What is the session expiry threshold for OTP flow?** - Blocks: UPEX-100 (1 Story) - Suggested action: PO + Security decision; default-safe = 5 minutes. ## Aggregated Technical Questions for Dev (deduped) 1. **Q: Which Auth0 SDK version is pinned?** - Blocks: UPEX-100, UPEX-114 (2 Stories) 2. ... ## Risk distribution | Level | Count | Stories | |-------|-------|---------| | HIGH | 2 | UPEX-100, UPEX-101 | | MEDIUM | 1 | UPEX-102 | | LOW (skipped) | 1 | UPEX-117 | ## Data feasibility blockers Stories flagged with `DATA-FEASIBILITY-RISK` during selection that still carry the flag post-refinement: | Story | Blocker | Required pre-work | |-------|---------|-------------------| | UPEX-101 | Refund-history fixture missing in staging | Backend team to seed; ETA: ? | ## Recommended sprint-planning order Sorted by risk + dependency: 1. **UPEX-100** — HIGH, no dependencies, refinement clean → ready for estimation 2. **UPEX-101** — HIGH but data-feasibility flag → estimate after pre-work confirmed 3. **UPEX-102** — MEDIUM → can wait, low coupling ## Next steps - [ ] PO answers Aggregated Critical Questions before sprint planning - [ ] Dev lead answers Aggregated Tech Questions before estimation - [ ] When each Story reaches `Ready For QA`, run `/sprint-testing` — Stage 1 will detect `shift-left-reviewed` label and short-circuit Phases 1-3 of `acceptance-test-planning.md` - [ ] If any Story still carries a data-feasibility blocker at sprint-planning time, consider moving it to a later sprint ``` ### Posting rules 1. **Always write** the file under `.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/batch-report.md`. 2. **If all refined Stories share ONE parent epic**, post the batch report as a comment on that epic — the team grooms by epic, so the report lands where decision-makers look. 3. **If Stories span multiple epics or have no common parent**, do NOT post to any epic — deliver inline to the user as the session-closing message. 4. **Never post the report on every Story**. The per-Story handoff comment already landed in Step 3; the batch report is a session-level artifact. --- ## Resume protocol The session is resumable. If interrupted (subagent failure, user abort, network drop), the next invocation: 1. Reads the session's `progress.md` (`.session/shift-left-testing/<batch-id>/`) to know which Stories are pending vs handed-off. 2. Reads each pending Story's `shift-left-refinement.md` (if it exists) to know Phase 2 is done. 3. Continues from the first incomplete step (Step 1 if description not yet updated, Step 2 if ATP not populated, ...). Each step is idempotent: | Step | Idempotency rule | |------|------------------| | Step 1 description | If "QA Refinements (Shift-Left Analysis)" section already present → skip | | Step 2 ATP field | Overwrite OK — field is single-value | | Step 3 comment | If a comment headed `## Acceptance Test Plan (ATP)` already exists → skip | | Step 4 labels | acli labels operation is set-based; re-running adds nothing | | Step 5 transition | Read current status before transitioning; skip if already at target | | Step 5b subtask | Find by exact title (created in Phase 1); skip transition if already Done; append annotations as a new comment, never overwrite | | Step 6 trace | Always re-verify | --- ## Gotchas 1. **Description append, never overwrite.** Read first, append second. 2. **The comment is a pointer, not a mirror.** When `{{jira.acceptance_test_plan}}` exists, the handoff comment only points to the field — never paste the full body. The full body goes in the comment ONLY in fallback mode (field absent). `fix-traceability` checks the field, or the fallback comment when the field is absent. 3. **Transition guardrail.** STOP at `estimation`. Stories past that point keep the refinement (description + field + comment + labels) but skip transition. 4. **No TMS items pre-sprint.** This protocol never creates the Test Plan issue — `/sprint-testing` Stage 1 creates it from the `{{jira.acceptance_test_plan}}` field content. If an older session already left a pre-sprint Test Plan on the Story, leave it, note it in the per-Story log, and let Stage 1 reconcile. 5. **Mention discipline.** Only mention PO/Dev-lead handles that are explicitly listed in `.agents/project.yaml`. No guessing. 6. **Dated label** (`shift-left-{date}`) is APPENDED on every refinement. `/sprint-testing` reads the most recent one. Old dated labels are NOT pruned by this skill. 7. **Resume safety.** Every step is idempotent — re-running a partially-completed Story should converge to the same final state. 8. **Language**: all Jira content English. Mirror user's language only in conversation. AGENTS.md §1 Rule #14. 9. **NO git commit.** Jira is the source of truth. The local `shift-left-refinement.md` is a gitignored working artifact. 10. **Epic comment posting** is a courtesy. If it fails (epic doesn't exist, permission denied, network), DO NOT abort — log a warning and proceed; the local report is still authoritative. 11. **Subtask close is best-effort.** The `[QA] Shift-Left Review` subtask holds the exhaustive session annotations and shows the team QA worked the Story pre-sprint. No subtask work type in the catalog → Step 5b skips with a warning; the handoff still completes. --- ## Checklist before closing the session - [ ] All accepted Stories ran the per-Story handoff - [ ] Each per-Story log captured in the session's `progress.md` - [ ] No transition advanced beyond `{{jira.status.story.estimation}}` - [ ] No Test Plan item created (field-first — the item is `/sprint-testing` Stage 1's job) - [ ] `[QA] Shift-Left Review` subtask per Story: annotations posted + transitioned to Done (or skipped with warning) - [ ] Batch report written to `.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/batch-report.md` - [ ] Batch report posted to parent epic (if all Stories share one) OR delivered inline - [ ] User informed: when each Story reaches `Ready For QA`, run `/sprint-testing` (short-circuit thanks to `shift-left-reviewed`) - [ ] Warnings + errors surfaced explicitly in the user-facing session-close message -
refinement-playbook.md 16.1 KB
# Refinement Playbook — Phase 2 (Per-Story Subagent) > **Subagent context**: this file is the primary "Context docs" reference for the Phase 2 Refinement subagent (see `shift-left-testing/SKILL.md` §Subagent Dispatch Strategy). It is a thin wrapper that CITES `sprint-testing/references/acceptance-test-planning.md` rather than duplicating it. The Refinement subagent runs ONCE PER STORY. It reads the existing acceptance-test-planning playbook from sprint-testing, applies the shift-left-mode delta, and writes a single output file: `shift-left-refinement.md` inside the Story's PBI folder. This reference defines the delta only. Anything not overridden here behaves exactly as in `acceptance-test-planning.md`. --- ## Reuse contract ``` Source of truth (cited, not duplicated): .agents/skills/sprint-testing/references/acceptance-test-planning.md Phases run by the Refinement subagent: Phase 1 Critical Analysis -> run as-is, light code reads only Phase 2 Story Quality Analysis -> run as-is (this is the heart of shift-left) Phase 3 Refined ACs -> run as-is (Given/When/Then with data) Phase 4 Test Design (outlines) -> outline NAMES + brief preconditions only Phase 5 Edge case summary -> edge-case NAMES + criticality only Phases skipped: Phase 0 Triage -> already done in shift-left-testing Phase 1 Phase 4 parametrization tables -> deferred to in-sprint Phase 4 per-outline test-data JSON -> deferred to in-sprint Phase 5 test-data generation strategy -> deferred to in-sprint Phase 6 Traceability + Ticket updates -> shift-left-testing Phase 3 owns this Phase 7 Final QA Feedback Report -> aggregated into batch report (Phase 3) Phase 8 Commit -> NO COMMIT, Jira is canonical ``` The Refinement subagent's job is to produce a high-signal artifact for PO + Dev BEFORE the feature exists. Anything that requires running the system (data generation, parametrization tuning, fixture design) belongs in `/sprint-testing` Stage 1, not here. --- ## Inputs (per Story) | Input | Source | |-------|--------| | Story (title, description, ACs, priority, labels, sprint, parent epic, comments) | `bun run jira:sync-issues get {STORY_KEY} --include-comments`, then read the synced `.md` (NEVER `acli view` — returns `null` for custom fields) | | Team Discussion | Synced `comments.md` — same extraction rules as `session-entry-points.md` §Step 1b | | Parent epic (if any) | `bun run jira:sync-issues get {EPIC_KEY}`, then read the synced epic `.md` — description + risk callouts only | | Project-wide context | `.context/business/business-data-map.md`, `.context/business/business-feature-map.md`, `.context/business/business-api-map.md`, `.context/master-test-plan.md` | | Module context (if it exists) | `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/module-context.md` (module = Epic, 1:1) | | Code (light read) | `{{BACKEND_REPO}}/{{BACKEND_ENTRY}}` + `{{FRONTEND_REPO}}/{{FRONTEND_ENTRY}}` — read enough to confirm feasibility, NOT to reproduce | | Common gap catalog | `refinement-questions.md` (this skill) | | ATP outline skeleton | `atp-outline-template.md` (this skill) | --- ## Output (per Story) ``` .context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/ context.md # NON-Jira working file — created if missing (minimal) shift-left-refinement.md # NON-Jira working file — this subagent's deliverable ``` `shift-left-refinement.md` follows the skeleton in `atp-outline-template.md`. Both files are NON-Jira working artifacts authored locally. Jira-mirrored files (story.md, acceptance-criteria.md, etc.) are produced ONLY by the sync — NEVER hand-write them. The subagent must NOT touch Jira. Phase 3 of the orchestrator skill owns all Jira mutations. --- ## Step 1 — Bootstrap the Story's PBI folder If `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/` does not exist: 1. Resolve the parent Epic — module = Epic (1:1). The epic folder is `EPIC-<EPIC_KEY>-<slug>` where `<slug>` is derived from the epic title (kebab-case). If the Story has no parent epic, use its module/component to name the epic folder. 2. Derive the story `<slug>` from the Story title — max 5 words, kebab-case. 3. Create the folder under `stories/STORY-<STORY_KEY>-<slug>/`. 4. Write a minimal `context.md` (template below). Do NOT include execution / smoke / evidence sections — those belong to `/sprint-testing` later. 5. Do NOT create `evidence/`. The Story has not shipped — there is nothing to capture. ### Minimal `context.md` template ```markdown # {{PROJECT_KEY}}-{n}: {Title} **Ticket:** {{PROJECT_KEY}}-{n} | **Module (= Epic):** {EPIC_KEY} {epic title} | **Status:** {Backlog | Shift-Left QA | Estimation | Ready For Dev} | **Sprint:** {n/a — pre-sprint} ## Acceptance Criteria (original) - AC1: ... - AC2: ... ## Team Discussion (from comments) {If comments exist: [Author] (date): key point. If none: "No team discussions found.")} ## Parent epic {EPIC_KEY}: {epic title} ## Pre-sprint status Shift-Left refinement: in progress (started {{YYYY-MM-DD}}) ``` If the folder already exists (because `/sprint-testing` ran on a sibling Story in the same module), DO NOT overwrite `context.md`. Just check it exists and proceed. --- ## Step 2 — Run `acceptance-test-planning.md` Phase 1: Critical Analysis Read sprint-testing/references/acceptance-test-planning.md §"Phase 1 — Critical Analysis". Apply as-is with these scoping notes: - **Code exploration is LIGHT.** Read entry files + 1-2 related files to confirm that the feature is implementable on the current codebase. Do NOT trace call graphs, do NOT open test files, do NOT measure performance. The cost ceiling is "would a senior dev say this is feasible in the current code". - **Epic-level inheritance** — if `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/feature-test-plan.md` exists, read it and reuse risks / integration points / PO answers verbatim. Cite, do not re-derive. - **Story complexity rating** drives the depth of Phase 2 below. Capture the output in the `## Phase 1 — Critical Analysis` section of `shift-left-refinement.md`. --- ## Step 3 — Run Phase 2: Story Quality Analysis (the heart of shift-left) This is where shift-left actually creates value. Read sprint-testing/references/acceptance-test-planning.md §"Phase 2 — Story Quality Analysis" — apply it MORE DEEPLY than the in-sprint variant, because PO is still able to act on the findings. For each: - **Ambiguities** — location in Story + question for PO/Dev + impact on testing + suggested clarification. - **Gaps (missing info)** — type (AC / technical detail / business rule) + why critical + what to add + risk if omitted. - **Edge cases not in Story** — scenario + expected behavior (best guess, flag for PO confirmation) + criticality + action (add to AC / test only / ask PO). - **Testability validation** — Yes / Partial / No + list of issues (vague AC, missing error messages, no test data examples, missing performance criteria, cannot isolate). - **Contradictions** — sections of the Story (description vs ACs vs comments vs Figma link if present) that disagree. Surface explicitly: "Description says X but AC3 says Y". Use the gap-catalog in `refinement-questions.md` as a checklist. The catalog is keyed by AC archetype (auth, money, search, state machine, list / table, integration). For each archetype matched in the Story, walk the corresponding question list. If the Story is genuinely clear, say so. A short "Story quality: good — no significant gaps" is better than inventing questions. Quality > quantity. Capture in `## Phase 2 — Story Quality Analysis` section of `shift-left-refinement.md`. --- ## Step 4 — Run Phase 3: Refined ACs Read sprint-testing/references/acceptance-test-planning.md §"Phase 3 — Refined Acceptance Criteria". Apply as-is — Given / When / Then with specific data. For each refined scenario: - **Type**: Positive / Negative / Boundary / Edge - **Priority**: Critical / High / Medium / Low - **Given**: initial system state + preconditions - **When**: triggering action with exact input values - **Then**: expected UI result + API status+body (if applicable) + DB changes + system state Mark any scenario that the refinement INFERRED (not literally in the original Story) with **NEEDS PO/DEV CONFIRMATION**. The marker survives into the Jira comment in Phase 3 — PO must see it during sprint planning. **1:N is the default — explode, then justify any collapse.** A non-trivial AC implies several scenarios. Derive them by the AC's shape (`agentic-qa-core/references/test-design-doctrine.md` Part 2): a range/limit → boundary scenarios (BVA); a status field → valid + invalid transition scenarios; 2+ interacting conditions → a decision-table set; 3+ factors → pairwise. Collapse an AC to a single scenario ONLY when it is trivially atomic (one boolean, no ranges/states/interactions) and say so. Counting estimate aside, a 1-line AC that hides a range or a state is NOT a 1-scenario AC. Capture in `## Phase 3 — Refined Acceptance Criteria` section of `shift-left-refinement.md`. --- ## Step 5 — Run Phase 4 (outline NAMES only) Read sprint-testing/references/acceptance-test-planning.md §"Phase 4 — Test Design". Apply with these deltas: | Phase 4 sub-step | Shift-Left action | |------------------|------------------| | Coverage estimate (Positive / Negative / Boundary / Integration / API counts) | **INCLUDE**. Drives PO estimation. | | Rationale paragraph | INCLUDE. Short, 2-3 sentences. | | Parametrization tables | **EXCLUDE**. Deferred to in-sprint. | | Per-outline structure (Title, Related scenario, Type, Priority, Test level, Preconditions, Test steps, Expected result, Test data JSON, Post-conditions) | **REPLACED** by outline NAME + 1-line precondition + 1-line expected. No test-data JSON, no numbered steps. | | Integration outlines (per integration point) | **NAMES ONLY**. List integration outlines as "Should validate {integration point} when {condition}". No mock strategy, no contract assertions. | ### Outline naming convention Format: `Should <BEHAVIOR> <CONDITION>`. Same as `acceptance-test-planning.md` §Phase 4. Examples: - `Should redirect to dashboard after successful OTP entry with valid code` - `Should display "Code expired" error after OTP entry with code older than 5 minutes` - `Should reject negative refund amount on POST /refunds` Capture in `## Phase 4 — Test Outlines (outline names only)` section of `shift-left-refinement.md`. Include the coverage estimate table at the top, then the outline list grouped by Type (Positive / Negative / Boundary / Integration). --- ## Step 6 — Run Phase 5 (edge-case NAMES only) Read sprint-testing/references/acceptance-test-planning.md §"Phase 5 — Edge case + Test-data summary". Apply with these deltas: | Phase 5 sub-step | Shift-Left action | |------------------|------------------| | Edge case table (case / in original story? / added to refined AC? / outline / priority) | **INCLUDE** but without "added to refined AC?" auto-resolution — that happens in-sprint. | | Test-data categories table | **EXCLUDE**. Deferred to in-sprint. | | Data generation strategy (Static / Faker / Cleanup) | **EXCLUDE**. Deferred to in-sprint. | Capture in `## Phase 5 — Edge Cases (outline)` section of `shift-left-refinement.md`. --- ## Step 7 — Summary block + open questions End `shift-left-refinement.md` with: - `## Story Quality Assessment` — one of: Good / Needs Improvement / Significant Issues. - `## Critical Questions for PO` — questions raised in Phase 2 that BLOCK sprint planning until answered. Format: question / context / impact-if-unanswered / suggested answer (if you have one). - `## Technical Questions for Dev` — questions that don't block PO but block implementation. - `## Suggested Story Improvements` — current state -> suggested change -> benefit. - `## Data feasibility flags` — if Phase 1 Selection marked `DATA-FEASIBILITY-RISK`, restate the risk here in concrete terms (which entity is missing, which API contract gap, which fixture needs to land first). - `## Recommended testing strategy` — pre / during / post implementation (high-level, NOT outline-level). Same as `acceptance-test-planning.md` §Phase 7. The subagent's RETURN TO ORCHESTRATOR is a compact summary derived from this block: ```json { "story": "UPEX-100", "quality_assessment": "Needs Improvement", "ambiguity_count": 4, "gap_count": 3, "inferred_edge_cases": 7, "critical_po_questions": 2, "tech_questions": 3, "outline_count_by_type": { "positive": 4, "negative": 5, "boundary": 2, "integration": 1 }, "data_feasibility_risk": false, "refinement_file": ".context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-UPEX-100-<slug>/shift-left-refinement.md", "open_blockers": [] } ``` The orchestrator presents the per-Story summary to the user, waits for OK, then dispatches the next Story's Refinement subagent (or moves to Phase 3 if this was the last Story). --- ## What the Refinement subagent must NOT do - **No Jira mutations.** All updates land in Phase 3. - **No test execution.** No smoke, no DB queries, no API calls. Refinement is doc-only. - **No parametrization tables, no test-data JSON, no Faker recipes.** Deferred to in-sprint planning. - **No TC creation.** TCs are formalized in Stage 4 (`/test-documentation`). - **No git operations.** No branch, no commit. - **No new ATP / ATR Jira issues.** Phase 3 publishes the ATP to the `{{jira.acceptance_test_plan}}` field only — nobody in this skill creates TMS items. The Test Plan issue is created later by `/sprint-testing` Stage 1 from the field content. - **No subtask mutations.** The `[QA] Shift-Left Review` tracking subtask is owned by the orchestrator (Phase 1: find-or-create → In Progress; Phase 3: annotations + Done) — the Refinement subagent never touches it. - **No `evidence/` folder.** Feature does not exist yet. - **No "approval from user" mid-refinement.** Subagents do not prompt the user — they finish and return. The orchestrator presents and waits. --- ## Gotchas 1. **Light code reads.** Reproduction-grade exploration is for `/sprint-testing` Stage 2. Here, "is this feasible in the current codebase" is the bar. 2. **PO/Dev confirmation marker.** Any inferred scenario MUST carry `NEEDS PO/DEV CONFIRMATION` — verbatim, in English, in the local file and (later) in the Jira comment. 3. **Quality via technique, not via minimization.** Outline count = whatever the technique triggers yield (EP partitions + BVA boundaries + state transitions + decision-table rules), not a number picked to "look thorough" OR to "stay lean". Padding (an outline exploring nothing new) and under-derivation (a range with no boundary outline, a status field with no invalid-transition outline) are BOTH failures. A 4-AC Story with two ranges and a state machine will legitimately exceed 6 outlines. 4. **Cite, do not duplicate.** `acceptance-test-planning.md` is the source of truth for Phases 1-5 mechanics. If a future change to that file affects shift-left, the change propagates automatically. 5. **Coverage estimate matters.** PO uses the per-Type counts to estimate Story points. Always include the table even if some Types are 0. 6. **Module-context reuse.** If `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/module-context.md` exists (module = Epic, 1:1), read it and skip module-level code exploration. Story-level reads only. 7. **Output language**: artifact + Jira-bound content in English. Mirror user's language only in conversation. AGENTS.md §1 Rule #14. --- ## Checklist before returning to the orchestrator - [ ] PBI folder bootstrapped (created or detected); minimal `context.md` written if missing - [ ] `shift-left-refinement.md` written with sections Phase 1 / Phase 2 / Phase 3 / Phase 4 (outline names only) / Phase 5 (outline) / Story Quality Assessment / Critical Questions / Tech Questions / Suggested Story Improvements / Data feasibility flags / Recommended testing strategy - [ ] All inferred scenarios + edge cases carry `NEEDS PO/DEV CONFIRMATION` - [ ] Phase 4 coverage estimate table included; per-outline test-data JSON NOT included - [ ] Phase 5 edge-case names included; test-data generation strategy NOT included - [ ] No Jira mutations performed - [ ] Compact JSON summary returned to the orchestrator -
refinement-questions.md 16.3 KB
# Refinement Question Catalog — Gap-Spotting Checklist by AC Archetype > **Subagent context**: cited by the Phase 2 Refinement subagent (see `refinement-playbook.md` §Step 3 — Story Quality Analysis) as a checklist. Walk the archetype list(s) that match the Story; the catalog surfaces the questions in-sprint testers usually only discover when execution fails. A Story usually maps to one or more **archetypes** (auth, money, search, state machine, list/table, integration, notification, permissions). For each archetype, this catalog provides a question set the refinement subagent must mentally walk through. Questions that have no answer in the Story description, ACs, or Team Discussion become PO / Dev open questions in the refinement output. Use this catalog as a **rubric**, not a script. If the Story is already explicit on a topic, do not re-ask it. --- ## How to use 1. Read the Story title + description + ACs. Identify the archetype(s) involved (often 2-3 per Story). 2. For each matched archetype, walk the question list. 3. Any question without an explicit answer in the Story = candidate gap. 4. Classify the gap: - **Critical** -> blocks sprint planning (write into Phase 2 Critical Questions for PO) - **Important** -> blocks implementation (Technical Questions for Dev) - **Edge** -> testable but not blocking (Edge Cases not in Story, with NEEDS PO/DEV CONFIRMATION) 5. If a question's answer can be inferred from `.context/business/*` or module-context, cite the source and skip the PO ask. 6. **Map each archetype to the formal technique it implies, then derive outlines by that technique** (canon: `agentic-qa-core/references/test-design-doctrine.md` Part 2). A gap is not just a PO question — it is also a missing outline. | Archetype | Formal technique it triggers | Outlines to derive | |---|---|---| | State Machine (order, subscription, KYC, dispute, document lifecycle) | **State-Transition** | one per valid transition + one per *invalid* transition (SM3 / SM10) | | Money / Billing, Search/Filter ranges, List CRUD limits | **Boundary Value Analysis** | `min-1·min·min+1 … max-1·max·max+1`, zero / empty / null | | Permissions / RBAC, Auth (role × state × flag) | **Decision Table** | one per surviving rule of the condition matrix | | Integration / multi-factor configs (provider × locale × plan) | **Pairwise** | all-pairs set; log the reduction | | Universal (U5 idempotency, U-series retries/concurrency) | **Error Guessing charter** | double-submit, retry, race, timeout, rollback | --- ## Universal questions (apply to every Story) | # | Question | Common failure mode if unanswered | |---|----------|-----------------------------------| | U1 | What is the EXACT error message text + status code for each failure path? | QA writes "shows error" assertion that passes on the wrong error | | U2 | Which user role / permission level is required? | Auth bypass discovered in production | | U3 | What happens on success — UI redirect target, toast text, page reload? | Test passes on "no error" but UI never updated | | U4 | What is the source of truth for state — server response, local state, URL param? | Race condition between FE state and BE state in production | | U5 | Is the operation idempotent? What if it is triggered twice (double-click, retry)? | Duplicate charges / duplicate records | | U6 | What is the audit / logging requirement (who, what, when)? | Compliance gap discovered post-release | | U7 | What is the rollback behavior on partial failure (e.g. 3 of 5 records fail)? | Half-applied mutations leave the system inconsistent | | U8 | What devices / browsers / viewports are in scope? | Mobile Safari breaks because nobody tested it | | U9 | What is the expected behavior in offline / network-error state? | App hangs forever on flaky connection | | U10 | Is there a "first-time" / empty / zero-data state distinct from steady state? | Onboarding flow shows scary empty table | --- ## Archetype: Authentication / Authorization / Session | # | Question | Why critical | |---|----------|--------------| | A1 | Are there role / permission gates on each AC? Specify the role per AC. | Authorization escalation | | A2 | What is the session timeout policy? How is timeout surfaced to the user? | Silent session loss | | A3 | Are tokens short-lived (JWT) or session-based (cookie)? What is the refresh flow? | Refresh-token leaks / forced logouts | | A4 | What is the password complexity / rotation / lockout policy? | Account takeover | | A5 | Is 2FA / MFA in scope? Backup-code flow defined? | User locked out forever | | A6 | What is the "logout" surface — does it clear server-side session + local state + 3rd-party cookies? | Stale session attacks | | A7 | Are there concurrent-session rules (one device only, multi-device)? | Account-sharing abuse | | A8 | What is the response on unauthenticated access — redirect, 401, 403, 404? | Information disclosure (403 vs 404 reveals existence) | | A9 | Are there CSRF / CORS requirements specific to this endpoint? | CSRF vulnerability | | A10 | What is the rate-limit + lockout threshold on failed attempts? | Credential stuffing | --- ## Archetype: Money / Billing / Payment | # | Question | Why critical | |---|----------|--------------| | M1 | What is the currency, precision (cents? mills?), rounding rule? | Off-by-one cent across millions of transactions | | M2 | Is partial refund supported? Maximum refundable amount? Time window? | Refund abuse / accounting drift | | M3 | What is the failure mode when the payment provider times out — do we retry, surface error, queue? | Double charges or silent loss | | M4 | What is the reconciliation / audit log requirement? Who can view it? | Compliance audit failure | | M5 | Are taxes / fees / discounts applied before or after the displayed amount? Where is the rule defined? | Customer disputes over invoice math | | M6 | What happens when the user's card is declined mid-flow — saved state, abandoned cart, retry path? | Lost revenue + customer rage | | M7 | Currency conversion: which rate, which provider, locked at quote time or transaction time? | FX exposure | | M8 | Refunds vs voids vs cancellations — semantics in this Story? | Wrong financial bucket | | M9 | Subscription proration on plan change / cancellation? | Customer billed for entire month after downgrade | | M10 | What is the chargeback / dispute flow downstream of this Story? | Disputed transactions never reach the right team | --- ## Archetype: Search / Filter / List / Table | # | Question | Why critical | |---|----------|--------------| | S1 | What is the default sort? Is it user-overridable? Persisted across sessions? | Inconsistent default surprises users | | S2 | Pagination model — offset, cursor, infinite scroll? What is the page size? | Skip / duplicate records on insert during pagination | | S3 | Search semantics — exact match, fuzzy, full-text, prefix? Case-sensitive? | "User can't find anything they typed correctly" | | S4 | What is the empty-state UI when 0 results — distinct from "still loading"? | Users assume "still loading" forever | | S5 | Are search results filtered by user permissions? What if 0 of N records are visible to me? | Data leak via search OR confused user | | S6 | Sorting on a nullable field — nulls first or last? | "My missing-date records disappeared" | | S7 | Filter combinations — AND or OR? Per-field operator selectable? | Wrong record set returned | | S8 | Bulk-select scope — current page only or entire result set? | Accidental bulk-delete of records the user never saw | | S9 | Export / download scope — current view, all filtered, raw all? Format? Limits? | Large export crashes browser | | S10 | Performance budget — max query time, max rows returned, server-side vs client-side filter? | Slow query crashes prod | --- ## Archetype: State Machine (order, subscription, KYC, dispute, document lifecycle) | # | Question | Why critical | |---|----------|--------------| | SM1 | What are ALL the valid states + ALL valid transitions? Diagram / table reference? | Hidden state surfaces in production | | SM2 | Which transitions are user-driven vs system-driven (cron, webhook, time-based)? | Race between user action and background job | | SM3 | What is the behavior on attempted invalid transition (e.g. cancel a delivered order)? | Quiet 200 OK + no state change leaves user confused | | SM4 | Are there terminal states (no further transitions)? Can users still view / search them? | Closed records vanish or remain mutable | | SM5 | Is there a "rollback" / "reopen" transition? Auth gate? | Support unable to fix customer error | | SM6 | What triggers each transition — explicit user action, API event, timer, external webhook? | Trigger source not testable in isolation | | SM7 | What is the time-to-live in each state before auto-transition (timeout, escalation)? | Records orphaned in intermediate state | | SM8 | Audit log per transition — actor, timestamp, reason? | Cannot reconstruct what happened | | SM9 | Notifications fired per transition — to whom, via which channel? | User missed key state change | | SM10 | Concurrent transitions — what if two actors transition simultaneously? Lock / queue / last-write-wins? | Data corruption under load | --- ## Archetype: External Integration (Stripe, Auth0, Salesforce, partner API) | # | Question | Why critical | |---|----------|--------------| | I1 | Which integration SDK / version? Sandbox + production credentials in `.env`? | Wrong env credentials hit prod | | I2 | What is the contract version pinned (OpenAPI spec, schema hash)? | Silent contract drift | | I3 | Synchronous (request/response) or asynchronous (webhook / queue)? | Different failure modes per pattern | | I4 | What is the timeout + retry policy? Backoff strategy? Max attempts? | Resilience or thundering herd | | I5 | How is webhook signature verified? Replay-attack protection? | Webhook spoofing | | I6 | Idempotency-key strategy — who generates, where stored, TTL? | Duplicate operations on retry | | I7 | What happens if the partner is down — degraded mode, queue + retry, hard error to user? | Cascading outage | | I8 | Rate limits on the partner side — current usage vs quota? | Hitting rate-limit blocks all users | | I9 | PII / sensitive-data scope sent to the partner — is the contract minimized? | Data leak via 3rd party | | I10 | Versioning + deprecation policy — how do we know the partner is breaking change? | Production breakage on partner upgrade | --- ## Archetype: List/Table CRUD (item create / update / delete) | # | Question | Why critical | |---|----------|--------------| | C1 | Is delete soft (flag) or hard (row gone)? Permission to undelete? | Accidental data loss | | C2 | Required vs optional fields — server-side validation enforced regardless of UI? | Bypass via API | | C3 | Maximum field lengths + character set (unicode, emoji, RTL)? | Truncation / encoding bugs | | C4 | Uniqueness constraints — case-sensitive? Trimmed whitespace? | Two users register with same email + space | | C5 | Foreign-key cascading on delete (orphan children / null-out / block)? | Orphan rows OR cascading delete surprises | | C6 | Audit history — every edit logged, viewable by whom? | Compliance gap | | C7 | Optimistic concurrency — last-write-wins, version field, conflict UI? | Lost updates | | C8 | Bulk operations — all-or-nothing or partial-success? | Half-applied bulk import | | C9 | Import / export — CSV format, header row, encoding, max size? | Excel auto-converts ID to date | | C10 | Visibility / sharing — who else sees this record? Default sharing? | Privacy leak | --- ## Archetype: Notification / Email / Push / In-app | # | Question | Why critical | |---|----------|--------------| | N1 | Which channels for this notification — email, push, in-app, SMS? Per user preference? | User missed critical alert | | N2 | Template language — i18n keys defined? Fallback locale? | English template sent to non-English user | | N3 | Delivery guarantees — at-most-once, at-least-once, exactly-once? Dedupe key? | Inbox spam from retries | | N4 | Rate limits per user (digest vs immediate)? | Notification fatigue | | N5 | Unsubscribe / preference center scope — per category or all-or-nothing? | CAN-SPAM / GDPR violation | | N6 | Failure handling — bounce, suppression list, retry queue? | Permanent bounces consume quota | | N7 | Sender reputation / SPF / DKIM checks in scope? | Email lands in spam folder | | N8 | Time-of-day rules / quiet hours by user timezone? | 3 AM push wakes the user up | | N9 | Personalization fields — what data is interpolated? Validation of interpolation? | "Hi {{firstName}}" sent literally | | N10 | Audit — was a notification sent? Who can see this log? | Customer disputes ("I never got that") | --- ## Archetype: Permissions / Roles / RBAC | # | Question | Why critical | |---|----------|--------------| | P1 | Per-AC role matrix — who can see, who can edit, who can delete? | Privilege escalation | | P2 | Owner-vs-other rules — can user only act on own records or any in scope? | Cross-tenant leak | | P3 | Org / tenant boundary — is the action allowed across boundaries? | Multi-tenant data leak | | P4 | Permission inheritance — does parent role inherit child permissions? | Forgotten permission grants | | P5 | Default permission for new role? Migration path on role rename? | Production silently broken on role-system change | | P6 | UI affordance vs server enforcement — both required? | Button hidden in UI but API allows | | P7 | Permission change propagation — immediate or session-based? | Revoked user can act until logout | | P8 | Audit of permission grant / revoke? | SOX / SOC2 finding | | P9 | Service-account / API-key permissions distinct from user permissions? | Bot exceeds intended scope | | P10 | Permission delegation / impersonation supported? Audit thereof? | Insider abuse | --- ## Archetype: Performance / Scale | # | Question | Why critical | |---|----------|--------------| | PF1 | What is the expected p95 / p99 latency budget? | Slow tail kills UX | | PF2 | What is the expected throughput (req/s, records/s)? | Load test scope missing | | PF3 | What is the expected dataset size in production for this entity? | Test data is 100 rows; prod is 100M | | PF4 | Caching strategy — server, CDN, browser? Invalidation rules? | Stale data shown forever | | PF5 | Async / background jobs — queue used, retry, dead-letter? | Failed jobs disappear | | PF6 | Database index implications on new query? | Sequential scan in prod | | PF7 | N+1 query risk? Eager-load strategy? | Latency cliffs | | PF8 | Memory / CPU budget per request? | OOM under load | | PF9 | Cold-start vs warm budgets (serverless)? | First-request latency | | PF10 | Observability — metrics + logs + traces emitted for this endpoint? | Cannot diagnose prod incident | --- ## Archetype: Accessibility / i18n | # | Question | Why critical | |---|----------|--------------| | AX1 | WCAG level target — A / AA / AAA? Per-AC? | Compliance audit failure | | AX2 | Keyboard-only navigation path defined per interactive element? | Keyboard users blocked | | AX3 | Screen-reader labels + ARIA roles defined? | Blind users blocked | | AX4 | Color contrast minimum (4.5:1 text, 3:1 large)? | WCAG failure | | AX5 | Locale fallback chain (en-GB -> en -> default)? | Missing translation lands in wrong language | | AX6 | RTL support? Layout mirroring? | Arabic / Hebrew users broken | | AX7 | Date / number / currency format per locale? | "1,000.00" vs "1.000,00" confusion | | AX8 | Timezone handling — server UTC, user local, both displayed? | Calendar conflicts | | AX9 | Plural / gender / context-aware translation? | "1 item" / "2 items" / Polish 5-form plurals | | AX10 | Translation length expansion handling (German +30%)? | UI overflow | --- ## Gotchas 1. **Archetype overlap is normal.** Most Stories touch 2-3 archetypes. Walk each; dedupe questions. 2. **Don't ask for project-wide answers.** If `business-data-map.md` already defines the currency / state machine, cite it. PO has finite patience. 3. **Critical vs Edge classification matters.** PO triages by criticality. Putting a U3 question (success path text) into "Critical Questions for PO" alongside an A1 (auth gate) signals lack of judgment. 4. **Universal questions are not auto-included.** Walk them ONLY when the Story doesn't already answer them. The Refinement file is high-signal, not exhaustive. 5. **This catalog is evergreen.** Add new archetypes when projects discover them (`/sync-ai-memory` workflow may surface candidates).
-
-
SKILL.md 52 KB
--- name: shift-left-testing description: "Orchestrates pre-sprint Shift-Left QA on a batch of backlog Stories. Use when the user wants to refine acceptance criteria, surface ambiguities + gaps, draft an ATP outline, and hand off to PO/Dev BEFORE the Story enters a sprint — so defects are prevented in the requirements, not detected after implementation. Triggers on: shift-left testing, shift-left these stories, groom the backlog, pre-sprint QA, refine these N stories, pre-sprint refinement batch, prepare backlog for sprint planning, run AC refinement on UPEX-100/101/102, run shift-left QA, do early-game testing, pre-sprint test planning. ALSO trigger when the user pastes a comma-separated list of Story IDs sitting in Backlog / Shift-Left QA / Estimation / Ready For Dev and asks any variant of \"refine\", \"groom\", \"clean these ACs\", \"shift-left these\". Do NOT use for: in-sprint manual QA per ticket (use /sprint-testing — entry status is Ready For QA, this skill's entry status is Backlog/Shift-Left QA), Stage 4 TMS documentation + ROI (test-documentation), Stage 5 automation code (test-automation), Stage 6 regression suite execution (regression-testing), bugs (this skill only accepts Stories — bugs are reactive and have no upstream ACs to refine), epic-level test strategy (use feature-test-planning inside /sprint-testing for that)." license: MIT compatibility: [claude-code, copilot, cursor, codex, opencode] complementary_categories: [testing-e2e, issue-tracker, tms] --- ## Inputs Read in order; stop earlier when the batch is small enough that later inputs add no signal. 1. `.context/business/business-feature-map.md` + `.context/business/business-data-map.md` + `.context/business/business-api-map.md` — domain vocabulary, entity model, CRUD matrix, auth model + endpoint contracts. Anchors refined ACs in real entities, flows, and API behavior. (All three are hard-required by the Readiness Preflight Gate + Phase 0.3.) 2. `.context/master-test-plan.md` — regression Epic + in-scope modules. Tells the refinement whether the Story falls inside an already-prioritized area. 3. The Story's Acceptance Criteria + `**Source spec:**` reference on Jira. Detailed read via `bun run jira:sync-issues get <STORY_KEY> --include-comments`, then read the synced `acceptance-criteria.md` (+ description). NEVER `acli view` for custom fields. Canonical input — every refined AC must trace back here. 4. `.context/PBI/epics/EPIC-<KEY>-<slug>/stories/STORY-<KEY>-<slug>/` if a PBI folder already exists for this Story (created by a prior `/sprint-testing` cycle). Carries earlier session notes worth honoring. 5. `.agents/jira-workflows.json` — Story workflow + valid transitions (`backlog -> shift_left_qa -> estimation`). Source of `{{jira.transition.story.*}}` slugs used in Phase 3. ALSO the resolver for the `[QA] Shift-Left Review` tracking subtask: read it to confirm a subtask work type exists (+ its transitions) before Phase 1 creates any subtask; if the catalog has no subtask work type, the subtask steps are skipped with a warning — never blocked on. 6. `.agents/jira-required.yaml` — canonical slug catalog. Source of `{{jira.acceptance_test_plan}}` and other Jira field slugs touched in handoff. --- ## Forbidden invocations **NEVER invoke `/sdd-*` skills from this workflow.** SDD is an optional user-installed ceremony; this skill ships self-contained and does not chain SDD under any condition. If you need to refactor KATA, fixtures, cli/, scripts/, or api/schemas/ pipeline, exit this skill first and invoke `/framework-development`. This boundary is mechanical, not advisory: `scripts/lint-skills.ts` rejects any `/sdd-` mention outside this section. See: `.agents/skills/agentic-qa-core/references/skill-composition-strategy.md` §4 (governs users who manually install SDD). # Shift-Left Testing — Pre-Sprint AC Refinement on a Backlog Batch Drive Stage 0 — the pre-sprint Shift-Left loop — on a set of backlog Stories. Three phases, always in this order: **Phase 1 Selection -> Phase 2 Refinement -> Phase 3 Handoff**. Hand off afterwards to `/sprint-testing` once each Story reaches `Ready For QA`. The skill is **batch-by-design**: one session refines N Stories from the backlog so PO + Dev lead can run a single grooming pass with the team. There is no single-issue mode — for a one-off urgent refinement, pass a list of length 1; the cadence stays the same. --- ## Why this skill exists (separation from `/sprint-testing`) | Eje | `/sprint-testing` | `/shift-left-testing` | |-----|-------------------|----------------------| | Cadence | In-sprint, ticket-by-ticket loop | Pre-sprint, batch grooming of N Stories | | Entry status | `{{jira.status.story.ready_for_qa}}` | `{{jira.status.story.backlog}}` / `shift_left_qa` / `estimation` / `ready_for_dev` | | Exit status | `{{jira.status.story.qa_approved}}` (full execution) | `{{jira.status.story.estimation}}` (refined, awaiting estimate by PO + Dev) | | Audience | Dev + tester | PO / BA + tester (Dev lead optional) | | Output | ATP + ATR + bugs + execution evidence | Refined ACs + risk map + pre-sprint ATP in the Story field (outlines only) + `[QA] Shift-Left Review` subtask + batch report | | Execution | Smoke + UI / API / DB exploration | NONE — feature does not exist yet | | Code reads | Deep, targeted (reproduce / verify) | Light (feasibility only — does the codebase support this?) | | TC creation | Yes (TCs created in Stage 1) | No — Stage 4 (`test-documentation`) creates TCs after the Story ships | | Sprint-testing later | Runs full pipeline | Short-circuits Phases 1-3 (label `shift-left-reviewed` detected) and just validates | The reuse story is **deliberate**: ~70% of the refinement logic already lives in `sprint-testing/references/acceptance-test-planning.md` Phases 1-3. This skill cites that reference instead of duplicating it — see Phase 2 below. --- ## Dependencies Requires `agentic-qa-core`. Loads on demand: - `agentic-qa-core/references/test-design-doctrine.md` — **MANDATORY before refining ACs or estimating outline coverage.** A refinement that does not surface risk-beyond-AC and 1:N coverage is incomplete. - `agentic-qa-core/references/defect-management-doctrine.md` — **MANDATORY for the QA-Assignee hook (Part 2).** This skill is the EARLIEST QA pickup of a backlog Story: when a QA takes a Story into Shift-Left refinement, set `qa_assignee` to the authenticated session user (self) — read-before-write, NEVER overwrite an existing owner except on explicit, justified handover. This skill still files NO Bug/Defect/Improvement (Phase 1 rejects non-Story types); only the QA-Assignee semantics of Part 2 apply here. - `agentic-qa-core/references/briefing-template.md`, `agentic-qa-core/references/dispatch-patterns.md`, `agentic-qa-core/references/orchestration-doctrine.md`, `agentic-qa-core/references/session-management.md`, `agentic-qa-core/references/preflight-gate.md` — cited inline by the sections that use them. ## Compact Rules **Test-design doctrine (binding — full canon: `agentic-qa-core/references/test-design-doctrine.md`):** - ACs are the FLOOR. Refinement's job is to push past the happy-path contract: surface the boundaries, exceptions, states, and anomalies the Story is silent on. - 1:N is the default: a non-trivial AC implies multiple outlines (valid partition + each distinct invalid + boundaries + states). A 1-outline AC requires a written "trivially atomic" justification — never the default. - Tag each refinement gap to a technique: ranges/limits → BVA; status/lifecycle fields → State-Transition; 2+ interacting conditions → Decision Table; 3+ combinable factors → Pairwise. - A refined AC (Given/When/Then) is the business assertion; the outline (`Should <behavior> <condition>`) is its exploration. Keep them distinct. **Shift-left operational rules:** - Stories ONLY (no bugs — nothing to refine upstream). Entry status Backlog / Shift-Left QA / Estimation / Ready For Dev. - Output = refined ACs + gap/ambiguity questions + the pre-sprint ATP in the `{{jira.acceptance_test_plan}}` field (outline NAMES + coverage estimate, no test code, no execution, NO Test Plan item — `/sprint-testing` Stage 1 creates the item from the field) + the closed `[QA] Shift-Left Review` subtask + the batch report. - Tracking subtask `[QA] Shift-Left Review` per accepted Story: find-or-create in Phase 1 (assignee = self; Jira's `create` lands it in `{{jira.status.subtask.active}}`), close in Phase 3 handoff via `{{jira.transition.subtask.complete}}` (-> `{{jira.status.subtask.close}}`). The subtask workflow's status NAMES are `ACTIVE` / `Close`, not "In Progress" / "Done". Exhaustive session annotations (long analysis, refinement traces) go on the SUBTASK, keeping the Story clean. Work type + transitions resolved from `.agents/jira-workflows.json`; no subtask work type in the catalog → skip with a warning, never block. - The heart of the skill (Phase 2) = edge cases not in story + ambiguities + gaps — feed them to PO/Dev as questions AND as derived outlines. - On taking a Story into refinement (first QA pickup), set `qa_assignee` to self — read-before-write, never overwrite an existing owner (`agentic-qa-core/references/defect-management-doctrine.md` Part 2). This skill files NO Bug/Defect/Improvement; only the QA-Assignee hook applies. - On completion: add label `shift-left-reviewed`; transition Backlog → Shift-Left QA → Estimation. **Read full SKILL.md when**: running the batch grooming pipeline, writing the per-Story `shift-left-refinement.md`, or handling the PO/Dev handoff. --- ## Subagent Dispatch Strategy > **Orchestration & Session contracts**: this skill follows `agentic-qa-core/references/orchestration-doctrine.md` (mandatory subagent dispatch — main thread is command center) AND `agentic-qa-core/references/session-management.md` (Phase 0 resume check, plan-first persistence at `.session/<skill-slug>/<scope>/`, archive on completion). Phase 0 (resume check) and Phase 1 (plan write) are NOT optional. The orchestrator also applies the per-stage **Definition-of-Done gates** in `agentic-qa-core/references/stage-gates.md`: verify a stage's DoD (planning stages include the Test-Design Checklist) BEFORE recording its progress checkpoint and advancing. This skill is **per-batch scope**: `<scope>` = `<YYYY-MM-DD>-<descriptor>` (e.g. `2026-05-20-payments-area`). Session state lives at `.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/{plan.md, progress.md}` per `agentic-qa-core/references/session-management.md` §3 + §9. The per-Story `shift-left-refinement.md` files stay under each Story's PBI folder (`[LOCAL]` staging buffer for Phase 2 → Phase 3 — disposable once Phase 3 publishes to Jira; see `references/atp-outline-template.md` and `.context/PBI/README.md`). This skill is compliant with the doctrine in `AGENTS.md` §"Orchestration Mode (Subagent Strategy)" and the session contract in `.agents/skills/agentic-qa-core/references/session-management.md`. Every dispatch follows the 7-component briefing format defined in `.agents/skills/agentic-qa-core/references/briefing-template.md`, and the pattern selected per phase matches the decision guide in `.agents/skills/agentic-qa-core/references/dispatch-patterns.md`. | Phase | Pattern | Subagent role | |-------|---------|---------------| | Phase 1 — Selection | Single | Backlog Selection subagent: pull candidate Stories via `[ISSUE_TRACKER_TOOL]`, apply veto + risk-score triage, return ranked candidate table. After user OK: find-or-create the `[QA] Shift-Left Review` subtask under each accepted Story with assignee = self, left at `{{jira.status.subtask.active}}` where `create` lands it (skip with warning if the catalog has no subtask work type) | | Phase 2 — Refinement (per Story) | Sequential — looped per Story | Refinement subagent: load `acceptance-test-planning.md` Phases 1-3 + outline-only Phase 4, write `shift-left-refinement.md`, append PO/Dev questions, return summary block. ONE subagent per Story. NEVER parallel across Stories (each subagent writes a different PBI file but the orchestrator must present each summary to the user sequentially before the next dispatch) | | Phase 3 — Handoff (per Story) | Sequential — looped per Story | Handoff subagent: update Jira description + `{{jira.acceptance_test_plan}}` custom field (both modalities — no Test Plan item pre-sprint) + handoff comment + labels + subtask annotations + subtask transition to Done + Story transition `backlog -> shift_left_qa -> estimation`. Returns transition log + trace verification | | Phase 3 — Batch report | Single | Batch Report subagent: aggregate per-Story summaries into `.session/shift-left-testing/<batch-id>/batch-report.md` + post to parent epic if Stories share one | > **Sequential by design**. Phase 2 refinement looks parallelizable (each Story is independent in Jira), but the orchestrator must present each Story's refinement summary to the user before moving on. This keeps the user in the loop, lets them veto a Story mid-batch, and matches the team-grooming cadence the skill is designed for. Parallelism would burn the user's attention budget. > **On any subagent failure**: STOP, report the partial state (which Stories refined, which Jira mutations landed), present retry / skip-story / abort options. Do NOT auto-fix nor auto-rollback. Jira mutations are recorded in the batch report so partial sessions are resumable. See `.agents/skills/agentic-qa-core/references/orchestration-doctrine.md`. --- ## Fleet seam (optional) Refinement is **sequential by design** (above) and stays that way by default. A fleet — one persistent worker session per Story, a conductor collecting their reports — is available only when the user explicitly asks for it on a large batch. Then: - **Topology: same checkout.** This skill writes nothing but Jira: no code, no branch, no git index to contend on. A worktree per Story would be pure cost. - The conductor writes `launch.txt` in `.session/shift-left-testing/<batch-id>/` — one self-contained line per Story — **always**, whether or not any orchestration transport exists on the machine. Launching, supervising and closing those sessions is `orca-orchestration/SKILL.md` (`[ORCHESTRATION_TOOL]`): supervised launch is the native path, and `launch.txt` is the payload for the human-paste fallback when nothing can launch it. - **The per-Story user OK does not disappear, it moves**: the conductor presents each worker's refinement summary as it lands and waits for the same approve / veto decision before that Story's Phase 3 handoff runs. A fleet that skips this is not this skill running faster, it is this skill not running. - **One writer per Story.** Two workers never refine the same Story, and every Jira write for a Story belongs to that Story's worker. The batch report and the epic comment stay with the conductor. - **Silence rule**: the absence of an orchestration transport is never named to the user, never appears in the preflight gate, and never appears in the batch report. --- ## Workflow — one pipeline, three phases ``` Phase 0 — Session resume check + Session Init (always first) -> Check .session/shift-left-testing/<batch-id>/progress.md → offer resume / restart / abort -> Resolve TMS modality (A: Xray / B: Jira-native — recorded for Stage 1; the pre-sprint ATP write is field-first in BOTH modalities) -> Load /acli (no /xray-cli — this skill creates no TMS items) -> Verify project-wide context files -> Resolve candidate list (explicit IDs OR backlog JQL) -> Create session folder .session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/ (writes plan.md after candidate list confirmed; progress.md appended per phase) Phase 1 — Selection -> Detailed-read each candidate via `bun run jira:sync-issues get <STORY> --include-comments` (batch: `jql "<backlog JQL>"`), then read the synced .md -> Reject non-Story types (Bug / Spike / Sub-task / Tech-debt) -> Apply veto + risk-score triage per candidate -> Present ranked candidate table -> WAIT for user OK -> Per accepted Story: find-or-create subtask "[QA] Shift-Left Review" -> assignee = self; stays at {{jira.status.subtask.active}} (skip + warn if no subtask work type in the catalog) Phase 2 — Refinement (loop per accepted Story) -> Dispatch Refinement subagent: produce shift-left-refinement.md (Critical Analysis + Story Quality Analysis + Refined ACs + ATP outlines) -> Reuses sprint-testing/references/acceptance-test-planning.md §Phases 1-3 with shift-left-mode delta (no test-data generation, no parametrization tables, outline names only) -> Present per-Story summary -> WAIT for user OK before next Story Phase 3 — Handoff -> Per Story sequentially: - Update Jira description with "QA Refinements (Shift-Left Analysis)" - Populate the ATP field {{jira.acceptance_test_plan}} (both modalities; fallback: "## Acceptance Test Plan (ATP)" comment when the field is absent. NO Test Plan item — /sprint-testing Stage 1 creates it from the field) - Labels: shift-left-reviewed + shift-left-{YYYY-MM-DD} - Transition: backlog -> shift_left_qa (analyze) -> estimation (estimate) - Subtask "[QA] Shift-Left Review": post session annotations -> complete -> close -> Light stage verifier (artifact-lifecycle.md §5) closes the stage - Verify trace -> Batch report posted to .session/shift-left-testing/<batch-id>/batch-report.md + posted as comment on parent epic if Stories share one -> Archive: orchestrator moves .session/shift-left-testing/<batch-id>/ to .session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/ per agentic-qa-core/references/session-management.md §8 ---> Cross-skill handoff (NOT this skill): When each Story later reaches Ready For QA: /sprint-testing reads label `shift-left-reviewed` and short-circuits Phases 1-3 to validation-only (sprint-testing/references/acceptance-test-planning.md §Phase 0). Stage 1 ALSO creates the Test Plan item (`ATP: {STORY-KEY}: {title}`) FROM the {{jira.acceptance_test_plan}} field content — the item is in-sprint work, never pre-sprint. ``` --- ## Readiness Preflight Gate (MANDATORY — runs before Phase 0) > Full doctrine: `agentic-qa-core/references/preflight-gate.md`. Runs FIRST, before the resume check. Two laws: (1) **args-as-answers** — treat anything the user already stated (the Story IDs, the modality, "groom the backlog") as provided args; ask only real gaps. (2) **probe, don't assume**. Surface gaps + REDs as ONE `AskUserQuestion` checklist; self-fix with approval + explanation; STOP on any blocking RED. This skill does NO live execution (no env/DB/API/browser), so its gate is light — it is mostly a tooling + context readiness check. **Generic baseline** (env resolution, test-user creds, secret/restart handling, the two laws, output contract) is inherited from the reference §3.1 — not repeated here. Below is only this skill's **specific capability delta**. | Capability | Need | Why here | |---|---|---| | Issue-tracker (`[ISSUE_TRACKER_TOOL]`) | REQUIRED | All refinement output lands on Jira (description, ATP field, comment, labels, transitions). Load `/acli`; validate setup via `bun run jira:check`. | | TMS modality resolved | REQUIRED | Recorded in `plan.md` and carried into the handoff so `/sprint-testing` Stage 1 knows which engine will materialize the Test Plan item later. The pre-sprint ATP write itself is modality-independent — field-first in both. 4-step probe; ask only if all auto-checks fail. | | `/xray-cli` + `XRAY_*` creds | NOT NEEDED | Shift-Left creates no TMS items in either modality. The pre-sprint ATP lives in the `{{jira.acceptance_test_plan}}` field (fallback: comment); the Test Plan item is created by `/sprint-testing` Stage 1 from the field content. | | Business context files | REQUIRED | `.context/business/*` + `.context/master-test-plan.md` — refinement without them produces low-value questions. Missing → hand off to `/project-discovery`. | | Candidate Story list | REQUIRED | Explicit IDs (args) or a backlog JQL. Confirm size with the user before Phase 1. | Env reachability, test-user creds, DBHub, OpenAPI/`API_TOKEN`, Playwright and `resend` are **N/A** here — shift-left never executes against a running system. After the gate clears (all REQUIRED GREEN), continue to Phase 0 below. --- ## Phase 0 — Session resume check + Session Init 0.0 **Session resume check** (per `agentic-qa-core/references/session-management.md` §4). Compute `<batch-id>` = `<YYYY-MM-DD>-<descriptor>` from the invocation context. Check `.session/shift-left-testing/<batch-id>/progress.md`. If it exists, read `plan.md` + the tail of `progress.md`, surface the last completed phase + next planned phase + any blocking notes, and offer **resume / restart / abort**. On `restart`, archive the current directory to `.session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>-aborted/` before proceeding. On `abort`, stop here. 0.1 **Resolve TMS modality**. Same 4-step probe as `sprint-testing` Session Start (`test-documentation/SKILL.md` §Phase 0). Persist the result in `.session/shift-left-testing/<batch-id>/plan.md` (under the `## Inputs` H2 — the plan.md is the canonical record per session-management §6). 0.2 **Load required tool skills**: - Always load `/acli` (custom-field update, comment, transition, label, subtask create — all writes; plus the trivial key+summary+status candidate search). Story DETAIL reads (description, ACs, scope, comments, parent epic) go through `bun run jira:sync-issues get/jql` — NOT `acli view`. - `/xray-cli` is NOT loaded by this skill. Shift-Left creates no TMS items in either modality: the pre-sprint ATP lives in the `{{jira.acceptance_test_plan}}` custom field (fallback: the `## Acceptance Test Plan (ATP)` comment when the field is absent). The Test Plan ITEM is created by `/sprint-testing` Stage 1 from the field content, once PO has estimated and the Story enters the sprint. - Both modalities: `/acli` alone covers every write this skill performs. This step is **mandatory before any pseudocode block below executes**. The skills carry the concrete syntax, flags, and JSON payloads this skill intentionally omits. 0.3 **Verify project-wide context files exist**: - `.context/business/business-data-map.md` - `.context/business/business-feature-map.md` - `.context/business/business-api-map.md` - `.context/master-test-plan.md` If any of these is missing, STOP and hand off to `project-discovery` (or the individual `/business-*-map` and `/master-test-plan` commands). Shift-left refinement without business context produces low-value PO/Dev questions and bloats the batch report. 0.4 **Resolve the candidate Story list**. Two modes: - **Explicit IDs** — user passes `UPEX-100,101,102,103` (or any natural-language list of Story keys). Use these verbatim; no JQL. - **Backlog JQL** — user says "groom the backlog" with no IDs. Build a JQL via `[ISSUE_TRACKER_TOOL]` filtering on: - `project = {{PROJECT_KEY}}` - `issueType = Story` - `status in ({{jira.status.story.backlog}}, {{jira.status.story.shift_left_qa}}, {{jira.status.story.estimation}}, {{jira.status.story.ready_for_dev}})` - Optionally `sprint in openSprints()` if the user says "next sprint candidates" - Sort by Priority DESC, then Created DESC - Confirm the resolved list size with the user before Phase 1 starts. A batch of 1-12 Stories is the practical sweet spot; >12 should be split into multiple sessions. 0.5 **Create the session folder + write `plan.md`**: ``` .session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/ plan.md # session-management.md §6 schema — Goal, Inputs, Approach, # Phase breakdown, Risks, Verification checklist, Cross-references. # Inputs includes TMS modality + candidate list. progress.md # append-only, one entry per phase (§7 schema) candidates.md # Phase 1 output (domain artifact) batch-report.md # Phase 3 final output (domain artifact) # Per-Story refinement files live under each Story's own PBI folder, # NOT inside the session folder: # .context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/shift-left-refinement.md ``` The `<descriptor>` is kebab-case (e.g. `morning`, `payments-area`) and lets two sessions on the same day stay independent. After this step, append the first entry to `progress.md`: `## Phase 0 — Session Init — <ISO-8601 UTC>` with `status: completed`, `next: Phase 1 — Selection`. Subsequent phases follow the same shape per `agentic-qa-core/references/session-management.md` §7. --- ## Phase 1 — Selection Decides which Stories actually enter the refinement loop and at what depth. 1. For each candidate ID, detailed-read via `bun run jira:sync-issues get <STORY_KEY> --include-comments` (or batch `bun run jira:sync-issues jql "<backlog JQL>"`) and read the synced `.md` (title, description, ACs, priority, type, labels, sprint, parent epic, comments). NEVER `acli view` — it returns `null` for custom fields. `acli search` is fine for the trivial key+summary+status candidate list only. 2. **Type filter (hard)**: reject anything where `issueType != Story`. Surface the rejected list to the user with a one-line reason. Do NOT silently drop them. 3. **Label filter**: any Story already carrying `shift-left-reviewed` AND a dated label `shift-left-{YYYY-MM-DD}` less than 30 days old is treated as **already refined** — surface it separately under "Already Shift-Left Reviewed (skip or refresh?)". The user decides per-Story whether to skip or re-refine. (Freshness comes from the dated label, never from the issue's `updated` timestamp — any comment or rank change resets `updated`; see `references/backlog-selection.md` §Step 2.) 4. **Triage per accepted candidate** (veto + risk score). Read `references/backlog-selection.md` for the full rubric. Outcomes: - **VETO -> SKIP**: pure CSS / docs / static copy / tech-debt with no behavior change -> drop from refinement set, log reason. - **REQUIRE FULL**: money / data integrity / auth / external integration / state machine / calculation -> force refinement regardless of score. - **Score 0-3 LOW** -> SKIP (PO/Dev can write ACs directly without QA refinement). - **Score 4-7 MEDIUM** -> Full refinement (standard). - **Score 8+ HIGH** -> Full refinement + extended ambiguity / edge-case scan. 5. **Present the ranked candidate table** (see `references/backlog-selection.md` §Output format) and **WAIT for user OK** before Phase 2. Same pattern as sprint-testing's Story Explanation gate. 6. **Tracking subtask per accepted Story** (after user OK): find-or-create a subtask titled `[QA] Shift-Left Review` under the Story, **with `assignee` = the authenticated session user** (`agentic-qa-core/references/artifact-lifecycle.md` §2 — an unassigned QA artifact is a blocker waiting to happen). Jira's `create` transition lands it in `{{jira.status.subtask.active}}`; leave it there, Phase 3 closes it. Resolve the subtask work type from `.agents/jira-workflows.json`; if the catalog has no subtask work type (or the project disallows subtasks), log a warning in `progress.md` + the batch report and SKIP — never block the batch. Find-or-create: match the Story's existing subtasks by exact title before creating; an existing one already at `{{jira.status.subtask.close}}` is re-opened with `{{jira.transition.subtask.reactive}}` (refresh run). This makes QA's pre-sprint work visible on the board, and the subtask later receives the exhaustive session annotations in Phase 3. Persist the accepted list into `plan.md` §Inputs so a resumed session reads the same canonical decision. After user OK, append a progress entry: `## Phase 1 — Selection — <ts>` with `status: completed`, `artifacts_touched: [candidates.md, plan.md]`, `next: Phase 2 — Refinement`. --- ## Phase 2 — Refinement (per Story) For each accepted Story, dispatch ONE Refinement subagent. The subagent loads the existing in-skill reference and applies a shift-left-mode delta. **Reuse contract**: the subagent reads `.agents/skills/sprint-testing/references/acceptance-test-planning.md` §Phases 1-3 + Phase 4 (outline names only). The delta for shift-left mode: | acceptance-test-planning.md Phase | Shift-Left adaptation | |-----------------------------------|----------------------| | Phase 0 — Triage | Already done in this skill's Phase 1. Skip. | | Phase 1 — Critical Analysis | Run as-is. Light code exploration only (feasibility check, not reproduction). | | Phase 2 — Story Quality Analysis | Run as-is. **This is the heart of shift-left** — ambiguities + gaps + edge cases not in story + testability validation. | | Phase 3 — Refined ACs | Run as-is — Given/When/Then with specific data. Mark inferred scenarios with **NEEDS PO/DEV CONFIRMATION**. | | Phase 4 — Test Design (outlines) | **OUTLINE NAMES ONLY**. No parametrization tables. No exhaustive per-outline test-data JSON. Coverage estimate (Positive / Negative / Boundary / Integration counts) IS included — it informs PO estimation. | | Phase 5 — Edge case + Test-data summary | **Edge-case names + criticality only**. No data generation strategy, no Faker recipes — feature does not exist yet. | | Phase 6 — Traceability + Ticket updates | Phase 3 of THIS skill owns this. Refinement subagent only WRITES the local file; Handoff subagent does Jira mutations. | | Phase 7 — Final QA Feedback Report | Per-Story summary returned to orchestrator. Aggregated into the batch report in Phase 3. | | Phase 8 — Commit | **SKIPPED**. Jira is canonical. No git branch, no commit. | **Staging file**: `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/shift-left-refinement.md` (module = Epic, 1:1). Author it locally; it is NOT a Jira mirror, so the hand-write ban does not apply to it. **It is a buffer, not a deliverable.** Phase 2 writes it, Phase 3 publishes its full body to the Jira `acceptance_test_plan` field. After that, Jira holds the canonical copy and the synced `acceptance-test-plan.md` is the readable one. The staging file lives under `.context/PBI/**`, which is gitignored, so it exists only on the machine that ran the batch. Two consequences that are easy to get wrong: - **Nothing downstream may depend on the staging file being on disk.** `/sprint-testing` Stage 1 short-circuits off the SYNCED `acceptance-test-plan.md`, never off `shift-left-refinement.md` — otherwise the short-circuit silently degrades to a full re-run on any other machine. - **There is ONE ATP per Story.** This skill authors it early into the `{{jira.acceptance_test_plan}}` field; `/sprint-testing` Stage 1 creates the Test Plan ITEM from that field content and refines the same ATP into the executable superset. No `(Shift-Left DRAFT)` variant, no second Test Plan to reconcile, no pre-sprint Test Plan issue at all. **Folder bootstrap**: if `.context/PBI/epics/EPIC-<EPIC_KEY>-<slug>/stories/STORY-<STORY_KEY>-<slug>/` does not exist yet (Story has not been through sprint-testing), the refinement subagent creates it. Jira-mirrored content (story.md, acceptance-criteria.md, parent epic, comments) comes from `bun run jira:sync-issues get <STORY_KEY> --include-comments` — NEVER hand-write those files. The only hand-authored files here are the NON-Jira working artifacts (`shift-left-refinement.md`, `context.md` with local session notes). This mirrors `sprint-testing/references/session-entry-points.md` §Step 7. The `evidence/` subfolder is NOT created — there is nothing to capture yet. **Story Explanation step**: replaced by the per-Story summary the orchestrator presents AFTER the subagent returns. The user OKs (or vetoes) each Story before the next refinement dispatch. This matches the "explain story -> WAIT for OK" rhythm in sprint-testing. **Progress checkpoint per Story**: after each Refinement subagent returns AND the user OKs the summary, the orchestrator appends a phase entry to `.session/shift-left-testing/<batch-id>/progress.md` per `agentic-qa-core/references/session-management.md` §7: `## Phase 2.<n> — Refine <STORY_KEY> — <ts>` with `status: completed`, `artifacts_touched: [.context/PBI/.../shift-left-refinement.md]`, `next: Phase 2.<n+1> | Phase 3`. This lets a mid-batch resume skip already-refined Stories. After Phase 2 finishes the full accepted list, the per-Story summaries feed Phase 3. --- ## Phase 3 — Handoff For each refined Story, dispatch a Handoff subagent. Sequential, one Story at a time, so the user can review the post-handoff Jira state before the next mutation. ### Per-Story handoff sequence > **Prerequisite**: Phase 0.2 already loaded `/acli`. Pseudocode below uses `[ISSUE_TRACKER_TOOL]` only — this skill creates no TMS items, so `[TMS_TOOL]` never fires pre-sprint. ``` 1. Write the refined ACs to the Jira acceptance_criteria field, then append the supporting analysis. Jira is source of truth — local Jira-mirrored .md files are read-only caches generated by the sync, never hand-written. [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} fields: {{jira.acceptance_criteria}}: <refined ACs from Phase 2> # FALLBACK (field absent): post as a structured comment headed # "## Acceptance Criteria" per .agents/jira-required.yaml fallback: key. Never block. Then append the "QA Refinements (Shift-Left Analysis)" supporting section to the Story description: - Edge Cases Identified (from Phase 2) - Clarified Business Rules (from Phase 2) - Open Questions for PO / Dev (from Phase 2) After writing, run `bun run jira:sync-issues get {STORY_KEY} --include-comments` and read back the synced `acceptance-criteria.md` to confirm the field landed. 2. Populate the ATP — field-first, IDENTICAL in both modalities. ONE ATP per Story, authored early: there is no separate DRAFT item and no `(Shift-Left DRAFT)` title. Pre-sprint the ATP's home is the `{{jira.acceptance_test_plan}}` custom field; the Test Plan ITEM is created by `/sprint-testing` Stage 1 FROM this field content once the Story enters the sprint. Full rationale + mutation sequence: `references/handoff-protocol.md` Step 2. [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} fields: {{jira.acceptance_test_plan}}: <full shift-left-refinement.md body> # FALLBACK (field absent on this instance): skip this write — step 3's comment # carries the full body inline per .agents/jira-required.yaml fallback. Never block. 3. Handoff notification on the Story (the ATP lives in {{jira.acceptance_test_plan}} — do NOT mirror it; inline the full body as a `## Acceptance Test Plan (ATP)` comment ONLY if that field is absent — fallback per jira-required.yaml): [ISSUE_TRACKER_TOOL] Add Comment: issue: {STORY_KEY} body: | ## Acceptance Test Plan (ATP) — ready for pre-sprint review The ATP lives in the {{jira.acceptance_test_plan}} field. # FALLBACK ONLY (field absent): replace the pointer line above with the full staged refinement body. 4. Labels: [ISSUE_TRACKER_TOOL] Update Issue: issue: {STORY_KEY} labels: +shift-left-reviewed, +shift-left-{{YYYY-MM-DD}} 5. Set QA Assignee + Transition (Story must be currently in backlog / shift_left_qa / estimation): # First QA pickup of the Story — set qa_assignee to the authenticated session # user (self), the same moment the QA pulls it into Shift-Left refinement # (backlog -> shift_left_qa, the earliest pickup). Read-before-write: # set ONLY when empty; NEVER overwrite an existing QA owner except on explicit, # justified handover. Per defect-management-doctrine.md Part 2. NOTE: acli # `workitem edit` CANNOT set customfields → use REST PUT /rest/api/3/issue/{KEY} # with { fields: { {{jira.qa_assignee}}: { accountId } } } (doctrine Part 6). [ISSUE_TRACKER_TOOL] Set qa_assignee = <authenticated session user> # read-before-write; skip if already owned # If currently in backlog: [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.analyze}} # backlog -> shift_left_qa [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.story.estimate}} # shift_left_qa -> estimation # If already in shift_left_qa or estimation, advance only the missing leg. # NEVER advance beyond estimation — PO/Dev lead estimates and moves to ready_for_dev. 6. Close the tracking subtask (created in Phase 1; skip with warning if it was skipped there): # Exhaustive session annotations (long analysis, refinement traces — anything too # verbose for the Story) go on the SUBTASK, keeping the Story clean. [ISSUE_TRACKER_TOOL] Add Comment / Update description: issue: {SUBTASK_KEY} # "[QA] Shift-Left Review" under {STORY_KEY} body: <exhaustive session annotations> [ISSUE_TRACKER_TOOL] Transition: {{jira.transition.subtask.complete}} # active -> close # Unmapped slug -> artifact-lifecycle.md §4 fallback (ask, never skip silently) 7. Verify trace (both modalities — field-first, no Test Plan item to trace pre-sprint): `bun run jira:sync-issues get {STORY_KEY} --include-comments`, then read back the synced acceptance-test-plan field file + handoff comment; confirm the field is populated and the comment points to it (full body in the comment ONLY in fallback mode — field absent); confirm the subtask (when created) is at {{jira.status.subtask.close}}. 8. Run the **light stage verifier** (`agentic-qa-core/references/artifact-lifecycle.md` §5) — the stage-specific lines are in `references/handoff-protocol.md` §Step 6b. ``` The Handoff subagent returns a per-Story log: `{story: KEY, atp_container: <field|fallback_comment>, subtask: <done|skipped>, labels_added: [...], transitions: [...], trace_status: ok|warning|fail}`. ### Batch report + Archive After all Stories handed off, dispatch ONE Batch Report subagent (`Single` pattern) to aggregate: ``` .session/shift-left-testing/<batch-id>/batch-report.md ``` Contents (see `references/handoff-protocol.md` §Batch report template): - Session metadata (date, mode, candidate count, accepted count, rejected count) - Per-Story line: ID, title, risk level, # gaps, # critical questions, transition status - Aggregated top PO/Dev open questions (deduped across Stories) - Risk distribution chart (LOW / MEDIUM / HIGH counts) - Blockers (Stories that surfaced data feasibility gaps — flagged for PO before sprint planning) - Recommended sprint-planning order (by risk + dependency) - Cross-skill pointer: "When each Story reaches Ready For QA, run `/sprint-testing` — it will short-circuit Phases 1-3 thanks to the `shift-left-reviewed` label." If all Stories in the batch share a single parent epic, ALSO post the batch report as a comment on that epic. Otherwise, deliver inline to the user as the session-closing message. After the batch report lands, append the final progress entry `## Phase 3 — Handoff + Batch report — <ts>` with `status: completed`, `next: stop`, then run **Archive** per `agentic-qa-core/references/session-management.md` §8: move `.session/shift-left-testing/<batch-id>/` to `.session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/` (two-file dir preserved) and call `mem_session_summary` with the session template + archive path. --- ## Gotchas — inline rules to apply every invocation 1. **Credentials always from `.env`.** Never hardcode. Same as sprint-testing. 2. **Stories only.** Bugs / Spikes / Sub-tasks / Tech-debt are rejected in Phase 1. Bugs are reactive — no upstream ACs to refine. If the user really wants to run shift-left on a Tech-debt with behavior changes, ask them to convert it to a Story first. 3. **Veto beats risk score.** Same rule as `acceptance-test-planning.md` §Phase 0. Money / data integrity / auth / external integrations / state machines / calculations -> FORCE Full refinement regardless of score. 4. **Already-reviewed Stories** (label `shift-left-reviewed` <30 days old) are NOT auto-skipped. Surface them and let the user pick: skip / refresh. 5. **Outline names only.** Phase 2 produces outline TITLES + brief preconditions per outline. NO parametrization tables, NO per-outline test-data JSON, NO Faker recipes. Those belong to in-sprint planning (`/sprint-testing`) or Stage 4 (`/test-documentation`). 6. **NEEDS PO/DEV CONFIRMATION**. Any AC or edge case the refinement infers (not literally in the original story) is flagged with this marker. The flag appears verbatim in the Jira description + comment + custom field, so PO sees it during sprint planning. 7. **No execution.** This skill does not run smoke, does not query DBs for data presence, does not run the app. Feasibility is established by READING code + APIs + DB schema only. 8. **Sequential Phase 2.** One refinement subagent at a time, even if the user is impatient. Parallelism would prevent per-Story user OK and break the grooming cadence. 9. **Transition guardrail.** Never advance beyond `{{jira.status.story.estimation}}`. PO/Dev lead owns `estimate -> ready_for_dev`. If a Story is already past `estimation` when the session starts, log a warning and SKIP the transition step — refinement still lands on Jira, but the workflow stays untouched. 10. **Label hygiene.** Always add BOTH `shift-left-reviewed` AND `shift-left-{{YYYY-MM-DD}}`. The dated label lets `/sprint-testing` decide whether the refinement is still fresh (<30 days) and short-circuit, or whether to redo Phases 1-3. 11. **Jira is canonical.** No git commit, no test branch. Local `shift-left-refinement.md` is a working artifact — gitignored under `.context/PBI/**`. The populated `{{jira.acceptance_test_plan}}` field (or its `## Acceptance Test Plan (ATP)` fallback comment when the field is absent) is the contract `fix-traceability` checks later. 12. **Language**: artifacts + Jira content always English. Mirror the user's language only in conversation (per AGENTS.md §1 Rule #14). 13. **Session-footer contract (mandatory at close).** The final phase is not done until the two chat-facing blocks from `../agentic-qa-core/references/session-footer-contract.md` are printed: (1) consolidated screenshot list — repo-relative paths, verified on disk, bug annotations first — plus in-flow surfacing of every capture's path the instant it lands; (2) Session Footer listing skills/MCPs/CLIs actually used + testing levels touched, with explicit "none" entries for expected-but-untouched levels. Framing for this skill: execution. Multi-subagent sessions: each stage report carries the five footer fields (`skills_loaded`, `mcps_used`, `clis_used`, `testing_levels_touched`, `screenshots_captured`); the orchestrator compiles the footer ONCE at close. Chat only — never in a Jira comment or ATR body. 14. **Subtask tracking is best-effort.** The `[QA] Shift-Left Review` subtask makes QA's pre-sprint work visible on the board and holds the exhaustive session annotations that would otherwise clutter the Story. If `.agents/jira-workflows.json` has no subtask work type (or the project disallows subtasks), warn once in the batch report and proceed — never block a refinement on subtask support. --- ## Anti-patterns — NEVER do these **L1.** NEVER force ambiguity questions onto a Story to fill a checklist — raise PO/Dev questions ONLY when a genuine gap, ambiguity, or untestable AC exists. Per AGENTS.md §1 Rule #4: shift-left adds value by surfacing real risk, not by inflating question counts. A clean Story exits with an empty question list and that is a valid outcome. **L2.** NEVER skip the `shift-left-reviewed` label when transitioning a Story out of Phase 3. `/sprint-testing` Phase 0 inspects that label to short-circuit Phases 1-3 of in-sprint planning; missing the label forces redundant work later and breaks the cadence this skill exists to enable. **L3.** NEVER mix Story refinement with bug retest in the same batch. `/shift-left-testing` accepts Stories only (Phase 1 type filter is a hard reject). Bugs are reactive — they have no upstream ACs to refine and belong to `/sprint-testing` instead. **L4.** NEVER hand-write the ATP body as raw ADF JSON. Author the body in Markdown locally (`shift-left-refinement.md`) and let `[ISSUE_TRACKER_TOOL]` convert via its md-to-ADF path on update. Hand-rolled ADF drifts from the field content that `fix-traceability` later validates. **L5.** NEVER transition a Story to `estimation` without a populated ATP (the `{{jira.acceptance_test_plan}}` custom field in BOTH modalities; the `## Acceptance Test Plan (ATP)` fallback comment when the field is absent). The pre-sprint ATP is what makes the Story estimable — without it, Dev and PO guess scope and the shift-left effort delivers no signal. **L8.** NEVER create the Test Plan item pre-sprint. The pre-sprint ATP's only home is the `{{jira.acceptance_test_plan}}` field (or its fallback comment); `/sprint-testing` Stage 1 creates the Test Plan issue FROM that field content once the Story enters the sprint. A pre-sprint item wastes an artifact on a Story whose scope may still shrink and creates a second copy Stage 1 must reconcile. **L6.** NEVER refine more than ~10-12 Stories in a single batch. Refinement quality degrades past that — user attention budget collapses, summaries blur, the batch report loses signal. Split larger groomings into multiple sessions with distinct `<descriptor>` values. **L7.** NEVER add a PO/Dev question that the AC body already answers in plain text. The reader's bandwidth is the scarcest resource in a grooming session; redundant questions train the team to skim future shift-left output. --- ## Cross-skill handoff — what this skill does NOT do | After Phase 3 you need... | Load this skill / command | Reason | |---------------------------|---------------------------|--------| | Wait for Dev to estimate + commit the Story into a sprint | (manual / PO) | This skill stops at `{{jira.status.story.estimation}}`. PO + Dev lead drive `estimate -> ready_for_dev` and sprint commitment. | | In-sprint manual QA once the Story reaches `Ready For QA` | `/sprint-testing` | Will detect label `shift-left-reviewed`, validate the refinement is still fresh, short-circuit Phases 1-3, and run Phases 4-8 + Stages 2 + 3 normally. Stage 1 also creates the Test Plan ITEM from the `{{jira.acceptance_test_plan}}` field this skill populated. | | Formal TC creation + ROI scoring after Story ships | `/test-documentation` | Stage 4 turns the outlines + refined ACs into formal Xray TCs (Modality jira-xray) or Jira Test issues (Modality jira-native) with ROI scoring. | | Automated test code | `/test-automation` | Stage 5. | | Regression suite execution | `/regression-testing` | Stage 6. | | Generate / refresh business + master test plan context | `/project-discovery` + `/business-*-map` + `/master-test-plan` | This skill consumes those; it does not create them. | | Adversarial dual-review of the refinement (optional) | `/judgment-day` | Useful when shift-left output goes to a high-risk Story. Not auto-invoked. | If Phase 0.3 reports any project-wide context file missing, STOP and hand off — refinement without business context produces vague PO questions and dilutes the batch report. --- ## Pseudocode tags used here | Tag | Resolves to | Defined in | |-----|-------------|------------| | `[ISSUE_TRACKER_TOOL]` | `acli`, Atlassian MCP, or `{{ISSUE_TRACKER_CLI}}` | `AGENTS.md` Tool Resolution | | `[TMS_TOOL]` | xray-cli skill (Modality jira-xray) OR `acli` (Modality jira-native) | `AGENTS.md` Tool Resolution | | `[ORCHESTRATION_TOOL]` | the multi-session orchestration CLI (fleet seam only) | `orca-orchestration/SKILL.md` | > **Reads vs writes split** (per `agentic-qa-core/references/acli-integration.md` §"Reads vs writes"): detailed reads (description, ACs, scope, comments, parent epic) → `bun run jira:sync-issues get/jql`, then read the synced `.md`. Writes (custom-field update, comment, transition, label, link) + the trivial key+summary+status candidate list → `acli`. NEVER `acli view` for a custom field. | `[DB_TOOL]` | DBHub MCP or Supabase MCP | `AGENTS.md` Tool Resolution | | `[API_TOOL]` | OpenAPI MCP, Postman, or curl | `AGENTS.md` Tool Resolution | Concrete tools (`bun`, `git`, `gh`) used literally. Project variables resolve from `.agents/project.yaml` (env-scoped vars resolve to the active environment). Jira variables (`{{jira.status.story.*}}`, `{{jira.transition.story.*}}`, `{{jira.acceptance_test_plan}}`) resolve from `.agents/jira-workflows.json` + `.agents/jira-fields.json`. --- ## References — read the narrow one for the situation All references are self-contained. Load one at a time. | Reference | Read when | |-----------|-----------| | `references/backlog-selection.md` | Phase 0.4 + Phase 1 — building the candidate JQL, applying veto + risk-score triage per candidate, formatting the candidate table for user approval. | | `references/refinement-playbook.md` | Phase 2 — running the per-Story refinement subagent. Cites `acceptance-test-planning.md` Phases 1-3 + outline-only Phase 4. Documents the shift-left deltas (no parametrization, no test-data gen, outline names only). | | `references/atp-outline-template.md` | Phase 2 — body skeleton for `shift-left-refinement.md` (the pre-sprint ATP at outline maturity). Different from sprint-testing's full ATP body. | | `references/refinement-questions.md` | Phase 2 — catalog of typical PO / Dev / Design gap-spotting questions, grouped by AC archetype (auth, money, search, state machine, etc.). Use as a checklist when the Story is sparse. | | `references/handoff-protocol.md` | Phase 3 — exact Jira mutation sequence per Story (field-first ATP write, subtask close), label + transition rules, batch report template + epic-comment posting rules. | | `../agentic-qa-core/references/artifact-lifecycle.md` | Before any transition — the Story + subtask lifecycle rows (§1), assignee-at-create (§2), the unmapped-status fallback protocol (§4), the light stage verifier template (§5). | | `../agentic-qa-core/references/session-management.md` | Phase 0 + Phase 4 — resume contract, plan.md/progress.md schemas, archive policy, Engram per-phase checkpoint. This skill is a producer of `session/shift-left-testing/<batch-id>/...` topic keys. | --- ## Pre-flight checklist - [ ] Session resume check ran (Phase 0.0); user chose resume / restart / abort if prior state existed - [ ] TMS modality resolved + persisted to `plan.md` §Inputs - [ ] `/acli` loaded (no `/xray-cli` — this skill creates no TMS items) - [ ] Project-wide context files present (else handed off to `/project-discovery`) - [ ] Candidate Story list resolved (explicit IDs or backlog JQL) + confirmed with user - [ ] Session folder `.session/shift-left-testing/<YYYY-MM-DD>-<descriptor>/` created with `plan.md` written - [ ] Phase 1 produced the ranked candidate table, user OK'd the refinement set - [ ] Phase 1 found-or-created the `[QA] Shift-Left Review` subtask per accepted Story, assignee = self, at `{{jira.status.subtask.active}}` (or skipped with warning — no subtask work type) - [ ] Phase 2 ran ONE refinement subagent per accepted Story, user OK'd each summary - [ ] Per-Story `shift-left-refinement.md` written under each Story's PBI folder - [ ] Phase 3 handoff applied per Story: Jira description + ATP field (`{{jira.acceptance_test_plan}}`, both modalities) + handoff comment + labels + transition (stops at `estimation`) — NO Test Plan item created - [ ] Subtask closed per Story: session annotations posted on it + `{{jira.transition.subtask.complete}}` fired → `{{jira.status.subtask.close}}` (or skipped with warning) - [ ] Light stage verifier run per `agentic-qa-core/references/artifact-lifecycle.md` §5 — every line YES or a stated N/A - [ ] Trace verified per Story (both modalities: field populated + pointer comment; full body in comment only in fallback mode) - [ ] Batch report written + posted to parent epic (if applicable) - [ ] Archive: `.session/shift-left-testing/<batch-id>/` moved to `.session/.archive/<YYYY-MM-DD>-shift-left-testing-<batch-id>/` and `mem_session_summary` called - [ ] No git commit (Jira is canonical for this skill) - [ ] User informed: when each Story reaches `Ready For QA`, run `/sprint-testing` (will short-circuit thanks to `shift-left-reviewed`) - [ ] Session footer + consolidated screenshot list printed in chat per session-footer-contract (never in a Jira comment)
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.