Claude
Skill
pr-test-impact-analysis
Use this skill when you need to determine test impact from a pull request or code diff; triggers include PR test impact analysis.
Virus-scanned
Reviewed automatically before listing.
Download
naodeng-awesome-qa-skills-skills_en_testing-types_pr-test-impact-analysis-c44b892.zip · 5 KB
Install
skills CLI
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/pr-test-impact-analysis
Claude Code
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
Git
git clone https://github.com/naodeng/awesome-qa-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
PR Test Impact Analysis
When to Use
- Use this skill when you need to derive the smallest sufficient test scope from a pull request's diff, dependencies, and historical risk.
- Use it to review an existing plan, result, or evidence set and produce actionable improvements.
- Use it when context is incomplete but a bounded first pass is still valuable.
Output Format Options
- Default to Markdown for review, execution, and incremental refinement.
- When the user requests tables, CSV, JSON, or ticket fields, preserve risk, evidence, priority, and boundary information.
- For machine-consumed output, confirm the schema, enums, and required fields first.
How to Use
- Read and follow
prompts/pr-test-impact-analysis.md, including its input contract, execution rules, minimum coverage, and output order. - Add only context that changes the decision: scope, environment, version, constraints, evidence, and success criteria.
- Audit the input, then separate confirmed facts, working assumptions, and open questions.
- Rank by risk and evidence strength, and produce an artifact that can be executed or reviewed directly.
- If information is missing, deliver a bounded first pass and state which conclusions remain unsupported.
Reference Files
- Always read
prompts/pr-test-impact-analysis.md; it is the complete execution specification for this skill. - For evaluation or regression, read
evals/eval.yamland the relevant cases underevals/cases/. - Load
references/,examples/,scripts/, oroutput-formats.mdonly when those directories exist and the task needs them.
Core Constraints
- do not infer only from filenames
- provide evidence for test selection
- expand scope when risk is high and evidence is weak
- Never invent system behavior, fields, data, metrics, or root causes absent from the evidence.
- Link important conclusions to evidence; mark unsupported conclusions as hypotheses with a verification method.
- Explain priority using business impact, likelihood, or detectability.
Delivery Checklist
- Covered: change intent, direct and transitive impact, public interfaces, data changes, configuration, security, rollback, test gaps.
- Separated facts, assumptions, gaps, and recommendations.
- Gave high-risk items a priority, evidence basis, owner or next action.
- Defined verifiable decision criteria instead of generic advice.
- Performed no unauthorized production writes or destructive actions.
Common Pitfalls
- Listing checks without preconditions, expected outcomes, or evidence.
- Marking everything high priority and avoiding tradeoffs.
- Substituting tool names or generic theory for domain reasoning.
- Refusing incomplete input, or pretending incomplete evidence supports certainty.
Best Practices
- Start with paths most likely to cause business loss, safety issues, or release blockage.
- Reduce uncertainty through the smallest verifiable experiment and record reproduction conditions.
- Make the artifact executable and independently reviewable by another engineer.
Files (awesome-qa-skills)
-
agents
-
openai.yaml 293 B
version: 1 metadata: key: pr-test-impact-analysis interface: display_name: "PR Test Impact Analysis" short_description: "Analyze test impact from a PR or diff" default_prompt: "Use the pr-test-impact-analysis skill to analyze PR test impact." policy: allow_implicit_invocation: true
-
-
evals
-
cases
-
basic-success.yaml 863 B
id: basic-success title: "PR Test Impact Analysis: domain-complete scenario" description: | Checks that the skill produces domain-specific, evidence-based, prioritized, executable output. input: prompt: | Use the pr-test-impact-analysis skill for this scenario: a PR changes a discount calculator, shared money type, and database migration, and removes two old tests. The basic scope is available. Provide risk priorities, core execution items, expected results or decision criteria, evidence, and open questions. expect: must_contain: - "Task Understanding" - "Input Audit" - "Risks" - "P0" - "expected" must_not_contain: - "TODO" - "I cannot" - "unable to help" judge: type: rule_based success: - output_contains: all: - "Task Understanding" - "Input Audit" - "Risks" -
edge-incomplete-input.yaml 904 B
id: edge-incomplete-input title: "PR Test Impact Analysis: bounded first pass with incomplete input" description: | Checks that missing context leads to a bounded first pass with assumptions and gaps, not refusal or invention. input: prompt: | Use the pr-test-impact-analysis skill. The only known fact is: a PR changes a discount calculator, shared money type, and database migration, and removes two old tests. No environment, version, or supporting evidence is available. Deliver a useful first pass and state which conclusions are unsupported. expect: must_contain: - "Working Assumptions" - "Open Questions" - "evidence" - "Next Actions" must_not_contain: - "TODO" - "I cannot" - "unable to help" judge: type: rule_based success: - output_contains: all: - "Working Assumptions" - "Open Questions" - "evidence" -
edge-risk-priority.yaml 863 B
id: edge-risk-priority title: "PR Test Impact Analysis: high-risk boundary and tradeoffs" description: | Checks that a constrained window produces risk focus plus stop, escalation, or human-handoff boundaries. input: prompt: | Use the pr-test-impact-analysis skill for: a PR changes a discount calculator, shared money type, and database migration, and removes two old tests. The execution window is half a day and no destructive production action is authorized. Bound the scope, prioritize P0/P1, and state residual risk plus stop or escalation conditions. expect: must_contain: - "P0" - "P1" - "Residual Risk" - "condition" must_not_contain: - "TODO" - "I cannot" - "unable to help" judge: type: rule_based success: - output_contains: all: - "P0" - "P1" - "Residual Risk"
-
-
eval.yaml 557 B
schema_version: v1alpha1 environment: type: none skills: - source: local_path path: . engine: name: claude_code # model is optional; omit to use engine default # model: # provider: anthropic # name: claude-sonnet-4-6 cases: files: - evals/cases/basic-success.yaml - evals/cases/edge-incomplete-input.yaml - evals/cases/edge-risk-priority.yaml defaults: timeout_seconds: 180 max_turns: 8 expect: exit_code: 0 must_not_contain: - "TODO" - "I cannot" report: formats: [json]
-
-
prompts
-
pr-test-impact-analysis.md 3.3 KB
# PR Test Impact Analysis Prompt Derive the smallest sufficient test scope from a pull request's diff, dependencies, and historical risk and produce an artifact that can be executed, reviewed, and tracked directly. ## Role You are a senior risk- and evidence-driven QA practitioner who controls conclusion boundaries when context is incomplete. ## Input Prefer real materials supplied by the user: - PR description - code diff - test mappings - dependency graph - defect history - CI results - scope, environment, version, time budget, toolchain, and prohibited actions - existing results, historical failures, monitoring evidence, and stakeholder concerns If critical input is absent, list `Working Assumptions` and `Open Questions`, then still deliver a bounded first pass. ## What to do 1. Restate the objective, subject, and success criteria in one sentence. 2. Audit input completeness, credibility, recency, and comparability. 3. Build a risk or failure model and prioritize high-impact, likely, or hard-to-detect issues. 4. Convert analysis into concrete scenarios, assertions, verification steps, or decision gates. 5. Report residual risk, evidence gaps, and next actions without presenting hypotheses as facts. ## Execution Rules - do not infer only from filenames - provide evidence for test selection - expand scope when risk is high and evidence is weak - Give an evidence basis for every important conclusion; label unsupported claims as `Hypothesis to Verify`. - Each scenario must include preconditions, action or stimulus, expected behavior, and required evidence. - Use P0/P1/P2/P3 or an equivalent scale and explain the ranking. - Reuse the current toolchain and assets; avoid large code samples unless the user requests them. - For production, security, or privacy work, default to least privilege, masked data, mocks, dry runs, or isolated environments. ## Minimum Coverage Checklist Unless the user narrows the scope, cover at least: - change intent - direct and transitive impact - public interfaces - data changes - configuration - security - rollback - test gaps - confirmed facts, working assumptions, and open questions - blockers for execution, release, or decision making - residual risk and how it will be accepted, mitigated, or investigated ## Output Use this order: ### 1. Task Understanding and Scope - objective, subject, success criteria, inclusions, and exclusions ### 2. Input Audit - confirmed facts, working assumptions, open questions, and evidence quality ### 3. Risks and Priorities - P0/P1/P2/P3, impact, rationale, and sequence ### 4. Core Analysis and Execution Items - PR summary - impact surface - required tests - optional tests - residual risk - review questions - include preconditions, steps, expected result or decision criterion, and evidence for each item ### 5. Blockers and Residual Risk - stop, escalation, rollback, or human-handoff conditions ### 6. Next Actions and Open Questions - smallest verification actions, suggested owners, and missing materials ## Quality Bar - Tailor the content to the input; do not merely rename a generic template. - Make high-risk paths concrete with failure modes, expected behavior, and evidence. - Never invent numbers, root causes, or system behavior. - Let an executor act without guessing and a reviewer trace every important judgment.
-
-
SKILL.md 3.2 KB
--- name: pr-test-impact-analysis description: Use this skill when you need to determine test impact from a pull request or code diff; triggers include PR test impact analysis. --- # PR Test Impact Analysis ## When to Use - Use this skill when you need to derive the smallest sufficient test scope from a pull request's diff, dependencies, and historical risk. - Use it to review an existing plan, result, or evidence set and produce actionable improvements. - Use it when context is incomplete but a bounded first pass is still valuable. ## Output Format Options - Default to Markdown for review, execution, and incremental refinement. - When the user requests tables, CSV, JSON, or ticket fields, preserve risk, evidence, priority, and boundary information. - For machine-consumed output, confirm the schema, enums, and required fields first. ## How to Use 1. Read and follow `prompts/pr-test-impact-analysis.md`, including its input contract, execution rules, minimum coverage, and output order. 2. Add only context that changes the decision: scope, environment, version, constraints, evidence, and success criteria. 3. Audit the input, then separate confirmed facts, working assumptions, and open questions. 4. Rank by risk and evidence strength, and produce an artifact that can be executed or reviewed directly. 5. If information is missing, deliver a bounded first pass and state which conclusions remain unsupported. ## Reference Files - Always read `prompts/pr-test-impact-analysis.md`; it is the complete execution specification for this skill. - For evaluation or regression, read `evals/eval.yaml` and the relevant cases under `evals/cases/`. - Load `references/`, `examples/`, `scripts/`, or `output-formats.md` only when those directories exist and the task needs them. ## Core Constraints - do not infer only from filenames - provide evidence for test selection - expand scope when risk is high and evidence is weak - Never invent system behavior, fields, data, metrics, or root causes absent from the evidence. - Link important conclusions to evidence; mark unsupported conclusions as hypotheses with a verification method. - Explain priority using business impact, likelihood, or detectability. ## Delivery Checklist - [ ] Covered: change intent, direct and transitive impact, public interfaces, data changes, configuration, security, rollback, test gaps. - [ ] Separated facts, assumptions, gaps, and recommendations. - [ ] Gave high-risk items a priority, evidence basis, owner or next action. - [ ] Defined verifiable decision criteria instead of generic advice. - [ ] Performed no unauthorized production writes or destructive actions. ## Common Pitfalls - Listing checks without preconditions, expected outcomes, or evidence. - Marking everything high priority and avoiding tradeoffs. - Substituting tool names or generic theory for domain reasoning. - Refusing incomplete input, or pretending incomplete evidence supports certainty. ## Best Practices - Start with paths most likely to cause business loss, safety issues, or release blockage. - Reduce uncertainty through the smallest verifiable experiment and record reproduction conditions. - Make the artifact executable and independently reviewable by another engineer.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.