Claude Skill

agent-failure-recovery-testing

Use this skill when you need evidence-bounded failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence; triggers include Agent 故障恢复 and Agent failure recovery.

LLM Mart · 0 points · 2 views 0 listing impressions 0 install-command copies
Download naodeng-awesome-qa-skills-skills_en_testing-types_agent-failure-recovery-testing-9df700b.zip · 12 KB

Install

skills CLI npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/agent-failure-recovery-testing
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
Git git clone https://github.com/naodeng/awesome-qa-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Agent Failure Recovery Testing

When to Use

  • Use this skill when you need evidence-bounded analysis, design, or validation preparation for failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence.
  • Use it to review an Agent, RAG, or LLM plan, result, or evidence set and produce actionable improvements.
  • Use it when context is incomplete but a bounded first pass with assumptions, gaps, and human-decision boundaries is still useful.

Output Format Options

  • Default to Markdown organized by domain risk, evidence state, priority, and boundary.
  • When the user requests tables, CSV, JSON, or ticket fields, preserve the same finding fields, evidence, and decision boundaries.
  • Before machine consumption, confirm the schema, enums, required fields, and evidence sources.

How to Use

  1. Read and follow prompts/agent-failure-recovery-testing.md, including its input audit, domain coverage, and output order.
  2. Extract scope, environment, version, time window, constraints, success criteria, and available evidence, with attention to failure classification, retry and fallback, escalation path, state consistency, user notice.
  3. Separate confirmed facts, evidence-backed inferences, candidate recommendations, and Human decisions before ranking by risk and evidence strength.
  4. Turn high-risk items into preconditions, steps, expected behavior or decision criteria, required evidence, and a validation method.
  5. When input is incomplete, deliver a bounded first pass, state unsupported conclusions, and never present static material as execution evidence.

Reference Files

  • When calibrating domain outcomes, valid controls, or evidence gaps, read references/domain-decision-example.md for sanitized input, per-item oracles, and reproducible calculations.

  • Always read prompts/agent-failure-recovery-testing.md; it is the complete execution specification for this skill.

  • For evaluation, read evals/eval.yaml and the matching cases under evals/cases/.

  • Load references/, examples/, scripts/, or output-formats.md only when those directories exist and the task needs them.

Domain Execution Essentials

  1. Build failure | confirmed write state | retryability | retry/compensate/escalate | recovered state | evidence from the project protocol. Separate transient, permanent, and indeterminate outcomes.
  2. A timeout does not mean no execution: query committed effects by operation_id/idempotency key before replay. Read back an existing result or retry safely; the same key must not produce a second business write.
  3. For partial success reconcile completed and remaining steps. Compensate only under the supplied protocol and authority, recording each outcome; failed compensation escalates.
  4. Stop unchanged retries for permanent errors and notify the owner. Compare business state, attempt count, user-visible outcome, and resources before/after recovery; missing logs leave recovery unverified.

Core Constraints

  • Keep the analysis focused on failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence; do not replace business owners or Human risk acceptance, exception approval, or safety decisions.
  • Never invent system behavior, fields, model outputs, data, thresholds, root causes, execution records, or pass claims.
  • Static design, plans, file presence, or a dry run retain their evidence state and cannot become proof of real execution.
  • When evidence is insufficient, use pending confirmation, blocked, unassessed, or NOT_SCORED and give the smallest validation method.
  • For user data, production, or safety work, use least privilege, masked data, mocks, dry runs, or isolation.

Delivery Checklist

  • Covered failure classification, retry and fallback, escalation path, state consistency, user notice, with source, evidence state, and validation method for each.
  • Separated facts, inferences, candidate recommendations, gaps, and Human decisions.
  • Gave high-risk items P0/P1/P2/P3 or an equivalent priority, owner role, and close condition.
  • Did not turn plans, static checks, or dry runs into test execution, all-passed, or safety-approved claims.
  • Stated residual risk, stop/escalation conditions, and next actions.

Common Pitfalls

  • Listing checks without triggers, expected concerns, owner roles, close conditions, and evidence.
  • Treating adjacent tests or model tools as a complete Agent failure recovery judgment.
  • Using unexplained numbers for false precision or writing correlation as causation.
  • Refusing incomplete input, or pretending that incomplete evidence is conclusive.

Best Practices

  • Start with paths most likely to cause user harm, business loss, or decision blockage.
  • Use the smallest verifiable experiment to reduce uncertainty and record conditions, versions, sources, and evidence.
  • Make the Skill independently installable, executable, and reviewable by another engineer.
Files (awesome-qa-skills)
  • agents
    • openai.yaml 350 B
      Loading…
  • evals
    • cases
      • basic-success.yaml 1.5 KB
        Loading…
      • domain-method-regression.yaml 1.4 KB
        Loading…
      • edge-incomplete-input.yaml 952 B
        Loading…
      • edge-scope-boundary.yaml 1012 B
        Loading…
    • fixtures
      • check-domain-decisions.sh 95 B
        Loading…
      • check_domain_decisions.py 3.7 KB
        Loading…
      • domain-oracle.json 129 B
        Loading…
    • eval.yaml 489 B
      Loading…
    • local-rules.json 149 B
      Loading…
    • trigger-prompts.csv 621 B · in bundle
  • prompts
    • agent-failure-recovery-testing.md 5.7 KB
      Loading…
  • references
    • domain-decision-example.md 1.5 KB
      Loading…
  • SKILL.md 5.5 KB
    ---
    name: agent-failure-recovery-testing
    description: Use this skill when you need evidence-bounded failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence; triggers include Agent 故障恢复 and Agent failure recovery.
    license: PolyForm-Noncommercial-1.0.0
    metadata:
      version: "1.0.0"
      compatibility: ""
      author: "Awesome QA Skills contributors"
      language: "en"
      category: "testing-types"
      source: "https://github.com/naodeng/awesome-qa-skills"
      license-url: "https://polyformproject.org/licenses/noncommercial/1.0.0"
    ---
    
    # Agent Failure Recovery Testing
    
    ## When to Use
    
    - Use this skill when you need evidence-bounded analysis, design, or validation preparation for failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence.
    - Use it to review an Agent, RAG, or LLM plan, result, or evidence set and produce actionable improvements.
    - Use it when context is incomplete but a bounded first pass with assumptions, gaps, and human-decision boundaries is still useful.
    
    ## Output Format Options
    
    - Default to Markdown organized by domain risk, evidence state, priority, and boundary.
    - When the user requests tables, CSV, JSON, or ticket fields, preserve the same finding fields, evidence, and decision boundaries.
    - Before machine consumption, confirm the schema, enums, required fields, and evidence sources.
    
    ## How to Use
    
    1. Read and follow `prompts/agent-failure-recovery-testing.md`, including its input audit, domain coverage, and output order.
    2. Extract scope, environment, version, time window, constraints, success criteria, and available evidence, with attention to failure classification, retry and fallback, escalation path, state consistency, user notice.
    3. Separate confirmed facts, evidence-backed inferences, candidate recommendations, and Human decisions before ranking by risk and evidence strength.
    4. Turn high-risk items into preconditions, steps, expected behavior or decision criteria, required evidence, and a validation method.
    5. When input is incomplete, deliver a bounded first pass, state unsupported conclusions, and never present static material as execution evidence.
    
    ## Reference Files
    
    - When calibrating domain outcomes, valid controls, or evidence gaps, read [references/domain-decision-example.md](references/domain-decision-example.md) for sanitized input, per-item oracles, and reproducible calculations.
    
    - Always read `prompts/agent-failure-recovery-testing.md`; it is the complete execution specification for this skill.
    - For evaluation, read `evals/eval.yaml` and the matching cases under `evals/cases/`.
    - Load `references/`, `examples/`, `scripts/`, or `output-formats.md` only when those directories exist and the task needs them.
    
    ## Domain Execution Essentials
    
    1. Build failure | confirmed write state | retryability | retry/compensate/escalate | recovered state | evidence from the project protocol. Separate transient, permanent, and indeterminate outcomes.
    2. A timeout does not mean no execution: query committed effects by operation_id/idempotency key before replay. Read back an existing result or retry safely; the same key must not produce a second business write.
    3. For partial success reconcile completed and remaining steps. Compensate only under the supplied protocol and authority, recording each outcome; failed compensation escalates.
    4. Stop unchanged retries for permanent errors and notify the owner. Compare business state, attempt count, user-visible outcome, and resources before/after recovery; missing logs leave recovery unverified.
    ## Core Constraints
    
    - Keep the analysis focused on failure classification, retry/fallback/escalation, state consistency, user notice, and recovery evidence; do not replace business owners or Human risk acceptance, exception approval, or safety decisions.
    - Never invent system behavior, fields, model outputs, data, thresholds, root causes, execution records, or pass claims.
    - Static design, plans, file presence, or a dry run retain their evidence state and cannot become proof of real execution.
    - When evidence is insufficient, use pending confirmation, blocked, unassessed, or NOT_SCORED and give the smallest validation method.
    - For user data, production, or safety work, use least privilege, masked data, mocks, dry runs, or isolation.
    
    ## Delivery Checklist
    
    - [ ] Covered failure classification, retry and fallback, escalation path, state consistency, user notice, with source, evidence state, and validation method for each.
    - [ ] Separated facts, inferences, candidate recommendations, gaps, and Human decisions.
    - [ ] Gave high-risk items P0/P1/P2/P3 or an equivalent priority, owner role, and close condition.
    - [ ] Did not turn plans, static checks, or dry runs into test execution, all-passed, or safety-approved claims.
    - [ ] Stated residual risk, stop/escalation conditions, and next actions.
    
    ## Common Pitfalls
    
    - Listing checks without triggers, expected concerns, owner roles, close conditions, and evidence.
    - Treating adjacent tests or model tools as a complete Agent failure recovery judgment.
    - Using unexplained numbers for false precision or writing correlation as causation.
    - Refusing incomplete input, or pretending that incomplete evidence is conclusive.
    
    ## Best Practices
    
    - Start with paths most likely to cause user harm, business loss, or decision blockage.
    - Use the smallest verifiable experiment to reduce uncertainty and record conditions, versions, sources, and evidence.
    - Make the Skill independently installable, executable, and reviewable by another engineer.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.