Claude Skill

automation-testing

Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing and test automation strategy.

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download naodeng-awesome-qa-skills-skills_en_testing-types_automation-testing-c44b892.zip · 23 KB
Part of naodeng/awesome-qa-skills — 97 skills

Install

skills CLI npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/automation-testing
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
Git git clone https://github.com/naodeng/awesome-qa-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.

README

Automation Testing

Skill Overview

Need help with automation testing in a real project context; Need an output that can be used directly for execution, review, or follow-up.

How to Use

  1. Open SKILL.md in this folder and confirm this skill fits your task.
  2. In your AI tool, call @skill automation-testing, then add your real project context and goal.
  3. If you need a specific output format (table, checklist, report), include it directly in your request.

One-Click Install Script

Run from the repository root:

macOS / Linux

bash ./scripts/install-skills-mac.sh --tool codex --lang en --skill automation-testing

Windows PowerShell

powershell -ExecutionPolicy Bypass -File .\scripts\install-skills-windows.ps1 -Tool codex -Lang en -Skill automation-testing

Skill manifest

Automation Testing (English)

Chinese version: See the corresponding Chinese skill.

When to Use

  • Need help with automation testing in a real project context.
  • Need an output that can be used directly for execution, review, or follow-up.

Workflow

  1. Read and follow the main prompt listed under Progressive disclosure (coverage, structure, quality bar).
  2. Add only project context that changes the result: scope, environment, constraints, risks, dependencies, expected deliverable.
  3. If input is incomplete, return a usable first draft and explicitly mark assumptions and gaps.
  4. Default to Markdown; switch formats only when the user asks.

Core Constraints

  • Prioritize by risk / business impact — do not treat everything equally.
  • Separate confirmed facts from current assumptions.
  • Do not invent endpoints, fields, environments, or root causes the user did not provide.
  • Keep output executable: concrete scenarios, clear priority, clear next steps.

Progressive Disclosure

  • Before producing output, read and follow prompts/automation-testing.md (minimum coverage, output structure, quality bar).
  • When Excel/CSV/JSON/Word is requested: read output-formats.md and honor the format.
  • When a ready-made template fits: use matching files under output-templates/.
  • For deep framework/troubleshoot/schema notes: read only the relevant file(s) under references/, do not load the whole directory.
  • For format conversion or helper checks: prefer existing scripts/ over reinventing.
  • For evaluating/regressing this skill: use evals/ with skill-up.

Pre-delivery Checklist

  • Followed the main prompt's output structure
  • Minimum coverage focus: scope and objective, automation candidates, manual-only areas, priority by risk and ROI, framework alignment, data and environment needs, flaky-test risks, CI or execution strategy, ... (details in main prompt)
  • Covered the minimum checklist, or explained omissions
  • High-risk items have explicit priority
  • Did not invent details the user did not provide
  • Assumptions and gaps are marked

Common Pitfalls

  • Do not pretend completeness when scope/context is missing.
  • Do not treat every item as equally important.
  • Do not skip assumptions and information gaps.
  • Do not dump generic theory unrelated to the current toolchain.
Files (awesome-qa-skills)
  • agents
    • openai.yaml 467 B
      version: 1
      metadata:
        key: "automation-testing"
      interface:
        display_name: "Automation Testing (English)"
        short_description: "Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing…"
        default_prompt: "Use the automation-testing skill to complete this testing task with structured outputs and practical next steps."
      policy:
        allow_implicit_invocation: true
      
  • evals
    • cases
      • basic-success.yaml 767 B
        id: basic-success
        title: "Automation: what first vs not yet"
        description: |
          Separate automate-first vs not-yet; include approach and guardrails.
        
        input:
          prompt: |
            Use automation-testing.
            Current: Playwright; CI can run smoke. Candidates: login, search, checkout, payment-callback checks, admin report export.
            Goal: stable smoke in two weeks. Return automate-first scope, not-yet scope, and guardrails.
        
        expect:
          must_contain:
            - "Automate First"
            - "Not to Automate"
            - "selector"
            - "wait"
            - "maintain"
          must_not_contain:
            - "TODO"
            - "I cannot"
        
        judge:
          type: rule_based
          success:
            - output_contains:
                all:
                  - "Task Understanding"
                  - "What to Automate"
                  - "selector"
                  - "wait"
        
      • edge-domain-boundary.yaml 774 B
        id: edge-domain-boundary
        title: "Automation: never recommend 100% automation"
        description: |
          Flaky/exploratory paths belong in not-yet—not a full-coverage slogan.
        
        input:
          prompt: |
            Use automation-testing. Stakeholders demand 100% automation from login to reports, including flaky payment state and exploratory UX.
            Explicitly separate automate-first vs not-yet and explain why full automation is wrong.
        
        expect:
          must_contain:
            - "Not to Automate"
            - "risk"
            - "scope"
            - "existing framework"
            - "do not rewrite"
          must_not_contain:
            - "TODO"
            - "I cannot"
        
        judge:
          type: rule_based
          success:
            - output_contains:
                all:
                  - "What to Automate"
                  - "Guardrail"
                  - "scope"
                  - "existing framework"
        
      • edge-incomplete-input.yaml 707 B
        id: edge-incomplete-input
        title: "Automation: draft without framework details"
        description: |
          Missing framework/CI details still yield a draft with gaps.
        
        input:
          prompt: |
            Use automation-testing. Ask is only 'automate regression' with unknown framework, language, and CI.
            Domain: points redemption. Draft recommendations and list must-confirm information.
        
        expect:
          must_contain:
            - "assumption"
            - "Open Questions"
            - "gaps"
            - "assumptions"
            - "confirm"
          must_not_contain:
            - "TODO"
            - "I cannot"
        
        judge:
          type: rule_based
          success:
            - output_contains:
                all:
                  - "assumption"
                  - "Open Questions"
                  - "gaps"
                  - "assumptions"
        
    • eval.yaml 559 B
      schema_version: v1alpha1
      
      environment:
        type: none
      
      skills:
        - source: local_path
          path: .
      
      engine:
        name: claude_code
        # model is optional; omit to use engine default
        # model:
        #   provider: anthropic
        #   name: claude-sonnet-4-6
      
      cases:
        files:
          - evals/cases/basic-success.yaml
          - evals/cases/edge-incomplete-input.yaml
          - evals/cases/edge-domain-boundary.yaml
        defaults:
          timeout_seconds: 180
          max_turns: 8
          expect:
            exit_code: 0
            must_not_contain:
              - "TODO"
              - "I cannot"
      
      report:
        formats: [json]
      
  • output-templates
    • template-csv.csv 206 B · in bundle
    • template-excel.tsv 358 B · in bundle
    • template-json.json 393 B
      {
        "meta": {
          "skill": "",
          "scope": "",
          "environment": "",
          "priority": ""
        },
        "inputs": {
          "requirement": "",
          "constraints": [],
          "risks": []
        },
        "execution": [
          { "step": 1, "action": "", "expected": "" }
        ],
        "results": {
          "status": "",
          "evidence": [],
          "defects": []
        },
        "next_actions": [
          { "owner": "", "eta": "", "action": "" }
        ]
      }
      
    • template-markdown.md 250 B
      # QA Output Template
      
      ## Summary
      - Skill:
      - Scope:
      - Environment:
      - Priority:
      
      ## Inputs
      - Requirement:
      - Constraints:
      - Risks:
      
      ## Execution
      1. Step 1
      2. Step 2
      3. Step 3
      
      ## Results
      - Status:
      - Evidence:
      - Defects:
      
      ## Next Actions
      - Owner:
      - ETA:
      
    • template-word.md 302 B
      QA Report
      =========
      
      1. Basic Information
      - Skill:
      - Scope:
      - Environment:
      - Priority:
      
      2. Requirement and Constraints
      - Requirement:
      - Constraints:
      - Risks:
      
      3. Test/Review Process
      - Step 1:
      - Step 2:
      - Step 3:
      
      4. Outcome
      - Status:
      - Evidence:
      - Defects:
      
      5. Follow-up Plan
      - Owner:
      - ETA:
      - Action:
      
    • template-xmind.md 317 B
      # QA Output Mindmap
      
      - QA Output
        - Meta
          - Skill
          - Scope
          - Environment
          - Priority
        - Inputs
          - Requirement
          - Constraints
          - Risks
        - Execution
          - Step 1
          - Step 2
          - Step 3
        - Results
          - Status
          - Evidence
          - Defects
        - Next Actions
          - Owner
          - ETA
          - Action
      
  • prompts
    • automation-testing.md 1.7 KB
      # Automation Testing Prompt
      
      Design or refine an automation testing approach that is worth building, maintainable, and focused on the right risks.
      
      ## Role
      
      - Act as a senior QA and automation testing expert who balances coverage value, stability, and maintenance cost.
      
      
      ## Input
      
      - product scope, user flows, APIs, or current test suite information
      - existing framework, language, CI setup, and team constraints
      - manual pain points, flaky areas, release expectations, and risk hotspots
      
      ## What to do
      
      1. Decide what should be automated first and what should stay manual for now.
      2. Optimize for value, stability, and maintenance cost instead of raw script count.
      3. Produce an execution-ready automation plan or recommendation.
      
      ## Execution Rules
      
      - Prioritize stable, high-value, repeatable checks.
      - Flag areas with poor automation ROI such as unstable UI or one-off flows.
      - Respect the current stack when possible instead of proposing a full rewrite.
      
      ## Minimum Coverage Checklist
      
      Unless the user explicitly narrows the scope, make sure the result addresses these items:
      - scope and objective
      - automation candidates
      - manual-only areas
      - priority by risk and ROI
      - framework alignment
      - data and environment needs
      - flaky-test risks
      - CI or execution strategy
      - maintenance concerns
      - missing information and assumptions
      
      ## Output
      
      Return the result in this order:
      
      ### 1. Task Understanding
      ### 2. What to Automate First
      ### 3. What Not to Automate Yet
      ### 4. Recommended Approach
      ### 5. Execution Plan
      ### 6. Risks and Guardrails
      
      ## Quality Bar
      
      - Explain why each automation recommendation is worth it.
      - Do not recommend automating everything.
      - Keep the result practical for the existing team.
      
  • references
    • local
      • testing-types-automation-testing-examples-selenium-pom-python-README.md 608 B
        # Local Example Reference
        
        This file keeps a short note for an example package or runnable sample connected to this skill.
        
        ## What It Is
        
        - A sample or example README reference.
        - Useful when you need an example structure, sample assets, or a runnable package to inspect.
        
        ## Prefer These Files
        
        - `SKILL.md`
        - `prompts/automation-testing.md`
        - `references/`
        
        ## Notes
        
        - This file is now a short local reference instead of a full snapshot copy.
        - Use it only when you need extra background beyond the main prompt and `SKILL.md`.
        - If the current prompt conflicts with this file, follow the current prompt.
        
    • troubleshooting.md 2.6 KB
      ## Troubleshooting
      
      ### Common Issues
      
      #### 1. Element Not Found (NoSuchElementException)
      
      **Problem:** Cannot find page element during test execution
      
      **Solution:**
      ```python
      # Use explicit wait
      from selenium.webdriver.support.ui import WebDriverWait
      from selenium.webdriver.support import expected_conditions as EC
      
      wait = WebDriverWait(driver, 20)
      element = wait.until(EC.presence_of_element_located((By.ID, "element")))
      
      # Or increase implicit wait time
      driver.implicitly_wait(10)
      ```
      
      #### 2. Element Not Clickable (ElementClickInterceptedException)
      
      **Problem:** Element is obscured by other elements
      
      **Solution:**
      ```python
      # Wait for element to be clickable
      wait.until(EC.element_to_be_clickable((By.ID, "button"))).click()
      
      # Or use JavaScript click
      driver.execute_script("arguments[0].click();", element)
      
      # Or scroll to element
      driver.execute_script("arguments[0].scrollIntoView(true);", element)
      ```
      
      #### 3. StaleElementReferenceException
      
      **Problem:** Element reference becomes stale after page refresh
      
      **Solution:**
      ```python
      # Re-find the element
      def safe_click(locator):
          for _ in range(3):
              try:
                  element = driver.find_element(*locator)
                  element.click()
                  break
              except StaleElementReferenceException:
                  time.sleep(0.5)
      ```
      
      #### 4. Browser Driver Version Mismatch
      
      **Problem:** `SessionNotCreatedException: session not created`
      
      **Solution:**
      ```bash
      # Use webdriver-manager to auto-manage drivers
      pip install webdriver-manager
      
      # Use in code
      from webdriver_manager.chrome import ChromeDriverManager
      driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
      ```
      
      #### 5. Slow Test Execution
      
      **Problem:** Tests take too long to execute
      
      **Solution:**
      - Use headless mode: `options.add_argument("--headless")`
      - Run tests in parallel: `pytest -n 4`
      - Reduce unnecessary wait times
      - Optimize locators (prioritize ID, Name)
      
      #### 6. Flaky Tests
      
      **Problem:** Tests sometimes pass, sometimes fail
      
      **Solution:**
      - Increase wait times
      - Use explicit waits instead of implicit waits
      - Check test data dependencies
      - Ensure test independence
      - Add retry mechanism: `@pytest.mark.flaky(reruns=3)`
      
      #### 7. Screenshot Functionality Not Working
      
      **Problem:** No screenshots generated when tests fail
      
      **Solution:**
      ```python
      # Add hook in conftest.py
      @pytest.hookimpl(tryfirst=True, hookwrapper=True)
      def pytest_runtest_makereport(item, call):
          outcome = yield
          rep = outcome.get_result()
          if rep.when == "call" and rep.failed:
              driver = item.funcargs.get("driver")
              if driver:
                  driver.save_screenshot(f"screenshots/{item.name}.png")
      ```
      
      
  • scripts
    • batch_convert_templates.py 2.7 KB
      #!/usr/bin/env python3
      import argparse
      import subprocess
      import sys
      from pathlib import Path
      
      
      def detect_from(file: Path) -> str:
          ext = file.suffix.lower()
          if file.name.endswith('.word.md'):
              return 'markdown'
          return {
              '.md': 'markdown',
              '.markdown': 'markdown',
              '.json': 'json',
              '.csv': 'csv',
              '.tsv': 'excel',
              '.docx': 'word',
              '.xlsx': 'excel',
              '.xmind': 'xmind',
          }.get(ext, 'markdown')
      
      
      def run_convert(convert_script: Path, src: Path, to_fmt: str, out: Path) -> int:
          cmd = [sys.executable, str(convert_script), str(src), '--from', detect_from(src), '--to', to_fmt, '--output', str(out)]
          return subprocess.call(cmd)
      
      
      def main() -> None:
          parser = argparse.ArgumentParser(description='Batch convert all template files into target formats.')
          parser.add_argument('--templates-dir', type=Path, default=Path('output-templates'))
          parser.add_argument('--artifacts-dir', type=Path, default=Path('artifacts'))
          parser.add_argument('--targets', default='word,excel,xmind,json,csv,markdown', help='comma-separated target formats')
          parser.add_argument('--skip-same', action='store_true', help='skip conversion when source format equals target format')
          args = parser.parse_args()
      
          cwd = Path.cwd()
          templates_dir = (cwd / args.templates_dir).resolve()
          artifacts_dir = (cwd / args.artifacts_dir).resolve()
          artifacts_dir.mkdir(parents=True, exist_ok=True)
      
          local_convert = (Path(__file__).resolve().parent / 'convert_formats.py').resolve()
          targets = [t.strip() for t in args.targets.split(',') if t.strip()]
      
          if not templates_dir.exists():
              raise SystemExit(f'templates directory not found: {templates_dir}')
      
          files = [p for p in sorted(templates_dir.iterdir()) if p.is_file()]
          total = 0
          failed = 0
      
          for src in files:
              src_fmt = detect_from(src)
              for to_fmt in targets:
                  if args.skip_same and src_fmt == to_fmt:
                      continue
                  out_ext = {
                      'json': '.json',
                      'csv': '.csv',
                      'excel': '.tsv',
                      'markdown': '.md',
                      'word': '.word.md',
                      'xmind': '.xmind.md',
                  }[to_fmt]
                  out = artifacts_dir / f"{src.stem}.to-{to_fmt}{out_ext}"
                  total += 1
                  rc = run_convert(local_convert, src, to_fmt, out)
                  if rc != 0:
                      failed += 1
                      print(f'[FAILED] {src.name} -> {to_fmt}')
                  else:
                      print(f'[OK] {src.name} -> {out.name}')
      
          print(f'\nDone. total={total}, failed={failed}, artifacts={artifacts_dir}')
          if failed:
              raise SystemExit(1)
      
      
      if __name__ == '__main__':
          main()
      
    • convert_formats.py 10.4 KB
      #!/usr/bin/env python3
      import argparse
      import csv
      import json
      import re
      import zipfile
      from pathlib import Path
      from typing import Any
      from xml.etree import ElementTree as ET
      
      
      # ---- parsing ----
      def parse_markdown(path: Path) -> dict[str, Any]:
          text = path.read_text(encoding="utf-8", errors="ignore")
          lines = text.splitlines()
          headings: list[dict[str, Any]] = []
          for line in lines:
              m = re.match(r"^(#{1,6})\s+(.*)$", line.strip())
              if m:
                  headings.append({"level": len(m.group(1)), "title": m.group(2).strip()})
          return {"title": headings[0]["title"] if headings else path.stem, "headings": headings, "text": text}
      
      
      def parse_json(path: Path) -> dict[str, Any]:
          data = json.loads(path.read_text(encoding="utf-8", errors="ignore"))
          return {"title": path.stem, "data": data}
      
      
      def parse_csv_file(path: Path) -> dict[str, Any]:
          with path.open("r", encoding="utf-8", errors="ignore", newline="") as f:
              reader = csv.DictReader(f)
              rows = list(reader)
          return {"title": path.stem, "columns": reader.fieldnames or [], "rows": rows}
      
      
      def parse_docx(path: Path) -> dict[str, Any]:
          paragraphs: list[str] = []
          with zipfile.ZipFile(path) as zf:
              with zf.open("word/document.xml") as f:
                  root = ET.fromstring(f.read())
          ns = {"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"}
          for p in root.findall(".//w:p", ns):
              texts = [t.text for t in p.findall(".//w:t", ns) if t.text]
              s = "".join(texts).strip()
              if s:
                  paragraphs.append(s)
          return {"title": path.stem, "paragraphs": paragraphs}
      
      
      def _shared_strings(zf: zipfile.ZipFile) -> list[str]:
          out: list[str] = []
          try:
              with zf.open("xl/sharedStrings.xml") as f:
                  root = ET.fromstring(f.read())
              ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
              for si in root.findall(".//a:si", ns):
                  out.append("".join((t.text or "") for t in si.findall(".//a:t", ns)))
          except KeyError:
              pass
          return out
      
      
      def parse_xlsx(path: Path) -> dict[str, Any]:
          rows: list[list[str]] = []
          with zipfile.ZipFile(path) as zf:
              shared = _shared_strings(zf)
              with zf.open("xl/worksheets/sheet1.xml") as f:
                  root = ET.fromstring(f.read())
          ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
          for row in root.findall(".//a:sheetData/a:row", ns):
              vals = []
              for c in row.findall("a:c", ns):
                  t = c.attrib.get("t")
                  v = c.find("a:v", ns)
                  if v is None or v.text is None:
                      vals.append("")
                  elif t == "s":
                      idx = int(v.text)
                      vals.append(shared[idx] if 0 <= idx < len(shared) else "")
                  else:
                      vals.append(v.text)
              rows.append(vals)
          return {"title": path.stem, "rows": rows}
      
      
      def parse_xmind(path: Path) -> dict[str, Any]:
          with zipfile.ZipFile(path) as zf:
              names = set(zf.namelist())
              topics: list[str] = []
              if "content.json" in names:
                  data = json.loads(zf.read("content.json").decode("utf-8", errors="ignore"))
      
                  def walk(node: Any):
                      if isinstance(node, dict):
                          t = node.get("title")
                          if isinstance(t, str) and t.strip():
                              topics.append(t.strip())
                          for k in ("children", "topics", "rootTopic", "attached"):
                              walk(node.get(k))
                      elif isinstance(node, list):
                          for i in node:
                              walk(i)
      
                  walk(data)
              elif "content.xml" in names:
                  root = ET.fromstring(zf.read("content.xml"))
                  topics = [e.text.strip() for e in root.findall(".//title") if e.text and e.text.strip()]
              else:
                  raise ValueError("Unsupported XMind package structure")
          return {"title": topics[0] if topics else path.stem, "topics": topics}
      
      
      def detect_in_format(path: Path, forced: str | None) -> str:
          if forced and forced != "auto":
              return forced
          return {
              ".md": "markdown",
              ".markdown": "markdown",
              ".json": "json",
              ".csv": "csv",
              ".docx": "word",
              ".xlsx": "excel",
              ".xmind": "xmind",
              ".tsv": "excel",
          }.get(path.suffix.lower(), "markdown")
      
      
      def normalize(parsed: dict[str, Any]) -> dict[str, Any]:
          title = parsed.get("title") or "QA Output"
          sections: list[dict[str, Any]] = []
      
          if "text" in parsed:
              sections.append({"name": "content", "items": [{"key": "text", "value": parsed["text"]}]})
          if "headings" in parsed:
              sections.append({"name": "headings", "items": [{"key": "heading", "value": h.get("title", "")} for h in parsed["headings"]]})
          if "data" in parsed:
              data = parsed["data"]
              if isinstance(data, dict):
                  items = [{"key": k, "value": v} for k, v in list(data.items())[:100]]
                  sections.append({"name": "json_object", "items": items})
              elif isinstance(data, list):
                  sections.append({"name": "json_array", "items": [{"key": "row", "value": v} for v in data[:200]]})
              else:
                  sections.append({"name": "json_value", "items": [{"key": "value", "value": data}]})
          if "rows" in parsed:
              rows = parsed["rows"]
              sections.append({"name": "rows", "items": [{"key": f"row_{i+1}", "value": r} for i, r in enumerate(rows[:200])]})
          if "columns" in parsed:
              sections.append({"name": "columns", "items": [{"key": "column", "value": c} for c in parsed["columns"]]})
          if "paragraphs" in parsed:
              sections.append({"name": "paragraphs", "items": [{"key": f"p{i+1}", "value": p} for i, p in enumerate(parsed["paragraphs"][:200])]})
          if "topics" in parsed:
              sections.append({"name": "topics", "items": [{"key": "topic", "value": t} for t in parsed["topics"][:300]]})
      
          return {"title": title, "sections": sections}
      
      
      # ---- writers ----
      def write_json(model: dict[str, Any], output: Path) -> None:
          output.write_text(json.dumps(model, ensure_ascii=False, indent=2), encoding="utf-8")
      
      
      def _scalar(v: Any) -> str:
          if isinstance(v, (dict, list)):
              return json.dumps(v, ensure_ascii=False)
          return str(v)
      
      
      def write_csv(model: dict[str, Any], output: Path) -> None:
          with output.open("w", encoding="utf-8", newline="") as f:
              writer = csv.writer(f)
              writer.writerow(["section", "key", "value"])
              for s in model.get("sections", []):
                  for item in s.get("items", []):
                      writer.writerow([s.get("name", ""), item.get("key", ""), _scalar(item.get("value", ""))])
      
      
      def write_excel_tsv(model: dict[str, Any], output: Path) -> None:
          lines = ["Section\tKey\tValue"]
          for s in model.get("sections", []):
              for item in s.get("items", []):
                  lines.append(f"{s.get('name','')}\t{item.get('key','')}\t{_scalar(item.get('value','')).replace(chr(9), ' ')}")
          output.write_text("\n".join(lines) + "\n", encoding="utf-8")
      
      
      def write_markdown(model: dict[str, Any], output: Path) -> None:
          lines = [f"# {model.get('title', 'QA Output')}", ""]
          for s in model.get("sections", []):
              lines.append(f"## {s.get('name', 'section')}")
              for item in s.get("items", []):
                  lines.append(f"- **{item.get('key','key')}**: {_scalar(item.get('value',''))}")
              lines.append("")
          output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
      
      
      def write_word_md(model: dict[str, Any], output: Path) -> None:
          lines = [model.get("title", "QA Output"), "=" * len(model.get("title", "QA Output")), ""]
          idx = 1
          for s in model.get("sections", []):
              lines.append(f"{idx}. {s.get('name', 'section').replace('_', ' ').title()}")
              for item in s.get("items", []):
                  lines.append(f"- {item.get('key','key')}: {_scalar(item.get('value',''))}")
              lines.append("")
              idx += 1
          output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
      
      
      def write_xmind_md(model: dict[str, Any], output: Path) -> None:
          lines = [f"# {model.get('title', 'QA Output')}", "", f"- {model.get('title', 'QA Output')}"]
          for s in model.get("sections", []):
              lines.append(f"  - {s.get('name', 'section')}")
              for item in s.get("items", []):
                  lines.append(f"    - {item.get('key','key')}: {_scalar(item.get('value',''))}")
          output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
      
      
      def default_output(input_path: Path, to_fmt: str) -> Path:
          ext = {
              "json": ".json",
              "csv": ".csv",
              "excel": ".tsv",
              "markdown": ".md",
              "word": ".word.md",
              "xmind": ".xmind.md",
          }[to_fmt]
          return input_path.with_name(input_path.stem + ".converted" + ext)
      
      
      def main() -> None:
          parser = argparse.ArgumentParser(description="Convert QA output files between common formats")
          parser.add_argument("input", type=Path, help="Input file path")
          parser.add_argument("--from", dest="from_fmt", default="auto", choices=["auto", "word", "excel", "xmind", "json", "csv", "markdown"])
          parser.add_argument("--to", required=True, choices=["word", "excel", "xmind", "json", "csv", "markdown"])
          parser.add_argument("--output", type=Path, help="Output file path")
          args = parser.parse_args()
      
          in_fmt = detect_in_format(args.input, args.from_fmt)
          if in_fmt == "word":
              parsed = parse_docx(args.input)
          elif in_fmt == "excel":
              if args.input.suffix.lower() == ".tsv":
                  rows = [line.rstrip("\n").split("\t") for line in args.input.read_text(encoding="utf-8", errors="ignore").splitlines() if line]
                  parsed = {"title": args.input.stem, "rows": rows}
              else:
                  parsed = parse_xlsx(args.input)
          elif in_fmt == "xmind":
              parsed = parse_xmind(args.input)
          elif in_fmt == "json":
              parsed = parse_json(args.input)
          elif in_fmt == "csv":
              parsed = parse_csv_file(args.input)
          else:
              parsed = parse_markdown(args.input)
      
          model = normalize(parsed)
          output = args.output or default_output(args.input, args.to)
      
          if args.to == "json":
              write_json(model, output)
          elif args.to == "csv":
              write_csv(model, output)
          elif args.to == "excel":
              write_excel_tsv(model, output)
          elif args.to == "markdown":
              write_markdown(model, output)
          elif args.to == "word":
              write_word_md(model, output)
          else:
              write_xmind_md(model, output)
      
          print(str(output))
      
      
      if __name__ == "__main__":
          main()
      
    • convert_output_formats.py 255 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL] + sys.argv[1:]))
      
    • convert_to_csv.py 270 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'csv'] + sys.argv[1:]))
      
    • convert_to_excel.py 272 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'excel'] + sys.argv[1:]))
      
    • convert_to_json.py 271 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'json'] + sys.argv[1:]))
      
    • convert_to_markdown.py 275 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'markdown'] + sys.argv[1:]))
      
    • convert_to_word.py 271 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'word'] + sys.argv[1:]))
      
    • convert_to_xmind.py 272 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'xmind'] + sys.argv[1:]))
      
    • parse_csv.py 272 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'csv'] + sys.argv[1:]))
      
    • parse_excel.py 274 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'excel'] + sys.argv[1:]))
      
    • parse_formats.py 5.9 KB
      #!/usr/bin/env python3
      import argparse
      import csv
      import json
      import re
      import zipfile
      from pathlib import Path
      from typing import Any
      from xml.etree import ElementTree as ET
      
      
      def parse_markdown(path: Path) -> dict[str, Any]:
          text = path.read_text(encoding="utf-8", errors="ignore")
          headings = []
          for line in text.splitlines():
              m = re.match(r"^(#{1,6})\s+(.*)$", line.strip())
              if m:
                  headings.append({"level": len(m.group(1)), "title": m.group(2).strip()})
          return {"format": "markdown", "headings": headings, "preview": text[:500]}
      
      
      def parse_json(path: Path) -> dict[str, Any]:
          data = json.loads(path.read_text(encoding="utf-8", errors="ignore"))
          if isinstance(data, dict):
              shape = {"type": "object", "keys": list(data.keys())[:50]}
          elif isinstance(data, list):
              shape = {"type": "array", "size": len(data)}
          else:
              shape = {"type": type(data).__name__}
          return {"format": "json", "shape": shape, "data": data}
      
      
      def parse_csv_file(path: Path) -> dict[str, Any]:
          with path.open("r", encoding="utf-8", errors="ignore", newline="") as f:
              reader = csv.DictReader(f)
              rows = list(reader)
          return {
              "format": "csv",
              "columns": reader.fieldnames or [],
              "row_count": len(rows),
              "sample_rows": rows[:10],
          }
      
      
      def parse_docx(path: Path) -> dict[str, Any]:
          paragraphs = []
          with zipfile.ZipFile(path) as zf:
              with zf.open("word/document.xml") as f:
                  root = ET.fromstring(f.read())
          ns = {"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"}
          for p in root.findall(".//w:p", ns):
              texts = [t.text for t in p.findall(".//w:t", ns) if t.text]
              joined = "".join(texts).strip()
              if joined:
                  paragraphs.append(joined)
          return {"format": "word", "paragraph_count": len(paragraphs), "paragraphs": paragraphs[:100]}
      
      
      def _read_shared_strings(zf: zipfile.ZipFile) -> list[str]:
          strings = []
          try:
              with zf.open("xl/sharedStrings.xml") as f:
                  root = ET.fromstring(f.read())
              ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
              for si in root.findall(".//a:si", ns):
                  parts = [t.text or "" for t in si.findall(".//a:t", ns)]
                  strings.append("".join(parts))
          except KeyError:
              pass
          return strings
      
      
      def parse_xlsx(path: Path) -> dict[str, Any]:
          rows_out: list[list[str]] = []
          with zipfile.ZipFile(path) as zf:
              shared = _read_shared_strings(zf)
              with zf.open("xl/worksheets/sheet1.xml") as f:
                  root = ET.fromstring(f.read())
          ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
          for row in root.findall(".//a:sheetData/a:row", ns):
              vals = []
              for c in row.findall("a:c", ns):
                  cell_type = c.attrib.get("t")
                  v = c.find("a:v", ns)
                  if v is None or v.text is None:
                      vals.append("")
                      continue
                  if cell_type == "s":
                      idx = int(v.text)
                      vals.append(shared[idx] if 0 <= idx < len(shared) else "")
                  else:
                      vals.append(v.text)
              rows_out.append(vals)
          return {"format": "excel", "row_count": len(rows_out), "sample_rows": rows_out[:20]}
      
      
      def parse_xmind(path: Path) -> dict[str, Any]:
          with zipfile.ZipFile(path) as zf:
              names = set(zf.namelist())
              if "content.json" in names:
                  data = json.loads(zf.read("content.json").decode("utf-8", errors="ignore"))
                  titles: list[str] = []
      
                  def walk(node: Any):
                      if isinstance(node, dict):
                          title = node.get("title")
                          if isinstance(title, str) and title.strip():
                              titles.append(title.strip())
                          for k in ("children", "topics", "rootTopic", "attached"):
                              walk(node.get(k))
                      elif isinstance(node, list):
                          for i in node:
                              walk(i)
      
                  walk(data)
                  return {"format": "xmind", "topic_count": len(titles), "topics": titles[:200]}
              if "content.xml" in names:
                  root = ET.fromstring(zf.read("content.xml"))
                  titles = [el.text.strip() for el in root.findall(".//title") if el.text and el.text.strip()]
                  return {"format": "xmind", "topic_count": len(titles), "topics": titles[:200]}
          raise ValueError("Unsupported XMind package structure")
      
      
      def detect_format(path: Path, forced: str | None) -> str:
          if forced and forced != "auto":
              return forced
          ext = path.suffix.lower()
          return {
              ".md": "markdown",
              ".markdown": "markdown",
              ".json": "json",
              ".csv": "csv",
              ".docx": "word",
              ".xlsx": "excel",
              ".xmind": "xmind",
          }.get(ext, "markdown")
      
      
      def main() -> None:
          parser = argparse.ArgumentParser(description="Parse common QA output formats into normalized JSON")
          parser.add_argument("input", type=Path, help="Input file path")
          parser.add_argument("--format", default="auto", choices=["auto", "word", "excel", "xmind", "json", "csv", "markdown"])
          parser.add_argument("--output", type=Path, help="Output JSON path (default: stdout)")
          args = parser.parse_args()
      
          fmt = detect_format(args.input, args.format)
          if fmt == "word":
              result = parse_docx(args.input)
          elif fmt == "excel":
              result = parse_xlsx(args.input)
          elif fmt == "xmind":
              result = parse_xmind(args.input)
          elif fmt == "json":
              result = parse_json(args.input)
          elif fmt == "csv":
              result = parse_csv_file(args.input)
          else:
              result = parse_markdown(args.input)
      
          result["source"] = str(args.input)
          out = json.dumps(result, ensure_ascii=False, indent=2)
          if args.output:
              args.output.write_text(out, encoding="utf-8")
          else:
              print(out)
      
      
      if __name__ == "__main__":
          main()
      
    • parse_json.py 273 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'json'] + sys.argv[1:]))
      
    • parse_markdown.py 277 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'markdown'] + sys.argv[1:]))
      
    • parse_output_formats.py 253 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL] + sys.argv[1:]))
      
    • parse_word.py 273 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'word'] + sys.argv[1:]))
      
    • parse_xmind.py 274 B
      #!/usr/bin/env python3
      import os
      import subprocess
      import sys
      LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py'))
      if __name__ == '__main__':
          raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'xmind'] + sys.argv[1:]))
      
  • output-formats.md 401 B
    # Output Format Options
    
    This skill **defaults to Markdown**. To get **Excel**, **CSV**, or **JSON**, add at the **end** of your request:
    
    - **Excel:** "Please output as tab-separated table for pasting into Excel."
    - **CSV:** "Please output as CSV (comma-separated, header row first)."
    - **JSON:** "Please output as JSON."
    
    See repo `skills/testing-types/_output-formats-template-en.md` for examples.
    
  • quick-start.md 2.2 KB
    # Automation Testing: Quick Start
    
    Use the `automation-testing` Skill to plan maintainable automation across the right test layers and existing toolchain.
    
    ## 1. Prepare the Minimum Context
    
    Provide what is available from this list:
    
    - application architecture
    - current tests
    - CI constraints
    - environments
    - failure history
    - scope, version, environment, time budget, and prohibited actions
    - known failures, existing tests, and evidence links
    
    If some information is unavailable, say so explicitly. The Skill should still return a bounded first pass with assumptions and open questions.
    
    ## 2. Invoke the Skill
    
    ```text
    @skill automation-testing
    
    Scenario: Automate a login-to-checkout smoke flow in the existing CI pipeline.
    Prioritize the highest risks, distinguish facts from assumptions, and provide an executable result with evidence requirements and open questions.
    ```
    
    ## 3. Follow the Execution Flow
    
    1. Confirm the objective, subject, scope, and success criteria.
    2. Audit the completeness and credibility of the supplied material.
    3. Identify failure modes and rank them by impact, likelihood, and detectability.
    4. Produce concrete scenarios, checks, or decisions with expected results.
    5. Record evidence gaps, residual risk, and the smallest useful next action.
    
    ## 4. Minimum Coverage
    
    Unless the request narrows scope, cover:
    
    - test pyramid
    - selector and contract stability
    - data isolation
    - retries
    - parallelism
    - reporting
    - maintenance cost
    - confirmed facts, working assumptions, and open questions
    - P0/P1 priorities and the rationale for lower-priority deferral
    - blockers, stop conditions, and residual risk
    
    ## 5. Review the Result
    
    - Every high-risk item has a concrete failure mode and expected behavior.
    - Priorities are justified rather than evenly distributed.
    - No field, API, metric, environment, or execution result is invented.
    - The artifact can be executed or reviewed without guessing the next step.
    - Production, security, and privacy work uses least privilege and sanitized data.
    
    ## 6. Continue
    
    Add missing evidence and ask the Skill to refine the same artifact. Keep confirmed facts separate from newly introduced assumptions so changes remain reviewable.
    
  • README.md 820 B
    # Automation Testing
    
    ## Skill Overview
    
    Need help with automation testing in a real project context; Need an output that can be used directly for execution, review, or follow-up.
    
    ## How to Use
    
    1. Open `SKILL.md` in this folder and confirm this skill fits your task.
    2. In your AI tool, call `@skill automation-testing`, then add your real project context and goal.
    3. If you need a specific output format (table, checklist, report), include it directly in your request.
    
    ## One-Click Install Script
    
    Run from the repository root:
    
    ### macOS / Linux
    
    ```bash
    bash ./scripts/install-skills-mac.sh --tool codex --lang en --skill automation-testing
    ```
    
    ### Windows PowerShell
    
    ```powershell
    powershell -ExecutionPolicy Bypass -File .\scripts\install-skills-windows.ps1 -Tool codex -Lang en -Skill automation-testing
    ```
    
  • SKILL.md 2.5 KB
    ---
    name: automation-testing
    description: Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing and test automation strategy.
    ---
    
    # Automation Testing (English)
    
    **Chinese version:** See the corresponding Chinese skill.
    
    ## When to Use
    
    - Need help with automation testing in a real project context.
    - Need an output that can be used directly for execution, review, or follow-up.
    
    ## Workflow
    
    1. Read and follow the main prompt listed under Progressive disclosure (coverage, structure, quality bar).
    2. Add only project context that changes the result: scope, environment, constraints, risks, dependencies, expected deliverable.
    3. If input is incomplete, return a usable first draft and explicitly mark assumptions and gaps.
    4. Default to Markdown; switch formats only when the user asks.
    
    ## Core Constraints
    
    - Prioritize by risk / business impact — do not treat everything equally.
    - Separate confirmed facts from current assumptions.
    - Do not invent endpoints, fields, environments, or root causes the user did not provide.
    - Keep output executable: concrete scenarios, clear priority, clear next steps.
    
    ## Progressive Disclosure
    
    - Before producing output, read and follow `prompts/automation-testing.md` (minimum coverage, output structure, quality bar).
    - When Excel/CSV/JSON/Word is requested: read `output-formats.md` and honor the format.
    - When a ready-made template fits: use matching files under `output-templates/`.
    - For deep framework/troubleshoot/schema notes: read only the relevant file(s) under `references/`, do not load the whole directory.
    - For format conversion or helper checks: prefer existing `scripts/` over reinventing.
    - For evaluating/regressing this skill: use `evals/` with skill-up.
    
    ## Pre-delivery Checklist
    
    - [ ] Followed the main prompt's output structure
    - [ ] Minimum coverage focus: scope and objective, automation candidates, manual-only areas, priority by risk and ROI, framework alignment, data and environment needs, flaky-test risks, CI or execution strategy, ... (details in main prompt)
    - [ ] Covered the minimum checklist, or explained omissions
    - [ ] High-risk items have explicit priority
    - [ ] Did not invent details the user did not provide
    - [ ] Assumptions and gaps are marked
    
    ## Common Pitfalls
    
    - Do not pretend completeness when scope/context is missing.
    - Do not treat every item as equally important.
    - Do not skip assumptions and information gaps.
    - Do not dump generic theory unrelated to the current toolchain.
    
  • tutorial.md 4.1 KB
    # Automation Testing: Practical Tutorial
    
    This tutorial shows how to plan maintainable automation across the right test layers and existing toolchain with a repeatable, evidence-driven workflow.
    
    ## Learning Objectives
    
    By the end, you should be able to:
    
    - frame a bounded testing objective;
    - identify the evidence needed before making decisions;
    - turn domain risk into executable scenarios and assertions;
    - choose priorities without treating every check as equally important;
    - report residual risk without inventing certainty.
    
    ## Scenario
    
    Automate a login-to-checkout smoke flow in the existing CI pipeline.
    
    Use the scenario as a starting point. Replace it with real project material whenever possible.
    
    ## Step 1: Establish the Input Contract
    
    Collect:
    
    - application architecture
    - current tests
    - CI constraints
    - environments
    - failure history
    - the exact build, version, environment, and test window
    - authorized accounts, sanitized data, and allowed side effects
    - existing tests, defects, monitoring, and rollback constraints
    
    Create three lists before analysis:
    
    1. **Confirmed facts** — directly supported by supplied material.
    2. **Working assumptions** — necessary for a first pass but not yet proven.
    3. **Open questions** — missing information that changes scope or risk.
    
    ## Step 2: Build the Risk Model
    
    For each capability or user journey, record:
    
    | Field | Meaning |
    | --- | --- |
    | Failure mode | What can go wrong |
    | Impact | User, business, security, or operational consequence |
    | Likelihood | Why the failure is plausible |
    | Detectability | Whether existing checks or telemetry reveal it |
    | Priority | P0, P1, P2, or P3 with rationale |
    | Evidence | What would confirm correct or incorrect behavior |
    
    Start with irreversible data loss, security exposure, financial impact, critical journey failure, and release blockers.
    
    ## Step 3: Design Coverage
    
    Minimum domain coverage:
    
    - test pyramid
    - selector and contract stability
    - data isolation
    - retries
    - parallelism
    - reporting
    - maintenance cost
    
    For each important scenario, specify:
    
    ```text
    Preconditions:
    Test data:
    Action or stimulus:
    Expected behavior:
    Evidence to capture:
    Cleanup or rollback:
    ```
    
    Avoid generic statements such as “test positive and negative cases” unless the concrete cases are listed.
    
    ## Step 4: Execute Safely
    
    - Use the lowest environment and privilege level that can answer the question.
    - Prefer mocks, sandboxes, dry runs, and synthetic or masked data.
    - Do not run destructive production actions without explicit authorization.
    - Record timestamps, versions, configuration, and identifiers needed for reproduction.
    - Stop when a predefined safety, data-integrity, or customer-impact condition is met.
    
    ## Step 5: Interpret Evidence
    
    Separate observation from explanation:
    
    ```text
    Observation: what the system or test actually showed
    Hypothesis: a possible explanation
    Counter-evidence: facts that weaken the hypothesis
    Verification: the smallest experiment that can distinguish alternatives
    Conclusion: only what the available evidence supports
    ```
    
    One failure message, high resource value, or correlated metric is not automatically a root cause.
    
    ## Step 6: Report the Result
    
    Use this order:
    
    1. Task understanding and scope
    2. Input audit
    3. Risks and priorities
    4. Executed or proposed coverage
    5. Results and evidence
    6. Blockers and residual risk
    7. Next actions and owners
    
    ## Reusable Request
    
    ```text
    @skill automation-testing
    
    Analyze this real project context using the Skill's complete prompt contract.
    Separate confirmed facts, working assumptions, and open questions.
    Prioritize P0/P1 risks, provide executable scenarios with expected results and evidence, and state stop conditions and residual risk.
    ```
    
    ## Completion Checklist
    
    - [ ] The objective and success criteria are explicit.
    - [ ] The environment, version, and data boundary are recorded.
    - [ ] High-risk paths have concrete scenarios and expected results.
    - [ ] Evidence supports conclusions.
    - [ ] Missing information and assumptions remain visible.
    - [ ] Cleanup, rollback, or human-handoff conditions are defined.
    - [ ] The next action is small, owned, and verifiable.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related