automation-testing
Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing and test automation strategy.
Install
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/automation-testing
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
git clone https://github.com/naodeng/awesome-qa-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.
README
Automation Testing
Skill Overview
Need help with automation testing in a real project context; Need an output that can be used directly for execution, review, or follow-up.
How to Use
- Open
SKILL.mdin this folder and confirm this skill fits your task. - In your AI tool, call
@skill automation-testing, then add your real project context and goal. - If you need a specific output format (table, checklist, report), include it directly in your request.
One-Click Install Script
Run from the repository root:
macOS / Linux
bash ./scripts/install-skills-mac.sh --tool codex --lang en --skill automation-testing
Windows PowerShell
powershell -ExecutionPolicy Bypass -File .\scripts\install-skills-windows.ps1 -Tool codex -Lang en -Skill automation-testing
Skill manifest
Automation Testing (English)
Chinese version: See the corresponding Chinese skill.
When to Use
- Need help with automation testing in a real project context.
- Need an output that can be used directly for execution, review, or follow-up.
Workflow
- Read and follow the main prompt listed under Progressive disclosure (coverage, structure, quality bar).
- Add only project context that changes the result: scope, environment, constraints, risks, dependencies, expected deliverable.
- If input is incomplete, return a usable first draft and explicitly mark assumptions and gaps.
- Default to Markdown; switch formats only when the user asks.
Core Constraints
- Prioritize by risk / business impact — do not treat everything equally.
- Separate confirmed facts from current assumptions.
- Do not invent endpoints, fields, environments, or root causes the user did not provide.
- Keep output executable: concrete scenarios, clear priority, clear next steps.
Progressive Disclosure
- Before producing output, read and follow
prompts/automation-testing.md(minimum coverage, output structure, quality bar). - When Excel/CSV/JSON/Word is requested: read
output-formats.mdand honor the format. - When a ready-made template fits: use matching files under
output-templates/. - For deep framework/troubleshoot/schema notes: read only the relevant file(s) under
references/, do not load the whole directory. - For format conversion or helper checks: prefer existing
scripts/over reinventing. - For evaluating/regressing this skill: use
evals/with skill-up.
Pre-delivery Checklist
- Followed the main prompt's output structure
- Minimum coverage focus: scope and objective, automation candidates, manual-only areas, priority by risk and ROI, framework alignment, data and environment needs, flaky-test risks, CI or execution strategy, ... (details in main prompt)
- Covered the minimum checklist, or explained omissions
- High-risk items have explicit priority
- Did not invent details the user did not provide
- Assumptions and gaps are marked
Common Pitfalls
- Do not pretend completeness when scope/context is missing.
- Do not treat every item as equally important.
- Do not skip assumptions and information gaps.
- Do not dump generic theory unrelated to the current toolchain.
Files (awesome-qa-skills)
-
agents
-
openai.yaml 467 B
version: 1 metadata: key: "automation-testing" interface: display_name: "Automation Testing (English)" short_description: "Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing…" default_prompt: "Use the automation-testing skill to complete this testing task with structured outputs and practical next steps." policy: allow_implicit_invocation: true
-
-
evals
-
cases
-
basic-success.yaml 767 B
id: basic-success title: "Automation: what first vs not yet" description: | Separate automate-first vs not-yet; include approach and guardrails. input: prompt: | Use automation-testing. Current: Playwright; CI can run smoke. Candidates: login, search, checkout, payment-callback checks, admin report export. Goal: stable smoke in two weeks. Return automate-first scope, not-yet scope, and guardrails. expect: must_contain: - "Automate First" - "Not to Automate" - "selector" - "wait" - "maintain" must_not_contain: - "TODO" - "I cannot" judge: type: rule_based success: - output_contains: all: - "Task Understanding" - "What to Automate" - "selector" - "wait" -
edge-domain-boundary.yaml 774 B
id: edge-domain-boundary title: "Automation: never recommend 100% automation" description: | Flaky/exploratory paths belong in not-yet—not a full-coverage slogan. input: prompt: | Use automation-testing. Stakeholders demand 100% automation from login to reports, including flaky payment state and exploratory UX. Explicitly separate automate-first vs not-yet and explain why full automation is wrong. expect: must_contain: - "Not to Automate" - "risk" - "scope" - "existing framework" - "do not rewrite" must_not_contain: - "TODO" - "I cannot" judge: type: rule_based success: - output_contains: all: - "What to Automate" - "Guardrail" - "scope" - "existing framework" -
edge-incomplete-input.yaml 707 B
id: edge-incomplete-input title: "Automation: draft without framework details" description: | Missing framework/CI details still yield a draft with gaps. input: prompt: | Use automation-testing. Ask is only 'automate regression' with unknown framework, language, and CI. Domain: points redemption. Draft recommendations and list must-confirm information. expect: must_contain: - "assumption" - "Open Questions" - "gaps" - "assumptions" - "confirm" must_not_contain: - "TODO" - "I cannot" judge: type: rule_based success: - output_contains: all: - "assumption" - "Open Questions" - "gaps" - "assumptions"
-
-
eval.yaml 559 B
schema_version: v1alpha1 environment: type: none skills: - source: local_path path: . engine: name: claude_code # model is optional; omit to use engine default # model: # provider: anthropic # name: claude-sonnet-4-6 cases: files: - evals/cases/basic-success.yaml - evals/cases/edge-incomplete-input.yaml - evals/cases/edge-domain-boundary.yaml defaults: timeout_seconds: 180 max_turns: 8 expect: exit_code: 0 must_not_contain: - "TODO" - "I cannot" report: formats: [json]
-
-
output-templates
-
template-csv.csv 206 B · in bundle
-
template-excel.tsv 358 B · in bundle
-
template-json.json 393 B
{ "meta": { "skill": "", "scope": "", "environment": "", "priority": "" }, "inputs": { "requirement": "", "constraints": [], "risks": [] }, "execution": [ { "step": 1, "action": "", "expected": "" } ], "results": { "status": "", "evidence": [], "defects": [] }, "next_actions": [ { "owner": "", "eta": "", "action": "" } ] } -
template-markdown.md 250 B
# QA Output Template ## Summary - Skill: - Scope: - Environment: - Priority: ## Inputs - Requirement: - Constraints: - Risks: ## Execution 1. Step 1 2. Step 2 3. Step 3 ## Results - Status: - Evidence: - Defects: ## Next Actions - Owner: - ETA: -
template-word.md 302 B
QA Report ========= 1. Basic Information - Skill: - Scope: - Environment: - Priority: 2. Requirement and Constraints - Requirement: - Constraints: - Risks: 3. Test/Review Process - Step 1: - Step 2: - Step 3: 4. Outcome - Status: - Evidence: - Defects: 5. Follow-up Plan - Owner: - ETA: - Action: -
template-xmind.md 317 B
# QA Output Mindmap - QA Output - Meta - Skill - Scope - Environment - Priority - Inputs - Requirement - Constraints - Risks - Execution - Step 1 - Step 2 - Step 3 - Results - Status - Evidence - Defects - Next Actions - Owner - ETA - Action
-
-
prompts
-
automation-testing.md 1.7 KB
# Automation Testing Prompt Design or refine an automation testing approach that is worth building, maintainable, and focused on the right risks. ## Role - Act as a senior QA and automation testing expert who balances coverage value, stability, and maintenance cost. ## Input - product scope, user flows, APIs, or current test suite information - existing framework, language, CI setup, and team constraints - manual pain points, flaky areas, release expectations, and risk hotspots ## What to do 1. Decide what should be automated first and what should stay manual for now. 2. Optimize for value, stability, and maintenance cost instead of raw script count. 3. Produce an execution-ready automation plan or recommendation. ## Execution Rules - Prioritize stable, high-value, repeatable checks. - Flag areas with poor automation ROI such as unstable UI or one-off flows. - Respect the current stack when possible instead of proposing a full rewrite. ## Minimum Coverage Checklist Unless the user explicitly narrows the scope, make sure the result addresses these items: - scope and objective - automation candidates - manual-only areas - priority by risk and ROI - framework alignment - data and environment needs - flaky-test risks - CI or execution strategy - maintenance concerns - missing information and assumptions ## Output Return the result in this order: ### 1. Task Understanding ### 2. What to Automate First ### 3. What Not to Automate Yet ### 4. Recommended Approach ### 5. Execution Plan ### 6. Risks and Guardrails ## Quality Bar - Explain why each automation recommendation is worth it. - Do not recommend automating everything. - Keep the result practical for the existing team.
-
-
references
-
local
-
testing-types-automation-testing-examples-selenium-pom-python-README.md 608 B
# Local Example Reference This file keeps a short note for an example package or runnable sample connected to this skill. ## What It Is - A sample or example README reference. - Useful when you need an example structure, sample assets, or a runnable package to inspect. ## Prefer These Files - `SKILL.md` - `prompts/automation-testing.md` - `references/` ## Notes - This file is now a short local reference instead of a full snapshot copy. - Use it only when you need extra background beyond the main prompt and `SKILL.md`. - If the current prompt conflicts with this file, follow the current prompt.
-
-
troubleshooting.md 2.6 KB
## Troubleshooting ### Common Issues #### 1. Element Not Found (NoSuchElementException) **Problem:** Cannot find page element during test execution **Solution:** ```python # Use explicit wait from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC wait = WebDriverWait(driver, 20) element = wait.until(EC.presence_of_element_located((By.ID, "element"))) # Or increase implicit wait time driver.implicitly_wait(10) ``` #### 2. Element Not Clickable (ElementClickInterceptedException) **Problem:** Element is obscured by other elements **Solution:** ```python # Wait for element to be clickable wait.until(EC.element_to_be_clickable((By.ID, "button"))).click() # Or use JavaScript click driver.execute_script("arguments[0].click();", element) # Or scroll to element driver.execute_script("arguments[0].scrollIntoView(true);", element) ``` #### 3. StaleElementReferenceException **Problem:** Element reference becomes stale after page refresh **Solution:** ```python # Re-find the element def safe_click(locator): for _ in range(3): try: element = driver.find_element(*locator) element.click() break except StaleElementReferenceException: time.sleep(0.5) ``` #### 4. Browser Driver Version Mismatch **Problem:** `SessionNotCreatedException: session not created` **Solution:** ```bash # Use webdriver-manager to auto-manage drivers pip install webdriver-manager # Use in code from webdriver_manager.chrome import ChromeDriverManager driver = webdriver.Chrome(service=Service(ChromeDriverManager().install())) ``` #### 5. Slow Test Execution **Problem:** Tests take too long to execute **Solution:** - Use headless mode: `options.add_argument("--headless")` - Run tests in parallel: `pytest -n 4` - Reduce unnecessary wait times - Optimize locators (prioritize ID, Name) #### 6. Flaky Tests **Problem:** Tests sometimes pass, sometimes fail **Solution:** - Increase wait times - Use explicit waits instead of implicit waits - Check test data dependencies - Ensure test independence - Add retry mechanism: `@pytest.mark.flaky(reruns=3)` #### 7. Screenshot Functionality Not Working **Problem:** No screenshots generated when tests fail **Solution:** ```python # Add hook in conftest.py @pytest.hookimpl(tryfirst=True, hookwrapper=True) def pytest_runtest_makereport(item, call): outcome = yield rep = outcome.get_result() if rep.when == "call" and rep.failed: driver = item.funcargs.get("driver") if driver: driver.save_screenshot(f"screenshots/{item.name}.png") ```
-
-
scripts
-
batch_convert_templates.py 2.7 KB
#!/usr/bin/env python3 import argparse import subprocess import sys from pathlib import Path def detect_from(file: Path) -> str: ext = file.suffix.lower() if file.name.endswith('.word.md'): return 'markdown' return { '.md': 'markdown', '.markdown': 'markdown', '.json': 'json', '.csv': 'csv', '.tsv': 'excel', '.docx': 'word', '.xlsx': 'excel', '.xmind': 'xmind', }.get(ext, 'markdown') def run_convert(convert_script: Path, src: Path, to_fmt: str, out: Path) -> int: cmd = [sys.executable, str(convert_script), str(src), '--from', detect_from(src), '--to', to_fmt, '--output', str(out)] return subprocess.call(cmd) def main() -> None: parser = argparse.ArgumentParser(description='Batch convert all template files into target formats.') parser.add_argument('--templates-dir', type=Path, default=Path('output-templates')) parser.add_argument('--artifacts-dir', type=Path, default=Path('artifacts')) parser.add_argument('--targets', default='word,excel,xmind,json,csv,markdown', help='comma-separated target formats') parser.add_argument('--skip-same', action='store_true', help='skip conversion when source format equals target format') args = parser.parse_args() cwd = Path.cwd() templates_dir = (cwd / args.templates_dir).resolve() artifacts_dir = (cwd / args.artifacts_dir).resolve() artifacts_dir.mkdir(parents=True, exist_ok=True) local_convert = (Path(__file__).resolve().parent / 'convert_formats.py').resolve() targets = [t.strip() for t in args.targets.split(',') if t.strip()] if not templates_dir.exists(): raise SystemExit(f'templates directory not found: {templates_dir}') files = [p for p in sorted(templates_dir.iterdir()) if p.is_file()] total = 0 failed = 0 for src in files: src_fmt = detect_from(src) for to_fmt in targets: if args.skip_same and src_fmt == to_fmt: continue out_ext = { 'json': '.json', 'csv': '.csv', 'excel': '.tsv', 'markdown': '.md', 'word': '.word.md', 'xmind': '.xmind.md', }[to_fmt] out = artifacts_dir / f"{src.stem}.to-{to_fmt}{out_ext}" total += 1 rc = run_convert(local_convert, src, to_fmt, out) if rc != 0: failed += 1 print(f'[FAILED] {src.name} -> {to_fmt}') else: print(f'[OK] {src.name} -> {out.name}') print(f'\nDone. total={total}, failed={failed}, artifacts={artifacts_dir}') if failed: raise SystemExit(1) if __name__ == '__main__': main() -
convert_formats.py 10.4 KB
#!/usr/bin/env python3 import argparse import csv import json import re import zipfile from pathlib import Path from typing import Any from xml.etree import ElementTree as ET # ---- parsing ---- def parse_markdown(path: Path) -> dict[str, Any]: text = path.read_text(encoding="utf-8", errors="ignore") lines = text.splitlines() headings: list[dict[str, Any]] = [] for line in lines: m = re.match(r"^(#{1,6})\s+(.*)$", line.strip()) if m: headings.append({"level": len(m.group(1)), "title": m.group(2).strip()}) return {"title": headings[0]["title"] if headings else path.stem, "headings": headings, "text": text} def parse_json(path: Path) -> dict[str, Any]: data = json.loads(path.read_text(encoding="utf-8", errors="ignore")) return {"title": path.stem, "data": data} def parse_csv_file(path: Path) -> dict[str, Any]: with path.open("r", encoding="utf-8", errors="ignore", newline="") as f: reader = csv.DictReader(f) rows = list(reader) return {"title": path.stem, "columns": reader.fieldnames or [], "rows": rows} def parse_docx(path: Path) -> dict[str, Any]: paragraphs: list[str] = [] with zipfile.ZipFile(path) as zf: with zf.open("word/document.xml") as f: root = ET.fromstring(f.read()) ns = {"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"} for p in root.findall(".//w:p", ns): texts = [t.text for t in p.findall(".//w:t", ns) if t.text] s = "".join(texts).strip() if s: paragraphs.append(s) return {"title": path.stem, "paragraphs": paragraphs} def _shared_strings(zf: zipfile.ZipFile) -> list[str]: out: list[str] = [] try: with zf.open("xl/sharedStrings.xml") as f: root = ET.fromstring(f.read()) ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"} for si in root.findall(".//a:si", ns): out.append("".join((t.text or "") for t in si.findall(".//a:t", ns))) except KeyError: pass return out def parse_xlsx(path: Path) -> dict[str, Any]: rows: list[list[str]] = [] with zipfile.ZipFile(path) as zf: shared = _shared_strings(zf) with zf.open("xl/worksheets/sheet1.xml") as f: root = ET.fromstring(f.read()) ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"} for row in root.findall(".//a:sheetData/a:row", ns): vals = [] for c in row.findall("a:c", ns): t = c.attrib.get("t") v = c.find("a:v", ns) if v is None or v.text is None: vals.append("") elif t == "s": idx = int(v.text) vals.append(shared[idx] if 0 <= idx < len(shared) else "") else: vals.append(v.text) rows.append(vals) return {"title": path.stem, "rows": rows} def parse_xmind(path: Path) -> dict[str, Any]: with zipfile.ZipFile(path) as zf: names = set(zf.namelist()) topics: list[str] = [] if "content.json" in names: data = json.loads(zf.read("content.json").decode("utf-8", errors="ignore")) def walk(node: Any): if isinstance(node, dict): t = node.get("title") if isinstance(t, str) and t.strip(): topics.append(t.strip()) for k in ("children", "topics", "rootTopic", "attached"): walk(node.get(k)) elif isinstance(node, list): for i in node: walk(i) walk(data) elif "content.xml" in names: root = ET.fromstring(zf.read("content.xml")) topics = [e.text.strip() for e in root.findall(".//title") if e.text and e.text.strip()] else: raise ValueError("Unsupported XMind package structure") return {"title": topics[0] if topics else path.stem, "topics": topics} def detect_in_format(path: Path, forced: str | None) -> str: if forced and forced != "auto": return forced return { ".md": "markdown", ".markdown": "markdown", ".json": "json", ".csv": "csv", ".docx": "word", ".xlsx": "excel", ".xmind": "xmind", ".tsv": "excel", }.get(path.suffix.lower(), "markdown") def normalize(parsed: dict[str, Any]) -> dict[str, Any]: title = parsed.get("title") or "QA Output" sections: list[dict[str, Any]] = [] if "text" in parsed: sections.append({"name": "content", "items": [{"key": "text", "value": parsed["text"]}]}) if "headings" in parsed: sections.append({"name": "headings", "items": [{"key": "heading", "value": h.get("title", "")} for h in parsed["headings"]]}) if "data" in parsed: data = parsed["data"] if isinstance(data, dict): items = [{"key": k, "value": v} for k, v in list(data.items())[:100]] sections.append({"name": "json_object", "items": items}) elif isinstance(data, list): sections.append({"name": "json_array", "items": [{"key": "row", "value": v} for v in data[:200]]}) else: sections.append({"name": "json_value", "items": [{"key": "value", "value": data}]}) if "rows" in parsed: rows = parsed["rows"] sections.append({"name": "rows", "items": [{"key": f"row_{i+1}", "value": r} for i, r in enumerate(rows[:200])]}) if "columns" in parsed: sections.append({"name": "columns", "items": [{"key": "column", "value": c} for c in parsed["columns"]]}) if "paragraphs" in parsed: sections.append({"name": "paragraphs", "items": [{"key": f"p{i+1}", "value": p} for i, p in enumerate(parsed["paragraphs"][:200])]}) if "topics" in parsed: sections.append({"name": "topics", "items": [{"key": "topic", "value": t} for t in parsed["topics"][:300]]}) return {"title": title, "sections": sections} # ---- writers ---- def write_json(model: dict[str, Any], output: Path) -> None: output.write_text(json.dumps(model, ensure_ascii=False, indent=2), encoding="utf-8") def _scalar(v: Any) -> str: if isinstance(v, (dict, list)): return json.dumps(v, ensure_ascii=False) return str(v) def write_csv(model: dict[str, Any], output: Path) -> None: with output.open("w", encoding="utf-8", newline="") as f: writer = csv.writer(f) writer.writerow(["section", "key", "value"]) for s in model.get("sections", []): for item in s.get("items", []): writer.writerow([s.get("name", ""), item.get("key", ""), _scalar(item.get("value", ""))]) def write_excel_tsv(model: dict[str, Any], output: Path) -> None: lines = ["Section\tKey\tValue"] for s in model.get("sections", []): for item in s.get("items", []): lines.append(f"{s.get('name','')}\t{item.get('key','')}\t{_scalar(item.get('value','')).replace(chr(9), ' ')}") output.write_text("\n".join(lines) + "\n", encoding="utf-8") def write_markdown(model: dict[str, Any], output: Path) -> None: lines = [f"# {model.get('title', 'QA Output')}", ""] for s in model.get("sections", []): lines.append(f"## {s.get('name', 'section')}") for item in s.get("items", []): lines.append(f"- **{item.get('key','key')}**: {_scalar(item.get('value',''))}") lines.append("") output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") def write_word_md(model: dict[str, Any], output: Path) -> None: lines = [model.get("title", "QA Output"), "=" * len(model.get("title", "QA Output")), ""] idx = 1 for s in model.get("sections", []): lines.append(f"{idx}. {s.get('name', 'section').replace('_', ' ').title()}") for item in s.get("items", []): lines.append(f"- {item.get('key','key')}: {_scalar(item.get('value',''))}") lines.append("") idx += 1 output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") def write_xmind_md(model: dict[str, Any], output: Path) -> None: lines = [f"# {model.get('title', 'QA Output')}", "", f"- {model.get('title', 'QA Output')}"] for s in model.get("sections", []): lines.append(f" - {s.get('name', 'section')}") for item in s.get("items", []): lines.append(f" - {item.get('key','key')}: {_scalar(item.get('value',''))}") output.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") def default_output(input_path: Path, to_fmt: str) -> Path: ext = { "json": ".json", "csv": ".csv", "excel": ".tsv", "markdown": ".md", "word": ".word.md", "xmind": ".xmind.md", }[to_fmt] return input_path.with_name(input_path.stem + ".converted" + ext) def main() -> None: parser = argparse.ArgumentParser(description="Convert QA output files between common formats") parser.add_argument("input", type=Path, help="Input file path") parser.add_argument("--from", dest="from_fmt", default="auto", choices=["auto", "word", "excel", "xmind", "json", "csv", "markdown"]) parser.add_argument("--to", required=True, choices=["word", "excel", "xmind", "json", "csv", "markdown"]) parser.add_argument("--output", type=Path, help="Output file path") args = parser.parse_args() in_fmt = detect_in_format(args.input, args.from_fmt) if in_fmt == "word": parsed = parse_docx(args.input) elif in_fmt == "excel": if args.input.suffix.lower() == ".tsv": rows = [line.rstrip("\n").split("\t") for line in args.input.read_text(encoding="utf-8", errors="ignore").splitlines() if line] parsed = {"title": args.input.stem, "rows": rows} else: parsed = parse_xlsx(args.input) elif in_fmt == "xmind": parsed = parse_xmind(args.input) elif in_fmt == "json": parsed = parse_json(args.input) elif in_fmt == "csv": parsed = parse_csv_file(args.input) else: parsed = parse_markdown(args.input) model = normalize(parsed) output = args.output or default_output(args.input, args.to) if args.to == "json": write_json(model, output) elif args.to == "csv": write_csv(model, output) elif args.to == "excel": write_excel_tsv(model, output) elif args.to == "markdown": write_markdown(model, output) elif args.to == "word": write_word_md(model, output) else: write_xmind_md(model, output) print(str(output)) if __name__ == "__main__": main() -
convert_output_formats.py 255 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL] + sys.argv[1:])) -
convert_to_csv.py 270 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'csv'] + sys.argv[1:])) -
convert_to_excel.py 272 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'excel'] + sys.argv[1:])) -
convert_to_json.py 271 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'json'] + sys.argv[1:])) -
convert_to_markdown.py 275 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'markdown'] + sys.argv[1:])) -
convert_to_word.py 271 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'word'] + sys.argv[1:])) -
convert_to_xmind.py 272 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'convert_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--to', 'xmind'] + sys.argv[1:])) -
parse_csv.py 272 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'csv'] + sys.argv[1:])) -
parse_excel.py 274 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'excel'] + sys.argv[1:])) -
parse_formats.py 5.9 KB
#!/usr/bin/env python3 import argparse import csv import json import re import zipfile from pathlib import Path from typing import Any from xml.etree import ElementTree as ET def parse_markdown(path: Path) -> dict[str, Any]: text = path.read_text(encoding="utf-8", errors="ignore") headings = [] for line in text.splitlines(): m = re.match(r"^(#{1,6})\s+(.*)$", line.strip()) if m: headings.append({"level": len(m.group(1)), "title": m.group(2).strip()}) return {"format": "markdown", "headings": headings, "preview": text[:500]} def parse_json(path: Path) -> dict[str, Any]: data = json.loads(path.read_text(encoding="utf-8", errors="ignore")) if isinstance(data, dict): shape = {"type": "object", "keys": list(data.keys())[:50]} elif isinstance(data, list): shape = {"type": "array", "size": len(data)} else: shape = {"type": type(data).__name__} return {"format": "json", "shape": shape, "data": data} def parse_csv_file(path: Path) -> dict[str, Any]: with path.open("r", encoding="utf-8", errors="ignore", newline="") as f: reader = csv.DictReader(f) rows = list(reader) return { "format": "csv", "columns": reader.fieldnames or [], "row_count": len(rows), "sample_rows": rows[:10], } def parse_docx(path: Path) -> dict[str, Any]: paragraphs = [] with zipfile.ZipFile(path) as zf: with zf.open("word/document.xml") as f: root = ET.fromstring(f.read()) ns = {"w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main"} for p in root.findall(".//w:p", ns): texts = [t.text for t in p.findall(".//w:t", ns) if t.text] joined = "".join(texts).strip() if joined: paragraphs.append(joined) return {"format": "word", "paragraph_count": len(paragraphs), "paragraphs": paragraphs[:100]} def _read_shared_strings(zf: zipfile.ZipFile) -> list[str]: strings = [] try: with zf.open("xl/sharedStrings.xml") as f: root = ET.fromstring(f.read()) ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"} for si in root.findall(".//a:si", ns): parts = [t.text or "" for t in si.findall(".//a:t", ns)] strings.append("".join(parts)) except KeyError: pass return strings def parse_xlsx(path: Path) -> dict[str, Any]: rows_out: list[list[str]] = [] with zipfile.ZipFile(path) as zf: shared = _read_shared_strings(zf) with zf.open("xl/worksheets/sheet1.xml") as f: root = ET.fromstring(f.read()) ns = {"a": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"} for row in root.findall(".//a:sheetData/a:row", ns): vals = [] for c in row.findall("a:c", ns): cell_type = c.attrib.get("t") v = c.find("a:v", ns) if v is None or v.text is None: vals.append("") continue if cell_type == "s": idx = int(v.text) vals.append(shared[idx] if 0 <= idx < len(shared) else "") else: vals.append(v.text) rows_out.append(vals) return {"format": "excel", "row_count": len(rows_out), "sample_rows": rows_out[:20]} def parse_xmind(path: Path) -> dict[str, Any]: with zipfile.ZipFile(path) as zf: names = set(zf.namelist()) if "content.json" in names: data = json.loads(zf.read("content.json").decode("utf-8", errors="ignore")) titles: list[str] = [] def walk(node: Any): if isinstance(node, dict): title = node.get("title") if isinstance(title, str) and title.strip(): titles.append(title.strip()) for k in ("children", "topics", "rootTopic", "attached"): walk(node.get(k)) elif isinstance(node, list): for i in node: walk(i) walk(data) return {"format": "xmind", "topic_count": len(titles), "topics": titles[:200]} if "content.xml" in names: root = ET.fromstring(zf.read("content.xml")) titles = [el.text.strip() for el in root.findall(".//title") if el.text and el.text.strip()] return {"format": "xmind", "topic_count": len(titles), "topics": titles[:200]} raise ValueError("Unsupported XMind package structure") def detect_format(path: Path, forced: str | None) -> str: if forced and forced != "auto": return forced ext = path.suffix.lower() return { ".md": "markdown", ".markdown": "markdown", ".json": "json", ".csv": "csv", ".docx": "word", ".xlsx": "excel", ".xmind": "xmind", }.get(ext, "markdown") def main() -> None: parser = argparse.ArgumentParser(description="Parse common QA output formats into normalized JSON") parser.add_argument("input", type=Path, help="Input file path") parser.add_argument("--format", default="auto", choices=["auto", "word", "excel", "xmind", "json", "csv", "markdown"]) parser.add_argument("--output", type=Path, help="Output JSON path (default: stdout)") args = parser.parse_args() fmt = detect_format(args.input, args.format) if fmt == "word": result = parse_docx(args.input) elif fmt == "excel": result = parse_xlsx(args.input) elif fmt == "xmind": result = parse_xmind(args.input) elif fmt == "json": result = parse_json(args.input) elif fmt == "csv": result = parse_csv_file(args.input) else: result = parse_markdown(args.input) result["source"] = str(args.input) out = json.dumps(result, ensure_ascii=False, indent=2) if args.output: args.output.write_text(out, encoding="utf-8") else: print(out) if __name__ == "__main__": main() -
parse_json.py 273 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'json'] + sys.argv[1:])) -
parse_markdown.py 277 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'markdown'] + sys.argv[1:])) -
parse_output_formats.py 253 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL] + sys.argv[1:])) -
parse_word.py 273 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'word'] + sys.argv[1:])) -
parse_xmind.py 274 B
#!/usr/bin/env python3 import os import subprocess import sys LOCAL = os.path.normpath(os.path.join(os.path.dirname(__file__), 'parse_formats.py')) if __name__ == '__main__': raise SystemExit(subprocess.call([sys.executable, LOCAL, '--format', 'xmind'] + sys.argv[1:]))
-
-
output-formats.md 401 B
# Output Format Options This skill **defaults to Markdown**. To get **Excel**, **CSV**, or **JSON**, add at the **end** of your request: - **Excel:** "Please output as tab-separated table for pasting into Excel." - **CSV:** "Please output as CSV (comma-separated, header row first)." - **JSON:** "Please output as JSON." See repo `skills/testing-types/_output-formats-template-en.md` for examples. -
quick-start.md 2.2 KB
# Automation Testing: Quick Start Use the `automation-testing` Skill to plan maintainable automation across the right test layers and existing toolchain. ## 1. Prepare the Minimum Context Provide what is available from this list: - application architecture - current tests - CI constraints - environments - failure history - scope, version, environment, time budget, and prohibited actions - known failures, existing tests, and evidence links If some information is unavailable, say so explicitly. The Skill should still return a bounded first pass with assumptions and open questions. ## 2. Invoke the Skill ```text @skill automation-testing Scenario: Automate a login-to-checkout smoke flow in the existing CI pipeline. Prioritize the highest risks, distinguish facts from assumptions, and provide an executable result with evidence requirements and open questions. ``` ## 3. Follow the Execution Flow 1. Confirm the objective, subject, scope, and success criteria. 2. Audit the completeness and credibility of the supplied material. 3. Identify failure modes and rank them by impact, likelihood, and detectability. 4. Produce concrete scenarios, checks, or decisions with expected results. 5. Record evidence gaps, residual risk, and the smallest useful next action. ## 4. Minimum Coverage Unless the request narrows scope, cover: - test pyramid - selector and contract stability - data isolation - retries - parallelism - reporting - maintenance cost - confirmed facts, working assumptions, and open questions - P0/P1 priorities and the rationale for lower-priority deferral - blockers, stop conditions, and residual risk ## 5. Review the Result - Every high-risk item has a concrete failure mode and expected behavior. - Priorities are justified rather than evenly distributed. - No field, API, metric, environment, or execution result is invented. - The artifact can be executed or reviewed without guessing the next step. - Production, security, and privacy work uses least privilege and sanitized data. ## 6. Continue Add missing evidence and ask the Skill to refine the same artifact. Keep confirmed facts separate from newly introduced assumptions so changes remain reviewable. -
README.md 820 B
# Automation Testing ## Skill Overview Need help with automation testing in a real project context; Need an output that can be used directly for execution, review, or follow-up. ## How to Use 1. Open `SKILL.md` in this folder and confirm this skill fits your task. 2. In your AI tool, call `@skill automation-testing`, then add your real project context and goal. 3. If you need a specific output format (table, checklist, report), include it directly in your request. ## One-Click Install Script Run from the repository root: ### macOS / Linux ```bash bash ./scripts/install-skills-mac.sh --tool codex --lang en --skill automation-testing ``` ### Windows PowerShell ```powershell powershell -ExecutionPolicy Bypass -File .\scripts\install-skills-windows.ps1 -Tool codex -Lang en -Skill automation-testing ``` -
SKILL.md 2.5 KB
--- name: automation-testing description: Use this skill when you need to design automation testing approaches using patterns like POM, data-driven testing, or BDD; triggers include automation testing and test automation strategy. --- # Automation Testing (English) **Chinese version:** See the corresponding Chinese skill. ## When to Use - Need help with automation testing in a real project context. - Need an output that can be used directly for execution, review, or follow-up. ## Workflow 1. Read and follow the main prompt listed under Progressive disclosure (coverage, structure, quality bar). 2. Add only project context that changes the result: scope, environment, constraints, risks, dependencies, expected deliverable. 3. If input is incomplete, return a usable first draft and explicitly mark assumptions and gaps. 4. Default to Markdown; switch formats only when the user asks. ## Core Constraints - Prioritize by risk / business impact — do not treat everything equally. - Separate confirmed facts from current assumptions. - Do not invent endpoints, fields, environments, or root causes the user did not provide. - Keep output executable: concrete scenarios, clear priority, clear next steps. ## Progressive Disclosure - Before producing output, read and follow `prompts/automation-testing.md` (minimum coverage, output structure, quality bar). - When Excel/CSV/JSON/Word is requested: read `output-formats.md` and honor the format. - When a ready-made template fits: use matching files under `output-templates/`. - For deep framework/troubleshoot/schema notes: read only the relevant file(s) under `references/`, do not load the whole directory. - For format conversion or helper checks: prefer existing `scripts/` over reinventing. - For evaluating/regressing this skill: use `evals/` with skill-up. ## Pre-delivery Checklist - [ ] Followed the main prompt's output structure - [ ] Minimum coverage focus: scope and objective, automation candidates, manual-only areas, priority by risk and ROI, framework alignment, data and environment needs, flaky-test risks, CI or execution strategy, ... (details in main prompt) - [ ] Covered the minimum checklist, or explained omissions - [ ] High-risk items have explicit priority - [ ] Did not invent details the user did not provide - [ ] Assumptions and gaps are marked ## Common Pitfalls - Do not pretend completeness when scope/context is missing. - Do not treat every item as equally important. - Do not skip assumptions and information gaps. - Do not dump generic theory unrelated to the current toolchain. -
tutorial.md 4.1 KB
# Automation Testing: Practical Tutorial This tutorial shows how to plan maintainable automation across the right test layers and existing toolchain with a repeatable, evidence-driven workflow. ## Learning Objectives By the end, you should be able to: - frame a bounded testing objective; - identify the evidence needed before making decisions; - turn domain risk into executable scenarios and assertions; - choose priorities without treating every check as equally important; - report residual risk without inventing certainty. ## Scenario Automate a login-to-checkout smoke flow in the existing CI pipeline. Use the scenario as a starting point. Replace it with real project material whenever possible. ## Step 1: Establish the Input Contract Collect: - application architecture - current tests - CI constraints - environments - failure history - the exact build, version, environment, and test window - authorized accounts, sanitized data, and allowed side effects - existing tests, defects, monitoring, and rollback constraints Create three lists before analysis: 1. **Confirmed facts** — directly supported by supplied material. 2. **Working assumptions** — necessary for a first pass but not yet proven. 3. **Open questions** — missing information that changes scope or risk. ## Step 2: Build the Risk Model For each capability or user journey, record: | Field | Meaning | | --- | --- | | Failure mode | What can go wrong | | Impact | User, business, security, or operational consequence | | Likelihood | Why the failure is plausible | | Detectability | Whether existing checks or telemetry reveal it | | Priority | P0, P1, P2, or P3 with rationale | | Evidence | What would confirm correct or incorrect behavior | Start with irreversible data loss, security exposure, financial impact, critical journey failure, and release blockers. ## Step 3: Design Coverage Minimum domain coverage: - test pyramid - selector and contract stability - data isolation - retries - parallelism - reporting - maintenance cost For each important scenario, specify: ```text Preconditions: Test data: Action or stimulus: Expected behavior: Evidence to capture: Cleanup or rollback: ``` Avoid generic statements such as “test positive and negative cases” unless the concrete cases are listed. ## Step 4: Execute Safely - Use the lowest environment and privilege level that can answer the question. - Prefer mocks, sandboxes, dry runs, and synthetic or masked data. - Do not run destructive production actions without explicit authorization. - Record timestamps, versions, configuration, and identifiers needed for reproduction. - Stop when a predefined safety, data-integrity, or customer-impact condition is met. ## Step 5: Interpret Evidence Separate observation from explanation: ```text Observation: what the system or test actually showed Hypothesis: a possible explanation Counter-evidence: facts that weaken the hypothesis Verification: the smallest experiment that can distinguish alternatives Conclusion: only what the available evidence supports ``` One failure message, high resource value, or correlated metric is not automatically a root cause. ## Step 6: Report the Result Use this order: 1. Task understanding and scope 2. Input audit 3. Risks and priorities 4. Executed or proposed coverage 5. Results and evidence 6. Blockers and residual risk 7. Next actions and owners ## Reusable Request ```text @skill automation-testing Analyze this real project context using the Skill's complete prompt contract. Separate confirmed facts, working assumptions, and open questions. Prioritize P0/P1 risks, provide executable scenarios with expected results and evidence, and state stop conditions and residual risk. ``` ## Completion Checklist - [ ] The objective and success criteria are explicit. - [ ] The environment, version, and data boundary are recorded. - [ ] High-risk paths have concrete scenarios and expected results. - [ ] Evidence supports conclusions. - [ ] Missing information and assumptions remain visible. - [ ] Cleanup, rollback, or human-handoff conditions are defined. - [ ] The next action is small, owned, and verifiable.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.