Claude Skill

windows-qa-engineer

Use when testing Windows 11 desktop apps (WinForms/WPF/UWP) via UFO UIA/Win32 automation MCP. Triggers on "test this Windows app", "QA the app", "run smoke test", "click the button", "fill the form", "check the UI", "Windows automation", "UFO QA", "verify the dialog", or any Wind

LLM Mart · 0 points · 3 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download codealive-ai-ai-driven-development-skills_windows-qa-engineer-68a302a.zip · 18 KB
Part of codealive-ai/ai-driven-development — 21 skills

Install

skills CLI npx skills add https://github.com/CodeAlive-AI/ai-driven-development/tree/main/skills/windows-qa-engineer
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install codealive-ai-ai-driven-development@llmmart
Git git clone https://github.com/CodeAlive-AI/ai-driven-development.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole codealive-ai/ai-driven-development collection as a plugin from our marketplace. Git is the plain clone.

README

Windows QA Engineer Skill

Skill that turns the agent into a manual QA operator for Windows 11 desktop apps. Runs on the same desktop as the SUT — no mocks, no browser-only tricks.

Tech Stack

Layer Technology Role
UI Automation Microsoft UFO Windows UI automation framework. Provides ControlInspectorFacade, ActionExecutor, AppPuppeteer, PhotographerFacade for real control discovery and interaction
Accessibility Backend UI Automation (UIA) / Win32 OS-level accessibility APIs that UFO uses to inspect and manipulate UI controls. UIA is default; Win32 available as fallback
MCP Servers UFO UICollector + HostUIExecutor + AppUIExecutor UFO's built-in MCP servers (ufo/client/mcp/local_servers/ui_mcp_server.py) registered via MCPRegistry. Provide tools: get_desktop_app_info, select_application_window, get_app_window_controls_info, click_input, set_edit_text, texts, capture_window_screenshot, etc.
Server Composition FastMCP mount() Composes UFO's 3 MCP servers into a single stdio endpoint so Claude Code needs only one .mcp.json entry
Protocol MCP (Model Context Protocol) over stdio Standard protocol connecting Claude Code to UFO's automation tools
Target Apps WinForms, WPF, UWP, Win32 Any Windows desktop app exposing an accessibility tree in the same interactive session

How It Works

Claude Code ──stdio──▶ FastMCP server ──mount()──▶ UFO UICollector
                                       ──mount()──▶ UFO HostUIExecutor
                                       ──mount()──▶ UFO AppUIExecutor
                                                         │
                                                    UIA / Win32
                                                         │
                                                   Windows Desktop
                                                    (real SUT)

The skill's MCP server (scripts/ufo_windows_qa_mcp_server.py) imports UFO's server factories via MCPRegistry, mounts all three into one FastMCP instance, and adds QA helper tools:

  • qa_refresh_and_list_windows — refresh + list in one call
  • qa_refresh_controls — re-collect control tree for selected window
  • qa_wait_for_text_contains — polling assertion (avoids arbitrary sleeps)

QA Workflow

1. Discover windows    →  qa_refresh_and_list_windows()
2. Select SUT          →  select_application_window(id, name)
3. Screenshot baseline →  capture_window_screenshot()
4. Collect controls    →  get_app_window_controls_info(field_list=[...])
5. Interact by id/name →  click_input / set_edit_text / keyboard_input
6. Assert              →  qa_wait_for_text_contains(id, name, expected)
7. Report              →  PASS/FAIL + screenshots + execution log

Requirements

  • Windows 11 or Windows Server 2025 with Desktop Experience
  • Python 3.10.x (required by the currently pinned pandas/faiss Windows wheels)
  • Microsoft UFO — git clone + pip install -r requirements.txt
  • FastMCP — pip install fastmcp
  • Pydantic — comes with FastMCP

Install

Via Skills CLI

npx skills add CodeAlive-AI/ai-driven-development@windows-qa-engineer -g -y

Manual

  1. Clone this repo
  2. Copy windows-qa-engineer/ to ~/.claude/skills/
  3. Add the MCP server config to your project .mcp.json:
{
  "mcpServers": {
    "ufo-windows-qa": {
      "type": "stdio",
      "command": "C:\\Users\\<you>\\UFO\\.venv\\Scripts\\python.exe",
      "args": ["C:\\path\\to\\windows-qa-engineer\\scripts\\ufo_windows_qa_mcp_server.py"],
      "env": {
        "UFO_ROOT": "C:\\Users\\<you>\\UFO",
        "PYTHONPATH": "C:\\Users\\<you>\\UFO",
        "CONTROL_BACKEND": "uia",
        "SHOW_VISUAL_OUTLINE_ON_SCREEN": "true"
      }
    }
  }
}
  1. Restart the MCP client and verify that the tools appear

Run the MCP client and the application under test in the same logged-in interactive Windows desktop session. A process started through SSH, WinRM, CI, or a Windows service runs headlessly and cannot provide a usable UIA window tree or screenshot.

For an end-to-end environment check, run scripts/interactive_smoke_test.py with UFO's venv Python from an interactive terminal. It launches Notepad, discovers and selects its window through the MCP server, captures a PNG, and writes a machine-readable report in the user's temporary directory.

Usage

/windows-qa-engineer Calculator "verify 2+2=4"

Or describe what to test:

"Test the login flow on MyApp — enter admin/password, click Login, verify the welcome screen"

Skill Contents

windows-qa-engineer/
├── SKILL.md                              # Workflow instructions for Claude
├── scripts/
│   ├── ufo_windows_qa_mcp_server.py      # FastMCP server (UFO mount composition)
│   └── doctor.ps1                        # Environment validation
├── references/
│   ├── setup.md                          # UFO install + MCP config
│   └── qa-workflows.md                   # Examples + locator strategy
└── assets/
    └── test-case.md                      # Test case output template

License

MIT

Skill manifest

Windows QA Engineer (UFO-powered)

You are an AI-QA operator on the same interactive Windows desktop as the SUT. All automation uses UFO's real MCP tools (UICollector, HostUIExecutor, AppUIExecutor) -- no mocks.

The agent, UFO MCP process, and SUT must run in the same logged-in interactive Windows desktop session. SSH, WinRM, CI services, and Session 0 are headless even when they can start the SUT process; use an interactive console, RDP session, or a secured virtual desktop. Never expose RDP publicly merely to satisfy this requirement.

Auto-Setup (when MCP tools are missing)

If UFO tools are NOT available as MCP tools, run setup before QA work:

  1. Run: python "<skill-dir>/scripts/skill_installer.py" --project-dir "<project-root>"
  2. Parse the JSON output — if success is true, restart the MCP client
  3. If failed, show the error and direct user to references/setup.md for manual install

Mandatory Workflow

Follow this sequence for every test run. Do not skip steps.

1. Discover windows

  • Call qa_refresh_and_list_windows()
  • Identify the SUT window by title hint from the user

2. Select window

  • Call select_application_window(id, name) (HostUIExecutor)
  • Call capture_window_screenshot() (UICollector) -- baseline screenshot

3. Collect controls

  • Call qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])
  • Anchor on id + control_text / automation_id when the returned tree is usable
  • If control collection returns an error or an empty tree for a large/legacy WinForms window, continue with screenshot inspection and coordinate actions; do not repeatedly force full UIA subtree scans

4. Interact

  • Use click_input(id, name), set_edit_text(id, name, text), keyboard_input(id, name, keys)
  • Coordinate actions only as last resort (document why)
  • Re-collect controls after navigation or dialog open

5. Assert

  • Read with texts(id, name) and compare against expected
  • Prefer qa_wait_for_text_contains(id, name, expected, timeout_s=10) over sleeps
  • Screenshot after each major checkpoint

6. Report

  • Fill assets/test-case.md template
  • Numbered execution log (step -> tool call -> result)
  • Final PASS/FAIL with exact failing assertion if applicable
  • Attach screenshot base64 strings from capture_window_screenshot()

Tool Reference

Tool Server Purpose
qa_refresh_and_list_windows QA helper Refresh + list all windows
select_application_window HostUIExecutor Select SUT by id+name
get_app_window_controls_info UICollector Raw control tree; use only when helper output is insufficient
capture_window_screenshot UICollector Screenshot selected window
click_input AppUIExecutor Click control by id+name
set_edit_text AppUIExecutor Type into control
keyboard_input AppUIExecutor Send keystrokes
texts AppUIExecutor Read control text
qa_wait_for_text_contains QA helper Poll until text matches
qa_refresh_controls QA helper Re-collect control tree with fail-soft parsing

Example: Login Smoke Test

User says: "Test the login flow on MyApp"

1. qa_refresh_and_list_windows() → find "MyApp - Login"
2. select_application_window(id="3", name="MyApp - Login")
3. capture_window_screenshot() → baseline
4. qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])
   → find username (id=12), password (id=14), login button (id=16)
5. set_edit_text(id="12", name="Username", text="testuser")
6. set_edit_text(id="14", name="Password", text="pass123")
7. click_input(id="16", name="Login")
8. qa_wait_for_text_contains(id="20", name="WelcomeLabel", expected_substring="Welcome", timeout_s=10)
   → {"ok": true, "text": "Welcome, testuser"}
9. capture_window_screenshot() → post-login
10. Report: PASS

Error Handling

No windows found: Re-check the SUT is running. Call qa_refresh_and_list_windows() again. If still empty, ask the user to confirm the app is open.

Empty control tree: The window may not have finished loading. Wait 2-3 seconds, then qa_refresh_controls(field_list=[...]). If still empty, try CONTROL_BACKEND=win32 (see setup.md). For large or legacy WinForms apps, avoid repeated full UIA subtree scans and use screenshot plus targeted coordinates.

Control not clickable / action fails: Re-collect controls (the tree may have changed after navigation). If the control lacks a usable id, fall back to coordinate-based action and document why.

MCP tools not found: Run auto-setup first (see Auto-Setup above). If auto-setup fails, direct the user to references/setup.md and run doctor.ps1.

Detailed Workflows

See references/qa-workflows.md for more examples, locator strategy, and common patterns.

Setup

See references/setup.md for UFO installation, MCP configuration, and diagnostics.

Files (ai-driven-development)
  • assets
    • test-case.md 402 B
      # Test Case: <title>
      
      ## Metadata
      - App / Build:
      - Windows:
      - Date:
      - Tester:
      - Type: Smoke / Regression / Exploratory
      
      ## Preconditions
      - ...
      
      ## Steps / Expected / Assertions
      1) Action:
         Expected:
         Assertion (tool + control):
      2) ...
      
      ## Artifacts
      - Screenshot before:
      - Screenshot after:
      
      ## Verdict
      PASS / FAIL
      
      ## Stability Notes
      - Controls used (ids + names):
      - Any coordinate fallback + why:
      
  • references
    • qa-workflows.md 1.4 KB
      # QA Workflows
      
      ## Table of Contents
      - [Smoke Test (Login Example)](#smoke-test-login-example)
      - [Locator Strategy](#locator-strategy)
      - [Common Patterns](#common-patterns)
      
      ## Smoke Test (Login Example)
      
      ```
      1) qa_refresh_and_list_windows()
      2) select_application_window(id, name)
      3) capture_window_screenshot()
      4) get_app_window_controls_info(field_list=["label","control_text","control_type","automation_id","control_rect"])
      5) set_edit_text(id, name, "username_value")
      6) set_edit_text(id, name, "password_value")
      7) click_input(id, name)   # login button
      8) qa_wait_for_text_contains(id, name, "Welcome", timeout_s=10)
      9) capture_window_screenshot()
      ```
      
      ## Locator Strategy
      
      1. Prefer control IDs from `get_app_window_controls_info`
      2. Use `automation_id` as a human-readable secondary anchor when available
      3. Re-collect controls after navigation or dialog open (control tree changes)
      4. Use coordinate actions only as last resort, with a comment explaining why
      
      ## Common Patterns
      
      **Form fill + submit**: collect controls -> set_edit_text for each field -> click_input on submit -> assert result text
      
      **Navigation**: click menu/tab -> re-collect controls -> verify new view via screenshot + texts()
      
      **Dialog handling**: after triggering dialog, re-collect controls (new control tree) -> interact -> close dialog -> re-collect parent window controls
      
      **Data grid verification**: collect controls -> find grid cells by control_text -> compare against expected values
      
    • setup.md 3.1 KB
      # Setup: UFO + MCP Server
      
      ## Table of Contents
      - [Install UFO](#install-ufo)
      - [Configure MCP in Claude Code](#configure-mcp-in-claude-code)
      - [Verify](#verify)
      - [Backend Selection](#backend-selection)
      
      ## Install UFO
      
      Prerequisites: Windows 11 or Windows Server 2025 with Desktop Experience, Python 3.10.x, and Git. The pinned UFO requirements currently need Python 3.10 on Windows: `faiss-cpu==1.8.0` is unavailable for Python 3.13 and `pandas==1.4.3` is unavailable for Python 3.11.
      
      ```powershell
      cd $env:USERPROFILE
      git clone https://github.com/microsoft/UFO.git
      cd UFO
      py -3.10 -m venv .venv
      .\.venv\Scripts\Activate.ps1
      pip install -r requirements.txt
      ```
      
      Quick check:
      ```powershell
      python -c "from ufo.client.mcp.local_servers import load_all_servers; load_all_servers(); print('OK')"
      python -c "from ufo.client.mcp.mcp_registry import MCPRegistry; print(MCPRegistry.list())"
      ```
      
      Expected: UICollector, HostUIExecutor, AppUIExecutor registered.
      
      ## Configure MCP in Claude Code
      
      Add to your project `.mcp.json`:
      
      ```json
      {
        "mcpServers": {
          "ufo-windows-qa": {
            "type": "stdio",
            "command": "C:\\Users\\<you>\\UFO\\.venv\\Scripts\\python.exe",
            "args": [
              "C:\\path\\to\\windows-qa-engineer\\scripts\\ufo_windows_qa_mcp_server.py"
            ],
            "env": {
              "UFO_ROOT": "C:\\Users\\<you>\\UFO",
              "PYTHONPATH": "C:\\Users\\<you>\\UFO",
              "CONTROL_BACKEND": "uia",
              "MAXIMIZE_WINDOW": "false",
              "SHOW_VISUAL_OUTLINE_ON_SCREEN": "true",
              "RUN_CONFIGS": "true"
            }
          }
        }
      }
      ```
      
      Replace `<you>` and the skill path with absolute paths, restart the MCP client, and confirm that the tools are available.
      
      ## Verify
      
      Run the doctor script from any directory; it selects UFO's venv and working directory automatically:
      ```powershell
      .\.claude\skills\windows-qa-engineer\scripts\doctor.ps1
      ```
      
      Or check in Claude Code that these tools appear:
      `get_desktop_app_info`, `select_application_window`, `get_app_window_controls_info`,
      `click_input`, `set_edit_text`, `texts`, `capture_window_screenshot`,
      `qa_refresh_and_list_windows`, `qa_refresh_controls`, `qa_wait_for_text_contains`.
      
      The doctor verifies imports and MCP registration but cannot create an interactive desktop. For an end-to-end check, run the MCP client and the application under test in the same logged-in Windows session, call `qa_refresh_and_list_windows()`, select a visible app such as Notepad, and capture its screenshot. An empty window list from SSH or a service session is expected and does not verify desktop automation.
      
      The bundled smoke test performs that complete Notepad workflow and writes a PNG plus JSON report under the user's temporary directory:
      
      ```powershell
      & "$env:USERPROFILE\UFO\.venv\Scripts\python.exe" `
        "<skill-dir>\scripts\interactive_smoke_test.py" `
        --project-dir "<project-root>"
      ```
      
      Run it from a terminal inside the same interactive desktop session, not through SSH or a service.
      
      ## Backend Selection
      
      - `CONTROL_BACKEND=uia` (default, recommended for WinForms/WPF stability)
      - `CONTROL_BACKEND=win32` (fallback if UIA fails for a specific SUT)
      
      Set via the `env` block in `.mcp.json`.
      
  • scripts
    • doctor.ps1 1 KB · in bundle
    • interactive_smoke_test.py 4.6 KB
      #!/usr/bin/env python3
      """Exercise UFO discovery, window selection, and capture against Windows Notepad."""
      
      from __future__ import annotations
      
      import argparse
      import asyncio
      import base64
      import json
      import subprocess
      import tempfile
      from enum import Enum
      from pathlib import Path
      from typing import Any
      
      from fastmcp import Client
      
      
      def result_data(result: Any) -> Any:
          structured = getattr(result, "structured_content", None)
          if isinstance(structured, dict) and set(structured) == {"result"}:
              return structured["result"]
          if structured is not None:
              return structured
          return getattr(result, "data", None)
      
      
      def to_plain(value: Any) -> Any:
          if value is None or isinstance(value, (bool, int, float, str)):
              return value
          if isinstance(value, Enum):
              return to_plain(value.value)
          if isinstance(value, Path):
              return str(value)
          root = getattr(value, "root", None)
          if root is not None:
              return to_plain(root)
          if hasattr(value, "model_dump"):
              return to_plain(value.model_dump())
          if isinstance(value, dict):
              return {str(key): to_plain(item) for key, item in value.items()}
          if isinstance(value, (list, tuple)):
              return [to_plain(item) for item in value]
          if isinstance(value, type):
              return f"{value.__module__}.{value.__qualname__}"
          attributes = getattr(value, "__dict__", None)
          if isinstance(attributes, dict):
              public_attributes = {
                  str(key): to_plain(item)
                  for key, item in attributes.items()
                  if not str(key).startswith("_")
              }
              if public_attributes:
                  return public_attributes
          return str(value)
      
      
      async def run(project_dir: Path, output_dir: Path) -> dict[str, Any]:
          config_path = project_dir / ".mcp.json"
          if not config_path.exists():
              raise FileNotFoundError(f"MCP configuration not found: {config_path}")
      
          output_dir.mkdir(parents=True, exist_ok=True)
          screenshot_path = output_dir / "notepad.png"
          report_path = output_dir / "report.json"
          notepad = subprocess.Popen(["notepad.exe"])
          report: dict[str, Any] = {"success": False}
      
          try:
              config = json.loads(config_path.read_text(encoding="utf-8"))
              async with Client(config, timeout=45) as client:
                  await asyncio.sleep(2)
                  tools = await client.list_tools()
                  report["tool_count"] = len(tools)
      
                  discovery = await client.call_tool("qa_refresh_and_list_windows", {})
                  windows = to_plain(result_data(discovery) or [])
                  report["windows"] = windows
                  match = next(
                      (
                          window
                          for window in windows
                          if "notepad" in json.dumps(window, ensure_ascii=False).lower()
                      ),
                      None,
                  )
                  if not isinstance(match, dict):
                      raise RuntimeError("Notepad was not found in the interactive window list")
      
                  window_id = str(match.get("id", ""))
                  window_name = str(match.get("name", ""))
                  if not window_id or not window_name:
                      raise RuntimeError(f"Unsupported window record: {match!r}")
      
                  await client.call_tool(
                      "select_application_window", {"id": window_id, "name": window_name}
                  )
                  capture = await client.call_tool("capture_window_screenshot", {})
                  encoded = to_plain(result_data(capture))
                  if not isinstance(encoded, str):
                      raise RuntimeError("UFO returned no screenshot string")
                  if encoded.startswith("data:image/"):
                      encoded = encoded.split(",", 1)[1]
                  screenshot_path.write_bytes(base64.b64decode(encoded, validate=True))
      
                  report.update(
                      {
                          "success": True,
                          "window": {"id": window_id, "name": window_name},
                          "screenshot": str(screenshot_path),
                      }
                  )
                  return report
          except Exception as exc:
              report["error"] = str(exc)
              raise
          finally:
              report_path.write_text(json.dumps(report, indent=2), encoding="utf-8")
              notepad.terminate()
      
      
      def main() -> None:
          parser = argparse.ArgumentParser()
          parser.add_argument("--project-dir", type=Path, default=Path.cwd())
          parser.add_argument(
              "--output-dir",
              type=Path,
              default=Path(tempfile.gettempdir()) / "windows-qa-engineer-smoke",
          )
          args = parser.parse_args()
          report = asyncio.run(run(args.project_dir.resolve(), args.output_dir.resolve()))
          print(json.dumps(report, indent=2))
      
      
      if __name__ == "__main__":
          main()
      
    • skill_installer.py 21.5 KB
      #!/usr/bin/env python3
      """
      skill_installer.py — General-purpose installer for agent skills.
      
      Reads install.yaml from a skill directory and executes the declared steps:
      platform check, python check, git clone, venv, pip install, MCP config, verify.
      
      Usage:
          python skill_installer.py [--skill-dir PATH] [--project-dir PATH]
      
      Options:
          --skill-dir   Path to the skill root (default: parent of this script)
          --project-dir Path to the project root for .mcp.json (default: cwd)
      """
      
      from __future__ import annotations
      
      import argparse
      import json
      import os
      import platform
      import subprocess
      import sys
      import tempfile
      from pathlib import Path
      from typing import Any
      
      
      def load_manifest(skill_dir: Path) -> dict[str, Any]:
          """Load and parse install.yaml from the skill directory."""
          manifest_path = skill_dir / "install.yaml"
          if not manifest_path.exists():
              fail(f"install.yaml not found at {manifest_path}")
      
          try:
              import yaml
          except ImportError:
              # Fall back to a minimal YAML parser for simple manifests
              return _parse_simple_yaml(manifest_path)
      
          with open(manifest_path) as f:
              return yaml.safe_load(f)
      
      
      def _parse_simple_yaml(path: Path) -> dict[str, Any]:
          """Minimal YAML parser for install.yaml without PyYAML dependency.
      
          Handles the subset of YAML used by install.yaml: scalars, lists,
          nested mappings (one level of indent), block scalars (|).
          """
          import re
      
          result: dict[str, Any] = {}
          current_key: str | None = None
          current_list: list | None = None
          current_map: dict | None = None
          current_map_key: str | None = None
          block_scalar_key: str | None = None
          block_scalar_lines: list[str] = []
          block_scalar_indent: int = 0
          list_of_maps: bool = False
          list_maps: list[dict] = []
          current_list_map: dict | None = None
          # Track nesting: top-level key that owns a map value
          map_owner: str | None = None
      
          lines = path.read_text().splitlines()
          i = 0
          while i < len(lines):
              line = lines[i]
              stripped = line.strip()
      
              # Skip comments and blank lines (unless in block scalar)
              if block_scalar_key:
                  indent = len(line) - len(line.lstrip())
                  if stripped == "" or indent >= block_scalar_indent:
                      block_scalar_lines.append(line[block_scalar_indent:] if indent >= block_scalar_indent else "")
                      i += 1
                      continue
                  else:
                      # Block scalar ended
                      text = "\n".join(block_scalar_lines).rstrip("\n") + "\n"
                      if map_owner and current_map is not None:
                          current_map[block_scalar_key] = text
                      else:
                          result[block_scalar_key] = text
                      block_scalar_key = None
                      block_scalar_lines = []
                      # Fall through to process current line
      
              if stripped == "" or stripped.startswith("#"):
                  i += 1
                  continue
      
              indent = len(line) - len(line.lstrip())
      
              # List item inside a list-of-maps (e.g., repos entries)
              if list_of_maps and indent >= 2 and stripped.startswith("- "):
                  # Save previous map
                  if current_list_map is not None:
                      list_maps.append(current_list_map)
                  current_list_map = {}
                  # Parse key: value from "- key: value"
                  item = stripped[2:].strip()
                  if ":" in item:
                      k, v = item.split(":", 1)
                      current_list_map[k.strip()] = _yaml_val(v.strip())
                  i += 1
                  continue
      
              if list_of_maps and current_list_map is not None and indent >= 4 and ":" in stripped:
                  k, v = stripped.split(":", 1)
                  v = v.strip()
                  if v == "|":
                      block_scalar_key = k.strip()
                      block_scalar_indent = indent + 2
                      block_scalar_lines = []
                      map_owner = None  # block scalar target is current_list_map handled separately
                      # Actually for list maps we need special handling
                      # For simplicity, read ahead
                      i += 1
                      while i < len(lines):
                          bl = lines[i]
                          bi = len(bl) - len(bl.lstrip())
                          if bl.strip() == "" or bi >= block_scalar_indent:
                              block_scalar_lines.append(bl[block_scalar_indent:] if bi >= block_scalar_indent else "")
                              i += 1
                          else:
                              break
                      current_list_map[k.strip()] = "\n".join(block_scalar_lines).rstrip("\n") + "\n"
                      block_scalar_key = None
                      block_scalar_lines = []
                      continue
                  else:
                      current_list_map[k.strip()] = _yaml_val(v)
                  i += 1
                  continue
      
              # End list-of-maps context on dedent
              if list_of_maps and indent == 0:
                  if current_list_map is not None:
                      list_maps.append(current_list_map)
                      current_list_map = None
                  result[current_key] = list_maps
                  list_of_maps = False
                  list_maps = []
                  current_key = None
                  # Fall through
      
              # End current map context on dedent
              if map_owner and indent == 0:
                  result[map_owner] = current_map
                  current_map = None
                  map_owner = None
                  current_key = None
      
              # End simple list on dedent
              if current_list is not None and indent == 0 and not stripped.startswith("-"):
                  result[current_key] = current_list
                  current_list = None
                  current_key = None
      
              # Simple list item (e.g., "  - fastmcp")
              if current_list is not None and stripped.startswith("- "):
                  current_list.append(_yaml_val(stripped[2:].strip()))
                  i += 1
                  continue
      
              # Map sub-key (indent >= 2, inside a map owner)
              if map_owner and current_map is not None and indent >= 2 and ":" in stripped:
                  k, v = stripped.split(":", 1)
                  v = v.strip()
                  if v == "|":
                      i += 1
                      scalar_lines = []
                      scalar_indent = indent + 2
                      while i < len(lines):
                          scalar_line = lines[i]
                          scalar_line_indent = len(scalar_line) - len(scalar_line.lstrip())
                          if scalar_line.strip() == "" or scalar_line_indent >= scalar_indent:
                              scalar_lines.append(
                                  scalar_line[scalar_indent:]
                                  if scalar_line_indent >= scalar_indent
                                  else ""
                              )
                              i += 1
                          else:
                              break
                      current_map[k.strip()] = "\n".join(scalar_lines).rstrip("\n") + "\n"
                      continue
                  if v == "" or v.startswith("#"):
                      i += 1
                      nested_map = {}
                      nested_indent = indent + 2
                      while i < len(lines):
                          nested_line = lines[i]
                          nested_stripped = nested_line.strip()
                          nested_line_indent = len(nested_line) - len(nested_line.lstrip())
                          if nested_stripped == "" or nested_stripped.startswith("#"):
                              i += 1
                              continue
                          if nested_line_indent < nested_indent:
                              break
                          if nested_line_indent == nested_indent and ":" in nested_stripped:
                              nested_key, nested_value = nested_stripped.split(":", 1)
                              nested_map[nested_key.strip()] = _yaml_val(nested_value.strip())
                          i += 1
                      current_map[k.strip()] = nested_map
                      continue
                  current_map[k.strip()] = _yaml_val(v)
                  i += 1
                  continue
      
              # Top-level key
              if indent == 0 and ":" in stripped and not stripped.startswith("-"):
                  k, v = stripped.split(":", 1)
                  k = k.strip()
                  v = v.strip()
      
                  if v == "" or v.startswith("#"):
                      # Next lines define a nested structure — peek ahead
                      current_key = k
                      j = i + 1
                      while j < len(lines) and (lines[j].strip() == "" or lines[j].strip().startswith("#")):
                          j += 1
                      if j < len(lines):
                          next_stripped = lines[j].strip()
                          if next_stripped.startswith("- url:") or next_stripped.startswith("- url :"):
                              list_of_maps = True
                              list_maps = []
                              current_list_map = None
                          elif next_stripped.startswith("- "):
                              current_list = []
                          else:
                              # Nested map
                              map_owner = k
                              current_map = {}
                  elif v == "|":
                      block_scalar_key = k
                      block_scalar_indent = 2
                      block_scalar_lines = []
                  else:
                      result[k] = _yaml_val(v)
                  i += 1
                  continue
      
              i += 1
      
          # Flush any remaining context
          if block_scalar_key:
              text = "\n".join(block_scalar_lines).rstrip("\n") + "\n"
              result[block_scalar_key] = text
          if list_of_maps:
              if current_list_map is not None:
                  list_maps.append(current_list_map)
              result[current_key] = list_maps
          if current_list is not None:
              result[current_key] = current_list
          if map_owner and current_map is not None:
              result[map_owner] = current_map
      
          return result
      
      
      def _yaml_val(s: str) -> Any:
          """Convert a YAML scalar string to a Python value."""
          # Strip inline comments
          if "  #" in s:
              s = s[: s.index("  #")].strip()
          if s in ("true", "True"):
              return True
          if s in ("false", "False"):
              return False
          if s in ("null", "None", "~"):
              return None
          # Strip quotes
          if (s.startswith('"') and s.endswith('"')) or (s.startswith("'") and s.endswith("'")):
              return s[1:-1]
          try:
              return int(s)
          except ValueError:
              pass
          try:
              return float(s)
          except ValueError:
              pass
          return s
      
      
      # ---------------------------------------------------------------------------
      # Steps
      # ---------------------------------------------------------------------------
      
      def check_platform(manifest: dict) -> dict:
          """Check that current platform matches the manifest requirement."""
          required = manifest.get("platform", "any")
          if required == "any":
              return {"step": "platform", "status": "ok", "detail": platform.platform()}
      
          current = sys.platform  # win32, darwin, linux
          mapping = {"windows": "win32", "macos": "darwin", "linux": "linux"}
          expected = mapping.get(required, required)
      
          if current == expected:
              return {"step": "platform", "status": "ok", "detail": platform.platform()}
      
          return {
              "step": "platform",
              "status": "failed",
              "detail": f"Requires {required} but running on {platform.platform()}",
          }
      
      
      def check_python(manifest: dict) -> dict:
          """Check that current Python meets version constraint."""
          constraint = manifest.get("python")
          if not constraint:
              return {"step": "python", "status": "ok", "detail": platform.python_version()}
      
          import re
      
          for clause in (part.strip() for part in constraint.split(",")):
              match = re.fullmatch(r"([><=!]+)([\d.]+)", clause)
              if not match:
                  return {
                      "step": "python",
                      "status": "failed",
                      "detail": f"Unsupported Python constraint: {constraint}",
                  }
      
              op, ver_str = match.groups()
              required = tuple(int(x) for x in ver_str.split("."))
              current = sys.version_info[:len(required)]
              comparisons = {
                  ">=": current >= required,
                  ">": current > required,
                  "<=": current <= required,
                  "<": current < required,
                  "==": current == required,
                  "!=": current != required,
              }
              if op not in comparisons or not comparisons[op]:
                  return {
                      "step": "python",
                      "status": "failed",
                      "detail": (
                          f"Requires Python {constraint} but running "
                          f"{platform.python_version()}"
                      ),
                  }
      
          return {"step": "python", "status": "ok", "detail": platform.python_version()}
      
      
      def process_repos(manifest: dict) -> tuple[list[dict], Path | None]:
          """Clone repos, create venvs, install requirements. Returns (steps, venv_python)."""
          repos = manifest.get("repos", [])
          steps = []
          venv_python: Path | None = None
      
          for repo in repos:
              url = repo["url"]
              target = Path(os.path.expanduser(repo["target"]))
      
              # Clone
              if target.exists():
                  steps.append({"step": "clone", "status": "skipped", "detail": f"{target} already exists"})
              else:
                  run(["git", "clone", url, str(target)])
                  steps.append({"step": "clone", "status": "ok", "detail": str(target)})
      
              # Venv
              if repo.get("venv"):
                  venv_dir = target / ".venv"
                  if venv_dir.exists():
                      steps.append({"step": "venv", "status": "skipped", "detail": f"{venv_dir} already exists"})
                  else:
                      run([sys.executable, "-m", "venv", str(venv_dir)])
                      steps.append({"step": "venv", "status": "ok", "detail": str(venv_dir)})
      
                  # Resolve venv python
                  if sys.platform == "win32":
                      venv_python = venv_dir / "Scripts" / "python.exe"
                  else:
                      venv_python = venv_dir / "bin" / "python"
      
              # Requirements
              if repo.get("requirements"):
                  req_file = target / repo["requirements"]
                  pip_exe = venv_python or Path(sys.executable)
                  run([str(pip_exe), "-m", "pip", "install", "-r", str(req_file)])
                  steps.append({"step": "deps", "status": "ok", "detail": str(req_file)})
      
          return steps, venv_python
      
      
      def install_pip_packages(manifest: dict, venv_python: Path | None) -> dict | None:
          """Install additional pip packages."""
          packages = manifest.get("pip", [])
          if not packages:
              return None
      
          pip_exe = venv_python or Path(sys.executable)
          run([str(pip_exe), "-m", "pip", "install"] + packages)
          return {"step": "pip", "status": "ok", "detail": ", ".join(packages)}
      
      
      def configure_mcp(
          manifest: dict,
          skill_dir: Path,
          project_dir: Path,
          venv_python: Path | None,
      ) -> dict | None:
          """Register the MCP server in .mcp.json."""
          mcp_conf = manifest.get("mcp")
          if not mcp_conf:
              return None
      
          mcp_json_path = project_dir / ".mcp.json"
      
          # Read existing
          existing: dict[str, Any] = {}
          if mcp_json_path.exists():
              with open(mcp_json_path) as f:
                  existing = json.load(f)
      
          servers = existing.setdefault("mcpServers", {})
      
          # Build command — use venv python if available
          python_cmd = str(venv_python) if venv_python else "python"
          script_path = str(skill_dir / mcp_conf["script"])
      
          # Build env
          env: dict[str, str] = {}
      
          # PYTHONPATH from repos with pythonpath: true
          pythonpaths = []
          for repo in manifest.get("repos", []):
              if repo.get("pythonpath"):
                  pythonpaths.append(str(Path(os.path.expanduser(repo["target"]))))
          if pythonpaths:
              env["PYTHONPATH"] = os.pathsep.join(pythonpaths)
      
          # Merge env from manifest
          if mcp_conf.get("env"):
              env.update(
                  {
                      key: os.path.expanduser(str(value))
                      for key, value in mcp_conf["env"].items()
                  }
              )
      
          # Build server entry
          entry: dict[str, Any] = {
              "type": "stdio",
              "command": python_cmd,
              "args": [script_path],
          }
          if env:
              entry["env"] = env
      
          servers[mcp_conf["name"]] = entry
      
          # Write atomically
          tmp_fd, tmp_path = tempfile.mkstemp(dir=project_dir, suffix=".mcp.json.tmp")
          try:
              with os.fdopen(tmp_fd, "w") as f:
                  json.dump(existing, f, indent=2)
                  f.write("\n")
              os.replace(tmp_path, mcp_json_path)
          except Exception:
              os.unlink(tmp_path)
              raise
      
          return {"step": "mcp", "status": "ok", "detail": f"{mcp_json_path} updated"}
      
      
      def run_verify(manifest: dict, venv_python: Path | None) -> dict | None:
          """Run the verify script."""
          verify = manifest.get("verify")
          if not verify or not verify.get("script"):
              return None
      
          python_cmd = str(venv_python) if venv_python else sys.executable
      
          # Build PYTHONPATH
          pythonpaths = []
          for repo in manifest.get("repos", []):
              if repo.get("pythonpath"):
                  pythonpaths.append(str(Path(os.path.expanduser(repo["target"]))))
      
          env = os.environ.copy()
          if pythonpaths:
              existing_pp = env.get("PYTHONPATH", "")
              new_pp = os.pathsep.join(pythonpaths)
              env["PYTHONPATH"] = f"{new_pp}{os.pathsep}{existing_pp}" if existing_pp else new_pp
      
          result = subprocess.run(
              [python_cmd, "-c", verify["script"]],
              capture_output=True,
              text=True,
              env=env,
              cwd=pythonpaths[0] if pythonpaths else None,
          )
      
          if result.returncode == 0:
              detail = result.stdout.strip() or "passed"
              return {"step": "verify", "status": "ok", "detail": detail}
          else:
              detail = result.stderr.strip() or result.stdout.strip() or "verification failed"
              return {"step": "verify", "status": "failed", "detail": detail}
      
      
      # ---------------------------------------------------------------------------
      # Helpers
      # ---------------------------------------------------------------------------
      
      def run(cmd: list[str]) -> subprocess.CompletedProcess:
          """Run a subprocess, raising on failure."""
          result = subprocess.run(cmd, capture_output=True, text=True)
          if result.returncode != 0:
              stderr = result.stderr.strip()
              stdout = result.stdout.strip()
              detail = stderr or stdout or f"Command failed with exit code {result.returncode}"
              raise RuntimeError(f"Command failed: {' '.join(cmd)}\n{detail}")
          return result
      
      
      def fail(message: str) -> None:
          """Print error JSON and exit."""
          report = {"success": False, "error": message, "steps": []}
          print(json.dumps(report, indent=2))
          sys.exit(1)
      
      
      # ---------------------------------------------------------------------------
      # Main
      # ---------------------------------------------------------------------------
      
      def main() -> None:
          parser = argparse.ArgumentParser(description="Install skill dependencies from install.yaml")
          parser.add_argument(
              "--skill-dir",
              type=Path,
              default=Path(__file__).resolve().parent.parent,
              help="Path to the skill root (default: parent of scripts/)",
          )
          parser.add_argument(
              "--project-dir",
              type=Path,
              default=Path.cwd(),
              help="Project root for .mcp.json placement (default: cwd)",
          )
          args = parser.parse_args()
      
          skill_dir = args.skill_dir.resolve()
          project_dir = args.project_dir.resolve()
      
          manifest = load_manifest(skill_dir)
          steps: list[dict] = []
          success = True
      
          # 1. Platform check
          step = check_platform(manifest)
          steps.append(step)
          if step["status"] == "failed":
              success = False
              report = {"success": False, "steps": steps, "mcp_configured": False, "restart_required": False}
              print(json.dumps(report, indent=2))
              sys.exit(1)
      
          # 2. Python check
          step = check_python(manifest)
          steps.append(step)
          if step["status"] == "failed":
              success = False
              report = {"success": False, "steps": steps, "mcp_configured": False, "restart_required": False}
              print(json.dumps(report, indent=2))
              sys.exit(1)
      
          # 3. Repos (clone, venv, requirements)
          try:
              repo_steps, venv_python = process_repos(manifest)
              steps.extend(repo_steps)
          except RuntimeError as e:
              steps.append({"step": "repos", "status": "failed", "detail": str(e)})
              success = False
              report = {"success": False, "steps": steps, "mcp_configured": False, "restart_required": False}
              print(json.dumps(report, indent=2))
              sys.exit(1)
      
          # 4. Additional pip packages
          try:
              pip_step = install_pip_packages(manifest, venv_python)
              if pip_step:
                  steps.append(pip_step)
          except RuntimeError as e:
              steps.append({"step": "pip", "status": "failed", "detail": str(e)})
              success = False
              report = {"success": False, "steps": steps, "mcp_configured": False, "restart_required": False}
              print(json.dumps(report, indent=2))
              sys.exit(1)
      
          # 5. MCP config
          mcp_configured = False
          try:
              mcp_step = configure_mcp(manifest, skill_dir, project_dir, venv_python)
              if mcp_step:
                  steps.append(mcp_step)
                  mcp_configured = True
          except Exception as e:
              steps.append({"step": "mcp", "status": "failed", "detail": str(e)})
              success = False
      
          # 6. Verify
          if success:
              verify_step = run_verify(manifest, venv_python)
              if verify_step:
                  steps.append(verify_step)
                  if verify_step["status"] == "failed":
                      success = False
      
          # 7. Report
          report = {
              "success": success,
              "steps": steps,
              "mcp_configured": mcp_configured,
              "restart_required": mcp_configured and success,
          }
          print(json.dumps(report, indent=2))
      
          if not success:
              sys.exit(1)
      
      
      if __name__ == "__main__":
          main()
      
    • ufo_windows_qa_mcp_server.py 5.2 KB
      """
      ufo_windows_qa_mcp_server.py
      
      Stdio MCP server that exposes UFO's real Windows automation tools to an MCP client.
      
      Composes UFO's UICollector, HostUIExecutor, AppUIExecutor into ONE server
      via FastMCP.mount(). No mocks, no re-implementation.
      
      Requires: UFO installed (pip install from repo), fastmcp, pydantic
      """
      
      from __future__ import annotations
      
      import asyncio
      import json
      import logging
      import os
      import sys
      import time
      from typing import Annotated, Any, Dict, List, Optional
      
      ufo_root = os.environ.get("UFO_ROOT")
      if ufo_root:
          os.chdir(ufo_root)
          if ufo_root not in sys.path:
              sys.path.insert(0, ufo_root)
      
      from fastmcp import FastMCP
      from pydantic import Field
      
      from ufo.client.mcp.mcp_registry import MCPRegistry
      from ufo.client.mcp.local_servers import load_all_servers
      
      logger = logging.getLogger(__name__)
      
      
      def _get_ufo_server(namespace: str) -> FastMCP:
          """Load all UFO servers, then fetch by namespace."""
          load_all_servers()
          if not MCPRegistry.is_registered(namespace):
              raise RuntimeError(
                  f"UFO MCP server '{namespace}' not registered. "
                  f"Available: {MCPRegistry.list()}"
              )
          return MCPRegistry.get(namespace)
      
      
      # Compose into one server
      mcp = FastMCP("UFO Windows QA (UIA/Win32)")
      
      mcp.mount(_get_ufo_server("UICollector"))
      mcp.mount(_get_ufo_server("HostUIExecutor"))
      mcp.mount(_get_ufo_server("AppUIExecutor"))
      
      
      # QA helper tools (thin wrappers around UFO tools)
      
      def _parse_tool_result(result: Any) -> Any:
          """Return structured FastMCP results when possible."""
          def to_plain(value: Any) -> Any:
              root = getattr(value, "root", None)
              if root is not None:
                  return to_plain(root)
              if hasattr(value, "model_dump"):
                  return to_plain(value.model_dump())
              if isinstance(value, dict):
                  return {str(key): to_plain(item) for key, item in value.items()}
              if isinstance(value, (list, tuple)):
                  return [to_plain(item) for item in value]
              return value
      
          structured = getattr(result, "structured_content", None)
          if structured is not None:
              if isinstance(structured, dict) and set(structured.keys()) == {"result"}:
                  return to_plain(structured["result"])
              return to_plain(structured)
      
          content = getattr(result, "content", None)
          if content:
              text = getattr(content[0], "text", None)
              if text is not None:
                  try:
                      return to_plain(json.loads(text))
                  except json.JSONDecodeError:
                      return text
      
          return to_plain(result)
      
      
      async def _call_mounted_tool(name: str, arguments: Dict[str, Any]) -> Any:
          """Call one of the mounted UFO tools through FastMCP's public Tool API."""
          tool = await mcp.get_tool(name)
          return await tool.run(arguments)
      
      
      @mcp.tool()
      async def qa_refresh_and_list_windows(
          remove_empty: Annotated[bool, Field(description="Drop empty/ghost windows.")] = True
      ) -> Annotated[List[Dict[str, Any]], Field(description="Window list.")]:
          """Refresh + list windows in one call. Wraps UICollector.get_desktop_app_info."""
          result = await _call_mounted_tool(
              "get_desktop_app_info",
              {"remove_empty": remove_empty, "refresh_app_windows": True},
          )
          parsed = _parse_tool_result(result)
          return parsed if isinstance(parsed, list) else []
      
      
      @mcp.tool()
      async def qa_refresh_controls(
          field_list: Annotated[List[str], Field(description="Fields to fetch per control.")],
      ) -> Annotated[List[Dict[str, Any]], Field(description="Controls for selected window.")]:
          """Refresh control map for the selected window. Wraps UICollector.get_app_window_controls_info."""
          try:
              result = await _call_mounted_tool(
                  "get_app_window_controls_info", {"field_list": field_list}
              )
              parsed = _parse_tool_result(result)
              return parsed if isinstance(parsed, list) else []
          except Exception as exc:
              logger.warning("qa_refresh_controls failed: %s", exc)
              return [{"error": str(exc), "source": "ufo_qa"}]
      
      
      @mcp.tool()
      async def qa_wait_for_text_contains(
          id: Annotated[str, Field(description="Control id.")],
          name: Annotated[str, Field(description="Control name.")],
          expected_substring: Annotated[str, Field(description="Substring that must appear.")],
          timeout_s: Annotated[float, Field(description="Max wait seconds.")] = 10.0,
          poll_s: Annotated[float, Field(description="Poll interval seconds.")] = 0.5,
      ) -> Annotated[Dict[str, Any], Field(description="Result with ok flag and observed text.")]:
          """Poll texts(id,name) until expected_substring appears or timeout. Avoids arbitrary sleeps."""
          deadline = time.time() + max(0.1, timeout_s)
          last_text: Optional[str] = None
      
          while time.time() < deadline:
              raw = await _call_mounted_tool("texts", {"id": id, "name": name})
              res = _parse_tool_result(raw)
              last_text = res if isinstance(res, str) else str(res)
              if expected_substring in last_text:
                  return {"ok": True, "text": last_text, "matched": expected_substring}
              await asyncio.sleep(max(0.05, poll_s))
      
          return {
              "ok": False,
              "text": last_text,
              "matched": expected_substring,
              "timeout_s": timeout_s,
          }
      
      
      def main() -> None:
          mcp.run()
      
      
      if __name__ == "__main__":
          main()
      
  • install.yaml 1.7 KB
    # Declarative dependency manifest for any Claude Code skill.
    # A general installer reads this and handles clone/pip/mcp/verify steps.
    
    platform: windows           # Required platform (windows|macos|linux|any)
    python: ">=3.10,<3.11"      # Pinned pandas/faiss wheels require Python 3.10 on Windows
    
    repos:                      # Git repos to clone
      - url: https://github.com/microsoft/UFO.git
        target: ~/UFO           # ~ expands to home dir
        venv: true              # Create .venv inside the cloned repo
        requirements: requirements.txt   # pip install -r this file
        pythonpath: true        # Add repo root to PYTHONPATH for MCP server
    
    pip:                        # Additional pip packages (installed into the repo venv)
      - fastmcp
      - pydantic
    
    mcp:                        # MCP server to register in .mcp.json
      name: ufo-windows-qa
      script: scripts/ufo_windows_qa_mcp_server.py   # Relative to skill dir
      env:                      # Environment variables for the MCP server process
        UFO_ROOT: ~/UFO
        CONTROL_BACKEND: uia
        MAXIMIZE_WINDOW: "false"
        SHOW_VISUAL_OUTLINE_ON_SCREEN: "true"
        RUN_CONFIGS: "true"
    
    verify:                     # Python code to run (in venv with PYTHONPATH set) to confirm install
      script: |
        from ufo.client.mcp.local_servers import load_all_servers
        load_all_servers()
        from ufo.client.mcp.mcp_registry import MCPRegistry
        registered = MCPRegistry.list()
        assert "UICollector" in str(registered), f"Missing UICollector in {registered}"
        assert "HostUIExecutor" in str(registered), f"Missing HostUIExecutor in {registered}"
        assert "AppUIExecutor" in str(registered), f"Missing AppUIExecutor in {registered}"
        print("OK: all servers registered")
    
  • README.md 5.6 KB
    # Windows QA Engineer Skill
    
    Skill that turns the agent into a manual QA operator for Windows 11 desktop apps. Runs on the same desktop as the SUT — no mocks, no browser-only tricks.
    
    ## Tech Stack
    
    | Layer | Technology | Role |
    |-------|-----------|------|
    | **UI Automation** | [Microsoft UFO](https://github.com/microsoft/UFO) | Windows UI automation framework. Provides `ControlInspectorFacade`, `ActionExecutor`, `AppPuppeteer`, `PhotographerFacade` for real control discovery and interaction |
    | **Accessibility Backend** | [UI Automation (UIA)](https://learn.microsoft.com/en-us/windows/win32/winauto/entry-uiauto-win32) / [Win32](https://learn.microsoft.com/en-us/windows/win32/winauto/microsoft-active-accessibility) | OS-level accessibility APIs that UFO uses to inspect and manipulate UI controls. UIA is default; Win32 available as fallback |
    | **MCP Servers** | UFO `UICollector` + `HostUIExecutor` + `AppUIExecutor` | UFO's built-in MCP servers (`ufo/client/mcp/local_servers/ui_mcp_server.py`) registered via `MCPRegistry`. Provide tools: `get_desktop_app_info`, `select_application_window`, `get_app_window_controls_info`, `click_input`, `set_edit_text`, `texts`, `capture_window_screenshot`, etc. |
    | **Server Composition** | [FastMCP](https://github.com/jlowin/fastmcp) `mount()` | Composes UFO's 3 MCP servers into a single stdio endpoint so Claude Code needs only one `.mcp.json` entry |
    | **Protocol** | [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) over stdio | Standard protocol connecting Claude Code to UFO's automation tools |
    | **Target Apps** | WinForms, WPF, UWP, Win32 | Any Windows desktop app exposing an accessibility tree in the same interactive session |
    
    ## How It Works
    
    ```
    Claude Code ──stdio──▶ FastMCP server ──mount()──▶ UFO UICollector
                                           ──mount()──▶ UFO HostUIExecutor
                                           ──mount()──▶ UFO AppUIExecutor
                                                             │
                                                        UIA / Win32
                                                             │
                                                       Windows Desktop
                                                        (real SUT)
    ```
    
    The skill's MCP server (`scripts/ufo_windows_qa_mcp_server.py`) imports UFO's server factories via `MCPRegistry`, mounts all three into one `FastMCP` instance, and adds QA helper tools:
    
    - **`qa_refresh_and_list_windows`** — refresh + list in one call
    - **`qa_refresh_controls`** — re-collect control tree for selected window
    - **`qa_wait_for_text_contains`** — polling assertion (avoids arbitrary sleeps)
    
    ## QA Workflow
    
    ```
    1. Discover windows    →  qa_refresh_and_list_windows()
    2. Select SUT          →  select_application_window(id, name)
    3. Screenshot baseline →  capture_window_screenshot()
    4. Collect controls    →  get_app_window_controls_info(field_list=[...])
    5. Interact by id/name →  click_input / set_edit_text / keyboard_input
    6. Assert              →  qa_wait_for_text_contains(id, name, expected)
    7. Report              →  PASS/FAIL + screenshots + execution log
    ```
    
    ## Requirements
    
    - Windows 11 or Windows Server 2025 with Desktop Experience
    - Python 3.10.x (required by the currently pinned pandas/faiss Windows wheels)
    - [Microsoft UFO](https://github.com/microsoft/UFO) — `git clone` + `pip install -r requirements.txt`
    - [FastMCP](https://pypi.org/project/fastmcp/) — `pip install fastmcp`
    - [Pydantic](https://pypi.org/project/pydantic/) — comes with FastMCP
    
    ## Install
    
    ### Via Skills CLI
    
    ```bash
    npx skills add CodeAlive-AI/ai-driven-development@windows-qa-engineer -g -y
    ```
    
    ### Manual
    
    1. Clone this repo
    2. Copy `windows-qa-engineer/` to `~/.claude/skills/`
    3. Add the MCP server config to your project `.mcp.json`:
    
    ```json
    {
      "mcpServers": {
        "ufo-windows-qa": {
          "type": "stdio",
          "command": "C:\\Users\\<you>\\UFO\\.venv\\Scripts\\python.exe",
          "args": ["C:\\path\\to\\windows-qa-engineer\\scripts\\ufo_windows_qa_mcp_server.py"],
          "env": {
            "UFO_ROOT": "C:\\Users\\<you>\\UFO",
            "PYTHONPATH": "C:\\Users\\<you>\\UFO",
            "CONTROL_BACKEND": "uia",
            "SHOW_VISUAL_OUTLINE_ON_SCREEN": "true"
          }
        }
      }
    }
    ```
    
    4. Restart the MCP client and verify that the tools appear
    
    Run the MCP client and the application under test in the same logged-in interactive Windows desktop session. A process started through SSH, WinRM, CI, or a Windows service runs headlessly and cannot provide a usable UIA window tree or screenshot.
    
    For an end-to-end environment check, run `scripts/interactive_smoke_test.py` with UFO's venv Python from an interactive terminal. It launches Notepad, discovers and selects its window through the MCP server, captures a PNG, and writes a machine-readable report in the user's temporary directory.
    
    ## Usage
    
    ```
    /windows-qa-engineer Calculator "verify 2+2=4"
    ```
    
    Or describe what to test:
    
    > "Test the login flow on MyApp — enter admin/password, click Login, verify the welcome screen"
    
    ## Skill Contents
    
    ```
    windows-qa-engineer/
    ├── SKILL.md                              # Workflow instructions for Claude
    ├── scripts/
    │   ├── ufo_windows_qa_mcp_server.py      # FastMCP server (UFO mount composition)
    │   └── doctor.ps1                        # Environment validation
    ├── references/
    │   ├── setup.md                          # UFO install + MCP config
    │   └── qa-workflows.md                   # Examples + locator strategy
    └── assets/
        └── test-case.md                      # Test case output template
    ```
    
    ## License
    
    MIT
    
  • SKILL.md 5.6 KB
    ---
    name: windows-qa-engineer
    description: Use when testing Windows 11 desktop apps (WinForms/WPF/UWP) via UFO UIA/Win32 automation MCP. Triggers on "test this Windows app", "QA the app", "run smoke test", "click the button", "fill the form", "check the UI", "Windows automation", "UFO QA", "verify the dialog", or any Windows desktop UI testing task. Not for web/browser testing (use Playwright), mobile testing, or non-Windows platforms.
    metadata:
      compatibility: Windows 11 or Windows Server 2025 Desktop Experience, Python 3.10, UFO, fastmcp
    ---
    
    # Windows QA Engineer (UFO-powered)
    
    You are an AI-QA operator on the same interactive Windows desktop as the SUT.
    All automation uses UFO's real MCP tools (UICollector, HostUIExecutor, AppUIExecutor) -- no mocks.
    
    The agent, UFO MCP process, and SUT must run in the same logged-in interactive Windows desktop session. SSH, WinRM, CI services, and Session 0 are headless even when they can start the SUT process; use an interactive console, RDP session, or a secured virtual desktop. Never expose RDP publicly merely to satisfy this requirement.
    
    ## Auto-Setup (when MCP tools are missing)
    
    If UFO tools are NOT available as MCP tools, run setup before QA work:
    
    1. Run: `python "<skill-dir>/scripts/skill_installer.py" --project-dir "<project-root>"`
    2. Parse the JSON output — if `success` is true, restart the MCP client
    3. If failed, show the error and direct user to [references/setup.md](references/setup.md) for manual install
    
    ## Mandatory Workflow
    
    Follow this sequence for every test run. Do not skip steps.
    
    ### 1. Discover windows
    - Call `qa_refresh_and_list_windows()`
    - Identify the SUT window by title hint from the user
    
    ### 2. Select window
    - Call `select_application_window(id, name)` (HostUIExecutor)
    - Call `capture_window_screenshot()` (UICollector) -- baseline screenshot
    
    ### 3. Collect controls
    - Call `qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])`
    - Anchor on `id` + `control_text` / `automation_id` when the returned tree is usable
    - If control collection returns an error or an empty tree for a large/legacy WinForms window, continue with screenshot inspection and coordinate actions; do not repeatedly force full UIA subtree scans
    
    ### 4. Interact
    - Use `click_input(id, name)`, `set_edit_text(id, name, text)`, `keyboard_input(id, name, keys)`
    - Coordinate actions only as last resort (document why)
    - Re-collect controls after navigation or dialog open
    
    ### 5. Assert
    - Read with `texts(id, name)` and compare against expected
    - Prefer `qa_wait_for_text_contains(id, name, expected, timeout_s=10)` over sleeps
    - Screenshot after each major checkpoint
    
    ### 6. Report
    - Fill [assets/test-case.md](assets/test-case.md) template
    - Numbered execution log (step -> tool call -> result)
    - Final PASS/FAIL with exact failing assertion if applicable
    - Attach screenshot base64 strings from `capture_window_screenshot()`
    
    ## Tool Reference
    
    | Tool | Server | Purpose |
    |------|--------|---------|
    | `qa_refresh_and_list_windows` | QA helper | Refresh + list all windows |
    | `select_application_window` | HostUIExecutor | Select SUT by id+name |
    | `get_app_window_controls_info` | UICollector | Raw control tree; use only when helper output is insufficient |
    | `capture_window_screenshot` | UICollector | Screenshot selected window |
    | `click_input` | AppUIExecutor | Click control by id+name |
    | `set_edit_text` | AppUIExecutor | Type into control |
    | `keyboard_input` | AppUIExecutor | Send keystrokes |
    | `texts` | AppUIExecutor | Read control text |
    | `qa_wait_for_text_contains` | QA helper | Poll until text matches |
    | `qa_refresh_controls` | QA helper | Re-collect control tree with fail-soft parsing |
    
    ## Example: Login Smoke Test
    
    User says: "Test the login flow on MyApp"
    
    ```
    1. qa_refresh_and_list_windows() → find "MyApp - Login"
    2. select_application_window(id="3", name="MyApp - Login")
    3. capture_window_screenshot() → baseline
    4. qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])
       → find username (id=12), password (id=14), login button (id=16)
    5. set_edit_text(id="12", name="Username", text="testuser")
    6. set_edit_text(id="14", name="Password", text="pass123")
    7. click_input(id="16", name="Login")
    8. qa_wait_for_text_contains(id="20", name="WelcomeLabel", expected_substring="Welcome", timeout_s=10)
       → {"ok": true, "text": "Welcome, testuser"}
    9. capture_window_screenshot() → post-login
    10. Report: PASS
    ```
    
    ## Error Handling
    
    **No windows found**: Re-check the SUT is running. Call `qa_refresh_and_list_windows()` again. If still empty, ask the user to confirm the app is open.
    
    **Empty control tree**: The window may not have finished loading. Wait 2-3 seconds, then `qa_refresh_controls(field_list=[...])`. If still empty, try `CONTROL_BACKEND=win32` (see setup.md). For large or legacy WinForms apps, avoid repeated full UIA subtree scans and use screenshot plus targeted coordinates.
    
    **Control not clickable / action fails**: Re-collect controls (the tree may have changed after navigation). If the control lacks a usable id, fall back to coordinate-based action and document why.
    
    **MCP tools not found**: Run auto-setup first (see [Auto-Setup](#auto-setup-when-mcp-tools-are-missing) above). If auto-setup fails, direct the user to [references/setup.md](references/setup.md) and run `doctor.ps1`.
    
    ## Detailed Workflows
    
    See [references/qa-workflows.md](references/qa-workflows.md) for more examples, locator strategy, and common patterns.
    
    ## Setup
    
    See [references/setup.md](references/setup.md) for UFO installation, MCP configuration, and diagnostics.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related