windows-qa-engineer
Use when testing Windows 11 desktop apps (WinForms/WPF/UWP) via UFO UIA/Win32 automation MCP. Triggers on "test this Windows app", "QA the app", "run smoke test", "click the button", "fill the form", "check the UI", "Windows automation", "UFO QA", "verify the dialog", or any Wind
Install
npx skills add https://github.com/CodeAlive-AI/ai-driven-development/tree/main/skills/windows-qa-engineer
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install codealive-ai-ai-driven-development@llmmart
git clone https://github.com/CodeAlive-AI/ai-driven-development.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole codealive-ai/ai-driven-development collection as a plugin from our marketplace. Git is the plain clone.
README
Windows QA Engineer Skill
Skill that turns the agent into a manual QA operator for Windows 11 desktop apps. Runs on the same desktop as the SUT — no mocks, no browser-only tricks.
Tech Stack
| Layer | Technology | Role |
|---|---|---|
| UI Automation | Microsoft UFO | Windows UI automation framework. Provides ControlInspectorFacade, ActionExecutor, AppPuppeteer, PhotographerFacade for real control discovery and interaction |
| Accessibility Backend | UI Automation (UIA) / Win32 | OS-level accessibility APIs that UFO uses to inspect and manipulate UI controls. UIA is default; Win32 available as fallback |
| MCP Servers | UFO UICollector + HostUIExecutor + AppUIExecutor |
UFO's built-in MCP servers (ufo/client/mcp/local_servers/ui_mcp_server.py) registered via MCPRegistry. Provide tools: get_desktop_app_info, select_application_window, get_app_window_controls_info, click_input, set_edit_text, texts, capture_window_screenshot, etc. |
| Server Composition | FastMCP mount() |
Composes UFO's 3 MCP servers into a single stdio endpoint so Claude Code needs only one .mcp.json entry |
| Protocol | MCP (Model Context Protocol) over stdio | Standard protocol connecting Claude Code to UFO's automation tools |
| Target Apps | WinForms, WPF, UWP, Win32 | Any Windows desktop app exposing an accessibility tree in the same interactive session |
How It Works
Claude Code ──stdio──▶ FastMCP server ──mount()──▶ UFO UICollector
──mount()──▶ UFO HostUIExecutor
──mount()──▶ UFO AppUIExecutor
│
UIA / Win32
│
Windows Desktop
(real SUT)
The skill's MCP server (scripts/ufo_windows_qa_mcp_server.py) imports UFO's server factories via MCPRegistry, mounts all three into one FastMCP instance, and adds QA helper tools:
qa_refresh_and_list_windows— refresh + list in one callqa_refresh_controls— re-collect control tree for selected windowqa_wait_for_text_contains— polling assertion (avoids arbitrary sleeps)
QA Workflow
1. Discover windows → qa_refresh_and_list_windows()
2. Select SUT → select_application_window(id, name)
3. Screenshot baseline → capture_window_screenshot()
4. Collect controls → get_app_window_controls_info(field_list=[...])
5. Interact by id/name → click_input / set_edit_text / keyboard_input
6. Assert → qa_wait_for_text_contains(id, name, expected)
7. Report → PASS/FAIL + screenshots + execution log
Requirements
- Windows 11 or Windows Server 2025 with Desktop Experience
- Python 3.10.x (required by the currently pinned pandas/faiss Windows wheels)
- Microsoft UFO —
git clone+pip install -r requirements.txt - FastMCP —
pip install fastmcp - Pydantic — comes with FastMCP
Install
Via Skills CLI
npx skills add CodeAlive-AI/ai-driven-development@windows-qa-engineer -g -y
Manual
- Clone this repo
- Copy
windows-qa-engineer/to~/.claude/skills/ - Add the MCP server config to your project
.mcp.json:
{
"mcpServers": {
"ufo-windows-qa": {
"type": "stdio",
"command": "C:\\Users\\<you>\\UFO\\.venv\\Scripts\\python.exe",
"args": ["C:\\path\\to\\windows-qa-engineer\\scripts\\ufo_windows_qa_mcp_server.py"],
"env": {
"UFO_ROOT": "C:\\Users\\<you>\\UFO",
"PYTHONPATH": "C:\\Users\\<you>\\UFO",
"CONTROL_BACKEND": "uia",
"SHOW_VISUAL_OUTLINE_ON_SCREEN": "true"
}
}
}
}
- Restart the MCP client and verify that the tools appear
Run the MCP client and the application under test in the same logged-in interactive Windows desktop session. A process started through SSH, WinRM, CI, or a Windows service runs headlessly and cannot provide a usable UIA window tree or screenshot.
For an end-to-end environment check, run scripts/interactive_smoke_test.py with UFO's venv Python from an interactive terminal. It launches Notepad, discovers and selects its window through the MCP server, captures a PNG, and writes a machine-readable report in the user's temporary directory.
Usage
/windows-qa-engineer Calculator "verify 2+2=4"
Or describe what to test:
"Test the login flow on MyApp — enter admin/password, click Login, verify the welcome screen"
Skill Contents
windows-qa-engineer/
├── SKILL.md # Workflow instructions for Claude
├── scripts/
│ ├── ufo_windows_qa_mcp_server.py # FastMCP server (UFO mount composition)
│ └── doctor.ps1 # Environment validation
├── references/
│ ├── setup.md # UFO install + MCP config
│ └── qa-workflows.md # Examples + locator strategy
└── assets/
└── test-case.md # Test case output template
License
MIT
Skill manifest
Windows QA Engineer (UFO-powered)
You are an AI-QA operator on the same interactive Windows desktop as the SUT. All automation uses UFO's real MCP tools (UICollector, HostUIExecutor, AppUIExecutor) -- no mocks.
The agent, UFO MCP process, and SUT must run in the same logged-in interactive Windows desktop session. SSH, WinRM, CI services, and Session 0 are headless even when they can start the SUT process; use an interactive console, RDP session, or a secured virtual desktop. Never expose RDP publicly merely to satisfy this requirement.
Auto-Setup (when MCP tools are missing)
If UFO tools are NOT available as MCP tools, run setup before QA work:
- Run:
python "<skill-dir>/scripts/skill_installer.py" --project-dir "<project-root>" - Parse the JSON output — if
successis true, restart the MCP client - If failed, show the error and direct user to references/setup.md for manual install
Mandatory Workflow
Follow this sequence for every test run. Do not skip steps.
1. Discover windows
- Call
qa_refresh_and_list_windows() - Identify the SUT window by title hint from the user
2. Select window
- Call
select_application_window(id, name)(HostUIExecutor) - Call
capture_window_screenshot()(UICollector) -- baseline screenshot
3. Collect controls
- Call
qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"]) - Anchor on
id+control_text/automation_idwhen the returned tree is usable - If control collection returns an error or an empty tree for a large/legacy WinForms window, continue with screenshot inspection and coordinate actions; do not repeatedly force full UIA subtree scans
4. Interact
- Use
click_input(id, name),set_edit_text(id, name, text),keyboard_input(id, name, keys) - Coordinate actions only as last resort (document why)
- Re-collect controls after navigation or dialog open
5. Assert
- Read with
texts(id, name)and compare against expected - Prefer
qa_wait_for_text_contains(id, name, expected, timeout_s=10)over sleeps - Screenshot after each major checkpoint
6. Report
- Fill assets/test-case.md template
- Numbered execution log (step -> tool call -> result)
- Final PASS/FAIL with exact failing assertion if applicable
- Attach screenshot base64 strings from
capture_window_screenshot()
Tool Reference
| Tool | Server | Purpose |
|---|---|---|
qa_refresh_and_list_windows |
QA helper | Refresh + list all windows |
select_application_window |
HostUIExecutor | Select SUT by id+name |
get_app_window_controls_info |
UICollector | Raw control tree; use only when helper output is insufficient |
capture_window_screenshot |
UICollector | Screenshot selected window |
click_input |
AppUIExecutor | Click control by id+name |
set_edit_text |
AppUIExecutor | Type into control |
keyboard_input |
AppUIExecutor | Send keystrokes |
texts |
AppUIExecutor | Read control text |
qa_wait_for_text_contains |
QA helper | Poll until text matches |
qa_refresh_controls |
QA helper | Re-collect control tree with fail-soft parsing |
Example: Login Smoke Test
User says: "Test the login flow on MyApp"
1. qa_refresh_and_list_windows() → find "MyApp - Login"
2. select_application_window(id="3", name="MyApp - Login")
3. capture_window_screenshot() → baseline
4. qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])
→ find username (id=12), password (id=14), login button (id=16)
5. set_edit_text(id="12", name="Username", text="testuser")
6. set_edit_text(id="14", name="Password", text="pass123")
7. click_input(id="16", name="Login")
8. qa_wait_for_text_contains(id="20", name="WelcomeLabel", expected_substring="Welcome", timeout_s=10)
→ {"ok": true, "text": "Welcome, testuser"}
9. capture_window_screenshot() → post-login
10. Report: PASS
Error Handling
No windows found: Re-check the SUT is running. Call qa_refresh_and_list_windows() again. If still empty, ask the user to confirm the app is open.
Empty control tree: The window may not have finished loading. Wait 2-3 seconds, then qa_refresh_controls(field_list=[...]). If still empty, try CONTROL_BACKEND=win32 (see setup.md). For large or legacy WinForms apps, avoid repeated full UIA subtree scans and use screenshot plus targeted coordinates.
Control not clickable / action fails: Re-collect controls (the tree may have changed after navigation). If the control lacks a usable id, fall back to coordinate-based action and document why.
MCP tools not found: Run auto-setup first (see Auto-Setup above). If auto-setup fails, direct the user to references/setup.md and run doctor.ps1.
Detailed Workflows
See references/qa-workflows.md for more examples, locator strategy, and common patterns.
Setup
See references/setup.md for UFO installation, MCP configuration, and diagnostics.
Files (ai-driven-development)
-
assets
-
test-case.md 402 B
Loading…
-
-
references
-
qa-workflows.md 1.4 KB
Loading… -
setup.md 3.1 KB
Loading…
-
-
scripts
-
doctor.ps1 1 KB · in bundle
-
interactive_smoke_test.py 4.6 KB
Loading… -
skill_installer.py 21.5 KB
Loading… -
ufo_windows_qa_mcp_server.py 5.2 KB
Loading…
-
-
install.yaml 1.7 KB
Loading… -
README.md 5.6 KB
# Windows QA Engineer Skill Skill that turns the agent into a manual QA operator for Windows 11 desktop apps. Runs on the same desktop as the SUT — no mocks, no browser-only tricks. ## Tech Stack | Layer | Technology | Role | |-------|-----------|------| | **UI Automation** | [Microsoft UFO](https://github.com/microsoft/UFO) | Windows UI automation framework. Provides `ControlInspectorFacade`, `ActionExecutor`, `AppPuppeteer`, `PhotographerFacade` for real control discovery and interaction | | **Accessibility Backend** | [UI Automation (UIA)](https://learn.microsoft.com/en-us/windows/win32/winauto/entry-uiauto-win32) / [Win32](https://learn.microsoft.com/en-us/windows/win32/winauto/microsoft-active-accessibility) | OS-level accessibility APIs that UFO uses to inspect and manipulate UI controls. UIA is default; Win32 available as fallback | | **MCP Servers** | UFO `UICollector` + `HostUIExecutor` + `AppUIExecutor` | UFO's built-in MCP servers (`ufo/client/mcp/local_servers/ui_mcp_server.py`) registered via `MCPRegistry`. Provide tools: `get_desktop_app_info`, `select_application_window`, `get_app_window_controls_info`, `click_input`, `set_edit_text`, `texts`, `capture_window_screenshot`, etc. | | **Server Composition** | [FastMCP](https://github.com/jlowin/fastmcp) `mount()` | Composes UFO's 3 MCP servers into a single stdio endpoint so Claude Code needs only one `.mcp.json` entry | | **Protocol** | [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) over stdio | Standard protocol connecting Claude Code to UFO's automation tools | | **Target Apps** | WinForms, WPF, UWP, Win32 | Any Windows desktop app exposing an accessibility tree in the same interactive session | ## How It Works ``` Claude Code ──stdio──▶ FastMCP server ──mount()──▶ UFO UICollector ──mount()──▶ UFO HostUIExecutor ──mount()──▶ UFO AppUIExecutor │ UIA / Win32 │ Windows Desktop (real SUT) ``` The skill's MCP server (`scripts/ufo_windows_qa_mcp_server.py`) imports UFO's server factories via `MCPRegistry`, mounts all three into one `FastMCP` instance, and adds QA helper tools: - **`qa_refresh_and_list_windows`** — refresh + list in one call - **`qa_refresh_controls`** — re-collect control tree for selected window - **`qa_wait_for_text_contains`** — polling assertion (avoids arbitrary sleeps) ## QA Workflow ``` 1. Discover windows → qa_refresh_and_list_windows() 2. Select SUT → select_application_window(id, name) 3. Screenshot baseline → capture_window_screenshot() 4. Collect controls → get_app_window_controls_info(field_list=[...]) 5. Interact by id/name → click_input / set_edit_text / keyboard_input 6. Assert → qa_wait_for_text_contains(id, name, expected) 7. Report → PASS/FAIL + screenshots + execution log ``` ## Requirements - Windows 11 or Windows Server 2025 with Desktop Experience - Python 3.10.x (required by the currently pinned pandas/faiss Windows wheels) - [Microsoft UFO](https://github.com/microsoft/UFO) — `git clone` + `pip install -r requirements.txt` - [FastMCP](https://pypi.org/project/fastmcp/) — `pip install fastmcp` - [Pydantic](https://pypi.org/project/pydantic/) — comes with FastMCP ## Install ### Via Skills CLI ```bash npx skills add CodeAlive-AI/ai-driven-development@windows-qa-engineer -g -y ``` ### Manual 1. Clone this repo 2. Copy `windows-qa-engineer/` to `~/.claude/skills/` 3. Add the MCP server config to your project `.mcp.json`: ```json { "mcpServers": { "ufo-windows-qa": { "type": "stdio", "command": "C:\\Users\\<you>\\UFO\\.venv\\Scripts\\python.exe", "args": ["C:\\path\\to\\windows-qa-engineer\\scripts\\ufo_windows_qa_mcp_server.py"], "env": { "UFO_ROOT": "C:\\Users\\<you>\\UFO", "PYTHONPATH": "C:\\Users\\<you>\\UFO", "CONTROL_BACKEND": "uia", "SHOW_VISUAL_OUTLINE_ON_SCREEN": "true" } } } } ``` 4. Restart the MCP client and verify that the tools appear Run the MCP client and the application under test in the same logged-in interactive Windows desktop session. A process started through SSH, WinRM, CI, or a Windows service runs headlessly and cannot provide a usable UIA window tree or screenshot. For an end-to-end environment check, run `scripts/interactive_smoke_test.py` with UFO's venv Python from an interactive terminal. It launches Notepad, discovers and selects its window through the MCP server, captures a PNG, and writes a machine-readable report in the user's temporary directory. ## Usage ``` /windows-qa-engineer Calculator "verify 2+2=4" ``` Or describe what to test: > "Test the login flow on MyApp — enter admin/password, click Login, verify the welcome screen" ## Skill Contents ``` windows-qa-engineer/ ├── SKILL.md # Workflow instructions for Claude ├── scripts/ │ ├── ufo_windows_qa_mcp_server.py # FastMCP server (UFO mount composition) │ └── doctor.ps1 # Environment validation ├── references/ │ ├── setup.md # UFO install + MCP config │ └── qa-workflows.md # Examples + locator strategy └── assets/ └── test-case.md # Test case output template ``` ## License MIT -
SKILL.md 5.6 KB
--- name: windows-qa-engineer description: Use when testing Windows 11 desktop apps (WinForms/WPF/UWP) via UFO UIA/Win32 automation MCP. Triggers on "test this Windows app", "QA the app", "run smoke test", "click the button", "fill the form", "check the UI", "Windows automation", "UFO QA", "verify the dialog", or any Windows desktop UI testing task. Not for web/browser testing (use Playwright), mobile testing, or non-Windows platforms. metadata: compatibility: Windows 11 or Windows Server 2025 Desktop Experience, Python 3.10, UFO, fastmcp --- # Windows QA Engineer (UFO-powered) You are an AI-QA operator on the same interactive Windows desktop as the SUT. All automation uses UFO's real MCP tools (UICollector, HostUIExecutor, AppUIExecutor) -- no mocks. The agent, UFO MCP process, and SUT must run in the same logged-in interactive Windows desktop session. SSH, WinRM, CI services, and Session 0 are headless even when they can start the SUT process; use an interactive console, RDP session, or a secured virtual desktop. Never expose RDP publicly merely to satisfy this requirement. ## Auto-Setup (when MCP tools are missing) If UFO tools are NOT available as MCP tools, run setup before QA work: 1. Run: `python "<skill-dir>/scripts/skill_installer.py" --project-dir "<project-root>"` 2. Parse the JSON output — if `success` is true, restart the MCP client 3. If failed, show the error and direct user to [references/setup.md](references/setup.md) for manual install ## Mandatory Workflow Follow this sequence for every test run. Do not skip steps. ### 1. Discover windows - Call `qa_refresh_and_list_windows()` - Identify the SUT window by title hint from the user ### 2. Select window - Call `select_application_window(id, name)` (HostUIExecutor) - Call `capture_window_screenshot()` (UICollector) -- baseline screenshot ### 3. Collect controls - Call `qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"])` - Anchor on `id` + `control_text` / `automation_id` when the returned tree is usable - If control collection returns an error or an empty tree for a large/legacy WinForms window, continue with screenshot inspection and coordinate actions; do not repeatedly force full UIA subtree scans ### 4. Interact - Use `click_input(id, name)`, `set_edit_text(id, name, text)`, `keyboard_input(id, name, keys)` - Coordinate actions only as last resort (document why) - Re-collect controls after navigation or dialog open ### 5. Assert - Read with `texts(id, name)` and compare against expected - Prefer `qa_wait_for_text_contains(id, name, expected, timeout_s=10)` over sleeps - Screenshot after each major checkpoint ### 6. Report - Fill [assets/test-case.md](assets/test-case.md) template - Numbered execution log (step -> tool call -> result) - Final PASS/FAIL with exact failing assertion if applicable - Attach screenshot base64 strings from `capture_window_screenshot()` ## Tool Reference | Tool | Server | Purpose | |------|--------|---------| | `qa_refresh_and_list_windows` | QA helper | Refresh + list all windows | | `select_application_window` | HostUIExecutor | Select SUT by id+name | | `get_app_window_controls_info` | UICollector | Raw control tree; use only when helper output is insufficient | | `capture_window_screenshot` | UICollector | Screenshot selected window | | `click_input` | AppUIExecutor | Click control by id+name | | `set_edit_text` | AppUIExecutor | Type into control | | `keyboard_input` | AppUIExecutor | Send keystrokes | | `texts` | AppUIExecutor | Read control text | | `qa_wait_for_text_contains` | QA helper | Poll until text matches | | `qa_refresh_controls` | QA helper | Re-collect control tree with fail-soft parsing | ## Example: Login Smoke Test User says: "Test the login flow on MyApp" ``` 1. qa_refresh_and_list_windows() → find "MyApp - Login" 2. select_application_window(id="3", name="MyApp - Login") 3. capture_window_screenshot() → baseline 4. qa_refresh_controls(field_list=["label","control_text","control_type","automation_id","control_rect"]) → find username (id=12), password (id=14), login button (id=16) 5. set_edit_text(id="12", name="Username", text="testuser") 6. set_edit_text(id="14", name="Password", text="pass123") 7. click_input(id="16", name="Login") 8. qa_wait_for_text_contains(id="20", name="WelcomeLabel", expected_substring="Welcome", timeout_s=10) → {"ok": true, "text": "Welcome, testuser"} 9. capture_window_screenshot() → post-login 10. Report: PASS ``` ## Error Handling **No windows found**: Re-check the SUT is running. Call `qa_refresh_and_list_windows()` again. If still empty, ask the user to confirm the app is open. **Empty control tree**: The window may not have finished loading. Wait 2-3 seconds, then `qa_refresh_controls(field_list=[...])`. If still empty, try `CONTROL_BACKEND=win32` (see setup.md). For large or legacy WinForms apps, avoid repeated full UIA subtree scans and use screenshot plus targeted coordinates. **Control not clickable / action fails**: Re-collect controls (the tree may have changed after navigation). If the control lacks a usable id, fall back to coordinate-based action and document why. **MCP tools not found**: Run auto-setup first (see [Auto-Setup](#auto-setup-when-mcp-tools-are-missing) above). If auto-setup fails, direct the user to [references/setup.md](references/setup.md) and run `doctor.ps1`. ## Detailed Workflows See [references/qa-workflows.md](references/qa-workflows.md) for more examples, locator strategy, and common patterns. ## Setup See [references/setup.md](references/setup.md) for UFO installation, MCP configuration, and diagnostics.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.