End-to-end testing with Playwright Test (@playwright/test, TS/JS). A Python flavor\n(pytest-playwright) exists with the same browser API but pytest-style fixtures — patterns here\ntranslate directly; runner config does not.
npm init playwright@latest # scaffold config + example test + GH Actions workflow\nnpx playwright test # run all tests, all projects\nnpx playwright test --project=chromium --grep \"@smoke\"\nnpx playwright test --ui # interactive UI mode (watch, time-travel)\nnpx playwright codegen https://app.local # record actions -> generated locators\nnpx playwright show-report # open last HTML report\nnpx playwright show-trace trace.zip # inspect a trace\nHierarchy — always prefer the highest tier that uniquely matches:
\n| Tier | \nLocator | \nWhen | \n
|---|---|---|
| 1 | \npage.getByRole('button', { name: 'Submit' }) | \nAnything with an ARIA role — buttons, links, headings, textboxes. Tests a11y for free | \n
| 2 | \npage.getByLabel('Password') | \nForm fields with labels | \n
| 3 | \npage.getByPlaceholder('name@example.com') | \nInputs without labels (fix the label instead, when you can) | \n
| 4 | \npage.getByText('Welcome back') | \nNon-interactive text content | \n
| 5 | \npage.getByTestId('cart-total') | \nStable hook when semantics don't disambiguate. Configure attribute via testIdAttribute | \n
| 6 | \npage.locator('css=...') / xpath= | \nLast resort. Coupled to DOM structure; breaks on refactor | \n
Why: tiers 1–4 locate the way a user perceives the page — resilient to markup changes, and\ngetByRole fails loudly when accessibility regresses. CSS/XPath encode implementation detail.
Narrowing without CSS:
\npage.getByRole('listitem')\n .filter({ hasText: 'Product 2' })\n .getByRole('button', { name: 'Add to cart' });\n\npage.getByRole('row').filter({ has: page.getByRole('cell', { name: 'Alice' }) });\n// BAD — checks once, races the render; sleeps are flake factories\nexpect(await page.getByText('welcome').isVisible()).toBe(true);\nawait page.waitForTimeout(2000);\n\n// GOOD — auto-retries until pass or timeout\nawait expect(page.getByText('welcome')).toBeVisible();\nawait expect(page.getByRole('list')).toHaveCount(3);\nawait expect(page).toHaveURL(/\\/dashboard/);\nawait expect.soft(page.getByTestId('status')).toHaveText('Active'); // don't stop test on failure\nActions (click, fill) auto-wait for actionability (visible, stable, enabled). If you feel the\nneed for waitForTimeout, you're missing an assertion or an await expect(...) on a state change.\nFor async non-DOM conditions use expect.poll(() => fn()) or expect(async () => {...}).toPass().
Lint guard: enable @typescript-eslint/no-floating-promises — a missing await on an assertion is\nthe most common silent-pass bug.
Full production template with comments: assets/playwright.config.template.ts
\nimport { defineConfig, devices } from '@playwright/test';\n\nexport default defineConfig({\n testDir: './tests',\n fullyParallel: true,\n forbidOnly: !!process.env.CI,\n retries: process.env.CI ? 2 : 0,\n workers: process.env.CI ? 1 : undefined,\n reporter: process.env.CI ? 'blob' : 'html',\n use: {\n baseURL: process.env.BASE_URL ?? 'http://localhost:3000',\n trace: 'on-first-retry',\n testIdAttribute: 'data-testid',\n },\n projects: [\n { name: 'setup', testMatch: /.*\\.setup\\.ts/ },\n {\n name: 'chromium',\n use: { ...devices['Desktop Chrome'], storageState: 'playwright/.auth/user.json' },\n dependencies: ['setup'],\n },\n ],\n webServer: {\n command: 'npm run dev',\n url: 'http://localhost:3000',\n reuseExistingServer: !process.env.CI,\n },\n});\nWhat do I need to share/setup?\n│\n├─ Per-test object (page object, seeded record)\n│ └─ test.extend() test-scoped fixture — setup, await use(x), teardown\n│\n├─ Expensive, safe-to-share resource (DB pool, test account)\n│ └─ Worker-scoped: [fn, { scope: 'worker' }] — once per worker process\n│\n├─ Side effect every test needs (log capture, network stub)\n│ └─ Automatic: [fn, { auto: true }] — runs without being referenced\n│\n├─ Config-tunable value (locale, default item)\n│ └─ Option: ['default', { option: true }] — override in projects[].use\n│\n├─ Fixtures from several modules\n│ └─ mergeTests(testA, testB)\n│\n└─ Auth state per test file/role\n └─ test.use({ storageState: 'playwright/.auth/admin.json' })\nPOM-as-fixture (modern recommendation) — page objects are fine; instantiating them by hand in\nevery test is not. Inject via fixture:
\n// fixtures.ts\nimport { test as base } from '@playwright/test';\nimport { TodoPage } from './pages/todo-page';\n\nexport const test = base.extend<{ todoPage: TodoPage }>({\n todoPage: async ({ page }, use) => {\n const todoPage = new TodoPage(page);\n await todoPage.goto();\n await use(todoPage); // test body runs here\n },\n});\nexport { expect } from '@playwright/test';\nPage objects should expose locators and actions, not assertions wrapped in try/catch, and never\nstore element handles. Details: references/fixtures-and-pom.md
\nNetwork need?\n│\n├─ Stub a third-party API → page.route('**/api/**', r => r.fulfill({ json }))\n├─ Tweak a real response → const res = await route.fetch(); route.fulfill({ response: res, json })\n├─ Simulate failure / offline → route.abort() / route.fulfill({ status: 500 })\n├─ Many endpoints, real shapes → HAR record + replay (page.routeFromHAR, update: true to record)\n├─ Pure API test (no browser) → request fixture / APIRequestContext\n├─ Seed data fast, assert via UI → hybrid: create via request, verify via page\n└─ WebSocket traffic → page.routeWebSocket(url, ws => ws.onMessage(...))\nHybrid seed-via-API, assert-via-UI — the single biggest speed win in most suites:
\ntest('shows new project', async ({ request, page }) => {\n const res = await request.post('/api/projects', { data: { name: 'Apollo' } });\n expect(res.ok()).toBeTruthy();\n await page.goto('/projects');\n await expect(page.getByRole('link', { name: 'Apollo' })).toBeVisible();\n});\nRule of thumb: mock third-party dependencies you don't own; exercise your own backend for real\n(or mock it deliberately in a separate \"frontend-isolated\" project).\nDetails: references/network-and-api.md
\nStandard pattern — login once in a setup project, reuse storageState everywhere:
// tests/auth.setup.ts\nimport { test as setup, expect } from '@playwright/test';\n\nsetup('authenticate', async ({ page }) => {\n await page.goto('/login');\n await page.getByLabel('Username').fill(process.env.E2E_USER!);\n await page.getByLabel('Password').fill(process.env.E2E_PASS!);\n await page.getByRole('button', { name: 'Sign in' }).click();\n await expect(page.getByTestId('user-menu')).toBeVisible(); // wait for auth to settle!\n await page.context().storageState({ path: 'playwright/.auth/user.json' });\n});\n| Pattern | \nUse when | \n
|---|---|
One setup project + storageState in use | \nOne shared account, tests don't mutate server-side user state | \n
Per-role files (admin.json, user.json) + test.use({ storageState }) | \nRole-based behavior under test | \n
Worker-scoped account fixture (testInfo.parallelIndex) | \nParallel tests mutate user state — one account per worker | \n
API login (request.post + request.storageState) | \nLogin endpoint exists; 10x faster than UI login | \n
Gotchas: add playwright/.auth/ to .gitignore. storageState captures cookies +\nlocalStorage — not sessionStorage (persist that manually via page.evaluate + init script).\nAlways assert a logged-in signal before saving state, or you save a half-logged-in race.
| Knob | \nSetting | \nNotes | \n
|---|---|---|
| Workers | \nworkers: process.env.CI ? 1 : undefined | \nLocal: half the logical CPU cores. CI runners are small — shard machines instead of oversubscribing | \n
| File-level parallel | \nfullyParallel: true | \nAlso makes sharding split per-test, not per-file | \n
| Sharding | \nnpx playwright test --shard=1/4 | \nOne shard per CI machine; merge blob reports after | \n
| Retries | \nretries: process.env.CI ? 2 : 0 | \nPair with trace: 'on-first-retry'; treat \"flaky\" status as a bug queue, not a fix | \n
| Serial | \ntest.describe.configure({ mode: 'serial' }) | \nSmell — usually means hidden inter-test coupling | \n
Isolation discipline: every test gets a fresh context/page (cookies, storage) — keep it\nthat way. No test reads state written by another test; shared server-side state is reset via API in\nbeforeEach or scoped per worker (test.info().parallelIndex in usernames/tenant IDs). A suite\nthat only passes single-worker is broken, not \"sensitive\".
Flake diagnosis: trace: 'on-first-retry' → npx playwright show-trace (DOM snapshots,\nnetwork, console per action). Local: npx playwright test --ui or PWDEBUG=1 / page.pause().\nRepro: --repeat-each=20 --workers=4. Playbook: references/flake-hunting.md
Triage a whole run without eyeballing the report — generate the JSON reporter output, then\nrank the offenders with the bundled triage tool (scripts/triage-flakes.py):
\nnpx playwright test --reporter=json > results.json # or reporter: [['json', { outputFile: 'results.json' }]]\nscripts/triage-flakes.py results.json # flaky tests first, then hard fails\nIt emits a ranked TSV (or --json envelope, schema claude-mods.playwright-ops.flake-triage/v1):\nflaky tests (passed only on retry) first — ordered by retry count then duration — followed by\nunexpected hard failures, each with file:line, the status sequence (failed->passed), and total\nduration. Exit 10 means flakes/fails were found (the triage signal — go fix them); exit 0 means a\nclean suite. --outcome all includes the passing tests for context; -n N caps rows.
- uses: actions/checkout@v5\n- uses: actions/setup-node@v5\n with: { node-version: lts/* }\n- run: npm ci\n- run: npx playwright install --with-deps chromium # only browsers you test\n- run: npx playwright test\n- uses: actions/upload-artifact@v4\n if: ${{ !cancelled() }}\n with: { name: playwright-report, path: playwright-report/, retention-days: 30 }\n| Decision | \nGuidance | \n
|---|---|
| Container vs install-deps | \nmcr.microsoft.com/playwright:vX.Y.Z-jammy image pins browser+OS (best for visual tests); install --with-deps is simpler and fine otherwise. Pin image tag to your @playwright/test version | \n
| Browser caching | \nCache ~/.cache/ms-playwright keyed on Playwright version; skip when using the container | \n
| Sharded reports | \nreporter: 'blob' on shards → upload blob-report/ → merge job: npx playwright merge-reports --reporter html ./all-blob-reports | \n
| Fail-fast vs full suite | \nPRs: fail-fast: false + --max-failures=10 per shard — see all failures in one round-trip. Smoke gates: fail fast | \n
Full workflows (sharding matrix, merge job, caching): references/ci-patterns.md
\nawait expect(page).toHaveScreenshot('landing.png', {\n maxDiffPixels: 100, // or maxDiffPixelRatio / threshold\n mask: [page.getByTestId('ad-banner')], // black-box dynamic regions\n fullPage: true,\n});\nnpx playwright test --update-snapshotslanding-chromium-darwin.png) — baselines\ngenerated on macOS will not match Linux CI. Fix: generate baselines inside the same Docker image\nCI uses, or run visual tests only in the containertoHaveScreenshot defaults animations: 'disabled'; hide dynamic bits with\nmask or stylePath (CSS applied at capture time)expect: { toHaveScreenshot: { maxDiffPixels: 100 } } in configtoMatchSnapshot() for non-image data (text/buffers)@playwright/experimental-ct-react (also vue/svelte) mounts components in a real browser —\nstill experimental; for component-level work, Vitest browser mode or Testing Library are the\nsafer default, with Playwright covering E2E.
| Factor | \nPlaywright | \nCypress | \n
|---|---|---|
| Browsers | \nChromium, Firefox, WebKit (real Safari engine) | \nChrome-family, Firefox; WebKit experimental | \n
| Parallelism | \nFree, built-in, shardable | \nPaid Cloud for parallel orchestration | \n
| Multi-tab / multi-origin / iframes | \nNative | \nHistorically constrained | \n
| API testing | \nBuilt-in request context | \nVia cy.request, less ergonomic | \n
| Component testing | \nExperimental | \nMature, first-class | \n
| In-browser interactive DX | \nUI mode (excellent) | \nThe original benchmark; some teams still prefer it | \n
Reach for Cypress when component testing maturity or an existing Cypress investment dominates;\notherwise Playwright is the default for new E2E suites. (Repo also has a sibling cypress-ops skill.)
| Tool | \nCommand | \nUse | \n
|---|---|---|
| UI mode | \nnpx playwright test --ui | \nWatch mode, time-travel, pick locators | \n
| Inspector | \nPWDEBUG=1 npx playwright test or page.pause() | \nStep through actions live | \n
| Codegen | \nnpx playwright codegen <url> | \nRecords actions, emits role-based locators — treat output as a draft, refactor into POMs/fixtures | \n
| Trace viewer | \nnpx playwright show-trace trace.zip | \nPost-mortem: snapshots, network, console | \n
| Headed + slow | \n--headed --debug | \nEyeball a single test | \n
| VS Code extension | \n— | \nRun/debug tests, pick locators in-editor | \n
An official Playwright MCP server (@playwright/mcp) also exists for agent-driven browser\nautomation — distinct from the test runner; don't conflate browsing automation with the test suite.
| File | \nContents | \n
|---|---|
| references/fixtures-and-pom.md | \nFixture scopes/options/merging, POM-as-fixture architecture, anti-patterns | \n
| references/network-and-api.md | \nroute/fulfill/abort, HAR replay, API testing, hybrid seeding, WebSocket | \n
| references/ci-patterns.md | \nFull GH Actions workflows: basic, sharded+merge, container, caching, reporters | \n
| references/flake-hunting.md | \nSystematic flake diagnosis: traces, repro loops, common causes + fixes | \n
| scripts/triage-flakes.py | \nParse a Playwright JSON report and rank flaky/failing tests (exit 10 = findings); see Flake diagnosis above | \n
| assets/playwright.config.template.ts | \nCommented production config template | \n