---
name: "Backup and Disaster Recovery"
version: "1.2.0"
updated: "2026-04-23"
description: "Single-zip restore plan: D1 Time Travel (30-day) + D1→R2 export, KV dump, wrangler.toml + secrets list. Cron-based automated backups. Recovery runbook."
---

# Backup and Disaster Recovery

## Single-Zip Restore Plan

```
backup-domain-YYYY-MM-DD.zip
├── db/database.sql            # D1 export
├── kv/kv-dump.json            # KV pairs
├── r2/manifest.json           # R2 object list
├── config/wrangler.toml, secrets.txt (NAMES only), dns-records.json
├── src/                       # Full source
├── restore.sh                 # One-command restore
└── README.md                  # Recovery instructions
```

## Backup Commands

```bash
npx wrangler d1 export DB --output=backup/db/database.sql
npx wrangler r2 object list BUCKET --json > backup/r2/manifest.json
curl "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/dns_records" -H "Authorization: Bearer $CF_API_TOKEN" | jq '.result' > backup/config/dns-records.json
```

## Restore Script

```bash
#!/bin/bash
set -e
npm install && npx wrangler deploy                    # 1. Deploy Worker
npx wrangler d1 execute DB --file=backup/db/database.sql  # 2. Restore D1
# 3. Restore KV (iterate kv-dump.json)
# 4. Manual: re-enter secrets from password manager
# 5. Purge cache
```

## Automated Backups (Worker Cron)

```toml
[triggers] crons = ["0 3 * * *"]  # Daily 3AM UTC
```

Export D1 tables + KV dump to R2 at `backups/daily/{date}/`.

## Retention Policy

- **Daily** — 7 days
- **Weekly (Sunday)** — 12 weeks
- **Monthly (1st)** — 12 months

Cleanup: delete daily backups older than 7 days after each run.

## Recovery Runbook

- **Worker deleted** — `npx wrangler deploy`
- **D1 corrupted** — restore from R2 backup
- **KV lost** — restore from R2 backup
- **DNS lost** — re-create from `dns-records.json`
- **Secrets lost** — re-enter from password manager
- **Total loss** — run `restore.sh`

## MCP Tools for Backups

- `mcp__coolify__database_backups` — trigger Coolify DB backups
- `mcp__coolify__diagnose_server` — check disk space
- `mcp__claude_ai_Cloudflare_Developer_Platform__d1_database_query` — export D1
- `mcp__claude_ai_Cloudflare_Developer_Platform__kv_namespace_get` — read KV
- `mcp__claude_ai_Cloudflare_Developer_Platform__r2_buckets_list` — list R2

## R2 Storage Structure

```
r2://backups/{daily|weekly|monthly}/{date}/{db|kv|coolify|config}/
```

## Acceptance Criteria

- Daily backup runs automatically
- D1 row count matches
- KV key count matches
- Coolify DBs triggered
- Retention enforced
- `restore.sh` works end-to-end
- Size <100MB (alert on 10x spike)
- Secrets list has names only
- DNS records valid

---
name: "Changelog and Releases"
version: "1.1.0"
updated: "2026-04-23"
description: "Auto-generate changelog from conventional commits. Public /changelog page, GitHub Releases, semver rules, social announcement on minor+ releases."
---

# Changelog and Releases

## Conventional Commits

All commits should follow conventional commits format:

```
feat: add donation progress bar
fix: contact form validation on mobile
docs: update README with new features
perf: compress hero image to 150KB
chore: update dependencies
```

### Commit Types

- **`feat`** — New feature — shows in changelog as "New"
- **`fix`** — Bug fix — shows in changelog as "Fixed"
- **`perf`** — Performance improvement — shows in changelog as "Improved"
- **`docs`** — Documentation — NOT in changelog
- **`chore`** — Maintenance — NOT in changelog
- **`refactor`** — Code restructure — NOT in changelog
- **`test`** — Tests — NOT in changelog

## Auto-Generate Changelog

```bash
# Using git log (no dependencies)
git log --pretty=format:"%h %s (%an, %ar)" --since="30 days ago" | \
  grep -E "^[a-f0-9]+ (feat|fix|perf):" | \
  sed 's/feat:/✨/; s/fix:/🐛/; s/perf:/⚡/'
```

### Changelog Page (/changelog)

```typescript
app.get('/changelog', async (c) => {
  // Read from a changelog.json or D1 table
  const entries = await getChangelogEntries(c.env);
  return c.html(renderChangelog(entries));
});
```

## GitHub Releases

```bash
# Create a release after significant deploys
gh release create v1.2.0 --title "v1.2.0 — Donation Progress Bars" --notes "$(cat <<'EOF'
## What's New
- ✨ Real-time donation progress bar with Stripe webhooks
- ✨ Multi-language support (EN + ES)

## Fixed
- 🐛 Contact form validation on mobile Safari
- 🐛 OG image not showing on LinkedIn

## Improved
- ⚡ Hero image compressed from 400KB to 150KB
- ⚡ Lighthouse score: 72 → 91
EOF
)"
```

## Semantic Versioning

- **Major** (1.0.0 → 2.0.0) — breaking changes, redesign
- **Minor** (1.0.0 → 1.1.0) — new features
- **Patch** (1.0.0 → 1.0.1) — bug fixes

For most Emdash projects: start at 1.0.0, bump minor for features, patch for fixes.

### Version Bump Rules

- Multiple `feat` commits since last release → bump **minor**
- Only `fix` / `perf` commits since last release → bump **patch**
- Any commit with `BREAKING CHANGE:` in body or `!` after type → bump **major**
- Pre-release tags — use `-beta.1`, `-rc.1` for staging / preview deploys

## MCP Tools Available

### GitHub MCP (`mcp__github-mcp__*`)

- **`list_releases`** — list existing releases to determine next version
- **`get_latest_release`** — get the latest release tag for version bumping
- **`get_release_by_tag`** — fetch a specific release's notes
- **`list_tags`** — list all tags to check version history
- **`get_tag`** — get details of a specific tag
- **`list_commits`** — fetch commits since last release for changelog generation
- **`get_commit`** — get details of a specific commit
- **`create_or_update_file`** — update `CHANGELOG.md` in the repo
- **`push_files`** — push changelog + version bump in one commit

## Automated Changelog Generation from Git Log

### Step-by-step workflow

1. **Get latest release tag** — `mcp__github-mcp__get_latest_release` → extract tag name
2. **List commits since that tag** — `mcp__github-mcp__list_commits` with `sha: main` and filter by date
3. **Parse conventional commits** — categorize into `feat` / `fix` / `perf` / `breaking`
4. **Determine version bump** — apply semver rules above
5. **Generate changelog entry** — format as markdown grouped by type
6. **Update CHANGELOG.md** — prepend new entry, push via `mcp__github-mcp__push_files`
7. **Create GitHub Release** — use `gh release create` with generated notes

```typescript
// Parsing conventional commits from git log
function parseCommits(commits: Array<{ message: string; sha: string; author: string }>) {
  const categories = { feat: [], fix: [], perf: [], breaking: [] };
  for (const c of commits) {
    const match = c.message.match(/^(feat|fix|perf|docs|chore|refactor|test)(!?):\s*(.+)/);
    if (!match) continue;
    const [, type, bang, description] = match;
    if (bang === '!' || c.message.includes('BREAKING CHANGE:')) {
      categories.breaking.push({ description, sha: c.sha.slice(0, 7) });
    }
    if (type in categories) {
      categories[type].push({ description, sha: c.sha.slice(0, 7) });
    }
  }
  return categories;
}
```

## Acceptance Criteria

1. Every `feat` / `fix` / `perf` commit appears in changelog — diff changelog against git log, zero missing entries
2. Version follows semver rules — parse version string, verify bump type matches commit types
3. GitHub Release exists for every minor+ version — `mcp__github-mcp__list_releases` count matches expected releases
4. Release notes match changelog entry — diff release body against `CHANGELOG.md` section, identical content
5. `/changelog` page renders correctly — Playwright screenshot shows formatted entries, no empty state
6. Changelog page has valid date ordering — entries sorted newest-first, no date inversions
7. Social announcement fires for minor+ releases — 09/social-automation triggered, post confirmed on at least 2 platforms
8. No `docs` / `chore` / `refactor` / `test` commits leak into changelog — parse changelog, only `feat` / `fix` / `perf` / `breaking` entries present

---
name: "CI/CD Pipeline"
version: "1.2.0"
updated: "2026-04-23"
description: "Workers Builds (native, preferred) + GitHub Actions fallback. Auto-deploy on main push, E2E on PR, branch previews, Lighthouse audit, auto-merge on passing PRs."
---

# CI/CD Pipeline

## Note on Usage

Brian deploys live from CLI (`npx wrangler deploy`). This pipeline exists for:

- Future contributors who use PRs
- Safety net — auto-test on push
- Branch previews for review
- Lighthouse tracking over time

## GitHub Actions Workflow

### `.github/workflows/deploy.yml`

```yaml
name: Deploy
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - run: npx tsc --noEmit
      - run: npx eslint . --max-warnings=0
      - run: npx prettier --check .
      - name: Install Playwright
        run: npx playwright install --with-deps chromium
      - name: Run E2E tests
        run: npx playwright test
        env:
          PROD_URL: ${{ secrets.PROD_URL }}

  deploy:
    needs: test
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - name: Deploy to Cloudflare
        run: npx wrangler deploy
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
      - name: Purge Cache
        run: |
          curl -s -X POST "https://api.cloudflare.com/client/v4/zones/${{ secrets.CF_ZONE_ID }}/purge_cache" \
            -H "Authorization: Bearer ${{ secrets.CLOUDFLARE_API_TOKEN }}" \
            --data '{"purge_everything":true}'

  lighthouse:
    needs: deploy
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: treosh/lighthouse-ci-action@v12
        with:
          urls: ${{ secrets.PROD_URL }}
          uploadArtifacts: true
```

## Required GitHub Secrets

- **`CLOUDFLARE_API_TOKEN`** — Wrangler deploy token
- **`CF_ZONE_ID`** — for cache purge
- **`PROD_URL`** — `https://domain.com`

Set via: `gh secret set CLOUDFLARE_API_TOKEN --body "..."` or through GitHub UI.

## Branch Preview Deploys (Optional)

```yaml
  preview:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - name: Deploy Preview
        run: npx wrangler deploy --env preview
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
      - name: Comment PR with preview URL
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: '🚀 Preview deployed: https://preview.domain.com'
            });
```

## MCP Tools Available

### GitHub MCP (`mcp__github-mcp__*`)

- **`list_pull_requests`** — check open PRs and their CI status
- **`pull_request_read`** — read PR details including check results
- **`create_pull_request`** — create PRs programmatically for feature branches
- **`merge_pull_request`** — auto-merge PRs that pass all checks
- **`add_issue_comment`** — comment CI results on PRs
- **`search_code`** — search for workflow files across repos
- **`create_or_update_file`** — create / update `.github/workflows/*.yml` files
- **`push_files`** — push workflow changes in a single commit

### Playwright MCP (`mcp__playwright__*`) — for E2E in CI verification

- **`browser_navigate`** — navigate to preview / production URL post-deploy
- **`browser_take_screenshot`** — screenshot pages for visual regression
- **`browser_snapshot`** — get accessibility tree for a11y checks
- **`browser_console_messages`** — check for JS errors on deployed pages
- **`browser_network_requests`** — verify API calls succeed (no 4xx / 5xx)

### Cloudflare MCP — for deployment verification

- **`workers_get_worker`** — verify Worker deployed successfully
- **`workers_list`** — list all Workers and their status

## Deployment Verification Patterns

### Post-Deploy Smoke Test (run after `wrangler deploy` in CI)

```yaml
  verify:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - name: Install Playwright
        run: npx playwright install --with-deps chromium
      - name: Wait for edge propagation
        run: sleep 5
      - name: Smoke test production
        run: |
          npx playwright test tests/smoke.spec.ts
        env:
          PROD_URL: ${{ secrets.PROD_URL }}
      - name: Check health endpoint
        run: |
          STATUS=$(curl -s -o /dev/null -w '%{http_code}' "${{ secrets.PROD_URL }}/health")
          if [ "$STATUS" != "200" ]; then
            echo "Health check failed with status $STATUS"
            exit 1
          fi
```

### PR Check Workflow (gate merges on quality)

```yaml
  pr-checks:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - run: npx tsc --noEmit
      - run: npx vitest run --coverage
      - name: Playwright E2E
        run: npx playwright test
      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
```

## Computer Use Integration

Use `mcp__computer-use__*` for debugging CI failures visually:

1. **GitHub Actions UI** — screenshot the Actions tab to see failed job steps when log parsing is insufficient
2. **Preview deploy verification** — open the preview URL and screenshot at multiple viewports to verify the deploy looks correct before merging
3. **Cloudflare dashboard** — screenshot Workers & Pages dashboard to verify deployment status and error rates

## Acceptance Criteria

1. CI runs on every push to main — GitHub Actions shows a workflow run for every main branch commit
2. CI runs on every PR — PRs show check status (pass / fail) before merge
3. TypeScript compilation passes — `npx tsc --noEmit` exits 0 in CI
4. E2E tests pass in CI — Playwright test job exits 0, report artifact uploaded
5. Deploy only happens on main push — deploy job has correct `if` condition, no deploy on PRs
6. Lighthouse score tracked — Lighthouse CI job runs on main, scores uploaded as artifacts
7. Health endpoint verified post-deploy — smoke test confirms `/health` returns 200
8. Preview deploys comment on PR — PR has a bot comment with preview URL
9. Secrets configured — `gh secret list` shows `CLOUDFLARE_API_TOKEN`, `CF_ZONE_ID`, `PROD_URL`
10. Failed CI blocks merge — branch protection requires CI to pass before merge is allowed

---
name: "Critical CSS Extraction"
version: "1.1.0"
updated: "2026-04-23"
description: "Inline above-fold CSS via critters at build time. Angular CLI built-in (v13+), Hono SSR manual. Pairs with font preloading. LCP target ≤2.5s."
---

# Critical CSS Extraction

## Angular CLI (Built-in critters)

```jsonc
// angular.json — critters is bundled since Angular v13
{
  "projects": {
    "app": {
      "architect": {
        "build": {
          "options": {
            "optimization": {
              "styles": {
                "minify": true,
                "inlineCritical": true  // Enables critters (default true in production)
              }
            }
          }
        }
      }
    }
  }
}
```

Angular SSR (`@angular/ssr`) automatically runs critters during server-side rendering. For prerendered routes (`ng build --prerender`), critical CSS is inlined at build time into each static HTML file. No runtime cost.

## Standalone critters (Non-Angular)

```bash
pnpm add -D critters
```

```typescript
// scripts/inline-critical-css.ts — run as build post-process
import Critters from 'critters';
import { readFileSync, writeFileSync, readdirSync } from 'node:fs';
import { join } from 'node:path';

const critters = new Critters({
  path: 'dist/',                    // Where CSS files live
  preload: 'swap',                  // font-display: swap for deferred CSS
  inlineFonts: false,               // We handle fonts separately (font-subsetting skill)
  compress: true,                   // Minify inlined CSS
  pruneSource: false,               // Keep full CSS file for non-critical
  reduceInlineStyles: true,         // Remove unused inline styles
  mergeStylesheets: true,           // Combine multiple <style> into one
  additionalStylesheets: [],        // Extra CSS to consider
});

async function processHtmlFiles(dir: string): Promise<void> {
  const files = readdirSync(dir, { recursive: true, encoding: 'utf8' })
    .filter((f) => f.endsWith('.html'));

  for (const file of files) {
    const fullPath = join(dir, file);
    const html = readFileSync(fullPath, 'utf8');
    const inlined = await critters.process(html);
    writeFileSync(fullPath, inlined);
    console.log(`Inlined critical CSS: ${file}`);
  }
}

await processHtmlFiles('dist/browser');
```

## How critters Works

1. Parses HTML, finds all `<link rel="stylesheet">` references
2. Loads referenced CSS files from disk
3. Renders page layout using a minimal DOM parser (no headless browser)
4. Identifies CSS rules that affect above-fold elements (viewport height heuristic)
5. Inlines critical rules into `<style>` in `<head>`
6. Converts remaining `<link>` to `<link rel="preload" as="style" onload="this.rel='stylesheet'">` with `<noscript>` fallback

## Hono SSR Manual Critical CSS

```typescript
// For Hono-served HTML pages (non-Angular)
// Pre-extract critical CSS per route at build time, store as strings

import { criticalCssMap } from './critical-css-map'; // Generated at build

app.get('/', (c) => {
  const criticalCss = criticalCssMap['/'] ?? '';
  return c.html(`<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <style>${criticalCss}</style>
  <link rel="preload" href="/fonts/Sora-Regular-subset.woff2" as="font" type="font/woff2" crossorigin>
  <link rel="preload" href="/styles/main.css" as="style">
  <link rel="stylesheet" href="/styles/main.css" media="print" onload="this.media='all'">
  <noscript><link rel="stylesheet" href="/styles/main.css"></noscript>
</head>
<body><!-- ... --></body>
</html>`);
});
```

## Build Integration (package.json)

```jsonc
{
  "scripts": {
    "build": "ng build --configuration=production",
    "postbuild": "node scripts/inline-critical-css.ts",  // Only if not using Angular SSR
    "deploy": "pnpm build && wrangler deploy"
  }
}
```

## Optimal Load Sequence

1. Inline critical CSS in `<style>` (0ms — already in HTML)
2. Preload above-fold fonts (parallel with HTML parse)
3. Deferred full stylesheet via `<link rel="preload" as="style">` (non-blocking)
4. `font-display: swap` prevents invisible text during font load
5. Below-fold images — lazy load
6. **Result:** first meaningful paint with styled content + system font → swap to custom font

## Verification

```bash
# Measure before/after with Lighthouse
npx lighthouse https://example.com --only-categories=performance --output=json | jq '.audits["render-blocking-resources"]'

# Check that critical CSS is actually inlined
curl -s https://example.com | grep -c '<style>' # Should be >= 1
curl -s https://example.com | grep -c 'media="print" onload' # Deferred stylesheets
```

### Targets

- LCP — ≤2.5s
- CLS — ≤0.1 (critical CSS prevents layout shift from late-loading styles)
- FCP — ≤1.8s
- Combined with font subsetting + preloading, typical FCP improvement: 300-800ms

---
name: "Font Subsetting"
version: "1.1.0"
updated: "2026-04-23"
description: "glyphhanger → WOFF2 → R2 self-host. Sora, Space Grotesk, JetBrains Mono subset to Latin+common. Preload critical, font-display:swap. Budget ≤100KB."
---

# Font Subsetting

## glyphhanger Subset Command

```bash
# Install
npm install -g glyphhanger
pip install fonttools brotli zopfli

# Subset to Latin + common symbols (covers 99%+ of English content)
glyphhanger --whitelist=US_ASCII --formats=woff2 --subset=Sora-Regular.ttf
glyphhanger --whitelist=US_ASCII --formats=woff2 --subset=Sora-Medium.ttf
glyphhanger --whitelist=US_ASCII --formats=woff2 --subset=SpaceGrotesk-SemiBold.ttf
glyphhanger --whitelist=US_ASCII --formats=woff2 --subset=SpaceGrotesk-Bold.ttf
glyphhanger --whitelist=US_ASCII --formats=woff2 --subset=JetBrainsMono-Regular.ttf

# Output: *-subset.woff2 files (typically 60-80% smaller)

# For sites with specific character needs, crawl first:
glyphhanger https://example.com --formats=woff2 --subset=*.ttf
```

## @font-face Declarations

```css
/* src/styles/fonts.css — self-hosted from R2 */
@font-face {
  font-family: 'Sora';
  src: url('/fonts/Sora-Regular-subset.woff2') format('woff2');
  font-weight: 400;
  font-style: normal;
  font-display: swap;
  unicode-range: U+0000-007F, U+00A0-00FF, U+2000-206F, U+2190-21FF, U+2200-22FF;
}

@font-face {
  font-family: 'Sora';
  src: url('/fonts/Sora-Medium-subset.woff2') format('woff2');
  font-weight: 500;
  font-style: normal;
  font-display: swap;
  unicode-range: U+0000-007F, U+00A0-00FF, U+2000-206F;
}

@font-face {
  font-family: 'Space Grotesk';
  src: url('/fonts/SpaceGrotesk-SemiBold-subset.woff2') format('woff2');
  font-weight: 600;
  font-style: normal;
  font-display: swap;
  unicode-range: U+0000-007F, U+00A0-00FF;
}

@font-face {
  font-family: 'Space Grotesk';
  src: url('/fonts/SpaceGrotesk-Bold-subset.woff2') format('woff2');
  font-weight: 700;
  font-style: normal;
  font-display: swap;
  unicode-range: U+0000-007F, U+00A0-00FF;
}

@font-face {
  font-family: 'JetBrains Mono';
  src: url('/fonts/JetBrainsMono-Regular-subset.woff2') format('woff2');
  font-weight: 400;
  font-style: normal;
  font-display: swap;
  unicode-range: U+0000-007F, U+00A0-00FF;
}
```

## Preload Critical Fonts (HTML head)

```html
<!-- Preload only above-fold fonts (body + heading) — max 2-3 -->
<link rel="preload" href="/fonts/Sora-Regular-subset.woff2" as="font" type="font/woff2" crossorigin>
<link rel="preload" href="/fonts/SpaceGrotesk-Bold-subset.woff2" as="font" type="font/woff2" crossorigin>
```

`crossorigin` attribute required even for same-origin fonts (CORS fetch mode). Only preload fonts used above the fold — preloading all fonts wastes bandwidth and delays LCP.

## @fontsource Alternative (pre-subset, tree-shakeable)

```bash
pnpm add @fontsource-variable/sora @fontsource-variable/space-grotesk @fontsource/jetbrains-mono
```

```typescript
// In Angular styles.css or main.ts
import '@fontsource-variable/sora/wght.css';           // Variable font, all weights
import '@fontsource-variable/space-grotesk/wght.css';
import '@fontsource/jetbrains-mono/400.css';            // Single weight
```

@fontsource ships WOFF2-only, pre-subset to Latin by default. Faster setup than manual glyphhanger. Trade-off: less control over exact subset, slightly larger than custom subset.

## R2 Upload + Cache Headers

```bash
# Upload subset fonts to R2 with immutable caching
for f in fonts/*-subset.woff2; do
  wrangler r2 object put "assets/fonts/$(basename $f)" --file "$f" \
    --content-type "font/woff2" \
    --cache-control "public, max-age=31536000, immutable"
done
```

## Worker Font Serving (if not using R2 public bucket)

```typescript
app.get('/fonts/:filename', async (c) => {
  const object = await c.env.R2.get(`fonts/${c.req.param('filename')}`);
  if (!object) return c.notFound();
  return new Response(object.body, {
    headers: {
      'Content-Type': 'font/woff2',
      'Cache-Control': 'public, max-age=31536000, immutable',
      'Access-Control-Allow-Origin': '*',
    },
  });
});
```

## Size Budget

- **Target per font file** — 15-25KB (subset WOFF2)
- **Total all fonts** — ≤100KB

### Typical breakdown

- Sora 400 (~18KB)
- Sora 500 (~18KB)
- Space Grotesk 600 (~16KB)
- Space Grotesk 700 (~16KB)
- JetBrains Mono 400 (~20KB)
- **Total: ~88KB**

If over budget: drop least-used weight, use `font-synthesis: weight` for minor weight differences.

Never use Google Fonts CDN (privacy, extra DNS lookup, no HTTP/3 multiplexing with your origin). Self-host always.

---
name: "GH Fix CI"
version: "1.1.0"
updated: "2026-04-23"
description: "Debug failing GitHub PR checks via gh CLI. Fetch GH Actions logs, summarize failure snippet, draft fix plan, implement. External providers (Buildkite) out of scope — report URL only."
---

# Gh Pr Checks Plan Fix

## Overview

Use `gh` to locate failing PR checks, fetch GitHub Actions logs for actionable failures, summarize the failure snippet, then propose a fix plan and implement after explicit approval.

- If a plan-oriented skill (for example `create-plan`) is available, use it; otherwise draft a concise plan inline and request approval before implementing.

**Prereq:** authenticate with the standard GitHub CLI once (for example, run `gh auth login`), then confirm with `gh auth status` (repo + workflow scopes are typically required).

## Inputs

- **`repo`** — path inside the repo (default `.`)
- **`pr`** — PR number or URL (optional; defaults to current branch PR)
- **`gh` authentication** — for the repo host

## Quick start

- `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "<number-or-url>"`
- Add `--json` if you want machine-friendly output for summarization

## Workflow

### 1. Verify gh authentication

- Run `gh auth status` in the repo
- If unauthenticated, ask the user to run `gh auth login` (ensuring repo + workflow scopes) before proceeding

### 2. Resolve the PR

- Prefer the current branch PR — `gh pr view --json number,url`
- If the user provides a PR number or URL, use that directly

### 3. Inspect failing checks (GitHub Actions only)

**Preferred:** run the bundled script (handles `gh` field drift and job-log fallbacks):

- `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "<number-or-url>"`
- Add `--json` for machine-friendly output

**Manual fallback:**

- `gh pr checks <pr> --json name,state,bucket,link,startedAt,completedAt,workflow`
- If a field is rejected, rerun with the available fields reported by `gh`
- For each failing check, extract the run id from `detailsUrl` and run:
  - `gh run view <run_id> --json name,workflowName,conclusion,status,url,event,headBranch,headSha`
  - `gh run view <run_id> --log`
- If the run log says it is still in progress, fetch job logs directly:
  - `gh api "/repos/<owner>/<repo>/actions/jobs/<job_id>/logs" > "<path>"`

### 4. Scope non-GitHub Actions checks

- If `detailsUrl` is not a GitHub Actions run, label it as external and only report the URL
- Do not attempt Buildkite or other providers; keep the workflow lean

### 5. Summarize failures for the user

- Provide the failing check name, run URL (if any), and a concise log snippet
- Call out missing logs explicitly

### 6. Create a plan

- Use the `create-plan` skill to draft a concise plan and request approval

### 7. Implement after approval

- Apply the approved plan, summarize diffs / tests, and ask about opening a PR

### 8. Recheck status

- After changes, suggest re-running the relevant tests and `gh pr checks` to confirm

## Bundled Resources

### scripts/inspect_pr_checks.py

Fetch failing PR checks, pull GitHub Actions logs, and extract a failure snippet. Exits non-zero when failures remain so it can be used in automation.

Usage examples:

- `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "123"`
- `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --pr "https://github.com/org/repo/pull/123" --json`
- `python "<path-to-skill>/scripts/inspect_pr_checks.py" --repo "." --max-lines 200 --context 40`

---
name: "Launch Day Sequence"
version: "1.1.0"
updated: "2026-04-23"
description: "Go-live checklist: sitemap → GSC, robots.txt unblock, Postiz social, Resend launch email, full quality gate, uptime monitoring setup. Nothing forgotten."
---

# Launch Day Sequence

## Pre-Launch Verification

- [ ] All pages return 200
- [ ] All forms submit correctly (8-point test matrix — 06/contact-forms-and-endpoints)
- [ ] All images load (no broken images)
- [ ] No placeholder content (Lorem, TODO, coming soon)
- [ ] Mobile responsive at 375px
- [ ] Desktop looks good at 1280px
- [ ] Accessibility — axe-core 0 violations (07/accessibility-gate)
- [ ] SEO — Yoast checklist passes on all pages (09/seo-and-keywords)
- [ ] Performance — Lighthouse report generated
- [ ] Security — CSP headers, Turnstile on forms
- [ ] Legal — privacy policy + terms present
- [ ] Easter egg — at least one hidden delight (06/easter-eggs)
- [ ] Error pages — branded 404 + 500 (06/custom-error-pages)
- [ ] Contact form — working and tested (06/contact-forms-and-endpoints)
- [ ] Web property completeness (06/web-manifest-system):
  - `site.webmanifest` validates in Chrome DevTools (0 warnings)
  - PWA screenshots taken with Playwright (wide + narrow `form_factor`)
  - 4+ JSON-LD blocks per page (Organization, WebSite + SearchAction, WebPage, domain-specific)
  - OG images at 1200×630, visually verified on Twitter Card Validator + Facebook Debugger
  - Infrastructure files — `humans.txt`, `security.txt`, `browserconfig.xml`, `opensearch.xml`
  - Cross-site alternate links present
  - Sitemap submitted to Google Search Console

## Launch Sequence (Automated)

### Step 1: Final Deploy + Purge

```bash
npx wrangler deploy
curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/purge_cache" \
  -H "Authorization: Bearer $CF_API_TOKEN" --data '{"purge_everything":true}'
sleep 5
```

### Step 2: Ensure Search Engines Can Crawl

```bash
# Verify robots.txt allows crawling
curl -s "https://domain.com/robots.txt" | grep "Allow: /"
# Verify sitemap exists and is valid XML
curl -s "https://domain.com/sitemap.xml" | head -5
```

### Step 3: Submit Sitemap to Google Search Console

```bash
# Via GSC API (GCP service account required)
curl -X PUT "https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fdomain.com/sitemaps/https%3A%2F%2Fdomain.com%2Fsitemap.xml" \
  -H "Authorization: Bearer $GSC_TOKEN"

# Or via ping (no auth needed)
curl "https://www.google.com/ping?sitemap=https://domain.com/sitemap.xml"
```

### Step 4: GitHub Auto-Config

```bash
gh repo edit --description "$(curl -s https://domain.com | grep -oP '(?<=<meta name="description" content=")[^"]*')"
gh repo edit --homepage "https://domain.com"
gh repo edit --add-topic "cloudflare,hono,emdash,typescript"
```

### Step 5: Generate README (09/documentation-and-codebase-hygiene)

Auto-generate branded README with install.doctor template, aqua dividers, badges.

### Step 6: Social Announcement (09/social-automation)

```bash
# Auto-post via Postiz
curl -X POST "https://postiz.megabyte.space/api/posts" \
  -H "Authorization: Bearer $POSTIZ_API_KEY" \
  -d '{
    "content": "Just shipped: domain.com — [product description]",
    "platforms": ["twitter", "linkedin"],
    "media": ["https://domain.com/og/homepage.png"],
    "schedule": "now"
  }'
```

### Step 7: Launch Email (09/email-templates)

Send branded launch announcement via Resend to newsletter subscribers (if Listmonk is set up).

### Step 8: Production E2E Suite

```bash
PROD_URL=https://domain.com npx playwright test
```

### Step 9: Cross-Browser Smoke (first deploy only)

```bash
npx playwright test --project=chromium --project=firefox --project=webkit
```

### Step 10: Final Report

```markdown
## Launch Report — domain.com
### Status: ✅ LIVE
- URL: https://domain.com
- Deploy: [timestamp]
- Lighthouse: [score]
- E2E: [pass/fail]
- Sitemap: submitted to GSC
- Social: posted to [platforms]
- README: generated

### What Was Built
- [feature list]

### Next Steps
- [improvements from idea engine — 14-independent-idea-engine]
```

### Step 11: Notify Connected Services

```typescript
// Slack deploy notification
await notifySlack(env, `🚀 *${domain}* is LIVE\n<https://${domain}|Visit site>\nBuilt with projectsites.dev`);

// Discord community notification (if applicable)
await notifyDiscord(env, `${domain} launched!`, `Check it out: https://${domain}`);

// Zapier webhook (triggers any connected automations)
await triggerZapier(env, 'site_launched', { domain, url: `https://${domain}` });
```

### Step 12: Psychology-Optimized Launch (04/wisdom-and-human-psychology)

- **Peak-End Rule** — the launch announcement IS the peak moment. Make it count.
- **Social Proof** — include user count or testimonial in the social post
- **Reciprocity** — share something valuable in the announcement (tip, insight, free tool)
- **Unity** — frame as "we built this" not "I built this" — shared identity with community

### Brand Amplification

Every launch amplifies the projectsites.dev brand:

- Social posts mention "Built with projectsites.dev" when appropriate
- README includes projectsites.dev badge
- Footer includes projectsites.dev attribution
- The quality of the launch IS the marketing

# Pipeline Health Check (***FIRST ACTION EVERY SESSION INVOLVING BUILDS/JOBS***)

Long-running pipelines (CF Workflows, build queues, generation jobs, container orchestrators) must be inspected for wedged / error rows BEFORE any new trigger. Universal protocol — see `~/.claude/rules/failed-pipeline-protocol.md` for the full rule. This file is the project-sites runbook: exact commands, exact tables, exact thresholds.

## Detect — One-Line D1 Query

```bash
cd <repo> && set -a && source .env.local && set +a
npx wrangler d1 execute project-sites-db-production --remote --env production --json --command \
  "SELECT id, slug, status, datetime(updated_at) AS u FROM sites WHERE status IN ('building','queued','generating','imaging','uploading','collecting') AND updated_at < datetime('now','-30 minutes') AND deleted_at IS NULL ORDER BY updated_at"
```

- Any row returned = wedged
- If zero rows: pipeline is healthy, proceed
- **Cron unsticker check:** dashboard → Workers → project-sites → Triggers → cron `*/30 * * * *` last-run timestamp must be ≤30min old
- **Workflow instance status** (when needed): `instance.status()` from worker code OR `wrangler workflows instances describe <workflow> <id> --env production`

## Diagnose — Five Canonical Failure Modes

Order by historical frequency at project-sites:

1. **Schema CHECK constraint silently rejecting status writes** — `SELECT sql FROM sqlite_master WHERE name='sites'` → confirm allowed values include EVERY status the code writes (`draft|queued|collecting|imaging|generating|building|uploading|published|error|archived`). If missing any, table needs recreation.
2. **No hard wall-time cap in workflow** — `apps/project-sites/src/workflows/site-generation.ts` → `MAX_POLLS × interval` must be ≤30min and must call `updateSiteStatus(..., 'error')` + `notifyBuildFailure()` on cap hit.
3. **Cron unsticker threshold drift** — `apps/project-sites/src/index.ts` cron handler must use `datetime('now','-30 minutes')`, never -60.
4. **Missing failure notification** — every error path in workflow must `await notifyBuildFailed({ email, siteName, slug, reason })` from `services/notifications.ts`.
5. **Container / orchestrator OOM / credit-balance** — Anthropic credit hold, R2 quota, container OOM. `audit_logs WHERE site_id=<id> ORDER BY created_at DESC LIMIT 50` shows the last action before silence.

## Fix — Schema Recreation Pattern (D1 has no `ALTER CHECK`)

```sql
PRAGMA foreign_keys=OFF;
CREATE TABLE sites_new ( /* full schema with expanded CHECK */ );
INSERT INTO sites_new SELECT <every column in order> FROM sites;
DROP TABLE sites;
ALTER TABLE sites_new RENAME TO sites;
PRAGMA foreign_keys=ON;
```

Run via `wrangler d1 execute project-sites-db-production --remote --env production --file /tmp/migrate.sql`. Always test on local D1 first (`--local`).

## Verify Fix — Status Write Smoke Test

```bash
npx wrangler d1 execute project-sites-db-production --remote --env production --command \
  "UPDATE sites SET status='error' WHERE id='<test-id>'; SELECT status FROM sites WHERE id='<test-id>'"
```

Must return `error`, not silently fail. Then deploy: `cd apps/project-sites && npx wrangler deploy --env production` (sandbox often blocks log writes — use `dangerouslyDisableSandbox: true`).

## Retrigger — Session Mint + Direct Worker URL

- Public hostname `https://projectsites.dev` triggers CF Bot Fight challenge for scripted POSTs → use direct worker URL `https://project-sites.manhattan.workers.dev`
- Reference one-shot script: `/tmp/claude/retrigger-builds.mjs` — mints session via D1 INSERT (sha256 token_hash), POSTs `/api/sites/<id>/reset` with Bearer token
- Always pair with a background monitor task polling D1 every 60s, max 35min

## Operator Stance

This work IS the business. Detect → diagnose → fix → verify → retrigger → monitor → report URLs + codes in one session, no handoff. Document new failure modes in this file the same prompt they occur. Stale runbook = bug.

## Canonical Incident: 2026-05-01 (4 wedged sites, 15+ hours)

**Root cause:** `sites.status` CHECK allowed only 5 values, code wrote 10. Every error-path write silently rejected → cron saw fresh heartbeat `updated_at` → never fired unsticker. Compounded by 60min threshold + no `MAX_POLLS` cap + no `notifyBuildFailed` at error paths.

**Fix bundle:**

- Expanded CHECK
- `MAX_POLLS=30`
- Cron 30min
- Email at every error path
- Direct worker URL for retrigger

**Lesson:** when wedge persists past SLA, FIRST suspect schema silently rejecting writes. `sqlite_master` query is cheap, the answer is usually there.

# R2 Lifecycle Management (***UNIVERSAL — EVERY CLOUDFLARE WORKER WITH R2***)

R2 buckets accumulate stale deploy artifacts (Angular chunks, source-map files, old marketing builds) at one generation per deploy. Without lifecycle hygiene a 200-route site bucket bloats to 4,000+ objects in 6 months, $0.015/GB/month adds up, and audit / cleanup work explodes.

**Wrangler v4 does NOT list R2 objects** (`wrangler r2 object` = get | put | delete only) — use REST API or S3-compat CLI for inventory, wrangler for delete.

## Inventory (***THREE CANONICAL PATHS — PICK FIRST AVAILABLE***)

### (A) S3-compat CLI — preferred

When `aws-cli` or `rclone` installed. Set `~/.aws/credentials [r2]` profile with R2 access-key-id + secret from CF dashboard r2 API tokens page.

```bash
aws s3 ls "s3://project-sites-production/marketing/" --recursive \
  --endpoint-url "https://${CLOUDFLARE_ACCOUNT_ID}.r2.cloudflarestorage.com" --profile r2 > /tmp/r2-marketing.txt
```

### (B) REST list — works with GLOBAL KEY, no install

Cursor pagination in 6 lines of bash.

```bash
ACCOUNT=84fa0d1b16ff8086dd958c468ce7fd59 BUCKET=project-sites-production PREFIX=marketing/
CURSOR=""; > /tmp/r2-marketing.json
echo "[" > /tmp/r2-marketing.json
FIRST=1
while :; do
  RESP=$(curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT/r2/buckets/$BUCKET/objects?per_page=1000&prefix=$PREFIX${CURSOR:+&cursor=$CURSOR}" \
    -H "X-Auth-Email: $CLOUDFLARE_EMAIL" -H "X-Auth-Key: $CLOUDFLARE_API_KEY")
  KEYS=$(echo "$RESP" | jq -c '.result[]')
  [ -n "$KEYS" ] && { [ $FIRST -eq 0 ] && echo ","; echo "$KEYS" | paste -sd ',' -; FIRST=0; } >> /tmp/r2-marketing.json
  CURSOR=$(echo "$RESP" | jq -r 'if .result_info.is_truncated then .result_info.cursor else empty end')
  [ -z "$CURSOR" ] && break
done
echo "]" >> /tmp/r2-marketing.json
```

### (C) Quick key-only inventory (no metadata)

```bash
ACCOUNT=84fa0d1b16ff8086dd958c468ce7fd59 BUCKET=project-sites-production PREFIX=marketing/
CURSOR=""; > /tmp/r2-keys.txt
while :; do
  RESP=$(curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT/r2/buckets/$BUCKET/objects?per_page=1000&prefix=$PREFIX${CURSOR:+&cursor=$CURSOR}" \
    -H "X-Auth-Email: $CLOUDFLARE_EMAIL" -H "X-Auth-Key: $CLOUDFLARE_API_KEY")
  echo "$RESP" | jq -r '.result[].key' >> /tmp/r2-keys.txt
  CURSOR=$(echo "$RESP" | jq -r 'if .result_info.is_truncated then .result_info.cursor else empty end')
  [ -z "$CURSOR" ] && break
done
wc -l /tmp/r2-keys.txt
```

### Common prefixes in `project-sites-production`

- `sites/<slug>/` — client data, PRESERVE
- `marketing/` — homepage assets
- `templates/<category>/`
- `app/` — legacy admin SPA
- `container/` — build artifacts
- `retrospectives/` — debug dumps
- `test/` — probes

## Stale-asset classification (***Angular/Vite hash-named chunks accumulate fastest***)

```bash
# Identify hash-named build artifacts: chunk-ABC123.js, main-XYZ456.js, polyfills-DEF.js
grep -E "/(chunk|main|polyfills)-[A-Z0-9]+\.js$" /tmp/r2-keys.txt > /tmp/stale-chunks.txt
# Identify all CSS bundles (Angular/Vite generate one per deploy)
grep -E "\.css$" /tmp/r2-keys.txt > /tmp/stale-css.txt
# Cross-reference against the LIVE homepage references — anything live = preserve
curl -fsSL https://projectsites.dev/ > /tmp/live-homepage.html
grep -oE 'src="[^"]+"|href="[^"]+"' /tmp/live-homepage.html | sed 's/^[^"]*"//; s/"$//' | sort -u > /tmp/live-refs.txt
# Final delete list = stale candidates MINUS live refs
comm -23 <(sort /tmp/stale-chunks.txt) <(sort /tmp/live-refs.txt) > /tmp/r2-delete.txt
```

## Batch delete (***ONLY AFTER USER CONFIRMATION on production buckets***)

```bash
# Dry-run first
wc -l /tmp/r2-delete.txt # confirm count matches expectation
head -10 /tmp/r2-delete.txt # confirm pattern matches expectation
# Execute with parallel concurrency. wrangler r2 object delete IS native (verified v4.63.0).
# P=4 safe, P=8 OK, never >P=16 — R2 rate-limits at 1000 ops/s/bucket
xargs -I{} -P 4 npx wrangler r2 object delete "<bucket>/{}" --remote < /tmp/r2-delete.txt
# Verify post-delete by re-listing
# (re-run inventory script above)
```

## Lifecycle policy (***SHIP WITH EVERY NEW BUCKET***)

Cloudflare R2 supports object lifecycle rules via `wrangler r2 bucket lifecycle`. Apply at bucket creation:

```bash
# Auto-delete deploy artifacts older than 30 days
npx wrangler r2 bucket lifecycle add <bucket> --prefix marketing/ --age-days 30 --action delete --remote
# Preserve client data (sites/) — no lifecycle rule, manual purge only on subscription end
# Transition rarely-accessed templates to Infrequent Access tier after 60 days
npx wrangler r2 bucket lifecycle add <bucket> --prefix templates/ --age-days 60 --action transition-ia --remote
```

Without lifecycle rules every deploy is a permanent cost. **Verify with** `npx wrangler r2 bucket lifecycle list <bucket> --remote`.

## Prevention (***BUILD-CONFIG CHANGES***)

- Vite / Angular `outDir` should write to a versioned subdirectory (`marketing/v<commit-sha>/`), with the worker resolving `marketing/index.html` → latest version. Old versions become trivially purgeable by directory.
- Worker upload script (`scripts/deploy-r2.mjs`) MUST delete old artifacts before uploading new ones — never just `--no-prefix` upload that accumulates.
- Per `~/.claude/rules/builtin-tools-first.md` — probe `wrangler <subcommand> --help` before citing — wrangler v4 r2 object has no `list`. REST endpoint or `aws s3 ls --endpoint-url` are the real list paths.

## See Also

- `~/.claude/rules/builtin-tools-first.md` (universal vendor-CLI-first rule)
- `~/.agentskills/08-deploy-and-runtime-verification/SKILL.md` (deploy gate)
- `migrations/_applied.md` (wrangler global-key rejection on D1 migrations only — does NOT extend to R2)

---
name: "Service Worker and Offline"
version: "1.2.0"
updated: "2026-04-23"
description: "Workbox cache strategies: CacheFirst static (30-day), NetworkFirst API/HTML, offline fallback, background sync, push via Novu, Angular ngsw integration, CF Workers edge+client SW coordination."
---

# Service Worker and Offline

## Workbox Configuration (workbox-config.js)

```javascript
// workbox-config.js — Vite/webpack plugin feeds this
module.exports = {
  globDirectory: 'dist/',
  globPatterns: ['**/*.{html,js,css,woff2,ico,png,svg}'],
  swDest: 'dist/sw.js',
  skipWaiting: true,
  clientsClaim: true,
  runtimeCaching: [
    {
      urlPattern: /\.(?:js|css|woff2)$/,
      handler: 'CacheFirst',
      options: {
        cacheName: 'static-assets',
        expiration: { maxAgeSeconds: 30 * 24 * 60 * 60, maxEntries: 100 },
      },
    },
    {
      urlPattern: /\.(?:png|jpg|jpeg|webp|avif|svg|gif|ico)$/,
      handler: 'CacheFirst',
      options: {
        cacheName: 'images',
        expiration: { maxAgeSeconds: 90 * 24 * 60 * 60, maxEntries: 200 },
      },
    },
    {
      urlPattern: /\/api\//,
      handler: 'NetworkFirst',
      options: {
        cacheName: 'api-responses',
        expiration: { maxAgeSeconds: 5 * 60, maxEntries: 50 },
        networkTimeoutSeconds: 3,
      },
    },
    {
      urlPattern: /\//,
      handler: 'NetworkFirst',
      options: {
        cacheName: 'html-pages',
        expiration: { maxAgeSeconds: 24 * 60 * 60, maxEntries: 30 },
        networkTimeoutSeconds: 3,
      },
    },
  ],
};
```

## Service Worker Entry (sw.ts)

```typescript
// src/sw.ts — compiled by Workbox webpack/vite plugin
import { precacheAndRoute, cleanupOutdatedCaches } from 'workbox-precaching';
import { registerRoute, NavigationRoute } from 'workbox-routing';
import { CacheFirst, NetworkFirst, StaleWhileRevalidate } from 'workbox-strategies';
import { ExpirationPlugin } from 'workbox-expiration';
import { BackgroundSyncPlugin } from 'workbox-background-sync';
import { CacheableResponsePlugin } from 'workbox-cacheable-response';

declare const self: ServiceWorkerGlobalScope;

// Precache critical shell (auto-injected by Workbox build)
precacheAndRoute(self.__WB_MANIFEST);
cleanupOutdatedCaches();

// Static assets: CacheFirst, 30-day TTL
registerRoute(
  ({ request }) => ['script', 'style', 'font'].includes(request.destination),
  new CacheFirst({
    cacheName: 'static-v1',
    plugins: [
      new ExpirationPlugin({ maxAgeSeconds: 30 * 24 * 60 * 60, maxEntries: 100 }),
      new CacheableResponsePlugin({ statuses: [0, 200] }),
    ],
  })
);

// Images: CacheFirst, 90-day TTL (R2 URLs are content-addressed)
registerRoute(
  ({ request }) => request.destination === 'image',
  new CacheFirst({
    cacheName: 'images-v1',
    plugins: [
      new ExpirationPlugin({ maxAgeSeconds: 90 * 24 * 60 * 60, maxEntries: 200 }),
      new CacheableResponsePlugin({ statuses: [0, 200] }),
    ],
  })
);

// API responses: NetworkFirst, 5-minute TTL
registerRoute(
  ({ url }) => url.pathname.startsWith('/api/'),
  new NetworkFirst({
    cacheName: 'api-v1',
    networkTimeoutSeconds: 3,
    plugins: [new ExpirationPlugin({ maxAgeSeconds: 5 * 60, maxEntries: 50 })],
  })
);

// Non-critical API: StaleWhileRevalidate
registerRoute(
  ({ url }) => url.pathname.startsWith('/api/public/'),
  new StaleWhileRevalidate({
    cacheName: 'api-public-v1',
    plugins: [new ExpirationPlugin({ maxAgeSeconds: 15 * 60, maxEntries: 30 })],
  })
);

// HTML pages: NetworkFirst, fallback to cached shell
const htmlStrategy = new NetworkFirst({
  cacheName: 'pages-v1',
  networkTimeoutSeconds: 3,
  plugins: [new ExpirationPlugin({ maxAgeSeconds: 24 * 60 * 60, maxEntries: 30 })],
});

const navigationRoute = new NavigationRoute(htmlStrategy, {
  denylist: [/\/api\//, /\/admin\//],
});
registerRoute(navigationRoute);

// Background sync: queue failed form submissions, retry when online
const bgSyncPlugin = new BackgroundSyncPlugin('form-submissions', {
  maxRetentionTime: 24 * 60, // 24 hours in minutes
  onSync: async ({ queue }) => {
    let entry;
    while ((entry = await queue.shiftRequest())) {
      try {
        await fetch(entry.request);
      } catch {
        await queue.unshiftRequest(entry);
        throw new Error('Replay failed');
      }
    }
  },
});

registerRoute(
  ({ url }) => url.pathname.startsWith('/api/forms/'),
  new NetworkFirst({ plugins: [bgSyncPlugin] }),
  'POST'
);

// Offline fallback page
self.addEventListener('fetch', (event) => {
  if (event.request.mode === 'navigate') {
    event.respondWith(
      fetch(event.request).catch(() => caches.match('/offline.html') as Promise<Response>)
    );
  }
});

// Immediate activation: skipWaiting + clients.claim
self.addEventListener('install', () => self.skipWaiting());
self.addEventListener('activate', (event) => {
  event.waitUntil(self.clients.claim());
});
```

## Offline Fallback Page (offline.html)

```html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Offline — Megabyte Labs</title>
  <style>
    * { margin: 0; padding: 0; box-sizing: border-box; }
    body { background: #060610; color: #fff; font-family: 'Space Grotesk', system-ui; display: flex; align-items: center; justify-content: center; min-height: 100vh; }
    .container { text-align: center; padding: 2rem; }
    h1 { font-family: 'Sora', system-ui; font-size: 2rem; margin-bottom: 1rem; }
    p { color: #A0A0B8; margin-bottom: 2rem; }
    .accent { color: #00E5FF; }
    button { background: #00E5FF; color: #060610; border: none; padding: 12px 32px; font-size: 1rem; font-weight: 700; border-radius: 8px; cursor: pointer; }
    button:hover { background: #50AAE3; }
  </style>
</head>
<body>
  <div class="container">
    <h1>You're <span class="accent">offline</span></h1>
    <p>Check your connection and try again. Cached pages are still available.</p>
    <button onclick="window.location.reload()">Retry</button>
  </div>
</body>
</html>
```

## Angular Integration (ngsw-config.json)

```json
{
  "$schema": "./node_modules/@angular/service-worker/config/schema.json",
  "index": "/index.html",
  "assetGroups": [
    {
      "name": "shell",
      "installMode": "prefetch",
      "updateMode": "prefetch",
      "resources": {
        "files": ["/index.html", "/main*.js", "/polyfills*.js", "/styles*.css"],
        "urls": ["/assets/fonts/*.woff2"]
      }
    },
    {
      "name": "assets",
      "installMode": "lazy",
      "updateMode": "lazy",
      "resources": {
        "files": ["/assets/**", "/**/*.png", "/**/*.svg", "/**/*.ico"]
      }
    }
  ],
  "dataGroups": [
    {
      "name": "api-fresh",
      "urls": ["/api/**"],
      "cacheConfig": { "strategy": "freshness", "maxAge": "5m", "maxSize": 50, "timeout": "3s" }
    },
    {
      "name": "api-cached",
      "urls": ["/api/public/**"],
      "cacheConfig": { "strategy": "performance", "maxAge": "15m", "maxSize": 30 }
    }
  ],
  "navigationUrls": ["/**", "!/api/**", "!/admin/**"],
  "navigationRequestStrategy": "freshness"
}
```

## Push Notifications (Novu Integration)

```typescript
// src/sw-push.ts — append to sw.ts or separate push handler
self.addEventListener('push', (event) => {
  const data = event.data?.json() ?? { title: 'Megabyte Labs', body: 'New notification' };
  event.waitUntil(
    self.registration.showNotification(data.title, {
      body: data.body,
      icon: '/assets/icons/icon-192.png',
      badge: '/assets/icons/badge-72.png',
      data: { url: data.url || '/' },
      actions: data.actions || [],
    })
  );
});

self.addEventListener('notificationclick', (event) => {
  event.notification.close();
  const url = event.notification.data?.url || '/';
  event.waitUntil(self.clients.openWindow(url));
});
```

```typescript
// Server-side: trigger push via Novu
import { Novu } from '@novu/node';
const novu = new Novu(env.NOVU_API_KEY);

await novu.trigger('push-notification', {
  to: { subscriberId: userId },
  payload: { title: 'Update Available', body: 'New features shipped.', url: '/changelog' },
});
```

## SW Registration (main.ts)

```typescript
// Register service worker in Angular main.ts
if ('serviceWorker' in navigator && environment.production) {
  window.addEventListener('load', async () => {
    const reg = await navigator.serviceWorker.register('/sw.js');
    reg.addEventListener('updatefound', () => {
      const newWorker = reg.installing;
      newWorker?.addEventListener('statechange', () => {
        if (newWorker.state === 'activated' && navigator.serviceWorker.controller) {
          // New version available — prompt user or auto-reload
          if (confirm('New version available. Reload?')) window.location.reload();
        }
      });
    });
  });
}
```

## CF Workers + Client SW Coordination

- **Edge Worker handles** — routing, auth, cache headers, HTML streaming, API
- **Client SW handles** — offline fallback, asset caching, background sync, push
- **No overlap** — edge sets `Cache-Control`, client SW respects it
- Edge never caches HTML (`max-age=0`), client SW caches shell for offline
- Edge handles `/api/` auth + rate limiting, client SW caches safe GET responses

## Vite Plugin Setup

```typescript
// vite.config.ts
import { VitePWA } from 'vite-plugin-pwa';

export default defineConfig({
  plugins: [
    VitePWA({
      strategies: 'injectManifest',
      srcDir: 'src',
      filename: 'sw.ts',
      injectManifest: { globPatterns: ['**/*.{html,js,css,woff2,ico,png,svg}'] },
    }),
  ],
});
```

---
name: "deploy-and-runtime-verification"
description: "MANDATORY deploy after every code change. Typecheck → deploy → purge CDN → E2E on production → visual verify → fix-forward loop. Workers Builds native CI/CD, D1 Time Travel PIT recovery, D1→R2 long-term backups, wrangler rollback, wrangler secrets management, structured observability, cross-browser smoke tests, rollback procedures, and GitHub auto-configuration."
metadata:
  version: "2.1.0"
  updated: "2026-05-03"
  effort: "high"
  model: "sonnet"
license: "Rutgers"
compatibility:
  claude-code: ">=2.0.0"
  agentskills: ">=1.0.0"
submodules:
  - backup-and-disaster-recovery.md
  - changelog-and-releases.md
  - ci-cd-pipeline.md
  - critical-css.md
  - font-subsetting.md
  - gh-fix-ci.md
  - launch-day-sequence.md
  - pipeline-health-check.md
  - r2-lifecycle.md
  - service-worker.md
  - uptime-and-health.md
priority: 2
pack: "backend"
stage: stable
triggers:
  - "deploy"
  - "wrangler deploy"
  - "rollback"
paths:
  - "concern:cloudflare-workers"
---

# 08 — Deploy and Runtime Verification

Enforce the mandatory typecheck→deploy→CDN-purge→prod-E2E→visual-verify loop after every code change; never mark done without a live PROD assertion.

## Mandatory deploy loop (every code change)

1. Build + typecheck
2. `wrangler deploy` (or platform equivalent)
3. Purge CDN (`wrangler cache purge` or `curl -X POST https://api.cloudflare.com/client/v4/zones/{id}/purge_cache`)
4. Fetch each changed route on PROD URL via curl + Playwright
5. Assert new content / headers / JSON-LD / status live
6. AI vision QA at 6 viewports (per `_kernel/standards.md#breakpoints`)
7. Fix-forward (max 3 redeploys) — never silently fail
8. Only then mark DONE

"Local typecheck + build pass" is NOT done. Per `rules/verification-loop.md`.

## Auth fallback chain

- `CLOUDFLARE_API_TOKEN` from `/Users/Apple/.local/bin/get-secret`
- On 401: `CLOUDFLARE_API_KEY` + `CLOUDFLARE_EMAIL`
- Both stale: prompt `! npx wrangler login`, resume deploy once fresh
- NEVER silently skip deploy because creds missing — surface as blocker

## Workers Builds (native CI/CD)

- Configure in `wrangler.jsonc` `build` block
- Auto-deploys on push to `main` per `rules/main-only-branch.md`
- Runs `npm install` + `npm run build` + `wrangler deploy`
- Secrets injected via dashboard or `wrangler secret put` per `rules/secret-provisioning.md`

## Secrets management

- `wrangler secret put KEY` — runtime secrets; `wrangler secret list` — names only, never values
- Two-way mirror: every prod secret also in chezmoi (`~/.local/share/chezmoi/home/.chezmoitemplates/secrets/{KEY}`)
- `scripts/check-secrets.mjs --audit` runs before every deploy
- Per `rules/secret-provisioning.md` + `rules/secret-auto-provisioning.md`

## D1 backup strategy

- **Time Travel** — 30-day PIT, free: `wrangler d1 time-travel restore <db> --timestamp=<ts>`
- **D1 → R2 long-term** — `wrangler d1 export <db> --output=backup.sql` + upload to R2 daily via cron
- **Pre-migration safety** — `wrangler d1 export` BEFORE any destructive migration
- 1 TB storage limit per account, 10 GB per database

## Rollback procedures

- **Worker** — `wrangler rollback <version-id>` in <30s
- **D1** — Time Travel PIT to known-good timestamp
- **R2** — bucket versioning enabled; revert via object version
- **Combined deploy** — log `{commit, version_id, timestamp}` to D1 after every deploy; rollback target one query away

## Auto-rollback gates (gradual deployment)

- 1% → watch error rate 5 min → 10% → watch 5 min → 100%
- Auto-rollback at p99 error >1% or LCP regression >20%

## Browser console gate

Per `rules/verification-loop.md` § Console-error gate — all must be 0:

- Console errors / CSP report-uri violations / Trusted Types violations / Deprecation warnings / Third-party script errors

Run as part of `npm run e2e:prod` Playwright suite.

## Cross-browser smoke

Per `_kernel/standards.md#breakpoints` × 3 browsers (Chromium, Firefox, WebKit):

- Homepage loads; primary CTA clickable; form submit succeeds (Turnstile invisible)
- No console errors at any breakpoint
- Lighthouse Perf ≥75 + A11y ≥95

## Observability check post-deploy

Verify each is firing (missing = blocker):

- Sentry — recent events in dashboard
- PostHog — recent pageviews + captures
- Workers Tracing — recent traces in Axiom (Tier 2)
- AI Gateway — recent LLM calls logged (Tier 3)
- GA4 — recent events (Tier 2)

## GitHub auto-configuration

- New project: `gh repo create` with template
- Add CF Workers Builds via dashboard or API
- Wire OIDC + `cloudflare/wrangler-action@v3` for CI
- Trust policy: GitHub OIDC token verifies repo + branch — no long-lived `CLOUDFLARE_API_TOKEN` in repo secrets

## Per-deploy CHANGELOG entry

Every deploy logs to `CHANGELOG.md` (`## [version] — YYYY-MM-DD` / `### Added / Changed / Fixed`). Auto-generated by `changelog-generator` agent from conventional commits.

## See submodules: workers-builds.md, d1-backups.md, rollback.md, github-config.md.

---
name: "Uptime and Health"
version: "1.2.0"
updated: "2026-04-23"
description: "Health endpoints on every Worker (/health + /health/deep), external monitoring (UptimeRobot, Better Stack), /status page, cron self-check with Resend alert escalation."
---

# Uptime and Health

## Health Endpoint (EVERY Worker)

```typescript
app.get('/health', (c) => {
  return c.json({
    status: 'ok',
    version: c.env.VERSION || 'unknown',
    timestamp: new Date().toISOString(),
    region: c.req.header('cf-ray')?.split('-')[1] || 'unknown',
  });
});

// Deep health check (optional — checks dependencies)
app.get('/health/deep', async (c) => {
  const checks: Record<string, string> = {};

  // D1
  try {
    await c.env.DB.prepare('SELECT 1').first();
    checks.d1 = 'ok';
  } catch { checks.d1 = 'error'; }

  // KV
  try {
    await c.env.KV.get('__health');
    checks.kv = 'ok';
  } catch { checks.kv = 'error'; }

  // R2
  try {
    await c.env.R2.head('__health');
    checks.r2 = 'ok';
  } catch { checks.r2 = 'ok'; } // head returns null for missing, not error

  const allOk = Object.values(checks).every(v => v === 'ok');
  return c.json({ status: allOk ? 'ok' : 'degraded', checks }, allOk ? 200 : 503);
});
```

## External Monitoring (Let Others Handle It)

### UptimeRobot (Free — 50 monitors)

Set up via their dashboard or API:

- Monitor — `https://domain.com/health`
- Interval — 5 minutes
- Alert — email to brian@megabyte.space

### Better Stack (Free tier)

Better UI, incident management, status pages.

- Monitor — `https://domain.com/health`
- Status page — auto-generated

### Cloudflare Health Checks (Built-in)

If using Cloudflare Load Balancing, health checks are built in.

## Status Page (Optional)

```typescript
app.get('/status', async (c) => {
  const health = await fetch(`https://${c.env.DOMAIN}/health/deep`).then(r => r.json());
  return c.html(renderStatusPage(health));
});
```

Simple status page showing:

- Overall status (operational / degraded / down)
- Individual service checks (D1, KV, R2)
- Last checked timestamp
- Uptime percentage (from KV counter)

## Cron Self-Check (Optional)

```typescript
// wrangler.toml: [triggers] crons = ["*/5 * * * *"]
export default {
  async scheduled(event, env, ctx) {
    const res = await fetch(`https://${env.DOMAIN}/health`);
    if (!res.ok) {
      // Alert via Resend
      await new Resend(env.RESEND_API_KEY).emails.send({
        from: 'Monitor <alerts@megabyte.space>',
        to: ['brian@megabyte.space'],
        subject: `⚠️ ${env.DOMAIN} is down`,
        html: `Health check failed with status ${res.status}`,
      });
    }
  },
};
```

## MCP Tools Available

### Coolify MCP (`mcp__coolify__*`) — for self-hosted service health

- **`diagnose_app`** — diagnose a Coolify-hosted app (Postiz, Listmonk, PostHog, etc.)
- **`diagnose_server`** — check overall server health (CPU, memory, disk)
- **`get_infrastructure_overview`** — full infrastructure summary — all apps, DBs, services
- **`application_logs`** — pull recent logs for a specific application
- **`find_issues`** — auto-detect issues across all services
- **`validate_server`** — validate server configuration and connectivity
- **`list_applications`** — list all running applications with status
- **`control`** — start / stop / restart any Coolify service

### Cloudflare MCP (`mcp__claude_ai_Cloudflare_Developer_Platform__*`)

- **`workers_list`** — list all Workers to verify they're deployed
- **`workers_get_worker`** — check a specific Worker's status

### Playwright MCP (`mcp__playwright__*`) — for visual health verification

- **`browser_navigate`** — navigate to `/health` or `/status` endpoint
- **`browser_take_screenshot`** — screenshot the status page for visual verification

## Health Endpoint Schema (Strict)

Every Worker MUST return this exact JSON shape from `/health`:

```typescript
interface HealthResponse {
  status: 'ok' | 'degraded' | 'error';
  version: string;          // semver from wrangler.toml or env
  timestamp: string;        // ISO 8601
  region: string;           // Cloudflare colo code from cf-ray header
  uptime?: number;          // seconds since last restart (optional)
}

interface DeepHealthResponse extends HealthResponse {
  checks: Record<string, 'ok' | 'error' | 'timeout'>;
  // Each key is a dependency: 'd1', 'kv', 'r2', 'stripe', 'postiz', etc.
}
```

### Rules

- `/health` returns 200 for `ok`, 503 for `degraded` or `error`
- Response time must be <500ms (no heavy queries in shallow health)
- `/health/deep` may take up to 5s (checks external dependencies)
- Never expose secrets, internal IPs, or stack traces in health responses

## Monitoring Integration Patterns

### Pattern 1: Coolify + Cloudflare Workers (hybrid)

```
Coolify services (Postiz, Listmonk, PostHog)
  → mcp__coolify__diagnose_app per service
  → mcp__coolify__find_issues for cross-service problems

Cloudflare Workers (site, API)
  → fetch /health endpoint directly
  → mcp__claude_ai_Cloudflare_Developer_Platform__workers_get_worker for deploy status
```

### Pattern 2: Cron self-check with escalation

```
Every 5 min: Worker cron hits /health
  → OK: log to KV (increment uptime counter)
  → FAIL: send Resend alert + log to KV
  → 3 consecutive FAILs: escalate (post to Slack via mcp__claude_ai_Slack__slack_send_message)
```

## Computer Use Integration

Use `mcp__computer-use__*` for visual verification of monitoring dashboards:

1. **Better Stack dashboard** — screenshot the status page to verify all monitors show green
2. **UptimeRobot dashboard** — visual check that no monitors are in alert state
3. **Coolify dashboard** — screenshot `https://coolify.megabyte.space` to see all service health at a glance

Best used during: incident investigation, post-deploy verification, weekly health audits.

## Acceptance Criteria

1. `/health` returns 200 with valid JSON — `curl /health` returns `{ status: 'ok', version, timestamp, region }`
2. `/health` responds in <500ms — `curl -w '%{time_total}' /health` <0.5
3. `/health/deep` checks all dependencies — response `checks` object has keys for every bound resource (D1, KV, R2)
4. 503 returned when any dependency is down — kill a dependency, confirm status flips to `degraded` and HTTP 503
5. Cron self-check fires every 5 minutes — check KV uptime counter increments every 5 min
6. Alert email sent on failure — simulate failure, confirm Resend email arrives within 5 minutes
7. External monitor configured — UptimeRobot or Better Stack has an active monitor for `/health`
8. Status page renders correctly — Playwright screenshot of `/status` shows current health, no errors