Claude Cursor opencode Skill

ultimate-browsing

Escalation skill for blocked or hard-to-reach web access — load it when a normal browse/fetch is blocked (WAF, 403, Cloudflare, JS-only render, login-gated, or a platform a generic fetcher cannot read). Tiered router: TIER 1 insane-search (headless extraction + WAF bypass via cur

LLM Mart · 0 points · 23 views 0 listing impressions 0 install-command copies
Download code-yeongyu-oh-my-openagent-packages_shared-skills_skills_ultimate-browsing-05dcba6.zip · 190 KB
This skill couldn't be refreshed from GitHub on the last check — you're seeing the last imported snapshot.

Install

skills CLI npx skills add https://github.com/code-yeongyu/oh-my-openagent/tree/dev/packages/shared-skills/skills/ultimate-browsing
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install code-yeongyu-oh-my-openagent@llmmart
Git git clone https://github.com/code-yeongyu/oh-my-openagent.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole code-yeongyu/oh-my-openagent collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Ultimate Browsing

Web access for everything a plain fetch cannot finish: a page that renders in JS, a click or a form, a screenshot, a login that must persist across pages, or a host that blocks generic fetchers (WAF / 403 / Cloudflare). Start at the cheapest tier that can do the job and climb only when it cannot:

Tier 1 — insane-search (headless extraction + WAF bypass) -> Tier 1.5 — agent-reach (platform-native APIs, esp. Chinese platforms) -> Tier 2 — a real browser through omowright from js eval: 2a the owned engine (a browser your code launches, CloakBrowser for stealth), 2b the attached engine (the user's own signed-in browser).

PHASE 0 — ROUTE FIRST (MANDATORY)

User request
  |
  +- extract text/data from a URL --------------------- TIER 1  insane-search
  +- URL blocked / 403 / Cloudflare / WAF ------------- TIER 1  insane-search
  +- YouTube/Vimeo/TikTok subtitles or metadata ------- TIER 1  insane-search (yt-dlp)
  +- read an article / blog / Reddit / HN / arXiv ----- TIER 1  insane-search
  |
  +- Chinese platform (xhs/douyin/weibo/bilibili/v2ex/wechat)  TIER 1.5 agent-reach
  +- podcast transcript / stock forum ----------------- TIER 1.5 agent-reach
  +- Twitter feed / LinkedIn profile / GitHub via CLI - TIER 1.5 agent-reach
  |
  +- Tier 1/1.5 returned empty or partial ------------- TIER 2  2a owned engine -> 2b attached engine
  +- click / fill form / scroll / interact ------------ TIER 2  2a owned engine -> 2b attached engine
  +- screenshot / render / play video ----------------- TIER 2  2a owned engine -> 2b attached engine
  +- login session across pages / the user's account --- TIER 2  2b attached engine (their browser)
  +- test web app / QA / dogfood ---------------------- TIER 2  2a owned engine -> 2b attached engine
  |
  +- simple search query ------------------------------ NOT this skill (use web-search)

Read the matching reference before acting: references/insane-search/README.md, references/agent-reach/README.md, or references/chrome-stealth.md.

Tier 1 — insane-search (headless extraction)

When: content extraction, blocked-URL bypass, media metadata — no browser UI needed. Why first: ~10x faster than a browser, no process spin-up; handles most "fetch this blocked page" requests via curl_cffi TLS impersonation, yt-dlp (1858 sites), official public APIs, mobile URL transforms, Phase-2.5 surrogate archives (Wayback / archive.today snapshots, provenance-tagged — see references/insane-search/cache-archive.md), a key-gated Jina Reader (JINA_API_KEY), and a Playwright real-Chrome fallback. The engine lives inside this skill at engine/ and is invoked as a module. Surrogate results are dated COPIES: a result whose provenance is snapshot must be reported with its snapshot_timestamp, never presented as the live page.

# Core command — auto-detects WAF, runs the full fetch grid (run from the skill dir):
python3 -m engine "https://example.com/blocked-page"
#   add --selector "<CSS>" for positive-proof validation, --device auto|desktop|mobile,
#   --trace to inspect every attempt, --json for machine-readable output.

# YouTube subtitles / metadata (no browser):
yt-dlp --write-sub --write-auto-sub --sub-lang "en,ko" --skip-download -o "/tmp/%(id)s" "<URL>"

# Reddit / HN / Bluesky / arXiv etc. use official public endpoints — see the Phase 0 index in
# references/insane-search/README.md (Twitter syndication, Reddit .json, HN Firebase, ...).

The full engine harness (rules R1-R7, the Phase 0 official-API index, the no-site-name rule, and the references/insane-search/*.md deep-dives for TLS, Playwright routing, Naver, media, etc.) is in references/insane-search/README.md. Read it before tuning the engine or adding a WAF profile.

Escalate to Tier 1.5 or Tier 2 when

  • The target is a Chinese / social platform with a native reader -> Tier 1.5.
  • insane-search returns empty/partial, or the page needs JS interaction, a screenshot, a persistent login, or media playback -> Tier 2.

Tier 1.5 — agent-reach (platform-native readers)

When: the target is a platform with a first-class API/CLI that beats generic fetching — especially Chinese platforms that stealth browsers still cannot reach cleanly. Several channels are zero-config (Douyin, V2EX, Reddit, RSS, YouTube); others need a one-time auth you supply via environment variables if you have access (JINA_API_KEY for Jina Reader — anonymous access is dead, see references/insane-search/jina.md; TWITTER_* for X; a transcription key for podcasts).

Category Platforms Entry
social xhs (Xiaohongshu), douyin, weibo, bilibili, V2EX, Reddit, Twitter/X references/agent-reach/social.md
web Jina Reader, WeChat articles, RSS references/agent-reach/web.md
video YouTube, Bilibili, podcast transcripts, Douyin video references/agent-reach/video.md
career LinkedIn references/agent-reach/career.md
dev GitHub (gh CLI) references/agent-reach/dev.md
search Exa AI references/agent-reach/search.md
mcporter call 'douyin.parse_douyin_video_info(url: "<URL>")'   # douyin, zero-config
curl -s "https://r.jina.ai/https://weibo.com/<uid>/<pid>"      # weibo via Jina
yt-dlp --dump-json "<bilibili-url>"                            # Bilibili (overseas: add --cookies-from-browser)
curl -s "https://www.v2ex.com/api/topics/hot.json"            # V2EX public API

Routing table, per-platform auth (set TWITTER_* env vars, gh auth login, a transcription key — only if you have access), rate-limit notes, and known version quirks are in references/agent-reach/README.md.

Tier 2 — a real browser (real interaction)

When: real interaction is needed (clicks, forms, screenshots, video, persistent login), or Tier 1/1.5 failed.

Both tiers are omowright, staged inside the browser skill and loaded from js eval:

const { loadOmowright } = await import("<browser-skill-root>/scripts/omowright.mjs")
const { omowright } = await loadOmowright()

Tier 2a — owned engine (default)

A browser your code launches with a task-owned profile. connectPipe opens no listening port; connectCloakProfile launches CloakBrowser with a pinned fingerprint seed and is the path for WAF, Cloudflare and bot-scored pages.

const browser = await omowright.connectPipe({ browserPath, browserArgs: ["--headless", `--user-data-dir=${profile}`], storageRoot: profile })
try {
  const page = await browser.newTab(url)
  const tree = omowright.compactSnapshot(await page.snapshot())   // the read; refs come from it
  const snoop = omowright.createNetworkSnoop(page)                  // read the API JSON instead of the DOM when there is one
  await page.locator("e3").click()
  await Bun.write(pngPath, await page.screenshot())
} finally {
  await browser.close()                                            // then rm -rf the profile
}

The rest of the surface (CUA coordinates, captcha solving, routes, traces, frames, human handoff) is in the browser skill's references/owned-engine/. A stealth binary is not proof of access: inspect the rendered result and report challenges that remain.

Tier 2b — attached engine (logged-in pages)

When the page needs the user's account, drive the browser they are already signed into instead of cloning their profile: connectBrowserSkill() → session.navigate → bskSnapshot(session) / session.observe() → session.click → session.stop(). NEVER launch against or clear cookies/cache/site data from the user's live profile, and never fall back to the owned engine for an authenticated criterion: if no extension is connected, run the browser skill's onboarding script and relay its one human step. The full loop is the browser skill.

Cookie login (cross-platform)

scripts/extract_cookies.py reads cookies from a local Chromium-family or Firefox-family browser and optionally injects them into the running CDP session. It resolves browser profile paths and decrypts cookie values per-OS (macOS Keychain, Linux libsecret, Windows DPAPI):

# Extract cookies to a file:
mkdir -p ~/.local/state/omo-cookies
python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --output ~/.local/state/omo-cookies/youtube.cookies.json
# Extract and inject into the running CDP session:
python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --inject --cdp 9242

Cookie export files are written with owner-only 0600 permissions. Do not place live auth cookies in shared temp directories or commit them to a repo. Cookie injection sends values to CDP over stdin rather than argv. Cookies apply on next navigation — reload after injecting. Google services use fingerprint-bound tokens that may not transfer across browser profiles. Limits in references/chrome-stealth.md.

Reference docs

File When to read
references/insane-search/README.md Tier-1 engine harness (R1-R7, Phase 0 API index, no-site-name rule) + its *.md deep-dives
references/agent-reach/README.md Tier-1.5 routing table, platform auth, per-category *.md
references/chrome-stealth.md Tier-2 stealth through omowright + CloakBrowser, cookie login limits

Environment variables

# agent-reach auth: set the channel-specific env vars from each tool's docs only if you have access
# insane-search needs no env vars — it auto-installs deps on first run

Anti-patterns

  • Do NOT launch Chrome stealth for plain text extraction — use Tier 1.
  • Use stealth plugins only in an explicitly installed script environment, not injected into WebView.
  • Close every WebView/browser context when done and remove only task-owned profile clones.
  • Do NOT inject cookies without reloading the page.
  • Do NOT hardcode site domains/selectors into engine/** or waf_profiles.yaml — runtime hints only (see the no-site-name rule in the insane-search reference).
Files (oh-my-openagent)
  • engine
    • templates
      • package.json 349 B
        Loading…
      • playwright_mobile_chrome.js 3.6 KB
        Loading…
      • playwright_real_chrome.js 5.6 KB
        Loading…
    • tests
      • fixtures
        • amp_redirect_stub.html 323 B · in bundle
        • search_interstitial.html 90.4 KB · in bundle
        • wayback_available.json 246 B
          Loading…
        • wayback_snapshot.html 181.4 KB · in bundle
      • test_fetch_chain.py 2.7 KB
        Loading…
      • test_playwright_stealth.py 3.9 KB
        Loading…
      • test_playwright_templates.py 10.7 KB
        Loading…
      • test_surrogate.py 11.3 KB
        Loading…
      • test_surrogate_validators.py 3.5 KB
        Loading…
    • AGENTS.md 10.4 KB
      Loading…
    • bias_check.py 7.1 KB
      Loading…
    • curl_probe.py 2.4 KB
      Loading…
    • executor.py 6.3 KB
      Loading…
    • fetch_chain.py 13 KB
      Loading…
    • referers.py 327 B
      Loading…
    • result_schema.py 1.5 KB
      Loading…
    • summary.py 1.2 KB
      Loading…
    • surrogate.py 7.9 KB
      Loading…
    • surrogates.yaml 2.3 KB
      Loading…
    • url_transforms.py 3.2 KB
      Loading…
    • validators.py 9.1 KB
      Loading…
    • waf_detector.py 6.9 KB
      Loading…
    • waf_profiles.yaml 6.2 KB
      Loading…
    • __init__.py 668 B
      Loading…
    • __main__.py 4.8 KB
      Loading…
  • references
    • agent-reach
      • career.md 712 B
        Loading…
      • dev.md 1.3 KB
        Loading…
      • README.md 2.2 KB
        Loading…
      • search.md 821 B
        Loading…
      • social.md 5.6 KB
        Loading…
      • video.md 2.7 KB
        Loading…
      • web.md 1.9 KB
        Loading…
    • insane-search
      • cache-archive.md 4.1 KB
        Loading…
      • fallback.md 6 KB
        Loading…
      • jina.md 3.8 KB
        Loading…
      • json-api.md 3.5 KB
        Loading…
      • media.md 3.5 KB
        Loading…
      • metadata.md 2.9 KB
        Loading…
      • naver.md 3.3 KB
        Loading…
      • playwright.md 6.7 KB
        Loading…
      • public-api.md 3.4 KB
        Loading…
      • README.md 19.5 KB
        Loading…
      • rss.md 2.9 KB
        Loading…
      • tls-impersonate.md 6.3 KB
        Loading…
      • twitter.md 3.5 KB
        Loading…
    • chrome-stealth.md 2.2 KB
      Loading…
  • scripts
    • tests
      • test_cookie_domain_filter.py 3.5 KB
        Loading…
      • test_extract_cookies.py 9.4 KB
        Loading…
    • cookie_crypto.py 2.9 KB
      Loading…
    • cookie_domains.py 1.1 KB
      Loading…
    • cookie_paths.py 3.8 KB
      Loading…
    • extract_cookies.py 9.3 KB
      Loading…
  • .gitignore 29 B · in bundle
  • ATTRIBUTION.md 2.5 KB
    Loading…
  • SKILL.md 10.7 KB
    ---
    name: ultimate-browsing
    description: "Renders, drives, and screenshots web pages: JS-rendered sources, clicks and forms, persistent logins, WAF-blocked hosts (platform-native readers, stealth Chrome), and the browsing lane of a research run, with screenshots as provenance. Not for plain search or unblocked static fetch."
    ---
    
    # Ultimate Browsing
    
    Web access for everything a plain fetch cannot finish: a page that renders in JS, a click or a form, a screenshot, a login that must persist across pages, or a host that blocks generic fetchers (WAF / 403 / Cloudflare). Start at the cheapest tier that can do the job and climb only when it cannot:
    
    **Tier 1 — insane-search** (headless extraction + WAF bypass) -> **Tier 1.5 — agent-reach** (platform-native APIs, esp. Chinese platforms) -> **Tier 2 — a real browser** through omowright from js eval: 2a the owned engine (a browser your code launches, CloakBrowser for stealth), 2b the attached engine (the user's own signed-in browser).
    
    ## PHASE 0 — ROUTE FIRST (MANDATORY)
    
    ```
    User request
      |
      +- extract text/data from a URL --------------------- TIER 1  insane-search
      +- URL blocked / 403 / Cloudflare / WAF ------------- TIER 1  insane-search
      +- YouTube/Vimeo/TikTok subtitles or metadata ------- TIER 1  insane-search (yt-dlp)
      +- read an article / blog / Reddit / HN / arXiv ----- TIER 1  insane-search
      |
      +- Chinese platform (xhs/douyin/weibo/bilibili/v2ex/wechat)  TIER 1.5 agent-reach
      +- podcast transcript / stock forum ----------------- TIER 1.5 agent-reach
      +- Twitter feed / LinkedIn profile / GitHub via CLI - TIER 1.5 agent-reach
      |
      +- Tier 1/1.5 returned empty or partial ------------- TIER 2  2a owned engine -> 2b attached engine
      +- click / fill form / scroll / interact ------------ TIER 2  2a owned engine -> 2b attached engine
      +- screenshot / render / play video ----------------- TIER 2  2a owned engine -> 2b attached engine
      +- login session across pages / the user's account --- TIER 2  2b attached engine (their browser)
      +- test web app / QA / dogfood ---------------------- TIER 2  2a owned engine -> 2b attached engine
      |
      +- simple search query ------------------------------ NOT this skill (use web-search)
    ```
    
    Read the matching reference before acting: [`references/insane-search/README.md`](references/insane-search/README.md), [`references/agent-reach/README.md`](references/agent-reach/README.md), or [`references/chrome-stealth.md`](references/chrome-stealth.md).
    
    ## Tier 1 — insane-search (headless extraction)
    
    **When**: content extraction, blocked-URL bypass, media metadata — no browser UI needed.
    **Why first**: ~10x faster than a browser, no process spin-up; handles most "fetch this blocked page" requests via curl_cffi TLS impersonation, yt-dlp (1858 sites), official public APIs, mobile URL transforms, **Phase-2.5 surrogate archives** (Wayback / archive.today snapshots, provenance-tagged — see [`references/insane-search/cache-archive.md`](references/insane-search/cache-archive.md)), a key-gated Jina Reader (`JINA_API_KEY`), and a Playwright real-Chrome fallback. The engine lives **inside this skill** at `engine/` and is invoked as a module. Surrogate results are dated COPIES: a result whose `provenance` is `snapshot` must be reported with its `snapshot_timestamp`, never presented as the live page.
    
    ```bash
    # Core command — auto-detects WAF, runs the full fetch grid (run from the skill dir):
    python3 -m engine "https://example.com/blocked-page"
    #   add --selector "<CSS>" for positive-proof validation, --device auto|desktop|mobile,
    #   --trace to inspect every attempt, --json for machine-readable output.
    
    # YouTube subtitles / metadata (no browser):
    yt-dlp --write-sub --write-auto-sub --sub-lang "en,ko" --skip-download -o "/tmp/%(id)s" "<URL>"
    
    # Reddit / HN / Bluesky / arXiv etc. use official public endpoints — see the Phase 0 index in
    # references/insane-search/README.md (Twitter syndication, Reddit .json, HN Firebase, ...).
    ```
    
    The full engine harness (rules R1-R7, the Phase 0 official-API index, the no-site-name rule, and the `references/insane-search/*.md` deep-dives for TLS, Playwright routing, Naver, media, etc.) is in [`references/insane-search/README.md`](references/insane-search/README.md). Read it before tuning the engine or adding a WAF profile.
    
    ### Escalate to Tier 1.5 or Tier 2 when
    - The target is a Chinese / social platform with a native reader -> Tier 1.5.
    - insane-search returns empty/partial, or the page needs JS interaction, a screenshot, a persistent login, or media playback -> Tier 2.
    
    ## Tier 1.5 — agent-reach (platform-native readers)
    
    **When**: the target is a platform with a first-class API/CLI that beats generic fetching — especially Chinese platforms that stealth browsers still cannot reach cleanly. Several channels are zero-config (Douyin, V2EX, Reddit, RSS, YouTube); others need a one-time auth you supply via environment variables if you have access (`JINA_API_KEY` for Jina Reader — anonymous access is dead, see `references/insane-search/jina.md`; `TWITTER_*` for X; a transcription key for podcasts).
    
    | Category | Platforms | Entry |
    |---|---|---|
    | social | xhs (Xiaohongshu), douyin, weibo, bilibili, V2EX, Reddit, Twitter/X | [references/agent-reach/social.md](references/agent-reach/social.md) |
    | web | Jina Reader, WeChat articles, RSS | [references/agent-reach/web.md](references/agent-reach/web.md) |
    | video | YouTube, Bilibili, podcast transcripts, Douyin video | [references/agent-reach/video.md](references/agent-reach/video.md) |
    | career | LinkedIn | [references/agent-reach/career.md](references/agent-reach/career.md) |
    | dev | GitHub (gh CLI) | [references/agent-reach/dev.md](references/agent-reach/dev.md) |
    | search | Exa AI | [references/agent-reach/search.md](references/agent-reach/search.md) |
    
    ```bash
    mcporter call 'douyin.parse_douyin_video_info(url: "<URL>")'   # douyin, zero-config
    curl -s "https://r.jina.ai/https://weibo.com/<uid>/<pid>"      # weibo via Jina
    yt-dlp --dump-json "<bilibili-url>"                            # Bilibili (overseas: add --cookies-from-browser)
    curl -s "https://www.v2ex.com/api/topics/hot.json"            # V2EX public API
    ```
    
    Routing table, per-platform auth (set `TWITTER_*` env vars, `gh auth login`, a transcription key — only if you have access), rate-limit notes, and known version quirks are in [references/agent-reach/README.md](references/agent-reach/README.md).
    
    ## Tier 2 — a real browser (real interaction)
    
    **When**: real interaction is needed (clicks, forms, screenshots, video, persistent login), or Tier 1/1.5 failed.
    
    Both tiers are omowright, staged inside the `browser` skill and loaded from js eval:
    
    ```js
    const { loadOmowright } = await import("<browser-skill-root>/scripts/omowright.mjs")
    const { omowright } = await loadOmowright()
    ```
    
    ### Tier 2a — owned engine (default)
    
    A browser your code launches with a task-owned profile. `connectPipe` opens no listening port; `connectCloakProfile` launches CloakBrowser with a pinned fingerprint seed and is the path for WAF, Cloudflare and bot-scored pages.
    
    ```js
    const browser = await omowright.connectPipe({ browserPath, browserArgs: ["--headless", `--user-data-dir=${profile}`], storageRoot: profile })
    try {
      const page = await browser.newTab(url)
      const tree = omowright.compactSnapshot(await page.snapshot())   // the read; refs come from it
      const snoop = omowright.createNetworkSnoop(page)                  // read the API JSON instead of the DOM when there is one
      await page.locator("e3").click()
      await Bun.write(pngPath, await page.screenshot())
    } finally {
      await browser.close()                                            // then rm -rf the profile
    }
    ```
    
    The rest of the surface (CUA coordinates, captcha solving, routes, traces, frames, human handoff) is in the `browser` skill's `references/owned-engine/`. A stealth binary is not proof of access: inspect the rendered result and report challenges that remain.
    
    ### Tier 2b — attached engine (logged-in pages)
    
    When the page needs the user's account, drive the browser they are already signed into instead of cloning their profile: `connectBrowserSkill()` → `session.navigate` → `bskSnapshot(session)` / `session.observe()` → `session.click` → `session.stop()`. NEVER launch against or clear cookies/cache/site data from the user's live profile, and never fall back to the owned engine for an authenticated criterion: if no extension is connected, run the `browser` skill's onboarding script and relay its one human step. The full loop is the `browser` skill.
    
    ### Cookie login (cross-platform)
    
    `scripts/extract_cookies.py` reads cookies from a local Chromium-family or Firefox-family browser and optionally injects them into the running CDP session. It resolves browser profile paths and decrypts cookie values per-OS (macOS Keychain, Linux libsecret, Windows DPAPI):
    
    ```bash
    # Extract cookies to a file:
    mkdir -p ~/.local/state/omo-cookies
    python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --output ~/.local/state/omo-cookies/youtube.cookies.json
    # Extract and inject into the running CDP session:
    python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --inject --cdp 9242
    ```
    
    Cookie export files are written with owner-only `0600` permissions. Do not place live auth cookies in shared temp directories or commit them to a repo. Cookie injection sends values to CDP over stdin rather than argv. Cookies apply on next navigation — reload after injecting. Google services use fingerprint-bound tokens that may not transfer across browser profiles. Limits in [references/chrome-stealth.md](references/chrome-stealth.md).
    
    ## Reference docs
    
    | File | When to read |
    |------|-------------|
    | [references/insane-search/README.md](references/insane-search/README.md) | Tier-1 engine harness (R1-R7, Phase 0 API index, no-site-name rule) + its `*.md` deep-dives |
    | [references/agent-reach/README.md](references/agent-reach/README.md) | Tier-1.5 routing table, platform auth, per-category `*.md` |
    | [references/chrome-stealth.md](references/chrome-stealth.md) | Tier-2 stealth through omowright + CloakBrowser, cookie login limits |
    
    ## Environment variables
    
    ```bash
    # agent-reach auth: set the channel-specific env vars from each tool's docs only if you have access
    # insane-search needs no env vars — it auto-installs deps on first run
    ```
    
    ## Anti-patterns
    
    - Do NOT launch Chrome stealth for plain text extraction — use Tier 1.
    - Use stealth plugins only in an explicitly installed script environment, not injected into WebView.
    - Close every WebView/browser context when done and remove only task-owned profile clones.
    - Do NOT inject cookies without reloading the page.
    - Do NOT hardcode site domains/selectors into `engine/**` or `waf_profiles.yaml` — runtime hints only (see the no-site-name rule in the insane-search reference).
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.