skill: api-connector-builder

should_trigger:
  - prompt: "Write a connector for the Linear GraphQL API that pulls every issue, not just the first page."
    why: "Core task: a typed client against a third-party GraphQL API with Relay cursor connections, walked to exhaustion. The skill's flagship deliverable."
  - prompt: "This vendor API keeps returning 429. Add backoff and respect their rate limit so we stop getting throttled."
    why: "Rate-limit + retry symptom. Honor Retry-After / X-RateLimit-* with jittered backoff is squarely this skill."
  - prompt: "We only ever get the first 100 results back from this API. Paginate through everything."
    why: "Pagination-to-exhaustion. The fixed-first-page bug is the canonical thing this skill fixes."
  - prompt: "Our access token expires halfway through a long run and the whole job dies. Add refresh."
    why: "Non-obvious phrasing — never says 'OAuth' or 'connector', but mid-run token expiry is the auth/refresh concern the skill owns (refresh rotation)."
  - prompt: "Conector para una API de terceros que nos banea la key cuando vamos demasiado rápido."
    why: "Spanish trigger combining rate-limit + auth (banned key from hammering). Vendor-agnostic outbound client = this skill."
  - prompt: "Wrap this vendor REST API as a typed SDK with retries and per-request timeouts."
    why: "Typed-client framing. 'Wrap a REST API as an SDK' with resilience is the connector-module deliverable."
  - prompt: "Build me a client for the Shopify Admin API that reads the cursor-based pages and handles the leaky-bucket rate limit."
    why: "Vendor with no dedicated skill, cursor pagination + bucket rate limit — the generic connector case this skill covers."

should_not_trigger:
  - prompt: "Verify the incoming webhook signature from Stripe on our /hooks endpoint and reject replays."
    route_to: webhooks
    why: "Inbound callback — receiving and verifying events the vendor POSTs to us, the inverse of calling their API."
  - prompt: "Design the versioning, resource model, and error envelope for OUR new public API."
    route_to: api-design
    why: "Authoring a contract others will consume, not consuming someone else's. Opposite side of the boundary."
  - prompt: "Chain Gmail to Sheets to Slack into one automated if-this-then-that workflow."
    route_to: automation-flows
    why: "Multi-service orchestration across many tools, not building one connector for one vendor."
  - prompt: "Scrape the product prices off this storefront — it has no public API at all."
    route_to: data-scraper
    why: "HTML/DOM scraping with no documented API; this skill requires a real API contract to obey."
  - prompt: "Pull the structured fields out of this LLM's free-text answer into JSON."
    route_to: structured-extraction
    why: "Extracting structure from a model response, not making HTTP calls to a third-party API."

capability:
  - scenario: "Build a Python connector for a paginated REST API (cursor-based) that requires a Bearer token and rate-limits at 60 req/min with X-RateLimit-* headers. It must fetch all records."
    must_include:
      - "Reads the Bearer token from an environment variable / secret store — never hardcoded in source."
      - "Sets a per-request timeout on every call (no unbounded socket wait)."
      - "Retries only transient statuses (429, 502, 503, 504) and network errors; never retries other 4xx."
      - "Uses exponential backoff WITH jitter and explicit stop conditions (max attempts 3–5 and/or a total deadline)."
      - "Honors Retry-After when present, otherwise computes the wait from X-RateLimit-Reset; throttles proactively on X-RateLimit-Remaining."
      - "Paginates by following the cursor until exhausted (no fixed page-count loop); ideally exposes a generator/iterator."
      - "Logs request id / status / attempt but never the raw token or Authorization header."
      - "Sends the token in the Authorization header, not a query string."
      - "Returns or streams the FULL record set across all pages, not just the first page."

# Evals — api-connector-builder

These cases are a hand-run rubric, not an automated test harness; no scoring
script ships. To run them: paste a `should_trigger` prompt into a fresh agent
session and confirm `api-connector-builder` is the skill that fires (and not a
sibling). Paste a `should_not_trigger` prompt and confirm the agent routes to the
named sibling instead. For the `capability` scenario, have the agent build the
connector, then grade the result against the `must_include` list — every bullet
should be satisfied by the code it produces. A connector that hardcodes the token,
retries all 4xx, or stops after one page fails the rubric regardless of how clean
it looks.

# Auth flows — per-flow walkthroughs

OAuth 2.1 baseline (oauth.net/2.1, accessed 2026-06-02):

- **PKCE is mandatory** for every authorization-code client, public and
  confidential alike.
- **Implicit grant and Resource-Owner-Password grant are removed.** If a vendor's
  docs still show them, do not use them — pick Auth-Code + PKCE or Client
  Credentials instead.
- **Bearer tokens must not travel in query strings.** Header only. Query strings
  leak into access logs, browser history, and `Referer` headers.

## API key in header

Simplest scheme. The vendor issues a static key; you send it on every request.
Store it in env, never source. There is no refresh — rotate manually when it
leaks or on a schedule.

```python
import os, httpx
KEY = os.environ["VENDOR_API_KEY"]
client = httpx.Client(base_url="https://api.vendor.com/v1",
                      headers={"X-API-Key": KEY})  # header name per vendor docs
```

Some vendors want `Authorization: Bearer <key>` instead of a custom header — read
the docs; do not guess the header name.

## Bearer static token (personal access token)

Treat exactly like an API key: long-lived, env-stored, no refresh. The only
difference is the standard `Authorization: Bearer <token>` header.

```typescript
const TOKEN = process.env.VENDOR_PAT!;
const headers = { Authorization: `Bearer ${TOKEN}` };
```

## OAuth2 Client Credentials (machine-to-machine)

No end user is involved — your service authenticates as itself. You hold a
`client_id` + `client_secret`, exchange them for a short-lived access token at the
token endpoint, cache it until it expires, and re-mint on expiry or a 401.

```python
import os, time, httpx

_cache = {"token": None, "exp": 0.0}

def access_token():
    if _cache["token"] and time.time() < _cache["exp"] - 30:  # 30s safety margin
        return _cache["token"]
    r = httpx.post("https://auth.vendor.com/oauth/token", data={
        "grant_type": "client_credentials",
        "client_id": os.environ["VENDOR_CLIENT_ID"],
        "client_secret": os.environ["VENDOR_CLIENT_SECRET"],
        "scope": "read:items",
    }, timeout=10.0)
    r.raise_for_status()
    body = r.json()
    _cache.update(token=body["access_token"], exp=time.time() + body["expires_in"])
    return _cache["token"]
```

## OAuth2 Authorization Code + PKCE (acting for a user)

The user-facing flow. PKCE prevents an intercepted authorization code from being
redeemed by an attacker.

1. Generate a `code_verifier` (43–128 chars, random) and its
   `code_challenge = base64url(sha256(verifier))`.
2. Redirect the user to the authorize endpoint with
   `response_type=code`, `code_challenge`, `code_challenge_method=S256`,
   `redirect_uri`, `scope`, and a random `state`.
3. On callback, verify `state`, then POST `grant_type=authorization_code` with
   the `code` **and the original `code_verifier`** to the token endpoint.
4. Store the resulting refresh token securely; use the access token for calls.

```python
import base64, hashlib, os
verifier = base64.urlsafe_b64encode(os.urandom(64)).rstrip(b"=").decode()
challenge = base64.urlsafe_b64encode(
    hashlib.sha256(verifier.encode()).digest()).rstrip(b"=").decode()
# -> send `challenge` (S256) on /authorize, keep `verifier` for the token exchange
```

### Refresh-token rotation (RFC 9700 / OAuth 2.0 Security BCP)

Refresh tokens for public clients must be **sender-constrained or one-time-use**:
every refresh returns a *new* refresh token and invalidates the old one. If the
old token is ever replayed, the server detects the reuse and revokes the whole
chain (a sign of theft). Persist the latest refresh token atomically — a crash
between "got new token" and "saved it" locks you out.

```python
def refresh(refresh_token: str) -> dict:
    r = httpx.post("https://auth.vendor.com/oauth/token", data={
        "grant_type": "refresh_token",
        "refresh_token": refresh_token,
        "client_id": os.environ["VENDOR_CLIENT_ID"],
    }, timeout=10.0)
    r.raise_for_status()
    body = r.json()
    save_refresh_token(body["refresh_token"])  # ROTATE: store the new one, drop old
    return body
```

Keep access tokens short-lived (minutes). Never log a raw token — log a partial
(`tok…3f9c`) or a hash if you must correlate across services.

## Device flow (input-constrained devices)

For CLIs, TVs, and devices with no browser. Request a device + user code, show the
user a URL and code to enter on another device, then **poll** the token endpoint
until they approve (respect the `interval`; back off on `slow_down`). Once
granted, you receive access + refresh tokens and rotate them exactly like the
auth-code flow above.

## DPoP (sender-constraining, brief)

DPoP (Demonstrating Proof-of-Possession) binds a token to a client-held key: each
request carries a signed `DPoP` header proving you hold the private key, so a
stolen bearer token alone is useless. If a vendor offers DPoP-bound access tokens,
prefer them for high-value scopes — it is the practical way to satisfy the
"sender-constrained" requirement without mTLS infrastructure.

## Storage rules (all flows)

- Secrets in env vars or a secret manager — never in source, never in
  `localStorage` (XSS-readable) for anything long-lived.
- One credential set per vendor per environment; never share a prod key into dev.
- Rotate on any suspected leak; the flows above make rotation cheap by design.

# Pagination — full code per style

The universal rule: **loop until the API signals no next page.** Never a fixed
page count. Expose results as a generator / async-iterator so callers stream
instead of buffering the whole set in memory.

(graphql.org/learn/pagination + REST pagination pattern guides, accessed
2026-06-02.)

## Offset / page number

Simple, but it **drifts** when rows are inserted or deleted mid-walk (you skip or
double-read) and gets **slow at large offsets** (the server scans and discards).
Prefer cursor/keyset when the API offers it.

```python
def iter_offset(client, path, page_size=100):
    offset = 0
    while True:
        body = client.get(path, params={"offset": offset, "limit": page_size}).json()
        rows = body["data"]
        if not rows:                  # empty page = exhausted
            return
        yield from rows
        offset += len(rows)
```

```typescript
export async function* iterOffset(get: (q: Record<string, number>) => Promise<any>, pageSize = 100) {
  let offset = 0;
  for (;;) {
    const body = await get({ offset, limit: pageSize });
    const rows = body.data as unknown[];
    if (rows.length === 0) return;     // exhausted
    yield* rows;
    offset += rows.length;
  }
}
```

## Cursor / keyset

The server returns an opaque cursor for the next page; you pass it back. **Stable
under concurrent writes and fixed cost regardless of depth** — the strategy to
prefer.

```python
def iter_cursor(client, path, page_size=100):
    cursor = None
    while True:
        body = client.get(path, params={"cursor": cursor, "limit": page_size}).json()
        yield from body["data"]
        cursor = body.get("next_cursor")
        if not cursor:                # cursor exhausted
            return
```

## Link header (REST)

The next page lives in the `Link` response header with `rel="next"`. Follow it
until the header has no `next`.

```python
import re

def iter_link(client, url):
    while url:
        resp = client.get(url)
        yield from resp.json()
        link = resp.headers.get("Link", "")
        m = re.search(r'<([^>]+)>;\s*rel="next"', link)
        url = m.group(1) if m else None   # no rel="next" = exhausted
```

```typescript
function nextLink(header: string | null): string | null {
  if (!header) return null;
  const m = header.match(/<([^>]+)>;\s*rel="next"/);
  return m ? m[1] : null;
}

export async function* iterLink(start: string, get: (u: string) => Promise<Response>) {
  let url: string | null = start;
  while (url) {
    const res = await get(url);
    yield* (await res.json()) as unknown[];
    url = nextLink(res.headers.get("Link"));
  }
}
```

## GraphQL — Relay Cursor Connections

The Relay spec returns `edges` (each with a `node` and `cursor`) and a `pageInfo`
with `endCursor` and `hasNextPage`. Pass `endCursor` as the `after` argument of
the next query and stop when `hasNextPage` is false.

```python
QUERY = """
query($after: String) {
  issues(first: 100, after: $after) {
    edges { node { id title } }
    pageInfo { endCursor hasNextPage }
  }
}
"""

def iter_relay(client, url):
    after = None
    while True:
        body = client.post(url, json={"query": QUERY, "variables": {"after": after}}).json()
        conn = body["data"]["issues"]
        for edge in conn["edges"]:
            yield edge["node"]
        info = conn["pageInfo"]
        if not info["hasNextPage"]:   # exhausted
            return
        after = info["endCursor"]
```

## Dedup on overlap

With offset pagination under concurrent inserts, the same row can appear on two
pages. If exactness matters, track seen ids and skip duplicates:

```python
def dedup(records, key="id"):
    seen = set()
    for r in records:
        k = r[key]
        if k in seen:
            continue
        seen.add(k)
        yield r
```

Cursor/keyset pagination does not have this problem for stable sort keys — another
reason to prefer it.

## Combine with retries + rate limits

Each `client.get(...)` above should go through the retry + rate-limit wrapper from
`SKILL.md` (transient-only retry with jittered backoff, `Retry-After` honored).
Pagination is the loop; the per-request call inside it is where resilience lives.

#!/usr/bin/env bash
#
# verify.sh — connector-module structure/banlist linter for `api-connector-builder`.
#
# WHAT IT DOES (read-only; never edits a file)
#   Lints the connector source the skill produces against the four-pillar shape:
#   secrets, retries, pagination, timeouts. It checks the artifact's STRUCTURE,
#   not its runtime behavior — appropriate for a connector emitted into an
#   arbitrary target repo. It never false-fails: a repo with no connector-like
#   source is simply skipped.
#
#   A connector file = a .py / .ts / .js / .mjs / .tsx file that looks like an
#   HTTP client (mentions a base URL, fetch/httpx/requests/undici/axios, or an
#   Authorization/Bearer header). Other files are ignored.
#
#   Checks per connector file:
#     1. FAIL  hardcoded secret  — an inline API key / bearer literal / long
#              token-looking string assigned in source (not read from env).
#     2. FAIL  no retry wired    — no tenacity import, no backoff loop, and no
#              undici/axios retry interceptor anywhere in the file.
#     3. FAIL  no pagination loop — uses pagination-style params (cursor/offset/
#              page/after/Link) but has no loop (while/for/async generator) to
#              walk them: a lone "get first page" that ignores the next cursor.
#     4. FAIL  no request timeout — issues HTTP calls but sets no timeout /
#              AbortSignal.timeout on them.
#     5. FAIL  token in localStorage  — long-lived token stored in localStorage.
#     6. WARN  token logging      — a log/console call that emits the token,
#              Authorization header, or Bearer value.
#
# HOW TO RUN (inside YOUR project, not the skills repo)
#   ./verify.sh                 # scan ./ for connector source
#   ./verify.sh --path src      # scan a subdirectory
#   ./verify.sh --strict        # treat any warning as a failure (exit 1)
#
# EXIT CODES
#   0  clean, or warnings only without --strict (also: nothing to check)
#   1  a structural failure, or --strict with a warning
#   2  bad usage
#
# Runs on stock macOS bash 3.2 — no mapfile, no associative arrays.

set -euo pipefail

if [ -t 1 ]; then
  RED=$'\033[31m'; GREEN=$'\033[32m'; YELLOW=$'\033[33m'; NC=$'\033[0m'
else
  RED=''; GREEN=''; YELLOW=''; NC=''
fi

ok_count=0; skip_count=0; warn_count=0; fail_count=0
ok()   { printf '%s[ ok ]%s %s\n' "$GREEN"  "$NC" "$*"; ok_count=$((ok_count + 1)); }
skip() { printf '%s[skip]%s %s\n' "$YELLOW" "$NC" "$*"; skip_count=$((skip_count + 1)); }
warn() { printf '%s[warn]%s %s\n' "$YELLOW" "$NC" "$*"; warn_count=$((warn_count + 1)); }
fail() { printf '%s[fail]%s %s\n' "$RED"    "$NC" "$*"; fail_count=$((fail_count + 1)); }

usage() { sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//'; }

SCAN_PATH="."
STRICT=0
while [ $# -gt 0 ]; do
  case "$1" in
    --path)    SCAN_PATH="${2:?--path needs a value}"; shift 2 ;;
    --strict)  STRICT=1; shift ;;
    -h|--help) usage; exit 0 ;;
    *) printf '%sUnknown argument: %s%s\n\n' "$RED" "$1" "$NC"; usage; exit 2 ;;
  esac
done

if [ ! -e "$SCAN_PATH" ]; then
  printf '%sPath not found: %s%s\n' "$RED" "$SCAN_PATH" "$NC"; exit 2
fi

TMPDIR_V="$(mktemp -d 2>/dev/null || printf '/tmp/apiconn-verify.%s' "$$")"
mkdir -p "$TMPDIR_V" 2>/dev/null || true
cleanup() { rm -rf "$TMPDIR_V" 2>/dev/null || true; }
trap cleanup EXIT

ALL="$TMPDIR_V/all"
FILES="$TMPDIR_V/files"

# Candidate source files (skip node_modules / vendored / build dirs).
find "$SCAN_PATH" -type f \
  \( -name '*.py' -o -name '*.ts' -o -name '*.tsx' -o -name '*.js' -o -name '*.mjs' \) \
  2>/dev/null \
  | grep -Ev '/(node_modules|\.git|dist|build|\.venv|venv|__pycache__)/' \
  > "$ALL" || true

# Keep only files that look like an HTTP client (a connector).
: > "$FILES"
while IFS= read -r f; do
  [ -z "$f" ] && continue
  if grep -Eiq 'https?://|\bfetch\(|\bhttpx\b|\brequests\.|\bundici\b|\baxios\b|Authorization|Bearer ' "$f" 2>/dev/null; then
    printf '%s\n' "$f" >> "$FILES"
  fi
done < "$ALL"

if [ ! -s "$FILES" ]; then
  skip "no connector-like source (.py/.ts/.js with an HTTP client) under: $SCAN_PATH"
  printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count"
  exit 0
fi

# --- per-file checks ---------------------------------------------------------
while IFS= read -r f; do
  [ -z "$f" ] && continue
  file_ok=1

  # 1. Hardcoded secret: an inline literal assigned to a key/token/secret-looking
  #    name, OR a Bearer/sk- literal embedded in source. Reading from env is fine.
  if grep -Eni \
      '(api[_-]?key|secret|token|bearer|password|client[_-]?secret)[[:space:]]*[:=][[:space:]]*["'\''][A-Za-z0-9_\-]{16,}["'\'']' \
      "$f" 2>/dev/null | grep -Eiv 'process\.env|os\.environ|getenv|import\.meta\.env|<[^>]*>|YOUR_|EXAMPLE|xxxx|\.\.\.' >/dev/null; then
    fail "hardcoded secret literal: $f"
    file_ok=0
  fi
  if grep -Eni '["'\''](sk_live_|sk_test_|ghp_|xox[baprs]-|AKIA)[A-Za-z0-9_\-]{8,}' "$f" 2>/dev/null >/dev/null; then
    fail "hardcoded provider token literal: $f"
    file_ok=0
  fi

  # 2. Retry mechanism wired somewhere in the file.
  if grep -Eiq 'tenacity|@retry|wait_exponential|backoff|retry_if|maxRetries|max_retries|interceptors\.|retry[_-]?interceptor|exponential' "$f" 2>/dev/null; then
    : # has a retry primitive
  elif grep -Eiq 'for[[:space:]].*attempt|while[[:space:]].*attempt|range\([[:space:]]*[0-9]|attempt[[:space:]]*[+<]' "$f" 2>/dev/null \
       && grep -Eiq 'sleep|setTimeout|delay|backoff' "$f" 2>/dev/null; then
    : # has a hand-rolled retry loop with a delay
  else
    fail "no retry mechanism wired (no tenacity/backoff/interceptor or retry loop): $f"
    file_ok=0
  fi

  # 3. Pagination loop, only required when the file uses pagination params.
  if grep -Eiq 'cursor|next[_-]?cursor|endCursor|hasNextPage|pageInfo|offset|\bpage\b|\bafter\b|rel="next"|Link' "$f" 2>/dev/null; then
    if grep -Eiq 'while[[:space:]]|for[[:space:]]|yield|async[[:space:]]*function\*|def[[:space:]]+iter|->[[:space:]]*Generator|AsyncIterator' "$f" 2>/dev/null; then
      : # walks the pages
    else
      fail "pagination params used but no loop to walk them (first-page-only): $f"
      file_ok=0
    fi
  fi

  # 4. Request timeout set on the HTTP calls.
  if grep -Eiq 'timeout|AbortSignal\.timeout|AbortController|signal[[:space:]]*[:=]|connect_timeout|read_timeout' "$f" 2>/dev/null; then
    : # a timeout is configured
  else
    fail "no request timeout set (a hung socket will stall the run): $f"
    file_ok=0
  fi

  # 5. Token in localStorage (long-lived secret in an XSS-readable store).
  if grep -Eiq 'localStorage\.(setItem|getItem)?[[:space:]]*\(?[^)]*\b(token|bearer|access[_-]?token|refresh[_-]?token|api[_-]?key)\b' "$f" 2>/dev/null \
     || grep -Eiq 'localStorage[^;]*\b(token|access_token|refresh_token|apiKey)\b' "$f" 2>/dev/null; then
    fail "token stored in localStorage (XSS-readable): $f"
    file_ok=0
  fi

  # 6. WARN: logging the token / Authorization header.
  if grep -Eni '(console\.(log|info|debug|warn|error)|logg?(er)?\.|print|logging\.)[^;\n]*\b(token|bearer|authorization)\b' "$f" 2>/dev/null \
     | grep -Eiv 'x-request-id|status|attempt|request[_-]?id|//|#' >/dev/null; then
    warn "possible token/Authorization logging: $f"
  fi

  if [ "$file_ok" -eq 1 ]; then
    ok "connector shape ok: $f"
  fi
done < "$FILES"

printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count"

if [ "$fail_count" -gt 0 ]; then exit 1; fi
if [ "$STRICT" -eq 1 ] && [ "$warn_count" -gt 0 ]; then exit 1; fi
exit 0