skill: cost-tracking

should_trigger:
  - prompt: "Wrap our OpenAI calls so we record token cost per request."
    why: Core capture — per-request token + cost capture around an SDK call is the skill's first step.
  - prompt: "Set a $500/month cap on the Claude API that hard-stops when we hit it."
    why: Budget plus a hard cap that refuses — the guard layer this skill owns.
  - prompt: "Our bill is 3x what we estimated — reconcile the billed usage against what we computed."
    why: Non-obvious — reconciliation of provider-billed vs ledgered usage is a direct tell for this skill, not generic debugging.
  - prompt: "Attribute our LLM spend per tenant so we know which customer is expensive."
    why: Ledger attribution — per-tenant cost rollup is exactly the attribution-key design here.
  - prompt: "Pon un tope de gasto a la API y avísame al 80%."
    why: Spanish — budget plus a soft alert threshold (80%), the alert/cap branch of the skill.
  - prompt: "Is prompt caching actually cheaper for our agent, and where's the break-even?"
    why: The levers section — caching break-even math is owned here, not asserted.
  - prompt: "A runaway agent loop burned through our token budget overnight with no guardrail."
    why: The surprise-bill scenario the hard cap exists to prevent; routes straight to budgets+caps.

should_not_trigger:
  - prompt: "What's our cash runway at this burn rate?"
    route_to: finance-ops
    why: Company-level cash, not the AI/cloud line item; cost-tracking is just one input feeding it.
  - prompt: "Compute the gross margin per subscription."
    route_to: unit-economics
    why: Margin math — cost-tracking only produces the cost numerator, it does not divide by units.
  - prompt: "Record this AWS invoice as an expense in the books."
    route_to: bookkeeping
    why: The accounting record, not the operational real-time meter.
  - prompt: "Decide what to charge customers for the pro plan."
    route_to: pricing
    why: Willingness-to-pay / what to charge, not controlling what we spend.
  - prompt: "Chart our monthly active users and retention curve."
    route_to: analytics
    why: Product/behavior telemetry, a different meter; the dollar capture is here but user-behavior charting is not.

capability:
  - scenario: "We have a Next.js app calling Claude + OpenAI with no cost visibility and a scary bill. Give us a metering + budget-cap design."
    must_include:
      - reads cost from the response `usage` object, not pre-send estimates (estimates only for the pre-flight cap check)
      - versioned pricing table where each model row carries effective_date + source, loaded as data not literals
      - loud failure on a model absent from the pricing table (never default to $0)
      - append-only ledger with an idempotency/request key and at least one attribution key (user/tenant/feature)
      - budget with BOTH a soft threshold (alert/degrade) and a hard threshold (refuse with a typed error)
      - the three levers (prompt caching with ~1-hit break-even, batch API 50% stacking with caching, model downshift)
      - notes cloud billing alerts (AWS CAD ~24h delay, GCP Pub/Sub) are a delayed backstop, not the real-time guard

# Evals for cost-tracking

These cases are eyeballed, not automated. Read `cases.yaml` and check three things by hand. First, every `should_trigger` prompt should make this skill the obvious pick — pay attention to the non-obvious ones (the "bill is 3x the estimate / reconcile billed vs response usage" line and the Spanish budget-cap line), since those are where a thinner trigger set would miss. Second, every `should_not_trigger` prompt should route cleanly to the named sibling (finance-ops, unit-economics, bookkeeping, pricing, analytics) — if any of those feels like it could legitimately land here, the description boundary needs tightening. Third, run the `capability` scenario against a generated answer and confirm each `must_include` rubric item is actually present, not hand-waved (especially: cost read from `usage`, dated+sourced pricing table, idempotency + attribution on the ledger, and a soft *and* hard cap). Note that `scripts/verify.sh` checks the emitted artifact (the pricing/ledger/budget config), not the prose — a great write-up that emits no checkable config has not finished the job.

# Cloud spend caps — recipes (accessed 2026-06-02)

App-level metering is the real-time guard. Everything here is the **slow backstop** — useful for catching what the app meter missed, never the thing standing between you and a runaway loop. The recurring caveat: cloud billing data lags, so by the time the cloud notices, the money is already spent.

## AWS

### Cost Anomaly Detection (ML-based)
- ML model, runs roughly **3x/day**, with **up to 24h data delay**.
- Emits to **SNS** → fan out to a **Lambda** that posts into your alert pipe (or revokes a key / disables a feature flag).
- Source: aws.amazon.com/aws-cost-management/aws-cost-anomaly-detection; docs.aws.amazon.com/cost-management/.../manage-ad.html. Accessed 2026-06-02.

### Budgets (threshold)
- Set 50/80/100% threshold alerts on a monthly budget → SNS → same Lambda.
- Pairs with CAD: Budgets catch the *expected* threshold; CAD catches the *unexpected* spike.

```text
AWS Budgets / Cost Anomaly Detection
        │ (SNS, up to ~24h delay)
        ▼
     Lambda  ──►  shared alert pipe (Slack/PagerDuty/webhook)
              └─► optional: disable feature flag / rotate-revoke API key
```

## GCP

- Budgets are **threshold-only** for alerting, but the budget can publish to **Pub/Sub**.
- Pub/Sub → **Cloud Function** is the one path that can actually **pause or throttle a workload** programmatically (e.g. scale a service to zero, disable a billing account binding).
- Source: costimizer.ai gcp guide. Accessed 2026-06-02.

```text
GCP Budget ──► Pub/Sub ──► Cloud Function
                            ├─► shared alert pipe
                            └─► throttle / pause workload (real control)
```

## Azure

- Threshold-only budgets with action groups. Same shape as GCP minus the clean Pub/Sub→pause path; wire the action group into the shared alert pipe.

## Alert-routing pattern (all providers)

Route **every** cloud alert into the **same pipe** as your in-app budget alerts, so there is one place to look and one severity ladder. Tag each alert with its source (`aws-cad`, `gcp-budget`, `app-meter`) so reconciliation can tell which guard fired.

## The delay caveat, stated plainly

| Guard | Latency | Can it stop spend? |
|---|---|---|
| App-level cap (this skill) | real-time | yes — refuse the request now |
| AWS CAD / Budgets | up to ~24h | no — alert only (Lambda can revoke after the fact) |
| GCP budget → Pub/Sub → Function | minutes-to-hours | partial — can pause a workload, but on lagged data |

Design accordingly: the cloud cap is insurance against what your app meter failed to count; it is not the meter.

# Pricing tables — PERISHABLE snapshot (dated 2026-06-02)

These rates move month to month. **Treat this file as data with an expiry, not a constant.** Every number below carries a source; re-fetch before trusting. The whole point of the skill is that prices live in a dated table, never as literals in logic.

## Anthropic (Claude) — per 1M tokens

| Model | Input | Output | Cache read | Source |
|---|---|---|---|---|
| Haiku 4.5 | $1.00 | $5.00 | $0.10 | cloudzero.com/blog/claude-api-pricing |
| Sonnet 4.6 | $3.00 | $15.00 | $0.30 | cloudzero.com/blog/claude-api-pricing |
| Opus 4.7 | $5.00 | $25.00 | $0.50 | evolink.ai/blog/claude-api-pricing-guide-2026 |

Cache writes: 5-minute = 1.25x base input; 1-hour = 2x base input. Cache reads = 0.1x base input. Break-even on a 5-min cache entry: after ~1 hit (platform.claude.com prompt-caching; finout.io anthropic-api-pricing). All accessed 2026-06-02.

## OpenAI — per 1M tokens

| Model | Input | Cached input | Source |
|---|---|---|---|
| GPT-5.5 | $5.00 | $0.50 | openai.com/api/pricing |
| GPT-5.4 | $2.50 | $0.25 | devtk.ai/en/blog/openai-api-pricing-guide-2026 |

Cached input = **90% off** standard input across the line. Output rates vary by model — read them off the source, do not assume. Model names and exact rates drift fast; this is precisely why the rate table is data and never a literal. Accessed 2026-06-02.

## Bedrock note

AWS Bedrock re-prices the same underlying models (often with its own regional/commitment rates) and bills on its own line. Do not assume Bedrock matches first-party Anthropic/OpenAI rates — pull the Bedrock model rate separately and tag the table row `source: bedrock`. Cloud-side caps for Bedrock spend belong in `cloud-caps.md`.

## Batch API discount

50% off on both OpenAI and Anthropic, returned async within 24h, and **stacks with prompt caching** (combined up to ~95% off). Source: finout.io anthropic-api-pricing, accessed 2026-06-02. Use for non-realtime work (evals, backfills, nightly jobs).

## The `usage` object — field map per provider

Bill against these, not pre-send estimates.

| Concept | Anthropic | OpenAI |
|---|---|---|
| Input tokens | `usage.input_tokens` | `usage.prompt_tokens` |
| Output tokens | `usage.output_tokens` | `usage.completion_tokens` |
| Cached (read) | `usage.cache_read_input_tokens` | `usage.prompt_tokens_details.cached_tokens` |
| Cache write | `usage.cache_creation_input_tokens` | n/a (no explicit write line) |
| Reasoning | included in output | `usage.completion_tokens_details.reasoning_tokens` |

Pre-flight estimate tools (estimates only): Anthropic `client.messages.count_tokens()` returns input tokens, is free, separate rate limit, **not** tiktoken-compatible; OpenAI/most use `tiktoken` locally. Sources: platform.claude.com token-counting; langfuse.com token-and-cost-tracking. Accessed 2026-06-02.

## How to refresh this file

1. Pull each provider's current pricing page (sources above).
2. Update the rate and set `effective_date` to today on every changed row.
3. Add any new model name as a new row — never leave a model out, or the lookup will hit the "unknown model" loud-fail path (which is correct behavior).
4. Re-run `scripts/verify.sh` against your `pricing.yaml` to confirm every row still carries `effective_date` + `source`.

#!/usr/bin/env bash
#
# verify.sh - lint a candidate cost-tracking config/ledger artifact.
#
# Usage:
#   cd <dir containing the config>
#   ./verify.sh [path-to-config]      # yaml | yml | json | ts | js
#
# Read-only: never mutates anything. Checks the artifact for:
#   1. every pricing entry carries effective_date AND source
#   2. no model referenced in logic is absent from the pricing table
#   3. the ledger schema has an idempotency/request key + >=1 attribution key
#   4. the budget declares BOTH a soft and a hard threshold
#   5. cost is derived from a usage/response field, not a len()/char estimate
#
# Emits [ ok ]/[fail] per check, then PASS or FAIL.
# Exits 0 on a clean config AND on no config found (no false failure).
# Exits 1 with the missing pieces on any violation.
#
# Portable: stock macOS bash 3.2 + grep + awk. No associative arrays, no mapfile.

set -euo pipefail

if [ -t 1 ]; then
  RED=$'\033[31m'; GREEN=$'\033[32m'; YELLOW=$'\033[33m'; RESET=$'\033[0m'
else
  RED=''; GREEN=''; YELLOW=''; RESET=''
fi

failed=0
fail() { printf '%s[fail]%s %s\n' "$RED" "$RESET" "$*"; failed=1; }
ok()   { printf '%s[ ok ]%s %s\n' "$GREEN" "$RESET" "$*"; }

# --- locate the config -------------------------------------------------------
file="${1:-}"
if [ -z "$file" ]; then
  for cand in cost-tracking.yaml cost-tracking.yml pricing.yaml pricing.yml \
              cost-config.yaml cost-config.yml cost-tracking.json cost-tracking.ts; do
    if [ -f "$cand" ]; then file="$cand"; break; fi
  done
fi

if [ -z "$file" ] || [ ! -f "$file" ]; then
  printf '%s[skip]%s no cost-tracking config found (cost-tracking.{yaml,json,ts} / pricing.yaml) - nothing to verify.\n' "$YELLOW" "$RESET"
  exit 0
fi

printf -- '----- linting %s\n' "$file"

# case-insensitive fixed-ish grep helper
has() { grep -Eiq "$1" "$file"; }

# --- 1. pricing entries carry effective_date AND source ----------------------
mentions_pricing=0
if has 'pric|input_per_mtok|in_per_mtok|rate_in|cost_per'; then mentions_pricing=1; fi
if [ "$mentions_pricing" -eq 1 ]; then
  miss=""
  has 'effective_date|effective-date|effectiveDate' || miss="$miss effective_date"
  has 'source' || miss="$miss source"
  if [ -z "$miss" ]; then
    ok "pricing entries carry effective_date + source"
  else
    fail "pricing table missing:$miss (each model row must be dated and sourced)"
  fi
else
  printf '%s[skip]%s no pricing table in this file (check 1)\n' "$YELLOW" "$RESET"
fi

# --- 2. no model referenced in logic absent from the table -------------------
# Heuristic: collect model-ish identifiers (claude-*/gpt-*/sonnet/opus/haiku/o-series)
# that appear OUTSIDE a "model:" / "- model:" declaration line, and warn if the
# file declares no model rows at all while still naming models.
declared=$(grep -Eio '(^|[^a-z])model[[:space:]]*[:=]' "$file" | wc -l | tr -d ' ')
named=$(grep -Eio '(claude-[a-z0-9.]+|gpt-[a-z0-9.]+|sonnet|opus|haiku|gemini-[a-z0-9.]+)' "$file" \
        | sort -u | wc -l | tr -d ' ')
if [ "$named" -gt 0 ]; then
  if [ "$declared" -eq 0 ]; then
    fail "models are named but no 'model:' table rows declared - unknown models would price as \$0"
  else
    ok "models named ($named) are declared in 'model:' rows ($declared) - lookup can fail-loud on unknowns"
  fi
else
  printf '%s[skip]%s no model identifiers found (check 2)\n' "$YELLOW" "$RESET"
fi

# --- 3. ledger: idempotency key + >=1 attribution key ------------------------
mentions_ledger=0
if has 'ledger|usage|input_tokens|output_tokens|cost_usd'; then mentions_ledger=1; fi
if [ "$mentions_ledger" -eq 1 ]; then
  miss=""
  has 'request_id|idempotency|request-id|requestId|dedup' || miss="$miss idempotency/request key"
  has 'user_id|tenant_id|tenant|feature|user-id|tenantId|userId' || miss="$miss attribution-key(user/tenant/feature)"
  if [ -z "$miss" ]; then
    ok "ledger has an idempotency key + at least one attribution key"
  else
    fail "ledger missing:$miss"
  fi
else
  printf '%s[skip]%s no ledger schema in this file (check 3)\n' "$YELLOW" "$RESET"
fi

# --- 4. budget: both a soft and a hard threshold -----------------------------
mentions_budget=0
if has 'budget|threshold|cap|alert'; then mentions_budget=1; fi
if [ "$mentions_budget" -eq 1 ]; then
  miss=""
  has 'soft|alert|warn|degrade|downshift' || miss="$miss soft(alert/degrade)"
  has 'hard|refuse|hard_cap|hard-cap|block|reject' || miss="$miss hard(refuse)"
  if [ -z "$miss" ]; then
    ok "budget declares both a soft and a hard threshold"
  else
    fail "budget missing:$miss threshold"
  fi
else
  printf '%s[skip]%s no budget config in this file (check 4)\n' "$YELLOW" "$RESET"
fi

# --- 5. cost derived from usage, not a len()/char estimate -------------------
if has 'usage|input_tokens|output_tokens|cached_tokens|prompt_tokens'; then
  if grep -Eiq 'len\(|\.length|char_count|charcount|//[[:space:]]*4|/[[:space:]]*4' "$file" \
     && ! has 'estimate|pre-?flight|preflight'; then
    fail "cost may be derived from a len()/char estimate - bill against the response usage object"
  else
    ok "cost derived from a usage/response field, not a char estimate"
  fi
else
  printf '%s[skip]%s no usage-based cost field found (check 5)\n' "$YELLOW" "$RESET"
fi

echo
if [ "$failed" -ne 0 ]; then
  printf '%sFAIL:%s cost-tracking config is missing required pieces (see [fail] lines above).\n' "$RED" "$RESET"
  exit 1
fi
printf '%sPASS:%s cost-tracking config carries the required guardrails.\n' "$GREEN" "$RESET"