Claude Skill

telemetry-canary

Observability and structured logging canary — checks for structured logs (JSON), OpenTelemetry metrics/traces, proper error stack traces, and flags empty catches or silent log swallowing. Triggers on keywords: "/telemetry-canary", "telemetry-canary", "observability audit", "struc

LLM Mart · 0 points · 2 views 0 listing impressions 0 install-command copies
Download TheColliery-CoalMine-skills_telemetry-canary-7953b7d.zip · 4 KB

Install

skills CLI npx skills add https://github.com/TheColliery/CoalMine/tree/main/skills/telemetry-canary
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install hetcreep-coalmine@llmmart
Git git clone https://github.com/TheColliery/CoalMine.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole hetcreep/coalmine collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Telemetry Canary (Observability & Logging Audit)

Audit code for proper telemetry instrumentation — ensure the app is not a black box in production, and that what it logs does not leak.

Auditing Categories

  1. Empty / Silent Catch — catch blocks that swallow exceptions without logging a stack trace or forwarding the error.
  2. Unstructured Logs — plain-string logging in server code (prefer JSON / structured key-value for cloud queries).
  3. No Correlation ID — operations crossing boundaries (HTTP/gRPC/threads) without propagating a trace/correlation ID.
  4. Missing Metrics — critical transactions (checkout, auth, errors) lacking counter/histogram instrumentation.
  5. No Stack Traces — errors logged without stack context (logger.error(e.message) instead of logger.error(e)).
  6. Sensitive data in logs — a log call in code a runtime loads whose argument is a request body, an Authorization/Cookie/Set-Cookie header (or the whole headers object), or a prompt or output value; logs that leave the machine keep it.

Per-stack grep patterns and right/wrong shapes per category: read references/checks.md before scanning.

Fix mode (choice-gated)

In Agent Context, after the report, present via ask_question:

  • Apply safe logs: insert error logging into empty catch blocks (standard logger template) + stack-trace mapping. Each fix: checkpoint (copy the touched file(s) aside, or use an isolated worktree — never git stash/git commit, which can hide or include unrelated staged/unstaged user work) → record a build+test BASELINE → apply → build + tests → auto-revert only if a NEW failure appeared versus the baseline.
  • Let me pick: user selects which telemetry gaps to resolve.
  • Report only: exit unchanged.

Grants & denials (CLASSIFY-BLOCK)

class step it powers grant on denial
read scan logging/metrics/error paths for the categories above Read·Grep·Glob refuse that file, name it — never a clean bill
write Fix mode's safe-log apply, incl. checkpoint → baseline → build+tests → auto-revert only on a NEW failure Edit·Bash (checkpoint/build/revert need exec) report the fix as NOT applied AND the checkpoint/revert as NOT available, never claim done

Output

| file:line | category | severity | finding | recommendation |

Severity: CRITICAL (swallowed error with state mutation; a confirmed secret value written to a log) · HIGH (missing stack trace in error logs; sensitive data logged where the Worker config ships logs off the machine) · MEDIUM (unstructured log in API boundary; sensitive data logged where logs are only seen live) · LOW (minor trace gaps)

Files (coalmine)
  • references
    • checks.md 5.7 KB
      Loading…
  • skill-meta.json 185 B
    Loading…
  • SKILL.md 3.3 KB
    ---
    name: telemetry-canary
    description: >-
      Observability and structured logging canary — checks for structured logs (JSON), OpenTelemetry metrics/traces, proper error stack traces, flags empty catches or silent log swallowing, and flags sensitive data written to logs. Triggers on keywords: "/telemetry-canary", "telemetry-canary", "observability audit", "structured logging". Use when adding or changing logging, metrics, tracing, or error-handling code.
    ---
    
    # Telemetry Canary (Observability & Logging Audit)
    
    <!-- SHARED:LANGUAGE_HEADER -->
    
    Audit code for proper telemetry instrumentation — ensure the app is not a black box in production, and that what it logs does not leak.
    
    ## Auditing Categories
    1. **Empty / Silent Catch** — catch blocks that swallow exceptions without logging a stack trace or forwarding the error.
    2. **Unstructured Logs** — plain-string logging in server code (prefer JSON / structured key-value for cloud queries).
    3. **No Correlation ID** — operations crossing boundaries (HTTP/gRPC/threads) without propagating a trace/correlation ID.
    4. **Missing Metrics** — critical transactions (checkout, auth, errors) lacking counter/histogram instrumentation.
    5. **No Stack Traces** — errors logged without stack context (`logger.error(e.message)` instead of `logger.error(e)`).
    6. **Sensitive data in logs** — a log call in code a runtime loads whose argument is a request body, an `Authorization`/`Cookie`/`Set-Cookie` header (or the whole headers object), or a prompt or output value; logs that leave the machine keep it.
    
    Per-stack grep patterns and right/wrong shapes per category: read `references/checks.md` before scanning.
    
    ## Fix mode (choice-gated)
    
    In Agent Context, after the report, present via `ask_question`:
    
    - **Apply safe logs:** insert error logging into empty catch blocks (standard logger template) + stack-trace mapping. Each fix: checkpoint (copy the touched file(s) aside, or use an isolated worktree — never `git stash`/`git commit`, which can hide or include unrelated staged/unstaged user work) → record a build+test BASELINE → apply → build + tests → auto-revert only if a NEW failure appeared versus the baseline.
    - **Let me pick:** user selects which telemetry gaps to resolve.
    - **Report only:** exit unchanged.
    
    ## Grants & denials (CLASSIFY-BLOCK)
    | class | step it powers | grant | on denial |
    |---|---|---|---|
    | read | scan logging/metrics/error paths for the categories above | `Read`·`Grep`·`Glob` | refuse that file, name it — never a clean bill |
    | write | Fix mode's safe-log apply, incl. checkpoint → baseline → build+tests → auto-revert only on a NEW failure | `Edit`·`Bash` (checkpoint/build/revert need exec) | report the fix as NOT applied AND the checkpoint/revert as NOT available, never claim done |
    
    <!-- SHARED:CLASSIFY_BLOCK -->
    
    ## Output
    `| file:line | category | severity | finding | recommendation |`
    
    Severity: CRITICAL (swallowed error with state mutation; a confirmed secret value written to a log) · HIGH (missing stack trace in error logs; sensitive data logged where the Worker config ships logs off the machine) · MEDIUM (unstructured log in API boundary; sensitive data logged where logs are only seen live) · LOW (minor trace gaps)
    
    <!-- SHARED:REPORTING_FOOTER -->
    
    <!-- SHARED:ORCHESTRATION -->
    
    <!-- SHARED:ESCALATION_FOOTER -->
    
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.