Claude Skill

telemetry-canary

Observability and structured logging canary — checks for structured logs (JSON), OpenTelemetry metrics/traces, proper error stack traces, and flags empty catches or silent log swallowing. Triggers on keywords: "/telemetry-canary", "telemetry-canary", "observability audit", "struc

LLM Mart · 0 points · 0 0 0

Virus-scanned Reviewed automatically before listing.

Full trust report

Download hetcreep-coalmine-skills_telemetry-canary-85306d7.zip · 3 KB

hetcreep/coalmine 13 2 forks Apache-2.0 Updated 7d ago

Part of hetcreep/coalmine — 18 skills

Install

skills CLI npx skills add https://github.com/TheColliery/CoalMine/tree/main/skills/telemetry-canary

Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install hetcreep-coalmine@llmmart

Git git clone https://github.com/TheColliery/CoalMine.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole hetcreep/coalmine collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Telemetry Canary (Observability & Logging Audit)

Audit code for proper telemetry instrumentation — ensure the app is not a black box in production.

Auditing Categories

Empty / Silent Catch — catch blocks that swallow exceptions without logging a stack trace or forwarding the error.
Unstructured Logs — plain-string logging in server code (prefer JSON / structured key-value for cloud queries).
No Correlation ID — operations crossing boundaries (HTTP/gRPC/threads) without propagating a trace/correlation ID.
Missing Metrics — critical transactions (checkout, auth, errors) lacking counter/histogram instrumentation.
No Stack Traces — errors logged without stack context (logger.error(e.message) instead of logger.error(e)).

Per-stack grep patterns and right/wrong shapes per category: read references/checks.md before scanning.

Fix mode (choice-gated)

In Agent Context, after the report, present via ask_question:

Apply safe logs: insert error logging into empty catch blocks (standard logger template) + stack-trace mapping. Each fix: checkpoint (git stash/commit in a git repo; else copy the file aside — never assume git) → apply → build + tests → auto-revert if newly red.
Let me pick: user selects which telemetry gaps to resolve.
Report only: exit unchanged.

Grants & denials (CLASSIFY-BLOCK)

class	step it powers	grant	on denial
read	scan logging/metrics/error paths for the categories above	`Read`·`Grep`·`Glob`	refuse that file, name it — never a clean bill
write	Fix mode's safe-log apply, incl. checkpoint → build+tests → auto-revert if newly red	`Edit`·`Bash` (checkpoint/build/revert need exec)	report the fix as NOT applied AND the checkpoint/revert as NOT available, never claim done

Output

Severity: CRITICAL (swallowed error with state mutation) · HIGH (missing stack trace in error logs) · MEDIUM (unstructured log in API boundary) · LOW (minor trace gaps)

Files (coalmine)

references

checks.md 2.2 KB

<!-- coalmine: verified 2026-06-12 · revalidate 90d · definition file for telemetry-canary -->
# Telemetry canary — concrete detection procedures

## 1. Empty / silent catch — grep first, then confirm by reading
| Stack | Patterns to grep |
|---|---|
| C#/.NET | `catch { }` · `catch (Exception) { }` · `catch (Exception ex) { }` with no `_logger`/throw inside |
| TS/JS | `catch {}` · `catch (e) {}` · `.catch(() => {})` · `.catch(console.log)` (downgrades errors) |
| Python | `except: pass` · `except Exception: pass` · bare `except:` |
| Go | `_ = err` · `if err != nil { }` empty body · err assigned then never checked |

Confirm: a catch is only SILENT if nothing inside logs with stack, rethrows, or sets a failure result.

## 2. Unstructured logs (server/API code only)
- String concatenation/interpolation into log calls: `log.info('user ' + id + ' did X')`, `$"..."`, f-strings.
- Structured replacements: .NET `ILogger` message templates (`_logger.LogInformation("User {UserId}", id)`) / Serilog · TS `pino`/`winston` object form (`log.info({userId}, 'did X')`) · Python `structlog` or `logger.info(..., extra={})` · Go `log/slog` / `zap` fields.
- CLI tools writing human output to stdout are NOT findings — scope this to services.

## 3. No correlation/trace ID across boundaries
- Outbound HTTP/gRPC/queue publish without propagating context: look for missing W3C `traceparent` header / OTel propagation (`propagation.inject`, .NET `Activity.Current`, Python `opentelemetry.propagate`).
- New thread/task/queue consumer that starts logging without the parent's correlation ID.

## 4. Missing metrics on critical transactions
- Identify money/auth/data-loss paths (checkout, login, write/delete APIs). Each should touch a counter or histogram (OTel `Counter`/`Histogram`, Prometheus client, StatsD).
- Error paths that increment nothing are invisible in dashboards — flag.

## 5. No stack traces
| Stack | Wrong | Right |
|---|---|---|
| TS/JS | `logger.error(e.message)` | `logger.error(e)` / `logger.error({err: e}, msg)` |
| Python | `logger.error(str(e))` | `logger.exception(...)` or `exc_info=True` |
| C# | `_logger.LogError(ex.Message)` | `_logger.LogError(ex, "msg")` |
| Go | `log.Error(err.Error())` only | wrap with `%w`, log with stack lib |

skill-meta.json 185 B

{ "lightIntent": "Spot telemetry check, key paths only", "standardIntent": "Balanced observability audit, multi-category", "heavyIntent": "Full 5-category audit + adversarial verify" }

SKILL.md 2.6 KB

---
name: telemetry-canary
description: >-
  Observability and structured logging canary — checks for structured logs (JSON), OpenTelemetry metrics/traces, proper error stack traces, and flags empty catches or silent log swallowing. Triggers on keywords: "/telemetry-canary", "telemetry-canary", "observability audit", "structured logging". Use when adding or changing logging, metrics, tracing, or error-handling code.
---

# Telemetry Canary (Observability & Logging Audit)

<!-- SHARED:LANGUAGE_HEADER -->

Audit code for proper telemetry instrumentation — ensure the app is not a black box in production.

## Auditing Categories
1. **Empty / Silent Catch** — catch blocks that swallow exceptions without logging a stack trace or forwarding the error.
2. **Unstructured Logs** — plain-string logging in server code (prefer JSON / structured key-value for cloud queries).
3. **No Correlation ID** — operations crossing boundaries (HTTP/gRPC/threads) without propagating a trace/correlation ID.
4. **Missing Metrics** — critical transactions (checkout, auth, errors) lacking counter/histogram instrumentation.
5. **No Stack Traces** — errors logged without stack context (`logger.error(e.message)` instead of `logger.error(e)`).

Per-stack grep patterns and right/wrong shapes per category: read `references/checks.md` before scanning.

## Fix mode (choice-gated)

In Agent Context, after the report, present via `ask_question`:

- **Apply safe logs:** insert error logging into empty catch blocks (standard logger template) + stack-trace mapping. Each fix: checkpoint (git stash/commit in a git repo; else copy the file aside — never assume git) → apply → build + tests → auto-revert if newly red.
- **Let me pick:** user selects which telemetry gaps to resolve.
- **Report only:** exit unchanged.

## Grants & denials (CLASSIFY-BLOCK)
| class | step it powers | grant | on denial |
|---|---|---|---|
| read | scan logging/metrics/error paths for the categories above | `Read`·`Grep`·`Glob` | refuse that file, name it — never a clean bill |
| write | Fix mode's safe-log apply, incl. checkpoint → build+tests → auto-revert if newly red | `Edit`·`Bash` (checkpoint/build/revert need exec) | report the fix as NOT applied AND the checkpoint/revert as NOT available, never claim done |

<!-- SHARED:CLASSIFY_BLOCK -->

## Output
`| file:line | category | severity | finding | recommendation |`

Severity: CRITICAL (swallowed error with state mutation) · HIGH (missing stack trace in error logs) · MEDIUM (unstructured log in API boundary) · LOW (minor trace gaps)

<!-- SHARED:REPORTING_FOOTER -->

<!-- SHARED:ORCHESTRATION -->

<!-- SHARED:ESCALATION_FOOTER -->