monte-carlo-prevent
Shift-left safety net for dbt/SQL model edits. Runs change impact assessment before edits, generates SQL validation queries after, and executes them via `/mc-validate run`. Delegates health and monitor creation to peer skills.
Install
npx skills add https://github.com/monte-carlo-data/mc-agent-toolkit/tree/main/skills/prevent
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install monte-carlo-data-mc-agent-toolkit@llmmart
git clone https://github.com/monte-carlo-data/mc-agent-toolkit.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole monte-carlo-data/mc-agent-toolkit collection as a plugin from our marketplace. Git is the plain clone.
README
Monte Carlo Prevent Skill
Bring Monte Carlo data observability into your editor — automatically, before you write a single line of code.
What this does
When you reference a dbt model or table, Monte Carlo context comes to you: table health, active alerts, lineage, and downstream blast radius. Your AI editor uses that context to shape the code it writes — not just surface it. If you try to rename a column with 500 downstream dependents, the editor recommends a safe transition strategy and explains why, citing the specific MC data it found. When you add new logic, it generates and deploys the right monitor for your logic — validation, metric, comparison, or custom SQL — before you merge. When you're done with a change, it generates targeted validation queries — tailored to the specific columns, filters, and business logic you modified — so you can verify the change behaved as intended before merging.
Editor & Stack Compatibility
The skill works with any AI editor that supports MCP and the Agent Skills format — including Claude Code, Cursor, and VS Code.
For data stacks, compatibility varies by how you work:
| Stack | Support | Notes |
|---|---|---|
| dbt + any MC-supported warehouse | ✅ Full | Optimized and tested |
| SQL-first, no dbt | 🟡 Partial | Core workflows work via explicit prompting; auto-triggers on file open coming soon |
| Databricks notebooks | 🟡 Partial | Health check, impact assessment, and alert triage work; file-based triggers coming soon |
| SQLMesh | 🟡 Partial | Core workflows work; native SQLMesh project structure support coming soon |
| PySpark / non-SQL pipelines | 🟠 Limited | Manual prompting only; broader support on the roadmap |
Coming shortly: Generic SQL file triggers, Databricks notebook support, and SQLMesh project structure support — so auto-activation works regardless of your transformation tool.
Core workflows — table health check, change impact assessment, alert triage, and monitor generation — work for any warehouse supported by Monte Carlo.
Prerequisites
- Claude Code, Cursor, VS Code or any editors with MCP support
- Monte Carlo account with Editor role or above
- MC CLI installed for monitor deployment (
pip install montecarlodata)
Setup
Via the mc-agent-toolkit plugin (recommended)
Install the plugin for your editor — it bundles the skill, hooks, MCP server, and permissions automatically. See the main README for editor-specific instructions.
Standalone
Configure the Monte Carlo MCP server:
claude mcp add --transport http monte-carlo-mcp https://mcp.getmontecarlo.com/mcpInstall the skill:
npx skills add monte-carlo-data/mc-agent-toolkit --skill preventAuthenticate: run
/mcpin your editor, selectmonte-carlo-mcp, and complete the OAuth flow.Verify: ask your editor "Test my Monte Carlo connection" — it should call
testConnectionand confirm.
Legacy: header-based auth (for MCP clients without HTTP transport)
If your MCP client doesn't support HTTP transport, use .mcp.json.example with npx mcp-remote and header-based authentication. See the MCP server docs for details.
How to use it
Open your dbt project (or any data engineering codebase) in your editor. Describe the change you want to make — or reference a model file together with an edit (@models/orders.sql add a column). The skill activates automatically when you express change intent; no special commands needed.
End-to-end flow
flowchart TD
A["Describe a<br/>change"] --> B["Fetch table<br/>context<br/>(silent)"]
B --> C["Impact<br/>assessment"]
C --> D{"Proceed?"}
D -- yes --> E["Edit<br/>applied"]
E --> P["Post-edit prompt:<br/>generate validation<br/>queries?<br/>add monitor?"]
P -- yes --> H["Generate<br/>validation queries"]
P -- yes --> G["Generate<br/>monitor"]
H --> R{"Run<br/>queries?"}
R -- yes --> S["Build & run"]
Impact assessment — Before any SQL edit (including filter changes, bugfixes, reverts, and parameter tweaks), prevent surfaces the change's blast radius: downstream models, active alerts, column exposure in recent queries, and monitor coverage. You get a risk tier (High / Medium / Low) and a recommendation tied to your specific change. If the data suggests your approach is risky, Claude proposes a safer alternative.
Validation queries — When you're ready to test a change, say "generate validation queries", "validate this change", or run /mc-validate. Prevent generates 3–5 targeted SQL queries based on what you actually changed — null checks, before/after row counts, distribution checks — saved to validation/<table_name>_<timestamp>.sql with inline comments describing a passing result.
Monitor coverage — After you finish an edit, if the impact assessment found a coverage gap, prevent prompts you to add a monitor. On yes, it hands off to monte-carlo-monitoring-advisor to produce a validation, metric, comparison, or custom SQL monitor as code.
Validate in sandbox (/mc-validate run) — Two-phase workflow. Run /mc-validate first to generate the queries, then /mc-validate run to execute them:
- Build — parses your
profiles.yml, classifies the resolved database, and runsdbt build --select <model>into your dev database. - Execute — substitutes
<YOUR_DEV_DATABASE>in the generated SQL with a user-confirmed value, runs each query through the Snowflake MCP, and reports findings.
⚠️ Heads up on prod vs. dev detection. The build phase classifies your resolved target as
personal/dev/shared-dev/prod/unknownfrom yourprofiles.ymland any hard-coded{{ config(database=...) }}, and hard-stops if it lands onprod. This is a safety net, not a guarantee — naming conventions vary across orgs and the classifier can be wrong (especially onunknown). You are still responsible for confirming the target database before approving the build. Read the value the skill surfaces and don't approve if it doesn't match where you intend to write.
Invocation modes:
| Command | What it does |
|---|---|
/mc-validate |
Default = generate. Runs query generation only. |
/mc-validate generate |
Explicit generate. Same as above. |
/mc-validate run |
Runs Build + Execute. Requires queries already generated. |
/mc-validate run --skip-build |
Runs Execute only — assumes you built manually. Requires queries already generated. |
/mc-validate run --dev-db <NAME> |
Same as run, bypasses the dev-database prompt in Execute. |
/mc-validate run prerequisites
The run subcommand only works if all three of the following are in place — otherwise the flow will fail mid-build with a confusing error. Verify before invoking:
- dbt installed and a
dbt_project.ymldiscoverable from the changed model (the workflow walks up from the model file to find it). profiles.ymlpresent (typically in~/.dbt/profiles.yml) with a working Snowflake target. The skill parses it to resolve your dev database.- Snowflake MCP server registered in the editor session — the skill detects this by looking for an
mcp__snowflake__*tool. Without it, queries cannot execute and the substituted SQL is left on disk for you to run manually.
The run subcommand performs a connection pre-flight check before kicking off the build. If any prerequisite is missing, it aborts early and tells you what to fix — rather than failing after a partial build.
Deploying generated monitors
When Claude generates a monitor, it saves the YAML to monitors/<table>.yml. Deploy with:
montecarlo monitors apply --dry-run # preview
montecarlo monitors apply --auto-yes # apply
Your project needs a montecarlo.yml config in the working directory:
version: 1
namespace: <your-namespace>
default_resource: <your-warehouse-name>
Troubleshooting
See TROUBLESHOOTING.md for common setup and runtime issues.
Skill manifest
Monte Carlo Prevent Skill
This skill brings Monte Carlo's data observability context directly into your editor. When you're modifying a dbt model or SQL pipeline, use it to surface table health, lineage, active alerts, and to generate monitors-as-code without leaving Claude Code.
Monte Carlo tool routing (required): Always call Monte Carlo MCP tools through this plugin's bundled server, whose fully-qualified tool names are
mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__<tool>(e.g.mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__get_alerts). Bare tool names used in this skill (get_alerts,search,get_table, …) refer to that bundled server. If the session also has a separately-configuredmonte-carlo-mcpserver, do not route to it — it may point at a different endpoint or credentials.
Reference files live next to this skill file. Use the Read tool (not MCP resources) to access them:
- Full workflow step-by-step instructions:
references/workflows.md(relative to this file) - MCP parameter details:
references/parameters.md(relative to this file) - Troubleshooting:
references/TROUBLESHOOTING.md(relative to this file)
When to activate this skill
Prevent is the edit-lifecycle skill. Activate only when the user expresses
intent to change a dbt model. Bare file mentions, table-name mentions in
passing, or general health questions are not prevent's territory — those
belong to monte-carlo-asset-health and will activate that skill on their own.
Do not wait to be asked. Run the appropriate workflow automatically whenever the user:
- Describes a planned change to a model (new column, join update, filter change, refactor) → STOP — run Workflow 1 first if it has not run for this table this session, then Workflow 2, before writing any code
- Adds a new column, metric, or output expression to an existing model → same rule: Workflow 1 first (if not yet run for this table), then Workflow 2; the post-edit hook will offer Workflow 5 (monitor generation) afterward
- References a model file with an edit verb in the same prompt (e.g.
@models/clients/client_hub.sql add an is_active column) → same rule: Workflow 1 first, then Workflow 2
Present the W2 impact assessment as context the engineer needs before proceeding — not as a response to a question.
Workflow 1 runs silently when chained to Workflow 2
When the user expresses change intent, Workflow 1 invokes monte-carlo-asset-health
purely as a data-gathering step. Read asset-health's report from your context, but
do not relay the full report to the engineer — the user-facing artifact is
Workflow 2's impact assessment, which already cites the relevant alerts / lineage /
monitors. Showing both creates duplicate reading.
Two exceptions where you must surface output from W1 to the engineer:
- Disambiguation prompt. If asset-health returns multiple matches and asks the engineer to pick one, surface that question — the user must choose.
- Stop-the-world signals. If the table is already on fire (active critical alerts firing, freshness severely stale), say so in one short line before W2.
If Workflow 1 already ran for this table earlier in the session, skip directly to Workflow 2 — re-running asset-health is redundant.
When NOT to activate this skill
Do not invoke Monte Carlo tools for:
- Seed files (files in seeds/ directory)
- Analysis files (files in analyses/ directory)
- One-off or ad-hoc SQL scripts not part of a dbt project
- Configuration files (dbt_project.yml, profiles.yml, packages.yml)
- Test files unless the user is specifically asking about data quality
If uncertain whether a file is a dbt model, check for {{ ref() }} or {{ source() }} Jinja references — if absent, do not activate.
Macros and snapshots — gate edits, skip auto-context
Macro files (macros/) and snapshot files (snapshots/) are not models, so
do not auto-fetch Monte Carlo context (Workflow 1) when they are opened. However,
macros are inlined into every model that calls them at compile time — a one-line
macro change can silently alter dozens of models. Snapshots control historical
tracking and are similarly sensitive.
The pre-edit hook gates these files. If the hook fires for a macro or snapshot, identify which models are affected and run the change impact assessment (Workflow 2) for those models before proceeding with the edit.
Peer-skill redirects
These requests have their own skills — do not run prevent for them:
- "How is table X doing?" / "is X healthy?" / "check status of X" →
monte-carlo-asset-health - "Create a monitor for X" / "what should I monitor?" / "set up freshness on X" (without an active edit context) →
monte-carlo-monitoring-advisor
Prevent invokes asset-health and monitoring-advisor itself when its workflows need them (W1, W5); it does not duplicate their entry points.
REQUIRED: Change impact assessment before any SQL edit
Before editing or writing any SQL for a dbt model or pipeline, you MUST run Workflow 2.
This applies whenever the user expresses intent to modify a model — including phrases like:
- "I want to add a column…"
- "Let me add / I'm adding…"
- "I'd like to change / update / rename…"
- "Can you add / modify / refactor…"
- "Let's add…" / "Add a
<column>column" - Any other description of a planned schema or logic change
- "Exclude / filter out / remove [records/customers/rows]…"
- "Adjust / increase / decrease [threshold/parameter/value]…"
- "Fix / bugfix / patch [issue/bug]…"
- "Revert / restore / undo [change/previous behavior]…"
- "Disable / enable [feature/logic/flag]…"
- "Clean up / remove [references/columns/code]…"
- "Implement [backend/feature] for…"
- "Create [models/dbt models] for…" (when modifying existing referenced tables)
- "Increase / decrease / change [max_tokens/threshold/date constant/numeric parameter]…"
- Any change to a hardcoded value, constant, or configuration parameter within SQL
- "Drop / remove / delete [column/field/table]"
- "Rename [column/field] to [new name]"
- "Add [column]" (short imperative form, e.g. "add a created_at column")
- Any single-verb imperative command targeting a column, table, or model (e.g. "drop X", "rename Y", "add Z", "remove W")
Parameter changes (threshold values, date constants, numeric limits) appear safe but silently change model output. Treat them the same as logic changes for impact assessment purposes.
Do not write or edit any SQL until the change impact assessment (Workflow 2) has been presented to the user. The assessment must come first — not after the edit, not in parallel.
Pre-edit gate — check before modifying any file
Before calling Edit, Write, or MultiEdit on any .sql or dbt model
file, you MUST check:
- Has the synthesis step been run for THIS SPECIFIC CHANGE in the current prompt?
- If YES → proceed with the edit
- If NO → stop immediately, run Workflow 2, present the full report with synthesis connected to this specific change. If risk is High or Medium: ask "Do you want me to proceed with the edit?" and wait for explicit confirmation. If risk is Low: use judgment — proceed if straightforward and no concerns found, otherwise ask before editing.
Important: "Workflow 2 already ran this session" is NOT sufficient to proceed. Each distinct change prompt requires its own synthesis step connecting the MC findings to that specific change.
The synthesis must reference the specific columns, filters, or logic being changed in the current prompt — not just general table health.
Example:
- ✅ "Given 34 downstream models depend on is_paying_workspace, adding 'MC Internal' to the exclusion list will exclude these workspaces from all downstream health scores and exports. Confirm?"
- ❌ "Workflow 2 already ran. Making the edit now."
The only exception: if the user explicitly acknowledges the risk and confirms they want to skip (e.g. "I know the risks, just make the change") — proceed but note the skipped assessment.
Available MCP tools
All tools are available via the monte-carlo-mcp MCP server.
| Tool | Purpose |
|---|---|
testConnection |
Verify auth and connectivity |
search |
Find tables/assets by name |
getTable |
Schema, stats, metadata for a table |
getAssetLineage |
Upstream/downstream dependencies (call with mcons array + direction) |
getAlerts |
Active incidents and alerts |
getMonitors |
Monitor configs — filter by table using mcons array |
getQueriesForTable |
Recent query history |
getQueryData |
Full SQL for a specific query |
createValidationMonitorMac |
Generate validation monitors-as-code YAML |
createMetricMonitorMac |
Generate metric monitors-as-code YAML |
createComparisonMonitorMac |
Generate comparison monitors-as-code YAML |
createCustomSqlMonitorMac |
Generate custom SQL monitors-as-code YAML |
getValidationPredicates |
List available validation rule types |
getAudiences |
List notification audiences |
getDomains |
List MC domains |
getUser |
Current user info |
Core workflows
Each workflow has detailed step-by-step instructions in references/workflows.md (Read tool).
1. Asset health pre-fetch (silent delegation to asset-health)
When: User expresses change intent for a table that hasn't been seen in this session.
What: Invokes monte-carlo-asset-health via the Skill tool to gather table state (health, upstream lineage, alerts, monitors). Then makes one direct get_asset_lineage(direction="DOWNSTREAM") call to complete the picture (asset-health only fetches upstream). The combined data is used as input to Workflow 2, not shown to the engineer. Two exceptions surface to the user: any disambiguation prompt, and stop-the-world signals (active critical alerts, severe staleness).
2. Change impact assessment — REQUIRED before modifying a model
When: Any intent to modify a dbt model's logic, columns, joins, or filters.
What: Surfaces blast radius, downstream dependencies, active incidents, monitor coverage, and query exposure. Reuses asset-health's data when Workflow 1 ran earlier this session; otherwise calls get_table / get_alerts / get_asset_lineage / get_monitors directly. Produces a risk-tiered report with synthesis connecting findings to specific code recommendations.
3. Change validation queries
When: Explicit engineer request only (e.g. "validate this change", "ready to commit"), or via /mc-validate run.
What: Generates 3–5 targeted SQL queries to verify the change behaved as intended. Uses Workflow 2 context — requires both impact assessment and file edit in session.
4. Validate change in sandbox — invoked by /mc-validate run
When: Only when the engineer invokes /mc-validate run (in any of its forms).
Pre-flight: run does not auto-generate. If no validation/<table>_<ts>.sql exists for the changed model(s), abort and tell the engineer to run /mc-validate (or /mc-validate generate) first.
What: Two-phase workflow.
- W4.1 — Build. Parses
profiles.yml, classifies the active database, detects hard-codeddatabase:in the model's{{ config() }}, then runsdbt build --select <model>into the engineer's dev database. Refuses to build against shared prod. Skipped automatically for YAML/docs-only diffs and for/mc-validate run --skip-build. - W4.2 — Execute validation queries. Substitutes
<YOUR_DEV_DATABASE>in Workflow-3 output with a user-confirmed value (or--dev-db <NAME>if supplied), runs a read-only pre-check on every query, executes via the Snowflake MCP, and reports per-query verdicts plus a consolidated summary.
Invocation matrix:
| Invocation | W3 (generate) | W4.1 (Build) | W4.2 (Execute) |
|---|---|---|---|
/mc-validate |
yes | — | — |
/mc-validate generate |
yes | — | — |
/mc-validate run |
no — must already exist | yes | yes |
/mc-validate run --skip-build |
no — must already exist | no | yes |
run accepts both flags together: /mc-validate run --skip-build --dev-db <NAME>.
5. Add monitor (delegated to monitoring-advisor, post-edit)
When: Post-edit hook injects the coverage prompt (driven by MC_MONITOR_GAP from Workflow 2), or the engineer explicitly asks to add a monitor.
What: Asks "Generate monitor definitions? (yes/no)". On yes, invokes monte-carlo-monitoring-advisor via the Skill tool with the model name and changed columns/logic. Prevent's responsibility ends at delegation — it does not wait for monitoring-advisor or emit a completion marker.
Workflow numbering note: numbers are assigned by execution order (W1 → W2 → optional W3 → optional W4 [W4.1 + W4.2] → optional W5), not by insertion order in this file.
references/workflows.mdis the source of truth.
Post-synthesis confirmation rules
Always end the synthesis with one clear, specific recommendation in plain English: "Given the above, I recommend: [specific action]"
If the risk is High or Medium: STOP and wait for confirmation before editing any file. You must ask the engineer and receive an explicit "yes", "go ahead", "proceed", or similar confirmation before making code changes. Say: "Do you want me to proceed with the edit?" Do NOT say: "Proceeding with the edit." — that skips the engineer's decision.
If the risk is Low: Use your judgment based on the synthesis findings. If the change is straightforward and the synthesis found no concerns, you may proceed. If anything is surprising or worth flagging, ask before editing.
Session markers
These markers coordinate between the skill and the plugin's hooks. Output each on its own line when the condition is met.
Impact check complete
After the engineer confirms (High/Medium) or after presenting the synthesis (Low), output one marker per assessed table. IMPORTANT: use only the table/model name, not the full MCON:
(Use the model filename without .sql extension — NOT "acme.analytics.orders" or "prod.public.client_hub")
How many markers to emit depends on how the assessment was triggered:
Hook-triggered (the pre-edit hook blocked an edit and instructed you to run the assessment): Be strict — only emit markers for tables whose lineage and monitor coverage were fetched directly via Monte Carlo tools in this session. If the engineer describes changes to multiple tables but only one was formally assessed, emit only one marker. The pre-edit hook will gate the other tables and prompt for their own Workflow 2 runs.
Voluntarily invoked (the engineer proactively asked for an impact assessment): Be looser — emit markers for all tables the assessment meaningfully covered, even if some were assessed via lineage context rather than direct MC tool calls. The engineer is already safety-conscious; don't force redundant assessments for tables they clearly considered.
Monitor coverage gap
When Workflow 2 finds zero custom monitors on a table's affected columns, output:
Use only the table/model name (NOT the full MCON). This allows the plugin's hooks to remind the engineer about monitor coverage at commit time. Only output this marker when the gap is specifically about the columns or logic being changed — not for general table-level monitor absence.
After the prompt is delivered, the post-edit / pre-commit hook clears the gap state internally so it won't re-prompt for the same gap; if the engineer edits the model again, Workflow 2 will re-evaluate from scratch and re-emit the marker only if a gap still exists.
Sandbox build ran (W4.1)
Emit after a successful dbt build in Workflow 4.1 (or after a deliberate skip
— e.g. YAML-only diff or --skip-build — including the skip reason). One marker
per model.
Validation executed (W4.2)
Emit after Workflow 4.2 finishes executing validation queries for a model, regardless of individual per-query verdicts. One marker per model.
Files (mc-agent-toolkit)
-
references
-
parameters.md 1 KB
# MCP Parameter Notes Important parameter details for Monte Carlo MCP tools. Consult when making API calls to avoid common mistakes. --- ## `getAlerts` — use snake_case parameters The MCP tool uses Python snake_case, **not** the camelCase params from the MC web UI: ``` ✓ created_after (not createdTime.after) ✓ created_before (not createdTime.before) ✓ order_by (not orderBy) ✓ table_mcons (not tableMcons) ``` Always provide `created_after` and `created_before`. Max window is 60 days. Pass ISO 8601 timestamps computed from the current date — e.g. for a 7-day window ending now: `created_after="2026-07-03T00:00:00Z"`, `created_before="2026-07-10T00:00:00Z"` (use the actual current date). --- ## `search` — finding the right table identifier MC uses MCONs (Monte Carlo Object Names) as table identifiers. Always use `search` first to resolve a table name to its MCON before calling `getTable`, `getAssetLineage`, or `getAlerts`. ``` search(query="orders_status") → returns mcon, full_table_id, warehouse ``` -
TROUBLESHOOTING.md 1.5 KB
## Troubleshooting ### MCP connection fails: ```bash # Verify the server is reachable curl -s -o /dev/null -w "%{http_code}" https://mcp.getmontecarlo.com/mcp/toolkit ``` **If using the plugin (OAuth):** Run `/mcp` in Claude Code, select the `monte-carlo-mcp` server, and re-authenticate. If the browser flow doesn't complete, copy the callback URL from your browser's address bar into the URL prompt that appears in Claude Code. **Legacy (header-based auth, for MCP clients without HTTP transport):** Check that `x-mcd-id` and `x-mcd-token` are set correctly in your MCP config. The key format is `<KEY_ID>:<KEY_SECRET>` — these are split across two separate headers. ### Monitor creation errors: **`montecarlo monitors apply` fails with "Unknown field":** Monitor definition files must have `montecarlo:` as the root key — do not copy the `validation:` or `custom_sql:` output from the MCP tools directly. Reformat using the `montecarlo: > custom_sql:` structure shown in Workflow 5. **`montecarlo monitors apply` fails with "Not a Monte Carlo project":** Ensure `montecarlo.yml` (the project config) exists in the working directory. This file must contain only `version`, `namespace`, and `default_resource` — not monitor definitions. **`createValidationMonitorMac` fails with a Snowflake error:** This tool validates the condition SQL against the live table. If the column doesn't exist yet (e.g. you're writing the monitor before deploying the model change), fall back to `createCustomSqlMonitorMac` with an explicit SQL query instead. -
workflows.md 36 KB
# Workflow Details Detailed step-by-step instructions for each Monte Carlo Prevent workflow. These are referenced from the main SKILL.md — consult the relevant section when executing a workflow. ## TodoWrite labels (applies to every workflow) When tracking progress through any workflow or sub-workflow with TodoWrite, use plain-English step labels — **not** internal workflow numbers like "W1" or "W4.2". Examples: - ✅ "Fetch asset health and downstream lineage" - ❌ "W1: Asset health pre-fetch" Internal numbering exists so skill authors reading this file can cross- reference; keep it out of user-visible UI. --- ## Workflow 1: Asset health pre-fetch (silent delegation) **Trigger:** The user expresses change intent. Workflow 1 only ever runs as a precursor to Workflow 2 — it does not run on bare file mentions or general "how is X doing" questions. Those go directly to `monte-carlo-asset-health` via its own activation rules. **Goal:** Gather Monte Carlo context (health, lineage, alerts, monitors) for the table being changed so Workflow 2 can incorporate it into the change-focused impact assessment. The report itself is data for W2 — not a separate user-facing artifact. ### Sequence 1. Invoke the `monte-carlo-asset-health` skill via the Skill tool. Pass the table name. Wait for the full health report. 2. Do **not** duplicate any of the MCP calls asset-health makes (`get_table`, `get_alerts`, `get_asset_lineage` upstream-only, `get_monitors`). Asset-health is the source of truth for those. 3. Asset-health only fetches **upstream** lineage. To complete the picture for Workflow 2's blast-radius synthesis, make one additional direct call: ``` get_asset_lineage(mcons=["<mcon resolved by asset-health>"], direction="DOWNSTREAM") ``` Use the MCON asset-health already resolved — do **not** re-call `search()`. If asset-health surfaced a disambiguation prompt and the engineer hasn't chosen yet, wait — do not run the downstream call until the MCON is fixed. 4. **Do NOT print, summarize, paraphrase, or relay asset-health's report.** Asset-health returns a long Markdown report (Health Check tables, monitor lists, recommendations) — that report is **internal data for prevent**, not user-facing output. Treat it the same way you would treat a raw MCP tool result: read it into context, then move on without echoing it. 5. Two exceptions where you **must** surface W1 output to the engineer: - **Disambiguation prompt.** If asset-health returns multiple matches, surface that question and wait for the answer before continuing. - **Stop-the-world signals.** If the table is already on fire (active critical alerts firing, freshness severely stale), say so in one short line before W2 begins. One line — not the full asset-health report. 6. **Immediately proceed to Workflow 2.** Do not pause, do not ask the engineer if they want to continue, do not summarize what W1 found. The user-facing artifact is W2's impact-assessment report, not asset-health's report. W1 is incomplete until W2 has been presented. ### What Workflow 1 does NOT do - Does not call MCP tools other than the single `get_asset_lineage(direction="DOWNSTREAM")` call in step 3. Everything else comes via asset-health. - Does not run standalone. W1 only fires as part of the W1 → W2 chain. **W1 finishing without W2 running is a workflow failure** — always continue to W2. - Does not produce a user-facing report. Asset-health's "Health Check" Markdown is data, not output. The user-facing artifact is W2's report. - Does not stop and wait for the engineer to confirm before W2. The transition W1 → W2 is automatic. - Does not handle new-model creation. Prevent's mission is preventing dangerous changes to existing models. If the engineer is authoring a brand-new model and wants to verify upstream health, that is a `monte-carlo-asset-health` question on each upstream — not a prevent workflow. --- ## Workflow 2: Change impact assessment — REQUIRED before modifying a model **Trigger:** Any expressed intent to add, rename, drop, or change a column, join, filter, or model logic. Run this immediately — before writing any code — even if the user hasn't asked for it. ### Bugfixes and reverts require impact assessment too When the user says "fix", "revert", "restore", or "undo", run this workflow before writing any code — even if the change seems small or safe. A revert that undoes a column addition or changes join logic has the same blast radius as the original change. Downstream models may have already adapted to the "incorrect" behavior, meaning the fix itself could break them. Pay special attention to: - Whether the revert removes a column other models now depend on - Whether downstream models reference the specific logic being reverted - Whether active alerts may be related to the change being reverted When the user is about to rename or drop a column, change a join condition, alter a filter, or refactor a model's logic, run this sequence to surface the blast radius before any changes are committed: **Data sources:** If asset-health (Workflow 1) ran for this table earlier in the session, reuse its lineage / alerts / monitors / table metadata. Do not re-fetch via MCP — the data is the same. If the asset-health report is stale (older than this turn's edit context) or covered a different table, re-invoke asset-health rather than running impact assessment on partial data. If asset-health did not run (the engineer invoked impact assessment directly, without a prior file-open trigger), call MCP tools yourself in this order: ``` 1. search(query="<table_name>") → list of candidate MCONs across MC connections. If multiple results are returned, present them in a table (full_table_id, warehouse, importance, key-asset flag) and ask the engineer which one to assess. Do not pick one automatically. Once they choose, call getTable(mcon="<mcon>") for that single MCON. → importance score, query volume (reads/writes per day), key asset flag 2. getAssetLineage(mcon="<mcon>") → full list of downstream dependents; for each, note whether it is a key asset 3. getTable(mcon="<downstream_mcon>") for each key downstream asset → importance score, last updated, monitoring status 4. getAlerts( created_after="<7 days ago>", created_before="<now>", table_mcons=["<mcon>", "<downstream_mcon_1>", ...], statuses=["NOT_ACKNOWLEDGED"] ) → any active incidents already affecting this table or its dependents 5. getQueriesForTable(mcon="<mcon>") → recent queries; scan for references to the specific columns being changed → use getQueryData(query_id="<id>") to fetch full SQL for ambiguous cases 5b. Supplementary local search for downstream dbt refs: - Search the local models/ directory for ref('<table_name>') (single-hop only) - Compare results against getAssetLineage output from step 2 - If any local models reference this table but are NOT in MC's lineage results: "⚠️ Found N local model(s) referencing this table not yet in MC's lineage: [list]" - If no models/ directory exists in the current project, skip silently - MC lineage remains the authoritative source — local grep is supplementary only 6. getMonitors(mcon="<mcon>") → which monitors are watching columns or metrics affected by the change ``` ### Risk tier assessment | Tier | Conditions | |---|---| | 🔴 High | Key asset downstream, OR active alerts already firing, OR >50 reads/day | | 🟡 Medium | Non-key assets downstream, OR monitors on affected columns, OR moderate query volume | | 🟢 Low | No downstream dependents, no active alerts, low query volume | ### Multi-model changes When the user is changing multiple models in the same session or same domain (e.g., 3 timeseries models, 4 criticality_score models): - Run a single consolidated impact assessment across all changed tables - Deduplicate downstream dependents — if two changed tables share a downstream dependent, count it once and note that it's affected by multiple upstream changes - Present a unified blast radius report rather than N separate reports - Escalate risk tier if the combined blast radius is larger than any individual table Example consolidated report header: "## Change Impact: 3 models in timeseries domain Combined downstream blast radius: 28 tables (deduplicated) Highest risk table: timeseries_detector_routing (22 downstream refs)" ### Report format ``` ## Change Impact: <table_name> Risk: 🔴 High / 🟡 Medium / 🟢 Low Downstream blast radius: - <N> tables depend on this model - Key assets affected: <list or "none"> Active incidents: - <alert title, status> or "none" Column exposure (for columns being changed): - Found in <N> recent queries (e.g. <query snippet>) Monitor coverage: - <monitor name> watches <metric> — will be affected by this change - If zero custom monitors exist → append: "⚠️ No custom monitors on this table. After making your changes, I'll suggest a monitor for the new logic — or say 'add a monitor' to do it now." Recommendation: - <specific callout, e.g. "Notify owners of downstream_table before deploying", "Coordinate with the freshness alert owner", "Add a monitor for the new column"> ``` If risk is 🔴 High: 1. Call `getAudiences()` to retrieve configured notification audiences 2. Include in the recommendation: "Notify: <audience names / channels>" 3. Proactively suggest: - Notifying owners of downstream key assets manually via the audience channels listed above (alert mutation is handled by `monte-carlo-incident-response`) - Adding a monitor for the new logic before deploying (Workflow 5) - Running `montecarlo monitors apply --dry-run` after changes to verify nothing breaks ### Synthesis: translate findings into code recommendations After presenting the impact report, use the findings to shape your code suggestion. Do not present MC data and then write code as if the data wasn't there. Explicitly connect each key finding to a specific recommendation: - Active alerts firing on the table: → Recommend deferring or minimally scoping the change until alerts are resolved → Explain: "There are N active alerts on this table — making this change now risks compounding an existing data quality issue" - Key assets downstream: → Recommend defensive coding patterns: null guards, backward-compatible changes, additive-only schema changes where possible → Explain: "X downstream key assets depend on this table — I'd recommend writing this as [specific pattern] to avoid breaking [specific dependent]" - Monitors on affected columns: → Call out that the change will affect monitor coverage → Recommend updating monitors alongside the code change (offer Workflow 5) → Explain: "The existing monitor on [column] will need to be updated to account for this change" - New output column or logic being added: → Always offer Workflow 5 after the impact assessment, regardless of existing monitor coverage → Do not skip this step even if risk tier is 🟢 Low → Say explicitly: "This adds new output logic — would you like me to generate a monitor for it? I can add a null check, range validation, or custom SQL rule." → Wait for the user's response before proceeding with the edit - High read volume (>50 reads/day): → Recommend extra caution around column renames or removals → Suggest backward-compatible transition (add new column, deprecate old one) → Explain: "This table has [N] reads/day — a column rename without a transition period would break downstream consumers immediately" - Column renames, even inside CTEs: → Never assume a CTE-internal rename is safe. Always check: 1. Does this column appear in the final SELECT, directly or via a CTE that feeds into the final SELECT? 2. If yes — treat as a breaking change. Recommend a backward-compatible transition: add the correctly-named column, keep the old one temporarily, remove in a follow-up PR. 3. If truly internal and never surfaces in output — confirm this explicitly before proceeding. → Explain: "Even though this column is defined in a CTE, if it surfaces in the final SELECT it is a public output column — renaming it breaks any downstream model selecting it by name." --- --- ## Workflow 3: Change validation queries — after a code change is made **Trigger:** Explicit engineer intent only. Activate when the engineer says something like: - "generate validation queries", "validate this change", "I'm done with this change" - "let me test this", "write queries to check this", "ready to commit" **Required session context — do not activate without both:** 1. Workflow 2 (change impact assessment) has run for this table in this session 2. A file edit was made to a `.sql` or dbt model file for that same table **Do NOT activate automatically after file edits. Do NOT proactively offer after Workflow 2 or file edits. The engineer asks when they are ready.** --- ### What this workflow does Using the context already in the session — the Workflow 2 findings, the file diff, and the `getTable` result — generate 3–5 targeted SQL validation queries that directly test whether this specific change behaved as intended. These are not generic templates. Use the semantic meaning of the change from Workflow 2 context: which columns changed and why, what business logic was affected, what downstream models depend on this table, and what monitors exist. A null check on a new `days_since_contract_start` column should verify it is never negative and never null for rows with a `contract_start_date` — not just check for nulls generically. --- ### Step 1 — Identify the change type from session context From Workflow 2 findings and the file diff, classify the primary change. A change may span multiple types — classify the dominant one and note secondaries: - **New column** — a new output column was added to the SELECT - **Filter change** — a WHERE clause, IN-list, or CASE condition was modified - **Join change** — a JOIN condition or join target was modified - **Column rename or drop** — an existing output column was renamed or removed - **Parameter change** — a hardcoded threshold, constant, or numeric value was changed - **New model** — the file was newly created, no production baseline exists --- ### Step 2 — Determine warehouse context from Workflow 2 From the `getTable` result already in session context, extract: - **Fully qualified table name** — e.g. `analytics.prod_internal_bi.client_hub_master` - **Warehouse type** — Snowflake, BigQuery, Redshift, Databricks - **Schema** — already resolved, do not re-derive Use the correct SQL dialect for the warehouse type. Key differences: | Warehouse | Date diff | Current timestamp | Notes | |---|---|---|---| | Snowflake | `DATEDIFF('day', a, b)` | `CURRENT_TIMESTAMP()` | `QUALIFY` supported | | BigQuery | `DATE_DIFF(a, b, DAY)` | `CURRENT_TIMESTAMP()` | Use subquery instead of `QUALIFY` | | Redshift | `DATEDIFF('day', a, b)` | `GETDATE()` | | | Databricks | `DATEDIFF(a, b)` | `CURRENT_TIMESTAMP()` | | For the dev database, use the placeholder `<YOUR_DEV_DATABASE>` with a comment instructing the engineer to replace it. Do not guess the dev database name. --- ### Step 3 — Apply database targeting rules (mandatory) These rules are not negotiable — violating them produces queries that will fail at runtime: - **Columns or logic that only exist post-change** → dev database only. Never query production for a column that doesn't exist there yet. - **Comparison queries (before vs after)** → both production and dev databases - **New model (no production baseline)** → dev database only for all queries - **Row count comparison** → always include, always query both databases --- ### Step 4 — Generate targeted validation queries Always include a row count comparison regardless of change type — it's the baseline signal that something unexpected happened. Then generate change-specific queries based on what needs to be validated for this change type. Use the exact conditions, column names, and business logic from the diff and Workflow 2 findings — not generic placeholders. The goal for each change type: **New column:** Verify the column is non-null where it should be non-null (based on its business meaning), that its value range is plausible, and that its distribution makes sense given the underlying data. Query dev only. **Filter change:** Verify that only the intended rows were reclassified — generate a before/after count showing how many rows were added or removed by the new condition using the exact filter logic from the diff, and a sample of the rows that changed classification. The sample helps the engineer confirm the right records moved. **Join change:** Verify that the join didn't introduce duplicates — a uniqueness check on the join key is essential. Also verify row count didn't change unexpectedly. Query dev for uniqueness, both databases for row count. **Column rename or drop:** Verify the old column name is absent and the new column (if renamed) is present in the dev schema. Also verify that downstream models referencing the old column name are identified — use the local ref() grep results from Workflow 2 if available. **Parameter or threshold change:** Verify the distribution of values affected by the change — how many rows moved above or below the new threshold, and whether the count matches the engineer's expectation. Query both databases to compare before and after. **New model:** No production comparison possible. Verify row count is non-zero and plausible, sample rows look correct, and key columns are non-null. Query dev only. --- ### Step 5 — Add change-specific context to each query For every query, include a SQL comment block that explains: - What the query is checking - What a healthy result looks like **for this specific change** - What would indicate a problem Derive this context from Workflow 2 findings. Use the business meaning of the change, not generic descriptions. For example, for adding `days_since_contract_start`: ```sql /* Null rate check: days_since_contract_start (new column, dev only) What to look for: - Null count should equal workspaces with no contract_start_date - All rows with contract_start_date should have a non-null, non-negative value - Values above 3650 (~10 years) are suspicious and may indicate a data issue */ ``` This is what differentiates these queries from generic validation — the comment tells the engineer exactly what pass and fail look like for their specific change. --- ### Step 6 — Save to local file Save all generated queries to: ``` validation/<table_name>_<YYYYMMDD_HHMM>.sql ``` Include a header at the top of the file: ```sql /* Validation queries for: <fully_qualified_table> Change type: <change type from Step 1> Generated: <timestamp> Workflow 2 risk tier: <tier from this session> Instructions: 1. Replace <YOUR_DEV_DATABASE> with your personal or branch database 2. Run the row count comparison first 3. Run change-specific queries to validate intended behavior 4. Unexpected results should be investigated before merging */ ``` Then tell the engineer: > "Validation queries saved to `validation/<table_name>_<timestamp>.sql`. > > What's next? Pick one: > - Say **continue** (or **yes**) — I'll run `/mc-validate run` for you (build + execute). > - Run `/mc-validate run` yourself — same as above. > - Run `/mc-validate run --skip-build` if you've already built the model and only want me to execute the queries. > - Run them manually: replace `<YOUR_DEV_DATABASE>` in the file and execute in Snowflake or your SQL client." **Always end Workflow 3 with the `/mc-validate run` offer**, regardless of how Workflow 3 was triggered (auto-activated or explicitly invoked). For YAML/docs-only diffs where no SQL validation is useful, skip query generation entirely and tell the engineer: "YAML-only diff; no SQL validation needed. `dbt test --select <model>` can still exercise newly-added schema tests." --- ### What this workflow does NOT do - Does not execute queries (Phase 2) - Does not require warehouse MCP connection - Does not generate Monte Carlo notebook YAML - Does not trigger automatically — only on explicit engineer request - Does not activate if Workflow 2 has not run for this table in this session --- ## Workflow 4: Validate change in sandbox — invoked by `/mc-validate run` **Trigger:** `/mc-validate run` (never automatic). **Required session context:** Workflow 3 has produced a `validation/<table>_<ts>.sql` for at least one changed model. ### Goal Build the changed model(s) into the engineer's dev database (W4.1), then substitute the dev-database placeholder in the generated queries, verify they are read-only, execute them via the Snowflake MCP, and present per-query verdicts plus a consolidated summary (W4.2). ### Pre-flight: validation queries must already exist `/mc-validate run` does **not** generate queries. Before any other step, verify at least one `validation/<table>_<ts>.sql` exists for the current session's changed models. If none exist, abort with: > "No validation queries found for <table_name>. Run `/mc-validate` (or > `/mc-validate generate`) first to generate them, then re-run > `/mc-validate run`." This applies to both `run` and `run --skip-build`. Auto-generating from `run` would silently mask the missing artifact and could surprise the engineer with queries they haven't reviewed. ### Invocation matrix | Invocation | Runs W4.1 (Build)? | Runs W4.2 (Execute)? | |---|---|---| | `/mc-validate run` | yes | yes | | `/mc-validate run --skip-build` | no | yes | | `/mc-validate run --dev-db <NAME>` | yes | yes (uses `<NAME>` directly, skips dev-db prompt) | For YAML/docs-only diffs W4.1 is automatically skipped (see W4.1 step 5); W4.2 still runs. --- ### Workflow 4.1: Build (materialize changed models into sandbox) **Trigger:** `/mc-validate run` without `--skip-build`. **Required session context:** Workflow 3 has produced a `validation/<table>_<ts>.sql` for at least one changed model. (Verified by the W4 pre-flight above — W4.1 itself does not re-check.) #### Goal Build the changed model(s) into the engineer's dev database with `dbt build --select <model>` so validation queries have something real to read from. Skip automatically for YAML/docs-only diffs. #### Sequence 0. **Pre-flight check (prerequisites).** Before any other step, verify: - `dbt` is installed and a `dbt_project.yml` is discoverable from the changed model. - The Snowflake MCP server is available in this session — look for any tool whose name starts with `mcp__snowflake__`. If absent, abort with: "Snowflake MCP is not registered in this session. `/mc-validate run` requires the Snowflake MCP server — see the prevent skill README's prerequisites. Aborting before the build so no work is lost." - `profiles.yml` exists where step 1 expects it. These are listed in `skills/prevent/README.md` under "`/mc-validate run` prerequisites". Failing fast here is much friendlier than failing deep inside `dbt build` or partway through query execution. 1. **Find `profiles.yml`.** Check `~/.dbt/profiles.yml` first, then the dbt project root (which is typically `analytics/` in MC's `dbt` repo — detect the same way `generate-validation-notebook` does, by walking up from the changed model file until a `dbt_project.yml` is found). 2. **Resolve the active target** using the sandbox script: ```bash python3 scripts/sandbox/parse_profiles.py <profiles.yml> ``` On error (missing file, unparseable YAML, unresolvable target), skip this step and ask the engineer for their dev database directly. 3. **Classify the resolved database:** ```bash python3 scripts/sandbox/classify_sandbox.py <database> ``` Categories: `personal`, `dev`, `shared-dev`, `prod`, `unknown`. 4. **Detect hard-coded `database:` in the model config:** ```bash python3 scripts/sandbox/detect_hardcoded_db.py <model.sql> ``` If a value is returned, surface it to the engineer and use it in place of the profile's database for this model. Warn that the build will land in the hard-coded location regardless of their profile. 5. **Decide whether to build** (diff-aware): - If the session diff is **YAML / markdown / docs only** → skip the build, note "no rebuild needed for YAML-only change," continue to Workflow 4.2. - Otherwise → show the resolved target context from step 2 (and the hard-coded `database:` from step 4, if any) and prompt for explicit confirmation. Never proceed on assumed defaults. ``` About to run: dbt build --select <model> Target: <target_name> (profile: <profile>) Warehouse: <warehouse> Database: <database> [hard-coded in model: <hardcoded_db>] Schema: <schema> Role: <role> Account: <account> Classification: <personal|dev|shared-dev|prod|unknown> Proceed? [y/N] ``` Default is **No**. Any answer other than an explicit `y`/`yes` aborts the build. Omit fields that `parse_profiles.py` returned as null; show the hard-coded-database note only when step 4 found one. 6. **Hard-stop for prod classification.** If the classifier returned `prod` (or the hard-coded `database:` value classifies as `prod`), **refuse regardless of the engineer's answer at step 5**: "Target resolves to shared prod. Aborting the build. Please fix your profiles.yml and re-run." Do not proceed to step 7. 7. **Execute the build.** Run from the dbt project root: ```bash dbt build --select <model> ``` For multiple models, pass them in one invocation: `--select m1 m2 m3`. Do not add `--full-refresh` or `+<model>` unless the engineer explicitly asked. Stream stdout to the user. 8. **Handle test failures.** `dbt build` may succeed the run phase but fail tests. Treat this as a soft block and prompt: ``` ✓ run succeeded ✗ N of M tests failed: <failing test names> Tests failed. Run validation queries anyway? [y/N] ``` 9. **Emit a session marker** on success (or on skip, with reason): ``` <!-- MC_BUILD_RAN: <table_name> --> ``` #### What this workflow does NOT do - Does not run `dbt run-operation`. If the engineer asks, refuse and instruct them to run it manually. - Does not auto-add `--full-refresh` or `+<model>` cascades. - Does not attempt to recover from `dbt debug` / connection failures; surface the error and stop. --- ### Workflow 4.2: Execute validation queries **Trigger:** `/mc-validate run` (with or without `--skip-build`). **Required session context:** Workflow 3 has produced a `validation/<table>_<ts>.sql`, and Workflow 4.1 has either completed, been explicitly skipped with `--skip-build`, or been no-op'd for a YAML-only diff. #### Goal Substitute the `<YOUR_DEV_DATABASE>` placeholder in the generated queries with a user-confirmed value, verify each query is read-only, execute them via the Snowflake MCP, and present per-query verdicts plus a consolidated summary. #### Sequence 1. **Propose a dev-database value.** If Workflow 4.1 resolved a database from `profiles.yml` (step 2–4 of W4.1), reuse that value. Otherwise, the engineer either passed `--dev-db <NAME>` or has not provided one — in which case prompt for it. 2. **Show the execution plan and require confirmation.** Scan the generated SQL for fully-qualified references and list every database that will be touched, so the engineer can see exactly where queries will run: ``` Execution plan: Dev database (from profiles.yml target 'prod'): → PERSONAL_ACHEN (classified: personal sandbox ✓) Other databases referenced literally in queries: → analytics (used in N query) Proceed? [Y / type new dev database / cancel] ``` Advisory text varies by classification: - `personal` / `dev` / `shared-dev` → `(classified: personal sandbox ✓)` etc. - `prod` → `⚠ classified as prod — this doesn't look like a dev database` - `unknown` → `(unrecognized — is this your dev database?)` If the engineer types a new value, re-classify it and re-confirm before continuing. 3. **Substitute placeholders:** ```bash python3 scripts/sandbox/substitute_placeholders.py \ validation/<table>_<ts>.sql --dev-db <CONFIRMED_DEV_DB> ``` This writes `validation/run/<table>_<ts>.run.sql` (the script creates the `run/` subdirectory if it doesn't exist) and reports the count of substitutions + the list of literal databases found. **All execution-time scratch output lives under `validation/run/`** so the main `validation/` directory stays clean with just the human-facing `.sql` generated by Workflow 3. 4. **Read-only pre-check** (mandatory): ```bash python3 scripts/sandbox/readonly_check.py \ validation/run/<table>_<ts>.run.sql ``` If the script exits non-zero, **abort execution**. Report the rejected keyword and the query it came from. Do not send anything to Snowflake MCP. **If the script exits zero, tell the engineer explicitly** — the point of this check is confidence, and a silent pass doesn't build it. Output one short line before step 5: > ✅ Read-only pre-check passed — N queries verified SELECT-only, no writes can reach Snowflake. 5. **Show the final SQL** (per query, as a fenced SQL block) to the engineer before sending to Snowflake MCP. This is the last point at which they can cancel. Having seen the ✓ from step 4 plus the exact SQL here, the engineer has everything they need to press proceed with confidence. 6. **Execute each query via Snowflake MCP** (e.g. the `mcp__snowflake__query` tool — confirm the exact tool name available in the session). Apply a 60s per-query timeout by default. On error (including timeout), continue with remaining queries but mark the failed one. **Prefer splitting queries in memory.** Read `validation/run/<table>_<ts>.run.sql` once, split the queries in memory (they're separated by blank lines between top-level statements; each query is preceded by a `/* ... */` comment block containing its name and "What to look for" guidance), and pass each query string directly to the Snowflake MCP tool. The comment is metadata for the verdict in step 7, not a file to write. **If you must write per-query scratch files** (e.g. because your MCP client only accepts file paths), put them under `validation/run/` alongside the `.run.sql` — never at the top level of `validation/`. The `validation/run/` directory is understood to be transient and safe to gitignore; top-level `validation/` is the durable human-facing artifact directory. 7. **Report per-query verdicts** using the "What to look for" comment block attached to each query in the generated `.sql`: - Print a short human heading per query (e.g. "Row count comparison: prod vs dev"). - Show the result as a compact table (cap 20 rows; truncate with a note above 20). - **Wide tables:** if the result has more than 12 columns, project to just the columns named in the query's `/* What to look for */` comment block (plus any obvious key columns like the join key or primary id). Mention the omission in the verdict line — e.g. "showing 6 of 84 columns; full result available by re-running the query directly." This keeps `SELECT *` against a 100-column table from burying the signal under width. - Emit one of `✅` / `⚠️` / `🔴` with a one-line reason grounded in the "What to look for" comment — do not invent "healthy" from nothing. - For `⚠️` and `🔴`, add a follow-up hint. 8. **Consolidated summary** at the end. Use one of the templates below verbatim for the final line — they're scoped to "what these queries checked," nothing more: - **All pass:** `Overall: N of N checks pass. No issues surfaced by the validation queries.` - **Mixed / failures:** `Overall: M of N checks pass. K warning(s)/failure(s) worth investigating — see the verdicts above.` Example (mixed): ``` ## Validation summary: <model> ✅ Row count comparison ✅ Sample data preview ⚠️ Null rate on days_since_contract_start (4.2% null in dev — expected ~0%) ✅ Core segmentation counts ✅ Uniqueness check on account_id Overall: 4 of 5 checks pass. 1 warning worth investigating — see the verdicts above. ``` **Do not state or imply a merge verdict** — no "safe to merge", "ready to ship", "looks good to ship", or similar phrases. The merge decision belongs to the engineer. 9. **Emit a session marker** per model after execution: ``` <!-- MC_VALIDATE_RAN: <table_name> --> ``` 10. **Feed 🔴 verdicts back to the change context.** If any query verdict was 🔴 (and only then), add a closing line to the consolidated summary that invites the engineer to revisit the change before merging: > "One or more checks failed. If these results suggest the change isn't > behaving as intended, consider re-running Workflow 2 (change impact > assessment) with the failing-query summary as additional input — that > can surface whether the failure is downstream-relevant — before > revising the code." Do not auto-invoke Workflow 2. Surface the option and let the engineer decide. This closes the loop between W4.2 (what the data says) and W2 (what to do about the change). For `⚠️`-only results, note them but don't suggest re-running W2 — yellow signals usually warrant investigation, not a re-assessment. #### Multi-model behavior If Workflow 3 generated files for multiple models, run steps 3–8 per model with its own substituted file and its own verdicts section. Finish with a top-level summary listing one status line per model. #### What this workflow does NOT do - Does not execute any statement that isn't read-only (rejected by step 4). - Does not guess a dev database when `profiles.yml` / `--dev-db` don't provide one — it asks. - Does not fall back to any execution path other than Snowflake MCP unless the MCP is unavailable, in which case it leaves the substituted `.run.sql` on disk and tells the engineer to run it manually. - Does not re-run queries — each invocation is a fresh execution of all queries in the current file. --- ## Workflow 5: Add monitor (delegated, post-edit) **Trigger:** *Never auto-invoked from a file-open or table-mention trigger.* W5 fires only when: 1. The post-edit / turn-end hook injects the monitor-coverage prompt — driven by the `MC_MONITOR_GAP` marker emitted during Workflow 2 — **or** 2. The engineer explicitly asks to add a monitor for the just-edited model (e.g. "add a monitor", "create a monitor for X"). **Required session context:** Workflow 2 has run for the model and identified a coverage gap, *or* the engineer is explicitly requesting monitor generation. ### Sequence 1. Ask the engineer: > "Generate monitor definitions for the new logic? (yes/no)" 2. On **no** → stop. The post-edit hook has already cleared the gap state; no further action. 3. On **yes** → invoke the `monte-carlo-monitoring-advisor` skill via the Skill tool. Pass: - The model name. - The specific columns / logic that changed (from the Workflow 2 synthesis output). 4. **Prevent's responsibility ends at the moment delegation fires.** Do not wait for monitoring-advisor to finish, do not emit any completion marker, do not insert any post-step. Monitor generation can take a while; prevent should not block on it. ### Re-edit behavior If the engineer edits the same model again, the pre-edit gate forces Workflow 2 to re-run, which re-evaluates monitor coverage via `get_monitors`. If the generated monitors now cover the changed columns, Workflow 2 will not re-emit `MC_MONITOR_GAP` — the gap is genuinely closed. If a fresh gap exists, Workflow 2 re-emits the marker and the post-edit hook prompts again. Self-healing — no explicit "already generated" tracking needed. ### What this workflow does NOT do - Does not generate monitor YAML itself. All generation is done by monitoring-advisor. - Does not modify the `monte-carlo-monitoring-advisor` skill in any way. - Does not emit `MC_MONITOR_GENERATED` or any other completion marker.
-
-
scripts
-
sandbox
-
tests
-
conftest.py 256 B
"""Shared fixtures for sandbox script tests.""" import sys from pathlib import Path # Make sandbox scripts importable for tests that want to call functions directly. _SANDBOX_DIR = Path(__file__).resolve().parents[1] sys.path.insert(0, str(_SANDBOX_DIR)) -
test_classify_sandbox.py 1.1 KB
"""Tests for classify_sandbox.py.""" import json import subprocess import sys from pathlib import Path import pytest SCRIPT = Path(__file__).resolve().parents[1] / "classify_sandbox.py" def _run(name: str) -> dict: result = subprocess.run( [sys.executable, str(SCRIPT), name], capture_output=True, text=True, check=True, ) return json.loads(result.stdout) @pytest.mark.parametrize("name,expected", [ ("PERSONAL_ACHEN", "personal"), ("personal_alice", "personal"), ("DEV_PLATFORM", "dev"), ("SANDBOX_42", "dev"), ("MY_DEV", "dev"), ("DBT_ACHEN", "shared-dev"), ("ANALYTICS", "prod"), ("RAW", "prod"), ("INGEST", "prod"), ("MONTECARLODATA_SHARED", "prod"), ("FOO_BAR", "unknown"), ("", "unknown"), ]) def test_classify(name, expected): assert _run(name) == {"database": name, "classification": expected} def test_classify_importable(): """Function form works for direct calls from other scripts.""" from classify_sandbox import classify assert classify("PERSONAL_ACHEN") == "personal" assert classify("ANALYTICS") == "prod" -
test_detect_hardcoded_db.py 1.6 KB
"""Tests for detect_hardcoded_db.py.""" import json import subprocess import sys from pathlib import Path SCRIPT = Path(__file__).resolve().parents[1] / "detect_hardcoded_db.py" def _run(path: Path) -> dict: out = subprocess.run( [sys.executable, str(SCRIPT), str(path)], capture_output=True, text=True, check=True, ) return json.loads(out.stdout) def test_no_hardcoded_db(tmp_path): f = tmp_path / "m.sql" f.write_text("{{ config(materialized='table', schema='prod') }}\nSELECT 1\n") assert _run(f) == {"database": None} def test_hardcoded_db_single_quotes(tmp_path): f = tmp_path / "m.sql" f.write_text("{{ config(materialized='table', database='MONTECARLODATA_SHARED', schema='exports') }}\nSELECT 1\n") assert _run(f) == {"database": "MONTECARLODATA_SHARED"} def test_hardcoded_db_double_quotes(tmp_path): f = tmp_path / "m.sql" f.write_text('{{ config(database="shared") }}\n') assert _run(f) == {"database": "shared"} def test_no_config_block(tmp_path): f = tmp_path / "m.sql" f.write_text("SELECT 1\n") assert _run(f) == {"database": None} def test_config_spread_across_lines(tmp_path): f = tmp_path / "m.sql" f.write_text( "{{ config(\n" " materialized='incremental',\n" " database='EXPORTS_DB',\n" " unique_key='id'\n" ") }}\n" "SELECT 1\n" ) assert _run(f) == {"database": "EXPORTS_DB"} def test_missing_file(tmp_path): result = subprocess.run( [sys.executable, str(SCRIPT), str(tmp_path / "nope.sql")], capture_output=True, text=True, ) assert result.returncode == 1 -
test_integration_smoke.py 2.8 KB
"""End-to-end smoke: simulate the Workflow 4.1/4.2 script chain on the happy path.""" import json import subprocess import sys from pathlib import Path SANDBOX_DIR = Path(__file__).resolve().parents[1] PROFILES = """ default: target: prod outputs: prod: type: snowflake database: personal_alice schema: prod role: DATA_ANALYST warehouse: research account: dka87615.us-east-1 """ MODEL_SQL = """\ {{ config(materialized='table', schema='prod') }} SELECT 1 AS account_id """ VALIDATION_SQL = """\ -- validation queries SELECT 'dev' AS src FROM <YOUR_DEV_DATABASE>.prod.client_hub UNION ALL SELECT 'prod' AS src FROM analytics.prod.client_hub WHERE <YOUR_DEV_DATABASE>.prod.client_hub.account_id IS NOT NULL; """ def _run(script: str, *args) -> tuple[int, str, str]: out = subprocess.run( [sys.executable, str(SANDBOX_DIR / script), *args], capture_output=True, text=True, ) return out.returncode, out.stdout, out.stderr def test_happy_path(tmp_path): profiles = tmp_path / "profiles.yml" profiles.write_text(PROFILES) model = tmp_path / "client_hub.sql" model.write_text(MODEL_SQL) sql = tmp_path / "client_hub_20260423.sql" sql.write_text(VALIDATION_SQL) # 1. Parse profiles. code, out, err = _run("parse_profiles.py", str(profiles)) assert code == 0, err profile_info = json.loads(out) assert profile_info["database"] == "personal_alice" # 2. Classify. code, out, _ = _run("classify_sandbox.py", profile_info["database"]) assert code == 0 assert json.loads(out)["classification"] == "personal" # 3. No hard-coded database in the model. code, out, _ = _run("detect_hardcoded_db.py", str(model)) assert code == 0 assert json.loads(out)["database"] is None # 4. Substitute placeholders. Default output lives in validation/run/. code, out, _ = _run("substitute_placeholders.py", str(sql), "--dev-db", "personal_alice") assert code == 0 sub_info = json.loads(out) assert sub_info["replaced_count"] == 2 assert "analytics" in sub_info["literal_databases"] from pathlib import Path as _P assert _P(sub_info["output_path"]).parent == sql.parent / "run" # 5. Read-only check on the substituted output. code, out, _ = _run("readonly_check.py", sub_info["output_path"]) assert code == 0 assert json.loads(out)["ok"] is True def test_prod_classification_gets_flagged(tmp_path): # Engineer's profile is mis-pointed at real prod. profiles = tmp_path / "profiles.yml" profiles.write_text(PROFILES.replace("personal_alice", "ANALYTICS")) code, out, _ = _run("parse_profiles.py", str(profiles)) assert code == 0 db = json.loads(out)["database"] code, out, _ = _run("classify_sandbox.py", db) assert json.loads(out)["classification"] == "prod" -
test_parse_profiles.py 3.1 KB
"""Tests for parse_profiles.py.""" import json import subprocess import sys from pathlib import Path import pytest SCRIPT = Path(__file__).resolve().parents[1] / "parse_profiles.py" def _run(profiles_path: Path, profile_name: str | None = None, target_name: str | None = None) -> tuple[int, dict, str]: args = [sys.executable, str(SCRIPT), str(profiles_path)] if profile_name: args += ["--profile", profile_name] if target_name: args += ["--target", target_name] result = subprocess.run(args, capture_output=True, text=True) return result.returncode, json.loads(result.stdout or "{}"), result.stderr SINGLE_TARGET = """ default: target: prod outputs: prod: type: snowflake account: dka87615.us-east-1 user: alice@example.com role: DATA_ANALYST database: personal_alice schema: prod warehouse: research threads: 2 """ TWO_TARGET = """ default: target: dev outputs: dev: type: snowflake account: hda34492.us-east-1 database: prod schema: dbt_alice role: data_analyst warehouse: dev local_prod: type: snowflake account: dka87615.us-east-1 database: personal_alice schema: prod role: developer warehouse: research """ def test_single_target(tmp_path): path = tmp_path / "profiles.yml" path.write_text(SINGLE_TARGET) code, data, _ = _run(path) assert code == 0 assert data == { "profile": "default", "target_name": "prod", "database": "personal_alice", "schema": "prod", "role": "DATA_ANALYST", "warehouse": "research", "account": "dka87615.us-east-1", } def test_two_target_default_active(tmp_path): path = tmp_path / "profiles.yml" path.write_text(TWO_TARGET) code, data, _ = _run(path) assert code == 0 assert data["target_name"] == "dev" assert data["database"] == "prod" assert data["schema"] == "dbt_alice" def test_explicit_target_override(tmp_path): path = tmp_path / "profiles.yml" path.write_text(TWO_TARGET) code, data, _ = _run(path, target_name="local_prod") assert code == 0 assert data["target_name"] == "local_prod" assert data["database"] == "personal_alice" def test_missing_file(tmp_path): code, _, err = _run(tmp_path / "nope.yml") assert code == 1 assert "not found" in err.lower() def test_unparseable_yaml(tmp_path): path = tmp_path / "profiles.yml" path.write_text("not: [valid: yaml") code, _, err = _run(path) assert code == 1 assert "parse" in err.lower() or "yaml" in err.lower() def test_target_not_defined(tmp_path): path = tmp_path / "profiles.yml" path.write_text("default:\n target: ghost\n outputs:\n other:\n database: foo\n") code, _, err = _run(path) assert code == 1 assert "ghost" in err def test_profile_not_found(tmp_path): path = tmp_path / "profiles.yml" path.write_text(TWO_TARGET) code, _, err = _run(path, profile_name="nonexistent_profile") assert code == 1 assert "nonexistent_profile" in err assert "not found" in err.lower() -
test_readonly_check.py 3.9 KB
"""Tests for readonly_check.py.""" import json import subprocess import sys from pathlib import Path SCRIPT = Path(__file__).resolve().parents[1] / "readonly_check.py" def _run(path: Path) -> tuple[int, dict]: out = subprocess.run( [sys.executable, str(SCRIPT), str(path)], capture_output=True, text=True, ) return out.returncode, json.loads(out.stdout or "{}") OK_SINGLE_SELECT = "SELECT 1\n" OK_WITH_CTE = "WITH a AS (SELECT 1) SELECT * FROM a;\n" OK_SHOW = "SHOW TABLES IN SCHEMA prod;\n" OK_COMMENT_THEN_SELECT = "-- comment\n/* block */\nSELECT 1;\n" BAD_INSERT = "INSERT INTO t VALUES (1);\n" BAD_UPDATE = "-- lead comment\nUPDATE t SET x = 1;\n" BAD_MERGE = "merge into t using s on s.id = t.id when matched then update set x = 1;" BAD_CREATE = "CREATE TABLE x (id INT);\n" BAD_DROP = "DROP TABLE x;\n" BAD_CALL = "CALL sp_do_thing();\n" BAD_USE = "USE DATABASE raw;\nSELECT 1;\n" OK_MULTI_SELECT = "SELECT 1;\nSELECT 2;\nSELECT 3;\n" MIXED_WRITE_IN_MULTI = "SELECT 1;\nDROP TABLE x;\nSELECT 2;\n" def test_ok_select(tmp_path): f = tmp_path / "q.sql" f.write_text(OK_SINGLE_SELECT) code, data = _run(f) assert code == 0 assert data == {"ok": True, "rejected": None} def test_ok_with_cte(tmp_path): f = tmp_path / "q.sql" f.write_text(OK_WITH_CTE) assert _run(f) == (0, {"ok": True, "rejected": None}) def test_ok_show(tmp_path): f = tmp_path / "q.sql" f.write_text(OK_SHOW) assert _run(f) == (0, {"ok": True, "rejected": None}) def test_ok_comment_then_select(tmp_path): f = tmp_path / "q.sql" f.write_text(OK_COMMENT_THEN_SELECT) assert _run(f) == (0, {"ok": True, "rejected": None}) def test_rejects_insert(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_INSERT) code, data = _run(f) assert code == 1 assert data["ok"] is False assert data["rejected"] == "INSERT" def test_rejects_update(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_UPDATE) assert _run(f)[1]["rejected"] == "UPDATE" def test_rejects_merge_lowercase(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_MERGE) assert _run(f)[1]["rejected"] == "MERGE" def test_rejects_create(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_CREATE) assert _run(f)[1]["rejected"] == "CREATE" def test_rejects_drop(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_DROP) assert _run(f)[1]["rejected"] == "DROP" def test_rejects_call(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_CALL) assert _run(f)[1]["rejected"] == "CALL" def test_rejects_use(tmp_path): f = tmp_path / "q.sql" f.write_text(BAD_USE) assert _run(f)[1]["rejected"] == "USE" def test_accepts_multi_select(tmp_path): """Multiple SELECT statements in one file are fine — Workflow 4.2 output shape.""" f = tmp_path / "q.sql" f.write_text(OK_MULTI_SELECT) code, data = _run(f) assert code == 0 assert data == {"ok": True, "rejected": None} def test_rejects_write_among_multi_statement(tmp_path): """A write statement anywhere in the file still gets rejected, even if other statements are pure reads.""" f = tmp_path / "q.sql" f.write_text(MIXED_WRITE_IN_MULTI) code, data = _run(f) assert code == 1 assert data["rejected"] == "DROP" def test_rejects_get_stage(tmp_path): f = tmp_path / "q.sql" f.write_text("GET @stage FILE 'out.csv';\n") assert _run(f)[1]["rejected"] == "GET" def test_rejects_put(tmp_path): f = tmp_path / "q.sql" f.write_text("PUT file:///tmp/x.csv @stage;\n") assert _run(f)[1]["rejected"] == "PUT" def test_rejects_unload(tmp_path): f = tmp_path / "q.sql" f.write_text("UNLOAD ('SELECT 1') TO '@stage';\n") assert _run(f)[1]["rejected"] == "UNLOAD" def test_rejects_set_session_var(tmp_path): f = tmp_path / "q.sql" f.write_text("SET v = 1;\nSELECT 1;\n") assert _run(f)[1]["rejected"] == "SET" -
test_substitute_placeholders.py 3.7 KB
"""Tests for substitute_placeholders.py.""" import json import subprocess import sys from pathlib import Path SCRIPT = Path(__file__).resolve().parents[1] / "substitute_placeholders.py" def _run(sql_path: Path, dev_db: str) -> tuple[int, dict, str]: out = subprocess.run( [sys.executable, str(SCRIPT), str(sql_path), "--dev-db", dev_db], capture_output=True, text=True, ) return out.returncode, json.loads(out.stdout or "{}"), out.stderr SAMPLE = """-- Validation queries SELECT * FROM <YOUR_DEV_DATABASE>.prod.client_hub; SELECT 'dev' AS source, COUNT(*) AS rows FROM <YOUR_DEV_DATABASE>.prod.client_hub UNION ALL SELECT 'prod' AS source, COUNT(*) AS rows FROM analytics.prod.client_hub; """ def test_substitutes_and_reports_literals(tmp_path): src = tmp_path / "queries.sql" src.write_text(SAMPLE) code, data, _ = _run(src, "PERSONAL_ACHEN") assert code == 0 assert data["dev_db"] == "PERSONAL_ACHEN" assert data["replaced_count"] == 2 assert sorted(data["literal_databases"]) == ["analytics"] out_path = Path(data["output_path"]) contents = out_path.read_text() assert "<YOUR_DEV_DATABASE>" not in contents assert "PERSONAL_ACHEN.prod.client_hub" in contents assert "analytics.prod.client_hub" in contents def test_no_placeholders_present(tmp_path): src = tmp_path / "q.sql" src.write_text("SELECT * FROM analytics.prod.orders;\n") code, data, _ = _run(src, "PERSONAL_ACHEN") assert code == 0 assert data["replaced_count"] == 0 assert data["literal_databases"] == ["analytics"] def test_output_dir_is_run_subdir_by_default(tmp_path): src = tmp_path / "q.sql" src.write_text("SELECT 1 FROM <YOUR_DEV_DATABASE>.prod.t;\n") _, data, _ = _run(src, "DEV_X") out_path = Path(data["output_path"]) assert out_path.parent == src.parent / "run" assert out_path.parent.exists() assert out_path.name == "q.run.sql" def test_explicit_output_path_respected(tmp_path): src = tmp_path / "q.sql" src.write_text("SELECT 1 FROM <YOUR_DEV_DATABASE>.prod.t;\n") custom = tmp_path / "custom" / "elsewhere.sql" result = subprocess.run( [sys.executable, str(SCRIPT), str(src), "--dev-db", "DEV_X", "--output", str(custom)], capture_output=True, text=True, ) assert result.returncode == 0 data = json.loads(result.stdout) out_path = Path(data["output_path"]) assert out_path == custom assert out_path.exists() def test_missing_file(tmp_path): code, _, err = _run(tmp_path / "nope.sql", "PERSONAL_ACHEN") assert code == 1 assert "not found" in err.lower() def test_string_literal_db_ref_not_listed(tmp_path): """Regression: a db.schema.table inside a string literal must not be reported as a literal database reference. Such text is data, not a ref.""" src = tmp_path / "q.sql" src.write_text( "SELECT * FROM <YOUR_DEV_DATABASE>.prod.t " "WHERE meta = 'analytics.prod.client_hub';\n" ) code, data, _ = _run(src, "PERSONAL_ACHEN") assert code == 0 # The string-literal `'analytics.prod.client_hub'` is data; it must NOT # surface as a real cross-database reference. assert data["literal_databases"] == [] def test_string_literal_does_not_mask_real_ref(tmp_path): """A real cross-DB ref alongside a string-literal red herring should still be reported.""" src = tmp_path / "q.sql" src.write_text( "SELECT * FROM <YOUR_DEV_DATABASE>.prod.t " "JOIN analytics.prod.client_hub USING (id) " "WHERE meta = 'staging.foo.bar';\n" ) code, data, _ = _run(src, "PERSONAL_ACHEN") assert code == 0 assert data["literal_databases"] == ["analytics"]
-
-
classify_sandbox.py 1.5 KB
#!/usr/bin/env python3 """ Classify a database name as personal / dev / shared-dev / prod / unknown. Usage: python3 classify_sandbox.py <database_name> Prints JSON: {"database": "<name>", "classification": "<label>"} Exits 0 always; caller decides how to handle 'prod' / 'unknown'. Rules (uppercase-insensitive): personal -> starts with PERSONAL_ dev -> starts with DEV_ / SANDBOX_ / ends with _DEV shared-dev -> starts with DBT_ prod -> exact match of any of: ANALYTICS, RAW, INGEST, MONTECARLODATA_SHARED unknown -> everything else (empty string included) """ import argparse import json import sys PROD_NAMES = {"ANALYTICS", "RAW", "INGEST", "MONTECARLODATA_SHARED"} def classify(database: str) -> str: if not database: return "unknown" name = database.upper() if name in PROD_NAMES: return "prod" if name.startswith("PERSONAL_"): return "personal" if name.startswith("DEV_") or name.startswith("SANDBOX_") or name.endswith("_DEV"): return "dev" if name.startswith("DBT_"): return "shared-dev" return "unknown" def main() -> int: p = argparse.ArgumentParser( description="Classify a database name as personal / dev / shared-dev / prod / unknown." ) p.add_argument("database", help="Database name to classify") args = p.parse_args() print(json.dumps({"database": args.database, "classification": classify(args.database)})) return 0 if __name__ == "__main__": sys.exit(main()) -
detect_hardcoded_db.py 1.2 KB
#!/usr/bin/env python3 """ Detect a hard-coded `database='...'` kwarg inside a dbt model's `{{ config(...) }}` block. Usage: python3 detect_hardcoded_db.py <model.sql> Prints JSON: {"database": "<value>"} or {"database": null} Exits 1 if the file is missing. """ import argparse import json import re import sys from pathlib import Path _CONFIG_RE = re.compile(r"\{\{\s*config\s*\((.*?)\)\s*\}\}", re.DOTALL) _DB_KWARG_RE = re.compile(r"""\bdatabase\s*=\s*(['"])([^'"]+)\1""") def detect(content: str) -> str | None: for config_match in _CONFIG_RE.finditer(content): kwargs = config_match.group(1) db_match = _DB_KWARG_RE.search(kwargs) if db_match: return db_match.group(2) return None def main() -> int: p = argparse.ArgumentParser( description="Detect a hard-coded database='...' in a dbt model's config() block." ) p.add_argument("path", type=Path, help="Path to the dbt model .sql file") args = p.parse_args() if not args.path.exists(): print(f"error: file not found: {args.path}", file=sys.stderr) return 1 result = detect(args.path.read_text()) print(json.dumps({"database": result})) return 0 if __name__ == "__main__": sys.exit(main()) -
parse_profiles.py 3.3 KB
#!/usr/bin/env python3 """ Parse a dbt profiles.yml and emit the active target's resolved context. Usage: python3 parse_profiles.py <profiles.yml path> [--profile <name>] [--target <name>] On success prints JSON: { "profile": "default", "target_name": "prod", "database": "personal_alice", "schema": "prod", "role": "DATA_ANALYST", "warehouse": "research", "account": "dka87615.us-east-1" } On error: exits 1 with a human message on stderr; stdout is empty. """ import argparse import json import sys from pathlib import Path try: import yaml # type: ignore except ImportError: print("error: pyyaml not installed; run `pip3 install pyyaml`", file=sys.stderr) sys.exit(1) def _select_profile(doc: dict, name: str | None) -> tuple[str, dict]: """Return (profile_name, profile_doc). If *name* is given it must match a top-level profile key; otherwise the first profile in the file is selected. """ if not isinstance(doc, dict) or not doc: raise ValueError("profiles.yml has no profiles defined") first = next(iter(doc)) if name: if name in doc: return name, doc[name] raise ValueError(f"profile '{name}' not found in profiles.yml") return first, doc[first] def _select_target(profile: dict, target: str | None) -> tuple[str, dict]: outputs = profile.get("outputs") or {} if not outputs: raise ValueError("profile has no 'outputs' defined") target_name = target or profile.get("target") if not target_name: raise ValueError("profile has no 'target' and --target not given") if target_name not in outputs: raise ValueError(f"target '{target_name}' not defined in profile outputs") return target_name, outputs[target_name] def parse(profiles_path: Path, profile: str | None, target: str | None) -> dict: if not profiles_path.exists(): raise FileNotFoundError(f"profiles.yml not found: {profiles_path}") try: raw = yaml.safe_load(profiles_path.read_text()) except yaml.YAMLError as exc: raise ValueError(f"could not parse yaml: {exc}") from exc profile_name, profile_doc = _select_profile(raw, profile) target_name, target_doc = _select_target(profile_doc, target) return { "profile": profile_name, "target_name": target_name, "database": target_doc.get("database"), "schema": target_doc.get("schema"), "role": target_doc.get("role"), "warehouse": target_doc.get("warehouse"), "account": target_doc.get("account"), } def main() -> int: p = argparse.ArgumentParser( description="Parse a dbt profiles.yml and emit the active target's resolved context as JSON." ) p.add_argument("profiles_path", type=Path, help="Path to profiles.yml") p.add_argument("--profile", default=None, help="Profile name (default: first profile in file)") p.add_argument("--target", default=None, help="Target name (default: profile's 'target' field)") args = p.parse_args() try: result = parse(args.profiles_path, args.profile, args.target) except (FileNotFoundError, ValueError) as exc: print(f"error: {exc}", file=sys.stderr) return 1 print(json.dumps(result)) return 0 if __name__ == "__main__": sys.exit(main()) -
readonly_check.py 2.3 KB
#!/usr/bin/env python3 """ Verify a .sql file contains only read-only statements. Usage: python3 readonly_check.py <path.sql> Exit 0 with {"ok": true, "rejected": null} if safe. Exit 1 with {"ok": false, "rejected": "<KEYWORD>"} if not. Rejects any write-like keyword: INSERT, UPDATE, DELETE, MERGE, CREATE, DROP, TRUNCATE, ALTER, COPY, PUT, GET, LIST, REMOVE, UNLOAD, GRANT, REVOKE, CALL, EXECUTE, USE, SET. Multi-statement files (several SELECTs separated by `;`) are accepted — the keyword scan catches a rogue write statement regardless of how many statements share the file. The caller is expected to split statements in memory and send them to the warehouse one at a time. """ import argparse import json import re import sys from pathlib import Path REJECTED_KEYWORDS = [ # Most-specific first: MERGE before UPDATE so "MERGE ... UPDATE SET" reports MERGE. "INSERT", "DELETE", "MERGE", "UPDATE", "CREATE", "DROP", "TRUNCATE", "ALTER", "COPY", "PUT", "GET", "LIST", "REMOVE", "UNLOAD", "GRANT", "REVOKE", "CALL", "EXECUTE", "USE", "SET", ] _LINE_COMMENT_RE = re.compile(r"--[^\n]*") _BLOCK_COMMENT_RE = re.compile(r"/\*.*?\*/", re.DOTALL) def _strip_sql(src: str) -> str: """Remove comments and string literals so keyword scan is false-positive-free.""" no_block = _BLOCK_COMMENT_RE.sub(" ", src) no_line = _LINE_COMMENT_RE.sub(" ", no_block) no_strings = re.sub(r"'(?:[^'\\]|\\.)*'", "''", no_line) no_strings = re.sub(r'"(?:[^"\\]|\\.)*"', '""', no_strings) return no_strings def check(sql: str) -> tuple[bool, str | None]: cleaned = _strip_sql(sql) upper = cleaned.upper() for kw in REJECTED_KEYWORDS: if re.search(rf"\b{kw}\b", upper): return False, kw return True, None def main() -> int: p = argparse.ArgumentParser( description="Verify a .sql file contains only read-only statements." ) p.add_argument("path", type=Path, help="Path to the .sql file to check") args = p.parse_args() if not args.path.exists(): print(f"error: file not found: {args.path}", file=sys.stderr) return 1 ok, rejected = check(args.path.read_text()) print(json.dumps({"ok": ok, "rejected": rejected})) return 0 if ok else 1 if __name__ == "__main__": sys.exit(main()) -
substitute_placeholders.py 3.7 KB
#!/usr/bin/env python3 """ Substitute <YOUR_DEV_DATABASE> in a validation .sql file with a confirmed dev database and report any remaining literal fully-qualified database references. Usage: python3 substitute_placeholders.py <path.sql> --dev-db <NAME> [--output <path>] Writes the substituted SQL to `<input>.run.sql` by default (or `--output`) and prints JSON: { "output_path": "<path>", "dev_db": "<NAME>", "replaced_count": <int>, "literal_databases": ["analytics", ...] } Exits 1 if the input file is missing. """ import argparse import json import re import sys from pathlib import Path _PLACEHOLDER = "<YOUR_DEV_DATABASE>" _FQ_RE = re.compile( r"\b([A-Za-z_][A-Za-z0-9_]*)\.[A-Za-z_][A-Za-z0-9_]*\.[A-Za-z_][A-Za-z0-9_]*\b" ) _LINE_COMMENT_RE = re.compile(r"--[^\n]*") _BLOCK_COMMENT_RE = re.compile(r"/\*.*?\*/", re.DOTALL) _SINGLE_QUOTED_RE = re.compile(r"'(?:[^'\\]|\\.)*'") _DOUBLE_QUOTED_RE = re.compile(r'"(?:[^"\\]|\\.)*"') def _strip_noncode(sql: str) -> str: """Remove comments and string literals so regex scans don't false-positive on identifiers that happen to appear inside literal text. Mirrors `readonly_check.py:_strip_sql` so the two scripts stay in lock-step on what counts as "code".""" no_block = _BLOCK_COMMENT_RE.sub(" ", sql) no_line = _LINE_COMMENT_RE.sub(" ", no_block) no_single = _SINGLE_QUOTED_RE.sub("''", no_line) no_double = _DOUBLE_QUOTED_RE.sub('""', no_single) return no_double # Backwards-compatible alias — older callers (and tests) may import this name. _strip_comments = _strip_noncode def substitute(sql: str, dev_db: str) -> tuple[str, int]: count = sql.count(_PLACEHOLDER) return sql.replace(_PLACEHOLDER, dev_db), count def find_literal_databases(sql: str, dev_db: str) -> list[str]: """Return distinct database names used in fully-qualified refs, excluding dev_db. Strips comments AND string literals before scanning, so that a SQL like ``WHERE meta = 'analytics.prod.client_hub'`` does not falsely surface ``analytics`` in the literal-databases list — that text is data, not a reference.""" code_only = _strip_noncode(sql) dbs = {m.group(1) for m in _FQ_RE.finditer(code_only)} dbs.discard(dev_db) return sorted(dbs) def main() -> int: p = argparse.ArgumentParser( description=( "Substitute <YOUR_DEV_DATABASE> in a validation .sql file with a confirmed " "dev database; report any remaining literal fully-qualified database references." ) ) p.add_argument("path", type=Path, help="Path to the validation .sql file") p.add_argument("--dev-db", required=True, help="Dev database name to substitute in") p.add_argument( "--output", type=Path, default=None, help="Output path (default: <input_dir>/run/<input_stem>.run.sql)", ) args = p.parse_args() if not args.path.exists(): print(f"error: file not found: {args.path}", file=sys.stderr) return 1 original = args.path.read_text() substituted, replaced_count = substitute(original, args.dev_db) literals = find_literal_databases(substituted, args.dev_db) if args.output is not None: out_path = args.output else: run_dir = args.path.parent / "run" run_dir.mkdir(parents=True, exist_ok=True) out_path = run_dir / (args.path.stem + ".run.sql") out_path.parent.mkdir(parents=True, exist_ok=True) out_path.write_text(substituted) print(json.dumps({ "output_path": str(out_path), "dev_db": args.dev_db, "replaced_count": replaced_count, "literal_databases": literals, })) return 0 if __name__ == "__main__": sys.exit(main())
-
-
-
README.md 8.1 KB
# Monte Carlo Prevent Skill Bring Monte Carlo data observability into your editor — automatically, before you write a single line of code. ## What this does When you reference a dbt model or table, Monte Carlo context comes to you: table health, active alerts, lineage, and downstream blast radius. Your AI editor uses that context to shape the code it writes — not just surface it. If you try to rename a column with 500 downstream dependents, the editor recommends a safe transition strategy and explains why, citing the specific MC data it found. When you add new logic, it generates and deploys the right monitor for your logic — validation, metric, comparison, or custom SQL — before you merge. When you're done with a change, it generates targeted validation queries — tailored to the specific columns, filters, and business logic you modified — so you can verify the change behaved as intended before merging. ## Editor & Stack Compatibility The skill works with any AI editor that supports MCP and the Agent Skills format — including Claude Code, Cursor, and VS Code. For data stacks, compatibility varies by how you work: | Stack | Support | Notes | |---|---|---| | dbt + any MC-supported warehouse | ✅ Full | Optimized and tested | | SQL-first, no dbt | 🟡 Partial | Core workflows work via explicit prompting; auto-triggers on file open coming soon | | Databricks notebooks | 🟡 Partial | Health check, impact assessment, and alert triage work; file-based triggers coming soon | | SQLMesh | 🟡 Partial | Core workflows work; native SQLMesh project structure support coming soon | | PySpark / non-SQL pipelines | 🟠 Limited | Manual prompting only; broader support on the roadmap | **Coming shortly:** Generic SQL file triggers, Databricks notebook support, and SQLMesh project structure support — so auto-activation works regardless of your transformation tool. Core workflows — table health check, change impact assessment, alert triage, and monitor generation — work for any warehouse supported by Monte Carlo. ## Prerequisites - Claude Code, Cursor, VS Code or any editors with MCP support - Monte Carlo account with Editor role or above - [MC CLI](https://docs.getmontecarlo.com/docs/using-the-cli) installed for monitor deployment (`pip install montecarlodata`) ## Setup ### Via the mc-agent-toolkit plugin (recommended) Install the plugin for your editor — it bundles the skill, hooks, MCP server, and permissions automatically. See the [main README](../../README.md#installing-the-plugin-recommended) for editor-specific instructions. ### Standalone 1. Configure the Monte Carlo MCP server: ``` claude mcp add --transport http monte-carlo-mcp https://mcp.getmontecarlo.com/mcp ``` 2. Install the skill: ```bash npx skills add monte-carlo-data/mc-agent-toolkit --skill prevent ``` 3. Authenticate: run `/mcp` in your editor, select `monte-carlo-mcp`, and complete the OAuth flow. 4. Verify: ask your editor "Test my Monte Carlo connection" — it should call `testConnection` and confirm. <details> <summary>Legacy: header-based auth (for MCP clients without HTTP transport)</summary> If your MCP client doesn't support HTTP transport, use `.mcp.json.example` with `npx mcp-remote` and header-based authentication. See the [MCP server docs](https://docs.getmontecarlo.com/docs/mcp-server) for details. </details> ## How to use it Open your dbt project (or any data engineering codebase) in your editor. Describe the change you want to make — or reference a model file together with an edit (`@models/orders.sql add a column`). The skill activates automatically when you express change intent; no special commands needed. ### End-to-end flow ```mermaid flowchart TD A["Describe a<br/>change"] --> B["Fetch table<br/>context<br/>(silent)"] B --> C["Impact<br/>assessment"] C --> D{"Proceed?"} D -- yes --> E["Edit<br/>applied"] E --> P["Post-edit prompt:<br/>generate validation<br/>queries?<br/>add monitor?"] P -- yes --> H["Generate<br/>validation queries"] P -- yes --> G["Generate<br/>monitor"] H --> R{"Run<br/>queries?"} R -- yes --> S["Build & run"] ``` **Impact assessment** — Before any SQL edit (including filter changes, bugfixes, reverts, and parameter tweaks), prevent surfaces the change's blast radius: downstream models, active alerts, column exposure in recent queries, and monitor coverage. You get a risk tier (High / Medium / Low) and a recommendation tied to your specific change. If the data suggests your approach is risky, Claude proposes a safer alternative. **Validation queries** — When you're ready to test a change, say "generate validation queries", "validate this change", or run `/mc-validate`. Prevent generates 3–5 targeted SQL queries based on what you actually changed — null checks, before/after row counts, distribution checks — saved to `validation/<table_name>_<timestamp>.sql` with inline comments describing a passing result. **Monitor coverage** — After you finish an edit, if the impact assessment found a coverage gap, prevent prompts you to add a monitor. On yes, it hands off to `monte-carlo-monitoring-advisor` to produce a validation, metric, comparison, or custom SQL monitor as code. **Validate in sandbox (`/mc-validate run`)** — Two-phase workflow. Run `/mc-validate` first to generate the queries, then `/mc-validate run` to execute them: - **Build** — parses your `profiles.yml`, classifies the resolved database, and runs `dbt build --select <model>` into your dev database. - **Execute** — substitutes `<YOUR_DEV_DATABASE>` in the generated SQL with a user-confirmed value, runs each query through the Snowflake MCP, and reports findings. > ⚠️ **Heads up on prod vs. dev detection.** The build phase classifies your > resolved target as `personal` / `dev` / `shared-dev` / `prod` / `unknown` > from your `profiles.yml` and any hard-coded `{{ config(database=...) }}`, > and hard-stops if it lands on `prod`. This is a safety net, not a > guarantee — naming conventions vary across orgs and the classifier can be > wrong (especially on `unknown`). **You are still responsible for confirming > the target database before approving the build.** Read the value the skill > surfaces and don't approve if it doesn't match where you intend to write. **Invocation modes:** | Command | What it does | |---|---| | `/mc-validate` | Default = generate. Runs query generation only. | | `/mc-validate generate` | Explicit generate. Same as above. | | `/mc-validate run` | Runs **Build + Execute**. Requires queries already generated. | | `/mc-validate run --skip-build` | Runs **Execute only** — assumes you built manually. Requires queries already generated. | | `/mc-validate run --dev-db <NAME>` | Same as `run`, bypasses the dev-database prompt in Execute. | #### `/mc-validate run` prerequisites The `run` subcommand only works if all three of the following are in place — otherwise the flow will fail mid-build with a confusing error. Verify before invoking: - **dbt installed** and a `dbt_project.yml` discoverable from the changed model (the workflow walks up from the model file to find it). - **`profiles.yml`** present (typically in `~/.dbt/profiles.yml`) with a working Snowflake target. The skill parses it to resolve your dev database. - **Snowflake MCP server** registered in the editor session — the skill detects this by looking for an `mcp__snowflake__*` tool. Without it, queries cannot execute and the substituted SQL is left on disk for you to run manually. The `run` subcommand performs a connection pre-flight check before kicking off the build. If any prerequisite is missing, it aborts early and tells you what to fix — rather than failing after a partial build. ### Deploying generated monitors When Claude generates a monitor, it saves the YAML to `monitors/<table>.yml`. Deploy with: ```bash montecarlo monitors apply --dry-run # preview montecarlo monitors apply --auto-yes # apply ``` Your project needs a `montecarlo.yml` config in the working directory: ```yaml version: 1 namespace: <your-namespace> default_resource: <your-warehouse-name> ``` ## Troubleshooting See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for common setup and runtime issues. -
SKILL.md 18.6 KB
--- name: monte-carlo-prevent description: Shift-left safety net for dbt/SQL model edits. Runs change impact assessment before edits, generates SQL validation queries after, and executes them via `/mc-validate run`. Delegates health and monitor creation to peer skills. when_to_use: | Invoke when the user expresses intent to change a dbt or SQL model — adding, dropping, renaming, refactoring a column or filter, fixing a bug in model logic, tweaking a parameter, or referencing a model file paired with an edit verb. Also invoke when the user asks to "validate this change", "verify my edit", or runs `/mc-validate` / `/mc-validate run`. Example triggers: "add an is_active column to client_hub", "refactor the join logic in stg_payments", "drop the legacy_id column from dim_users", "@models/orders.sql add a filter", "/mc-validate run". Do NOT invoke for: - Plain health questions about a table ("how is X doing?", "is X healthy?") — those go to monte-carlo-asset-health. - Alert investigation or incident triage ("freshness alert on X", "why did X fail?") — those go to automated-triage or monte-carlo-incident-response. - Standalone monitor creation requests without an edit context ("create a monitor for X", "what should I monitor?", "show coverage gaps") — those go to monte-carlo-monitoring-advisor. - Performance or pipeline diagnosis ("why is X slow?", "investigate the query plan") — those go to monte-carlo-performance-diagnosis. - Edits to non-model files: seed CSVs (seeds/), analysis files (analyses/), dbt config (dbt_project.yml, profiles.yml, packages.yml). - Bare file opens or reads without an edit verb ("open stg_orders.sql so I can see what it does") — that's navigation, not change intent. bucket: Prevent version: 1.0.0 --- # Monte Carlo Prevent Skill This skill brings Monte Carlo's data observability context directly into your editor. When you're modifying a dbt model or SQL pipeline, use it to surface table health, lineage, active alerts, and to generate monitors-as-code without leaving Claude Code. > **Monte Carlo tool routing (required):** Always call Monte Carlo MCP tools through this plugin's > bundled server, whose fully-qualified tool names are > `mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__<tool>` (e.g. > `mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__get_alerts`). Bare tool names used in this skill > (`get_alerts`, `search`, `get_table`, …) refer to that bundled server. If the session also has a > separately-configured `monte-carlo-mcp` server, do **not** route to it — it may point at a > different endpoint or credentials. Reference files live next to this skill file. **Use the Read tool** (not MCP resources) to access them: - Full workflow step-by-step instructions: `references/workflows.md` (relative to this file) - MCP parameter details: `references/parameters.md` (relative to this file) - Troubleshooting: `references/TROUBLESHOOTING.md` (relative to this file) ## When to activate this skill **Prevent is the edit-lifecycle skill.** Activate only when the user expresses intent to change a dbt model. Bare file mentions, table-name mentions in passing, or general health questions are **not** prevent's territory — those belong to `monte-carlo-asset-health` and will activate that skill on their own. **Do not wait to be asked.** Run the appropriate workflow automatically whenever the user: - Describes a planned change to a model (new column, join update, filter change, refactor) → **STOP — run Workflow 1 first if it has not run for this table this session, then Workflow 2, before writing any code** - Adds a new column, metric, or output expression to an existing model → same rule: Workflow 1 first (if not yet run for this table), then Workflow 2; the post-edit hook will offer Workflow 5 (monitor generation) afterward - References a model file with an edit verb in the same prompt (e.g. `@models/clients/client_hub.sql add an is_active column`) → same rule: Workflow 1 first, then Workflow 2 Present the W2 impact assessment as context the engineer needs before proceeding — not as a response to a question. ### Workflow 1 runs silently when chained to Workflow 2 When the user expresses change intent, Workflow 1 invokes `monte-carlo-asset-health` purely as a data-gathering step. Read asset-health's report from your context, but **do not relay the full report to the engineer** — the user-facing artifact is Workflow 2's impact assessment, which already cites the relevant alerts / lineage / monitors. Showing both creates duplicate reading. Two exceptions where you **must** surface output from W1 to the engineer: 1. **Disambiguation prompt.** If asset-health returns multiple matches and asks the engineer to pick one, surface that question — the user must choose. 2. **Stop-the-world signals.** If the table is already on fire (active critical alerts firing, freshness severely stale), say so in one short line before W2. If Workflow 1 already ran for this table earlier in the session, skip directly to Workflow 2 — re-running asset-health is redundant. ## When NOT to activate this skill Do not invoke Monte Carlo tools for: - Seed files (files in seeds/ directory) - Analysis files (files in analyses/ directory) - One-off or ad-hoc SQL scripts not part of a dbt project - Configuration files (dbt_project.yml, profiles.yml, packages.yml) - Test files unless the user is specifically asking about data quality If uncertain whether a file is a dbt model, check for {{ ref() }} or {{ source() }} Jinja references — if absent, do not activate. ### Macros and snapshots — gate edits, skip auto-context Macro files (`macros/`) and snapshot files (`snapshots/`) are **not** models, so do not auto-fetch Monte Carlo context (Workflow 1) when they are opened. However, macros are inlined into every model that calls them at compile time — a one-line macro change can silently alter dozens of models. Snapshots control historical tracking and are similarly sensitive. **The pre-edit hook gates these files.** If the hook fires for a macro or snapshot, identify which models are affected and run the change impact assessment (Workflow 2) for those models before proceeding with the edit. ### Peer-skill redirects These requests have their own skills — do not run prevent for them: - "How is table X doing?" / "is X healthy?" / "check status of X" → `monte-carlo-asset-health` - "Create a monitor for X" / "what should I monitor?" / "set up freshness on X" (without an active edit context) → `monte-carlo-monitoring-advisor` Prevent invokes asset-health and monitoring-advisor itself when its workflows need them (W1, W5); it does not duplicate their entry points. --- ## REQUIRED: Change impact assessment before any SQL edit **Before editing or writing any SQL for a dbt model or pipeline, you MUST run Workflow 2.** This applies whenever the user expresses intent to modify a model — including phrases like: - "I want to add a column…" - "Let me add / I'm adding…" - "I'd like to change / update / rename…" - "Can you add / modify / refactor…" - "Let's add…" / "Add a `<column>` column" - Any other description of a planned schema or logic change - "Exclude / filter out / remove [records/customers/rows]…" - "Adjust / increase / decrease [threshold/parameter/value]…" - "Fix / bugfix / patch [issue/bug]…" - "Revert / restore / undo [change/previous behavior]…" - "Disable / enable [feature/logic/flag]…" - "Clean up / remove [references/columns/code]…" - "Implement [backend/feature] for…" - "Create [models/dbt models] for…" (when modifying existing referenced tables) - "Increase / decrease / change [max_tokens/threshold/date constant/numeric parameter]…" - Any change to a hardcoded value, constant, or configuration parameter within SQL - "Drop / remove / delete [column/field/table]" - "Rename [column/field] to [new name]" - "Add [column]" (short imperative form, e.g. "add a created_at column") - Any single-verb imperative command targeting a column, table, or model (e.g. "drop X", "rename Y", "add Z", "remove W") Parameter changes (threshold values, date constants, numeric limits) appear safe but silently change model output. Treat them the same as logic changes for impact assessment purposes. **Do not write or edit any SQL until the change impact assessment (Workflow 2) has been presented to the user.** The assessment must come first — not after the edit, not in parallel. --- ## Pre-edit gate — check before modifying any file **Before calling Edit, Write, or MultiEdit on any `.sql` or dbt model file, you MUST check:** 1. Has the synthesis step been run for THIS SPECIFIC CHANGE in the current prompt? 2. **If YES** → proceed with the edit 3. **If NO** → stop immediately, run Workflow 2, present the full report with synthesis connected to this specific change. **If risk is High or Medium:** ask "Do you want me to proceed with the edit?" and wait for explicit confirmation. **If risk is Low:** use judgment — proceed if straightforward and no concerns found, otherwise ask before editing. **Important: "Workflow 2 already ran this session" is NOT sufficient to proceed.** Each distinct change prompt requires its own synthesis step connecting the MC findings to that specific change. The synthesis must reference the specific columns, filters, or logic being changed in the current prompt — not just general table health. Example: - ✅ "Given 34 downstream models depend on is_paying_workspace, adding 'MC Internal' to the exclusion list will exclude these workspaces from all downstream health scores and exports. Confirm?" - ❌ "Workflow 2 already ran. Making the edit now." The only exception: if the user explicitly acknowledges the risk and confirms they want to skip (e.g. "I know the risks, just make the change") — proceed but note the skipped assessment. ## Available MCP tools All tools are available via the `monte-carlo-mcp` MCP server. | Tool | Purpose | | ---------------------------- | -------------------------------------------------------------------- | | `testConnection` | Verify auth and connectivity | | `search` | Find tables/assets by name | | `getTable` | Schema, stats, metadata for a table | | `getAssetLineage` | Upstream/downstream dependencies (call with mcons array + direction) | | `getAlerts` | Active incidents and alerts | | `getMonitors` | Monitor configs — filter by table using mcons array | | `getQueriesForTable` | Recent query history | | `getQueryData` | Full SQL for a specific query | | `createValidationMonitorMac` | Generate validation monitors-as-code YAML | | `createMetricMonitorMac` | Generate metric monitors-as-code YAML | | `createComparisonMonitorMac` | Generate comparison monitors-as-code YAML | | `createCustomSqlMonitorMac` | Generate custom SQL monitors-as-code YAML | | `getValidationPredicates` | List available validation rule types | | `getAudiences` | List notification audiences | | `getDomains` | List MC domains | | `getUser` | Current user info | ## Core workflows Each workflow has detailed step-by-step instructions in `references/workflows.md` (Read tool). ### 1. Asset health pre-fetch (silent delegation to asset-health) **When:** User expresses change intent for a table that hasn't been seen in this session. **What:** Invokes `monte-carlo-asset-health` via the Skill tool to gather table state (health, upstream lineage, alerts, monitors). Then makes one direct `get_asset_lineage(direction="DOWNSTREAM")` call to complete the picture (asset-health only fetches upstream). The combined data is **used as input to Workflow 2**, not shown to the engineer. Two exceptions surface to the user: any disambiguation prompt, and stop-the-world signals (active critical alerts, severe staleness). ### 2. Change impact assessment — REQUIRED before modifying a model **When:** Any intent to modify a dbt model's logic, columns, joins, or filters. **What:** Surfaces blast radius, downstream dependencies, active incidents, monitor coverage, and query exposure. Reuses asset-health's data when Workflow 1 ran earlier this session; otherwise calls `get_table` / `get_alerts` / `get_asset_lineage` / `get_monitors` directly. Produces a risk-tiered report with synthesis connecting findings to specific code recommendations. ### 3. Change validation queries **When:** Explicit engineer request only (e.g. "validate this change", "ready to commit"), or via `/mc-validate run`. **What:** Generates 3–5 targeted SQL queries to verify the change behaved as intended. Uses Workflow 2 context — requires both impact assessment and file edit in session. ### 4. Validate change in sandbox — invoked by `/mc-validate run` **When:** Only when the engineer invokes `/mc-validate run` (in any of its forms). **Pre-flight:** `run` does **not** auto-generate. If no `validation/<table>_<ts>.sql` exists for the changed model(s), abort and tell the engineer to run `/mc-validate` (or `/mc-validate generate`) first. **What:** Two-phase workflow. - **W4.1 — Build.** Parses `profiles.yml`, classifies the active database, detects hard-coded `database:` in the model's `{{ config() }}`, then runs `dbt build --select <model>` into the engineer's dev database. Refuses to build against shared prod. Skipped automatically for YAML/docs-only diffs and for `/mc-validate run --skip-build`. - **W4.2 — Execute validation queries.** Substitutes `<YOUR_DEV_DATABASE>` in Workflow-3 output with a user-confirmed value (or `--dev-db <NAME>` if supplied), runs a read-only pre-check on every query, executes via the Snowflake MCP, and reports per-query verdicts plus a consolidated summary. **Invocation matrix:** | Invocation | W3 (generate) | W4.1 (Build) | W4.2 (Execute) | |---|---|---|---| | `/mc-validate` | yes | — | — | | `/mc-validate generate` | yes | — | — | | `/mc-validate run` | no — must already exist | yes | yes | | `/mc-validate run --skip-build` | no — must already exist | no | yes | `run` accepts both flags together: `/mc-validate run --skip-build --dev-db <NAME>`. ### 5. Add monitor (delegated to monitoring-advisor, post-edit) **When:** Post-edit hook injects the coverage prompt (driven by `MC_MONITOR_GAP` from Workflow 2), or the engineer explicitly asks to add a monitor. **What:** Asks "Generate monitor definitions? (yes/no)". On yes, invokes `monte-carlo-monitoring-advisor` via the Skill tool with the model name and changed columns/logic. Prevent's responsibility ends at delegation — it does not wait for monitoring-advisor or emit a completion marker. > **Workflow numbering note:** numbers are assigned by execution order (W1 → W2 → optional W3 → optional W4 [W4.1 + W4.2] → optional W5), not by insertion order in this file. `references/workflows.md` is the source of truth. --- ## Post-synthesis confirmation rules Always end the synthesis with one clear, specific recommendation in plain English: "Given the above, I recommend: [specific action]" **If the risk is High or Medium:** STOP and wait for confirmation before editing any file. You must ask the engineer and receive an explicit "yes", "go ahead", "proceed", or similar confirmation before making code changes. Say: "Do you want me to proceed with the edit?" Do NOT say: "Proceeding with the edit." — that skips the engineer's decision. **If the risk is Low:** Use your judgment based on the synthesis findings. If the change is straightforward and the synthesis found no concerns, you may proceed. If anything is surprising or worth flagging, ask before editing. --- ## Session markers These markers coordinate between the skill and the plugin's hooks. Output each on its own line when the condition is met. ### Impact check complete After the engineer confirms (High/Medium) or after presenting the synthesis (Low), output one marker per assessed table. **IMPORTANT: use only the table/model name, not the full MCON:** <!-- MC_IMPACT_CHECK_COMPLETE: <table_name> --> (Use the model filename without .sql extension — NOT "acme.analytics.orders" or "prod.public.client_hub") How many markers to emit depends on how the assessment was triggered: **Hook-triggered** (the pre-edit hook blocked an edit and instructed you to run the assessment): Be strict — only emit markers for tables whose lineage **and** monitor coverage were fetched directly via Monte Carlo tools in this session. If the engineer describes changes to multiple tables but only one was formally assessed, emit only one marker. The pre-edit hook will gate the other tables and prompt for their own Workflow 2 runs. **Voluntarily invoked** (the engineer proactively asked for an impact assessment): Be looser — emit markers for all tables the assessment meaningfully covered, even if some were assessed via lineage context rather than direct MC tool calls. The engineer is already safety-conscious; don't force redundant assessments for tables they clearly considered. ### Monitor coverage gap When Workflow 2 finds zero custom monitors on a table's affected columns, output: <!-- MC_MONITOR_GAP: <table_name> --> Use only the table/model name (NOT the full MCON). This allows the plugin's hooks to remind the engineer about monitor coverage at commit time. Only output this marker when the gap is specifically about the columns or logic being changed — not for general table-level monitor absence. After the prompt is delivered, the post-edit / pre-commit hook clears the gap state internally so it won't re-prompt for the same gap; if the engineer edits the model again, Workflow 2 will re-evaluate from scratch and re-emit the marker only if a gap still exists. ### Sandbox build ran (W4.1) Emit after a successful `dbt build` in Workflow 4.1 (or after a deliberate skip — e.g. YAML-only diff or `--skip-build` — including the skip reason). One marker per model. <!-- MC_BUILD_RAN: <table_name> --> ### Validation executed (W4.2) Emit after Workflow 4.2 finishes executing validation queries for a model, regardless of individual per-query verdicts. One marker per model. <!-- MC_VALIDATE_RAN: <table_name> -->
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.