{"slug":"data-2","title":"data","summary":"Pipelines, warehouses, dbt models and metrics, where failure is silently wrong numbers rather than a crash. Use when \"the dashboard is wrong\", \"the numbers do not match\", \"add data quality checks\", \"safe backfill\", or an upstream schema change broke a join. Covers freshness, row-","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-14T21:08:47.638134Z","repo":{"url":"https://github.com/rainmanjam/poka-yoke","stars":22,"forks":3,"license":"MIT","updatedAt":"2026-09-01T16:13:25Z"},"bodyHtml":"<hr>\n<h2>name: data\ndescription: &gt;-\nPipelines, warehouses, dbt models and metrics, where failure is silently wrong numbers rather than a crash. Use when \"the dashboard is wrong\", \"the numbers do not match\", \"add data quality checks\", \"safe backfill\", or an upstream schema change broke a join. Covers freshness, row-count and null-rate assertions, data contracts, reconciliation. For a crash rather than wrong numbers use audit.</h2>\n<h1>Poka-Yoke for Data</h1>\n<p>Data systems fail differently from application code, and that difference determines every\ndevice here. An application bug throws an exception, pages someone, and gets fixed. A data bug\nproduces a number. The number looks fine. Someone makes a decision with it. Three weeks later\na person notices revenue looks odd, and now you have three weeks of decisions to unwind and no\nway to know which were wrong.</p>\n<p><strong>In data, silence is the defect.</strong> A pipeline that fails loudly is working correctly. A\npipeline that succeeds while producing garbage is the thing to design against, so most\ndevices here are about converting silent wrongness into loud failure, which in Shingo's terms\nis buying yourself a Warning rung where you currently have nothing at all.</p>\n<h2>The four questions</h2>\n<p>Run these over any table or model. They map onto the standard lenses but the data-specific\nphrasing is what finds things.</p>\n<p><strong>Is it there?</strong> <em>(freshness)</em>, Did the data arrive at all, and recently enough to be worth\ntrusting? A stale table is the most dangerous artifact in a warehouse because it looks\ncompletely healthy. Every table needs a max-age assertion, and dashboards should surface\nlast-updated rather than hiding it.</p>\n<p><strong>Is there the right amount?</strong> <em>(volume, fixed-value lens)</em>, Row counts against expectation.\nThis catches the breakages that leave every individual row looking fine: a partial load, a\nfilter that silently matched nothing, a join that fanned out 100x. Assert both a floor and a\nceiling, and compare against the same weekday historically rather than against yesterday: most business data is weekly-seasonal and a naive day-over-day check will cry wolf every\nMonday.</p>\n<p><strong>Is it shaped right?</strong> <em>(schema and validity, contact lens)</em>, Types, nullability, accepted\nvalue sets, ranges. Negative quantities, percentages above 100, timestamps in the future,\ncurrency codes that don't exist, a <code>status</code> value nobody has seen before.</p>\n<p><strong>Does it agree?</strong> <em>(reconciliation)</em>, Does the warehouse total match the source system?\nDoes the sum of the parts match the whole? This is the only check that catches a logic error\nthe data still looks well-shaped after, everything above validates shape, and a wrong <code>JOIN</code>\nproduces perfectly well-shaped, wrong data. It catches what moves a total, not a\nmis-attribution that nets out. If you install one device, install this one on your\nrevenue-critical tables.</p>\n<h2>Devices, strongest first</h2>\n<h3>Constraints at the write, not tests after it</h3>\n<p>Where the warehouse supports it, <code>NOT NULL</code>, <code>UNIQUE</code>, <code>CHECK</code>, and primary keys are Control:\nthe bad row cannot be written. A dbt test is Detection: the bad row is already in the table\nand possibly already in a dashboard. Prefer the constraint; use the test where the engine\ngives you nothing better, which in several columnar warehouses is most of the time, say so\nexplicitly rather than pretending a test is prevention.</p>\n<h3>Data contracts at the boundary</h3>\n<p>The most common pipeline break is upstream changing a column without telling anyone. A\ncontract makes that break loud and attributable:</p>\n<ul>\n<li>The producer declares the schema, types, nullability, and semantics; changes go through\nversioning rather than through a surprise.</li>\n<li>The consumer validates on ingest and <strong>quarantines</strong> rather than dropping. Silently dropping\nmalformed rows is the data equivalent of <code>except: pass</code>: the pipeline goes green while the\nnumbers go wrong. Route bad rows to a dead-letter table with the reason, alert on the rate,\nand keep them for inspection.</li>\n<li>Additive changes are safe; renames and type narrowing are breaking. Treat a rename as a drop\nplus an add, because that is what downstream experiences.</li>\n</ul>\n<h3>Idempotent, resumable loads</h3>\n<p>Every incremental job should be safe to re-run over the same window. Pipelines get retried, by the scheduler, by an on-call engineer, by a backfill, and a non-idempotent load\ndouble-counts, which is a silently wrong number of exactly the worst kind.</p>\n<p>The device: partition-level replace, or <code>MERGE</code> on a real business key, rather than blind\n<code>INSERT</code>. Then a re-run converges rather than accumulating.</p>\n<h3>Backfills that cannot run away</h3>\n<p>Backfills are the data world's destructive operation. Before running one:</p>\n<ul>\n<li>Bound it explicitly: a date range with both ends, never open-ended.</li>\n<li>Batch it, with progress recorded, so a failure at 80% resumes rather than restarts.</li>\n<li>Write to a staging table and swap atomically, so consumers never see a half-populated table.</li>\n<li>Dry-run first, printing the partitions and row counts it will touch.</li>\n<li>Know the rollback: if the backfill is wrong, what restores the previous state? If the answer\nis \"nothing\", make a snapshot first. That snapshot <em>is</em> the device.</li>\n</ul>\n<h3>One definition per metric</h3>\n<p>If \"active user\" is defined in the dashboard, the model, and an analyst's spreadsheet, you have\nthree metrics with one name and they will disagree, usually in a meeting. Define each metric\nonce, in version-controlled code, and have every consumer reference that definition. A metric\nredefined in a BI tool is a copy that will silently drift.</p>\n<h3>Assertions in the pipeline, not beside it</h3>\n<p>The check must be able to <strong>stop the pipeline</strong>, not just report. A test suite that runs after\npublication and emails a failure lets bad data reach the dashboard, which is the whole problem.\nAssert between load and publish: build to staging, test staging, promote only on pass. That\nordering is the single most valuable structural change in most warehouses, and it costs no new\ntooling.</p>\n<h2>Auditing a pipeline</h2>\n<p>Read the DAG or the model files and work outward from what matters:</p>\n<ol>\n<li><strong>Which tables feed decisions or money?</strong> Start there; coverage everywhere is not the goal.</li>\n<li><strong>For each: freshness, volume, uniqueness on the key, null rate on required columns,\nreconciliation to source.</strong> Which exist? Which can actually block publication?</li>\n<li><strong>Where are rows silently dropped?</strong> Inner joins that should be left joins, <code>WHERE</code> clauses\nfiltering nulls, try/except around row parsing, <code>on_error='ignore'</code>. Each is a place the\ncount quietly shrinks.</li>\n<li><strong>What happens on re-run?</strong> Trace one job. Does it double-count?</li>\n<li><strong>What happens when upstream adds or renames a column?</strong> Break, or silently produce nulls?</li>\n<li><strong>Is anything in a dashboard that isn't in version control?</strong></li>\n</ol>\n<p>Report with the structure from <code>audit</code>, and be explicit about the rung, in data,\nmost devices you can actually install are Warning or Detection, and claiming Control for a\ndbt test overstates the protection.</p>\n<h2>The tone that matters here</h2>\n<p>When numbers have been wrong, the instinct is to find who wrote the bad join. Same rule as\neverywhere else in this plugin: the finding is that the pipeline could produce a wrong number\nwithout anyone noticing. That is a missing assertion, not a missing person.</p>\n","files":[{"path":"SKILL.md","sizeBytes":7266,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-14T21:08:56.106112Z","sha256":"850A1711BB17079DB8D18CA3CE256ED2E7ADFE985D5880A6C570421654721DC3","sizeBytes":3515},"review":null,"source":{"repositoryUrl":"https://github.com/rainmanjam/poka-yoke","path":"plugins/poka-yoke/skills/data","license":"MIT","commit":"726a575e3d48d07d908abfcbb192cae09671fff2","subtreeSha":"C5D4E79C42294996C69DBCD20BA27A3CC8C8034AAD6F6691893414075A8CC8EE","lastSyncedAt":"2026-09-27T19:47:53.390177Z"},"reviewedAt":"2026-09-14T21:09:14.88308Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/rainmanjam/poka-yoke/tree/main/plugins/poka-yoke/skills/data"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install rainmanjam-poka-yoke@llmmart"},{"target":"git","command":"git clone https://github.com/rainmanjam/poka-yoke.git"}]}