Make sense of a warehouse or a local DuckDB database the way an analytics\nengineer does: rank what matters, drill selectively, and persist a draft map.\nThis is the flagship, fully read-only skill. It absorbs profiling and\nrelationship inference as capabilities; they are not separate skills.
\nRun the engine through the wrapper. It prints one sanitized JSON envelope and\nnothing else; read the envelope and decide the next step.
\nuv run --no-project --script \"${CLAUDE_SKILL_DIR}/scripts/run.py\" <subcommand> [flags]\ndex runs its engine through uv, which is a prerequisite and is not installed by\nClaude Code. If the shell reports uv: command not found, stop and tell the user\nto install it (curl -LsSf https://astral.sh/uv/install.sh | sh, or\nbrew install uv, or pipx install uv), then re-run. Never fall back to raw\nPython, pip, or a database CLI to do the work another way: the guardrails live in\nthe engine, so any other path is unguarded.
The first command in a fresh environment installs the engine, so it can take tens\nof seconds where later ones take well under a second. --warm pays that install up\nfront and exits without running anything:
uv run --no-project --script \"${CLAUDE_SKILL_DIR}/scripts/run.py\" --warm\nOffer it once at setup. It is not something to run before an ordinary command.
\nIf the user has no warehouse to point at and wants to see what dex does, demo\ngenerates one: a seeded local DuckDB warehouse plus the .dex/config.yml for it,\nwith no credentials and no network, so every subcommand below then runs with no\nflags. It only ever creates, so it refuses rather than touch a file that already\nexists. Offer it rather than assuming it: a user who does have a warehouse wants\nthat one read, not a fixture built beside it.
Subcommands, in the usual order:
\nconnect test --path <file.duckdb> confirms a read-only connection and\nreports capabilities.
explore inventory --rank returns a ranked object summary (counts and sizes,\nnever rows).
explore profile <objects> (space- or comma-separated) returns column\nprofiles, PII flags recorded as (column, category, confidence) and never\nexample values, plus ranked candidate keys, the likely grain, key_evidence,\nand data-quality warnings (e.g. an id unique on all but 110 rows, which will\nfan out on joins). candidate_keys is ordered, tightest proven key first,\nand key_evidence gives one entry per combination considered with its\nstatus (reported or suppressed) and the reason. Read it before you\ntrust a composite: a combination unique only because one member is unique on\nalmost every row, or because a money column completes it, is suppressed\nrather than reported. Where a near-unique column is the real story the\nwarning says so with the ratio, the counts, and how many rows would have to\nbe removed for it to be unique. That last number is the one to act on: it\nnames a source defect to fix rather than a key to work around. A generic\n*_name flag's confidence is refined by value-shape evidence from the same\nscan, in both directions: person-shaped values corroborate it, a closed\nreference vocabulary or long labels de-rate it below the firewall's blocking\nthreshold, and missing evidence changes nothing (the flag itself is never\nremoved). Distinct counts\nare approximate for scale, but any column that looks unique within\napproximation noise is escalated to an exact COUNT(DISTINCT)\n(distinct_count_exact: true), so uniqueness and grain verdicts rest on\nproof; a ~ prefix marks a number that is still approximate, on a count and\non a percentage alike, so a figure quoted without one is exact arithmetic\nover an exact distinct count on a column with no nulls.\nA requested object whose cached profile is still fresh (same connector,\nschema unchanged, within profile_freshness_hours, default 24) is served\nfrom the cache (cache_hit_count) instead of re-scanned, so profiling a\ntable map just wrote costs nothing to spend; pass --refresh to force a\nre-scan when the source changed in a way the free metadata check cannot see.
explore relationships returns inferred and declared joins with confidences,\nplus notes explaining what the inference examined (so an empty list is\nmeaningful). Add --verify to measure each inferred join with an aggregate\noverlap probe (orphan fraction, confidence adjusted). A declared join has two\nsources: a relationships test, and (with --use-project) an entity two\nsemantic models share, which the layer states outright with the key named per\nmodel. declared_by on an edge names that entity, semantic_join_count says\nhow many came that way, and the notes call out the ones name-based inference\ndid not find, which is the interesting set: a semantic layer routinely joins\ncolumns that share no name at all.
explore map writes or updates the .dex/ cache and returns the map\n(--verify works here too). Alongside the counts, data.objects gives each\ntop-ranked object its row count, detected grain, best-ranked candidate key,\nnotable columns (each carrying the role that earned it a place: grain, key,\njoin, or a PII flag) and data-quality findings, and data.edges gives the\njoin edges in the same shape explore relationships returns. With\n--use-project each object also carries semantic_models, the semantic models\nthat sit on that relation, which is what separates a load-bearing table from a\nmerely large one: empty means nothing in the layer reads it. Read that\npayload instead of chaining profile and relationships to re-derive it;\ngo to those two when you need one object in full, or a value domain, which\nmap never carries. It is budgeted: 25 objects by rank, 12 columns per\nobject, 40 edges, 5 findings per object. Every cap binds in every mode and\nevery elision is counted in notes and in an elided_* field, so an empty\nnotes means nothing was cut. --detail widens the selection to every column\nand to objects that were inventoried but never profiled, and lifts no cap; it\nspends nothing, unlike --full. Past 50 objects it profiles only the top 25\nby rank and says so in notes (with skipped_count); pass --full to\nprofile everything. On a re-map, objects skipped this run keep their prior profiles\n(carried_forward_count), each stamped with its own profiled_at so\nstaleness is visible instead of column detail silently vanishing. A selected\nobject whose cached profile is still fresh (same connector, schema unchanged,\nprofiled within profile_freshness_hours, default 24) is reused without a\nre-scan (cache_hit_count), so re-runs cost nothing to spend; pass\n--refresh to force a full re-profile when the source changed in a way the\nfree metadata check cannot see (e.g. rows changed but the schema did not).\nexplore relationships and the standalone explore profile reuse fresh\nprofiles the same way.
explore diagram [--full] renders the cached map as a Mermaid ER diagram in\ndata.mermaid. Free and connectionless (it reads the cache, never the\nwarehouse), so it is safe to re-run while shaping the picture. Reproduce the\nstring verbatim in a fenced ```mermaid block so the human can see it, and\nwrite it to a .mmd or a markdown file when they want one on disk: the\nengine deliberately writes no file. Never redraw or \"tidy up\" the diagram\nby hand. The glyphs are claims the engine derived from evidence, and a\nplausible-looking cardinality you supplied is exactly the overclaim this\ncommand exists to prevent: declared joins are solid, inferred dotted, and an\nunverified inference never says \"exactly one\". A solid line labelled with a\nsemantic entity is a join the semantic layer declares; look the entity up with\nexplore semantic list. Read notes before presenting\nit, since it states any object or column that was left out; --full widens\nfrom the default (profiled, joined objects and their grain, key, join, and\nPII columns) to everything eligible.
explore query \"<SELECT ...>\" [\"<SELECT ...>\" ...] answers ad-hoc questions\nthe fixed commands don't cover: you write the SQL, the engine's query firewall\nrefuses or bounds it. Pass a statement per argument, or --sql-file <path>\nfor a longer list, and ask a whole chain of questions in one call rather than\none call each; each statement is judged and answered on its own, so a refusal\non one does not cost you the others, and data.results carries one entry per\nstatement. A table you have not profiled, including a model you just built, is\nprofiled for you and the statement then runs, so probing something new is one\ncall rather than three; the envelope says what it profiled, and on a metered\nconnector that profile is priced into the same confirmation as the statements.\nResults come back row-major and capped; a refusal names the offending column\nand the fix, so one rewrite is enough. Read ${CLAUDE_SKILL_DIR}/references/probe-playbook.md before\nwriting a probe: it maps common questions to effective probe shapes.
explore cluster <object> [--features a,b,c] [-k N] runs k-means over a\nbounded sample of the object's numeric columns and returns the segment\nstructure: per-cluster sizes and fractions, centroids (each coordinate is a\ncluster's mean of that feature, an aggregate), the silhouette score, and,\nwhen -k is omitted, the k it picked plus the silhouette sweep it chose from.\nRequires the .dex/ cache (run map/profile first) so features can be\nauto-selected from profiled numeric, non-PII, non-key columns; pass\n--features to choose them yourself (naming a PII column, or a key, opts it\nin deliberately, and only its mean is ever reported). A key is never a\nfeature: its mean is meaningless, and a fact table is mostly keys plus a\nhandful of measures, so clustering on them just partitions surrogate ranges.\nKeys are the unique columns, the columns that join out (from the joins map\ninferred), and the columns named like one; prefer map over a bare\nprofile here, because without inferred joins a foreign key is caught only\nif its name gives it away. The notes name every excluded column, so check\nthem before trusting a result. Two things the silhouette alone will not tell\nyou, both of which the notes will. A cluster holding under 1% of the sample\nis an outlier pocket, not a segment, and it pushes the score up precisely\nbecause it sits so far out: report that as outlier detection, or re-run with\n-k to split the bulk. And on connectors that cannot seed a sample the draw\nchanges per run, so two runs can disagree on k; the envelope's\nsample_repeatable says which case you are in, and comparing runs across\ndifferent draws is meaningless. Only aggregates cross the\nboundary: the sample rows are clustered in-process and never enter context.\nOn a metered connector it takes the same cost handshake as the scanning\ncommands below (only the feature columns are scanned, and a dialect-aware\nsample clause reads a fraction), so surface the estimate and get a budget\nfirst. Needs the [cluster] extra (scikit-learn); the wrapper installs it\nautomatically for this subcommand.
explore semantic list|values|query reach the semantic layer: the metrics an\nauthor defined, and the semantic models, measures, dimensions and entities\nthey are built out of. Distinct from the warehouse commands above, and from\nthe top-level semantic group, which authors the layer where this queries\nit.
list is discovery and returns the layer's objects rather than three lists of\nnames: semantic models (the unit the layer is organized around, each with the\ntransformation model it sits on, its default time dimension, and the physical\nrelation underneath), metrics (which dimensions each can be grouped by, the\nmeasures it reads, a ratio's two sides, any filter that makes it a subset, the\ngrains it can be queried at, and time_axis, the physical time column a time\ngrouping resolves to), dimensions (the token to group by, plus the bare\ndefinition, owning model, queryable grains and column behind it), entities\n(one declaration per semantic model, each with its own join key, so the\ndeclared join graph is readable), and measures (the aggregation and expression\nthe number is actually made of, which is often a conditional rather than a\ncolumn). An element defined as an expression carries no column rather than a\nguessed one. So \"which table is behind this metric\" is the metric's\nsemantic_models followed to their relations, and explore profile <relation>\nis the next call; --api exposes no relation at all and declares that in\nunavailable, so use --local when you need the physical side.
Three free ways to narrow it, and they compose. --metric <m> keeps those\nmetrics and what they reach. --for-dimension <d> asks the reverse question,\nreturning the metrics groupable by all the named tokens, which is what you want\nwhen you know the slice rather than the metric and is also the cheapest way to\nfind the metrics that can go on one chart against one axis. --search <t>\ntakes a word rather than a name and matches it against every element's name and\nagainst the project's own label and description. Each names its scope in the\npayload (scoped_to, for_dimensions, searched_for), so a subset is never\nmistaken for the layer; an unknown metric or dimension is refused by name,\nwhile a search term that matched nothing comes back as a note. The catalog is\nalso capped, with every cut counted in elided and named in notes and\n--full to lift the caps. elided is always present, so all zeros and no cap\nnotes is the positive statement that this is the whole layer. Prefer narrowing\nover --full: it decides which part comes back rather than letting a cap\ndecide.
values <dimension> returns that dimension's value domain, which is what you\nneed before writing a --where filter and the one thing no other dex command\ncan reach on a hosted layer (profile cannot see a semantic dimension). A\nPII-flagged dimension refuses this command outright rather than being screened,\nbecause the whole output is values.
query takes a positional metric after the explicit mode (with --metric kept\nfor compatibility), a --group-by <entity__dim>, and optional --where,\n--order-by, --grain and --limit, and returns the metric's values as a\ncapped columnar result. Name flags take a comma-separated list or a repeated\nflag (--group-by a,b is --group-by a --group-by b); --where is never\nsplit, because a filter clause carries its own commas. --grain is checked\nagainst the grains the layer reports for the metrics queried, so a refusal\nnames the ones that metric has.
Two payload fields carry legitimate differences between the backends rather\nthan leaving them to be inferred: dimension_scope says whether a dimension\nrow is one declaration or one groupable path, which is why two backends can\nreport different dimension counts for one layer, and unavailable names fields\na backend structurally cannot supply. --local resolves the join graph through\nMetricFlow where the [semantic] extra is installed, which is what makes its\ndimension lists the tokens a query can actually use; without it the payload says\ndeclarations and a note names the extra.
Three backends answer these commands, chosen by .dex/config.yml\nsemantic.vendor and semantic.deployment (the older semantic.backend\nspelling still works),\noverridable with --local / --api. Those two flags name who executes, not\nwhich vendor, and every result reports it as execution (dex or vendor).\n--local renders the SQL with MetricFlow and executes it through dex's own\nconnector and cost handshake, so cost is surfaced before spend (needs a dbt\nproject parsed at least once, and the [semantic] extra for values and\nquery; list reads the project and needs no extra). --api sends the query to\na hosted dbt Cloud deployment (needs a host, an environment id and a\nDBT_SL_TOKEN, plus [semantic-api], and no local project). The hosted backend\nis the one place the cost guard cannot apply: dbt Cloud executes server-side, so\nthe result carries an explicit warning that spend is governed there and no\n--confirm is asked. Either way a PII-shaped grouped or filtered dimension (for\nexample user__email) is refused before the query runs, and on --api the\nlayer's own PII metadata is fetched per metric so a multi-metric query stays\nauthoritative rather than falling back to names.
The third backend is semantic.vendor: ossie, native Apache Ossie documents\nread out of the repository with no dbt project and no MetricFlow in the path\n(needs the [ossie] extra). It is catalog-first: list answers, and values,\nquery and --for-dimension refuse by name, because Ossie specifies\ninterchange metadata and no portable query runtime. Those refusals are the\nformat's shape rather than a missing feature, and each one names the physical\nroute instead: a dimension carries its semantic_model, that model carries its\nrelation, and explore profile then explore query reach the values under\nthe firewall and the cost guard. --api is refused too; Ossie has no hosted\ndeployment.
Read ${CLAUDE_SKILL_DIR}/references/semantic-playbook.md before running a\nmetric query: a metric's time_axis, filter and measures decide what the\nnumber is, and the playbook covers the discovery order, the additivity and\ntime-axis traps this surface is full of, when values answers rather than a\nquery, and what changes when the layer is native Ossie.
Rules of engagement for query: prefer the fixed commands when they answer the\nquestion; one probe answers one question; batch related measures into a single\nquery rather than issuing many; aggregates over PII-flagged columns must be\nmeasuring (COUNT, APPROX_COUNT_DISTINCT, AVG(LENGTH(...))), never value-carrying\n(MIN, ANY_VALUE, STRING_AGG). The FROM clause may unnest JSON and array\ncolumns in the connector's native idiom, which is the right way to explore\nschemaless data (for example \"which keys appear across every row of this JSON\ncolumn\"): BigQuery t, UNNEST(JSON_KEYS(doc)) AS k, Snowflake\nt, LATERAL FLATTEN(input => doc) f, Databricks\nt LATERAL VIEW EXPLODE(json_object_keys(doc)) x AS k, Postgres\nt, jsonb_object_keys(doc) AS k, Redshift t, UNPIVOT t.doc AS v AT k,\nDuckDB t, UNNEST(json_keys(doc)) AS u(k), ClickHouse\nt ARRAY JOIN JSONExtractKeysAndValuesRaw(doc) AS kv (there is no lateral\njoin; ARRAY JOIN is the expansion). The unnested value must come from\na column of a table in the query (bare, or through a JSON/array function);\nunnesting a subquery, another table, a literal, or a generator is refused,\nand the unnest's outputs inherit the source column's PII flags. A column whose\nflag was de-rated below the blocking threshold projects normally, with an\nenvelope warning naming it; treat the warning as information for the user, not\nan error to fix. If the user says a refused column is not personal data,\nrecommend a pii_overrides entry in .dex/config.yml (fully qualified column,\noptional reason): it unblocks querying immediately, survives re-profiles, and is\nreviewable in git. Never hand-edit .dex/cache.json to clear a flag. Never fall\nback to raw Python or a database CLI to run SQL; the firewall path is the only\nsanctioned one.
A remote warehouse or database replaces --path with connector config. Start\nwith connect test --connector <name> (or set connector: plus the matching\nblock in .dex/config.yml: bigquery: with project and a datasets\nallowlist, snowflake: with the pinned warehouse and a databases\nallowlist, databricks: with the pinned SQL warehouse and a catalogs\nallowlist, postgres: with a schemas allowlist, redshift: with the\nServerless workgroup and a schemas allowlist). Credentials are\ndiscovered, never asked for: if the envelope reports missing or expired\ncredentials, relay the fix it names (for BigQuery\ngcloud auth application-default login; for Snowflake a connections.toml\nentry or SNOWFLAKE_* env; for Databricks databricks auth login or\nDATABRICKS_* env; for Postgres DATABASE_URL, PG* env, or a\npg_service.conf entry; for Redshift the AWS credential chain\n(aws configure, AWS_* env) or REDSHIFT_* env) and never ask the user to\npaste a key, token, or password.
On a metered connector, scanning commands (profile, map, relationships,\nquery) run a two-step handshake. The first call returns needs_confirmation\nwith an estimate in cost.estimate, a per-table breakdown where relevant, and\nthe unit it is counted in: bytes on BigQuery, warehouse-seconds on Snowflake\n(credits alongside) and Databricks (DBUs), compute-seconds on Redshift\n(RPU-hours), database-seconds on Postgres and ClickHouse (no dollars; the\nguarded quantity is load). Surface the estimate to the user in human units, get\nan explicit budget from them, and re-issue the same command with --confirm and\n--budget <magnitude> in that unit. Never invent a budget the user did not\nagree to, and never retry with a raised budget on an over-ceiling refusal\nwithout asking. Metadata is free (connect test, inventory run immediately),\nand OK envelopes report actual spend under data.spend.
An over-ceiling refusal now carries a calibration line drawn from\n.dex/spend.jsonl: what this connector's last few settled commands actually\nbilled as a fraction of what they were estimated at, or a sentence saying the\nproject has too little history to say. On a partitioned or clustered warehouse a\ndry-run estimate is an upper bound, so this is often the difference between a\nbudget that admits the work and one that does not. Relay it verbatim when you\nsurface the refusal, and note the part callers get wrong: the ceiling is checked\nagainst the estimate, so a budget set at the observed fraction of the estimate\nis refused again. It is still the user's decision, never yours.
When a needs_confirmation envelope carries suggested_session_ceiling, the\nproject has never decided whether the day's total spend is bounded, and this is\nthe one time it is asked. Surface it beside the per-command estimate and get the\nuser's answer: --session-ceiling <value> sets a cumulative cap for the project\n(the suggestion is five times this command's estimate, a starting point, not a\nrecommendation), and --no-session-ceiling records that the project runs\nunbounded. Either one is written to .dex/config.yml and reported as a diff, and\nnothing asks again. Add it to the same re-issue that carries --confirm --budget, or the confirmed run will stop once to ask. Never answer it on the\nuser's behalf: it is a durable project setting, not a per-command flag.
On BigQuery a profiling estimate holds a 10 MB floor per table for each\nescalation query a profile may still issue after its aggregate scan, so on a\nwarehouse of many small tables most of the number can be reserve for work that\nnever happens. Both the handshake and the over-ceiling refusal report that split\n(reserved_bytes and reserved_queries, and in the prose). Pass it on when you\nsurface the estimate: whether a number is scan or reserve changes whether\nraising the budget is buying work or headroom.
When an estimate is larger than the work deserves, narrow the scope rather than\nraise the budget. --scope (repeatable) bounds a command to part of the\nconfigured source allowlist, in the connector's own vocabulary: a dataset on\nBigQuery, a schema or database.schema on Snowflake, a catalog.schema on\nDatabricks, a schema on Postgres or Redshift, a database on ClickHouse (whose\nidentifiers are two-part database.table: there is no catalog level). It is\nfree to resolve, it can only narrow what\n.dex/config.yml already allows, and a scope that names nothing is refused with\nthe schemas that do exist listed. So explore map --scope <schema> is the first\nthing to reach for on a warehouse whose full map would be expensive.
pii_overrides\nentry clears a flag entirely; never suggest weakening the detection.references/pii-policy.md and references/cost-controls.md.