sota-cli-ux
State-of-the-art CLI and developer-tool UX guidance (2026) covering command and flag design, output and interaction (stdout/stderr, --json, TTY detection, exit codes, prompts), runtime behavior and lifecycle (signals, dry-run, idempotency, XDG paths, completions, telemetry), and
Install
npx skills add https://github.com/martinholovsky/SOTA-skills/tree/main/skills/sota-cli-ux
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install martinholovsky-sota-skills@llmmart
git clone https://github.com/martinholovsky/SOTA-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole martinholovsky/sota-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
SOTA CLI & Developer-Tool UX
Purpose
Expert-level rules for building and auditing command-line tools: the grammar of commands/flags/args, config layering, human-vs-machine output, TTY-aware interaction, exit codes, signal handling, lifecycle behavior, and distribution. The core thesis: a CLI has three users — a human at a TTY, a script in CI, and an AI agent driving the tool (a stricter script) — and every design decision must serve all three without flags-gymnastics. Rules are imperative with rationale and good/bad terminal examples; every rules file ends with an audit checklist. Load only the files relevant to the task via the index below.
BUILD mode
When designing or implementing a CLI:
- Design the command grammar before writing code. Write the
--helptext first: subcommand tree, every flag (short + long), args, examples. If the help is hard to write, the grammar is wrong (rules/01). - Defaults carry the common path. A new user must get a useful result with
zero flags. Anything required for the 80% case is a design bug (
rules/01§3). - Split the streams from day one: primary output → stdout, everything else
(logs, progress, prompts, errors) → stderr. Retrofitting this breaks users'
pipes (
rules/02§1). - Make it pipe-safe by default: detect TTY; disable color/progress/prompts
when piped; honor
NO_COLOR; ship--jsonwith every listing/reading command from v0.1, because output format is API (rules/02). - Treat exit codes, flags, env vars, and JSON shapes as a public API:
document them, version them, deprecate — never repurpose or remove without a
cycle (
rules/02§3,rules/03§7). - Build the unhappy paths with the happy path: Ctrl-C cleanup,
--dry-runon mutating commands, re-runnable/resumable operations, stdin-closed CI behavior, offline behavior (rules/03). - Plan distribution early: single static binary if the ecosystem allows,
checksums + signatures, completions and man page generated from the same
source as
--help(rules/03§8,rules/04). - Before declaring done, run every relevant Audit checklist against your
own tool, including the brutal smoke test:
tool cmd | cat,tool cmd > out.txt 2> err.txt,tool cmd < /dev/null,echo $?.
AUDIT mode
When reviewing an existing CLI:
- Identify the surface: parser setup (argparse/clap/cobra/etc.), main/entry point, output and error paths, signal handlers, config loading, install/ release scripts. Load the matching rules files.
- Run the tool, don't just read it. Minimum probe set:
tool --help,tool <sub> -h,tool --version,tool definitelynotacmdtool list | catandtool list | head -1(color codes? broken pipe panic?)tool list > /dev/null— does anything still reach the human? (it should, on stderr)tool mutate < /dev/null— hang waiting for a prompt = CI killerecho $?after success, after a usage error, after a real failure- Ctrl-C mid-operation: prompt state restored? partial state cleaned or resumable?
- Then verify in code what can't be probed: config precedence order, secret handling, temp/state file locations, update/telemetry behavior.
Severity conventions
- Critical — corrupts data or destroys trust: destructive op with no
confirmation/
--forceand no dry-run; non-zero work reported as exit 0 (or vice versa) so CI lies; prompt hangs forever with stdin closed; secrets echoed to terminal/logs/argv; auto-update or telemetry without disclosure. - High — breaks scripting or interrupts users: machine output polluted by
ANSI codes/log lines on stdout; no
--jsonon listing commands; SIGINT leaves corrupt partial state; undocumented/colliding exit codes; config precedence nondeterministic; breaking flag removal without deprecation. - Medium — erodes usability: missing long forms; required flags on common
path; no progress on >2s ops; errors without remediation;
$HOMEdotfile litter instead of XDG; no completions; no-q/-v; >500ms--help. - Low — polish: help without examples, no suggest-on-typo, inconsistent subcommand naming, missing man page, table borders in output.
Finding format
[SEVERITY] <one-line title>
Where: <file:line | command invocation that reproduces it>
Rule: <rules-file §section>
Issue: <what is wrong, with observed evidence (transcript or code)>
Impact: <who breaks: the human at the TTY, the script in CI, an agent driving the tool, or all>
Fix: <specific change; exact flag/stream/exit-code where load-bearing>
Order findings by severity; one finding per root cause; include the reproducing command line whenever the issue was observed by running the tool.
Rules index
| File | Read this when... |
|---|---|
rules/01-commands-flags-config.md |
Designing/auditing the command surface: subcommand grammar (noun-verb consistency), POSIX/GNU flag conventions, short/long forms, -- separator, args vs flags vs stdin, defaults, dangerous-op flags, config precedence chain, env var naming, help text quality, suggest-on-typo. |
rules/02-output-interaction.md |
Anything the tool prints or asks: stdout vs stderr contract, --json/--plain, TTY detection, color and NO_COLOR/TERM, exit codes as API, streaming vs buffering, progress indication, prompts and --yes/CI behavior, quiet/verbose levels, error message anatomy, --debug vs stack traces, agent-facing design (--fields, confirmation envelopes, frozen output contracts). |
rules/03-behavior-lifecycle.md |
Runtime behavior and tool lifecycle: startup latency, idempotent re-runnable commands, --dry-run, --self-test for tools whose output is a verdict, SIGINT/SIGTERM handling, crash-only resumable design, network-op responsiveness and offline behavior, self-update caution, telemetry consent, XDG base directories, shell completions, semantic versioning of the CLI surface. |
rules/04-distribution-docs.md |
Shipping and documenting: single-binary packaging, install channels (brew/curl-script/registry) with checksums + signatures, docs generated from the --help source of truth, man pages, README quickstart, version pinning for CI. |
Top-10 non-negotiables
- Primary output to stdout, everything else to stderr — logs, progress,
prompts, warnings, errors.
tool x | nextmust receive only data. (rules/02 §1) - Pipe-safe by default: TTY-detect; no color, no spinners, no prompts when
piped; honor
NO_COLOR(set and non-empty) andTERM=dumb;--jsonon every command that lists or reads data. (rules/02 §2, §4) - Exit 0 only on success; distinct, documented nonzero codes for usage
error vs operational failure; never
exit 0after printing an error. (rules/02 §3) - Every flag has a long form; short forms only for the frequent few;
GNU/POSIX syntax (
-f,--flag,--flag=value,--ends flags). (rules/01 §2) - No required flags on the common path — good defaults; and the reverse:
destructive operations require explicit
--force/typed confirmation on a TTY and refuse (don't hang) without one. (rules/01 §3, §5) - Every prompt is skippable:
--yes/--no-inputequivalent exists; stdin closed or non-TTY ⇒ use default or fail fast with the flag to pass — never block CI. (rules/02 §6) - Errors say what failed, why, and what to do next — path, value, and the
exact command to run; stack traces only under
--debug. (rules/02 §7) - Ctrl-C is sacred: first SIGINT exits promptly with cleanup and exit code 130, second SIGINT force-quits; terminal state always restored; interrupted work is resumable or rolled back. (rules/03 §3)
--dry-runon every mutating command, printing the real plan through the real code path. (rules/03 §2)- The CLI surface is a semver'd API: flags, exit codes, JSON fields, env vars. Deprecate with warnings for a full cycle; never silently change meaning. Config precedence is exactly flags > env > project config > user config > defaults, documented. (rules/01 §6, rules/03 §7)
Files (sota-skills)
-
rules
-
01-commands-flags-config.md 11.7 KB
# 01 — Commands, Flags, Args & Configuration Scope: subcommand grammar, POSIX/GNU flag conventions, args vs flags vs stdin, defaults, dangerous operations, config precedence, env var naming, help text. ## 1. Command grammar - Pick **one** subcommand ordering scheme and never mix: - `noun verb` (`tool repo clone`, `tool repo list`) — scales best when nouns multiply; group help by resource. Prefer this for tools with >1 resource type. - `verb noun` (`tool get pods`) — fine for query-centric tools; commit to it. - Bad: `tool repo clone` next to `tool list-users` next to `tool prune repos`. Three grammars = user must memorize every command individually. - Single-purpose tools need **no subcommands** (`grep`, `jq`). Don't invent `tool run` as the only subcommand; make the tool do its thing directly. - Names: lowercase, single word; verbs from the conventional set — `list, get, create, delete, update, add, remove, run, start, stop, status, init, config, login, logout, version, completion, help`. Don't coin `enumerate` when `list` exists; don't have both `delete` and `remove` meaning different things. - Aliases are fine (`ls` → `list`, `rm` → `delete`) but help shows the canonical name and the alias resolves everywhere (completion, docs). - Max depth: two levels of subcommand (`tool noun verb`). A third level (`tool a b c`) is a smell — usually the third word is really an argument. - Reserve and implement: `tool help <cmd>` ≡ `tool <cmd> --help`, `tool version` ≡ `tool --version`, `tool completion <shell>`. - **Suggest on typo** (Levenshtein over the command table), then exit nonzero: ``` $ tool stauts error: unknown command "stauts" Did you mean "status"? Run 'tool --help' for usage. $ echo $? 2 ``` Never auto-execute the guess — `tool prune` guessed from `tool prun` deleting data is unforgivable. (cobra: disable `SuggestionsMinimumDistance` auto-run is default-safe; clap: `suggestions` on by default; argparse: add a custom hook.) Agent-facing/machine modes may disable suggestions entirely (e.g. cobra's `DisableSuggestions`) — for an AI agent an unknown command should fail hard, not offer a guess it might adopt (rules/02 §9). ## 2. Flag syntax — POSIX/GNU, no improvisation - **Every flag has a long form** (`--output`); short forms (`-o`) only for the handful used constantly. Long-only flags are self-documenting in scripts and CI configs — that's where flags get read by other humans. - Support standard GNU behavior: bundling (`-abc` = `-a -b -c`), `--flag=value` and `--flag value`, `--` terminates flag parsing (everything after is positional — mandatory for tools that accept filenames, which can start with `-`): ``` $ tool delete -- --weird-filename ``` - Repeatable flags for lists: `-v -v` for verbosity, `--tag a --tag b` for values. Prefer repetition over ad-hoc delimiters (`--tag a,b` may be offered *additionally*, but commas appear in real values). - Never invent single-dash long flags (`-output`). Java-style `-Dkey=val` and Go's single-dash flags exist; don't propagate them in new tools. - Flag names: kebab-case (`--dry-run`, not `--dryRun`/`--dry_run`); booleans are plain switches with a `--no-<flag>` negation when the default is true (`--color` / `--no-color`). - **Same flag, same meaning, everywhere.** `-o` must not mean `--output` in one subcommand and `--organization` in another. Maintain a global flag registry; audit collisions in review. - Standard names users already know — don't get creative: `-h/--help`, `--version`, `-q/--quiet`, `-v/--verbose`, `-o/--output`, `-f/--force`, `-n/--dry-run` (or `--dry-run` long-only), `--json`, `--yes`, `--no-color`, `--config`, `-C <dir>` (run as if in dir). Note the trap: `-v` is verbose in most tools but version in some — pick verbose, give version only `--version` (and `-V` if you must). ## 3. Args vs flags vs stdin - Positional args only for the **primary object(s)** of the command, the thing you'd say in the sentence: `tool deploy <service>`, `rm file1 file2`. - One type of positional is fine; two types (`tool copy <src> <dst>`) is the ceiling; three or more positionals of different meanings is a design bug — switch the extras to flags. Order-dependence is where users make mistakes. - Everything optional, every modifier, every "how" → flag. Flags are self-documenting at the call site and reorderable. - **No required flags on the common path.** If `--region` is required for every invocation, it's not a flag, it's missing config/default. Required flags are acceptable only on rare expert subcommands. - stdin: accept `-` as a filename meaning stdin (GNU convention); read data from stdin when no file arg is given **and stdin is not a TTY**: ``` $ cat payload.json | tool apply # data via pipe $ tool apply -f payload.json # same, explicit $ tool apply # TTY stdin: don't hang — print usage ``` Never silently block reading a TTY stdin the user didn't intend to type into. - Secrets: **never as argv** (`--password hunter2` is visible in `ps`, shell history, CI logs). Accept via file (`--password-file`), stdin (`--password-stdin`, the docker pattern), or interactive no-echo prompt. Env vars for secrets are a last resort — they leak into child processes and crash dumps; if supported, document the risk. ## 4. Defaults - The zero-flag invocation must do the obviously useful, **safe** thing. Optimize for the 80% case; flags exist for the 20%. - Defaults must be visible: show effective values in `--help` (`--timeout <secs> request timeout (default: 30)`) and provide `tool config list`/`--show-config` to print the fully resolved configuration **with the source of each value** (flag/env/file/default) — this single feature kills most "works on my machine" config bugs. - Default to the least destructive interpretation. `tool clean` defaulting to "delete everything matching" is wrong; default narrow, widen with flags. ## 5. Dangerous operations - Destructive/irreversible commands (delete, overwrite, force-push, prune): - On a TTY: confirm interactively; for severe cases require typing the resource name (the GitHub repo-deletion pattern), not just `y`. - Non-TTY/script: **refuse with a clear error** naming the bypass flag — never hang on a prompt, never assume yes: ``` $ tool env delete prod < /dev/null error: refusing to delete environment "prod" without confirmation hint: pass --force to delete without prompting $ echo $? 1 ``` - `--force`/`-f` bypasses confirmation; `--yes`/`-y` answers benign prompts. Keep them distinct — `--yes` must not unlock destructive paths. - Big blast radius needs friction proportional to damage: deleting one item = `y/N`; deleting a namespace = type its name; `--all` + `--force` together for "everything". - Pair every dangerous op with `--dry-run` (rules/03 §2) and, where feasible, a grace mechanism (trash/soft-delete/`undo`, or print what to back up first). ## 6. Configuration precedence - Exactly this chain, highest wins, documented in `--help`/docs verbatim: 1. command-line flags 2. environment variables 3. project-level config (e.g. `./.toolrc`, `./tool.toml` — found by walking up from cwd, stopping at repo root or `$HOME`) 4. user-level config (`$XDG_CONFIG_HOME/tool/config.toml`, see rules/03 §6) 5. system-level config (`/etc/tool/…`) — only if genuinely needed 6. built-in defaults - Resolution must be **deterministic and explainable**. If two project files could apply, define the rule (nearest wins) and surface it in `--show-config`'s source column. - Every config key gets a flag; most get an env var. Not every flag needs a config key (one-shot flags like `--dry-run` shouldn't be configurable — a config file silently forcing dry-run, or worse `force=true`, is a trap). - One config format. TOML or YAML for human-edited config; JSON only if humans never touch it (no comments). Validate on load and report the file and key: `error: ~/.config/tool/config.toml: unknown key "tiemout" (did you mean "timeout"?)`. ## 7. Environment variables - Namespace everything: `TOOL_*` (`TOOL_API_URL`, `TOOL_LOG_LEVEL`). Unprefixed names (`TIMEOUT`, `DEBUG_MODE`) collide with the user's environment and other tools. `TOOL_DEBUG=1` not `DEBUG=1` — though *reading* the conventional generics below is fine. - Respect the conventional generics rather than reinventing them: `NO_COLOR`, `TERM`, `EDITOR`/`VISUAL`, `PAGER`, `HTTP_PROXY`/`HTTPS_PROXY`/ `NO_PROXY`, `TMPDIR`, `XDG_*`. Also honor `CI` (set by virtually every CI system) as a signal to disable interactivity and fancy output. - Env vars are for context that varies per environment (endpoints, proxies, CI), not for per-invocation behavior — that's what flags are for. - Boolean env vars: treat set-and-non-empty as true, but accept `0`/`false` as false to match user expectations; document which convention you use. ## 8. Help text - `-h` and `--help` work on the root and **every** subcommand, print to stdout, exit 0, and never trigger side effects (no config load failures, no network). - Invoked with no args and no obvious default action: print concise help (or a short usage + hint) and exit **nonzero** — no-args is usually a mistake, and exit 0 would hide it from scripts. If the tool *has* a sensible default action, do that instead. - Help structure, in order: one-line description; usage line (`tool <command> [flags] <arg>`); **examples first among the detail** — 2-3 real, copy-pasteable invocations covering the common cases; then flags (short, long, value placeholder, description, default); then a docs URL. - Usage-line notation: `<required>`, `[optional]`, `...` repeatable, `|` alternatives. Keep placeholders meaningful: `--output <format>` not `--output <value>`. - Wrong usage gets a targeted error + the relevant usage line + exit code 2 — not the full help dump (which scrolls the actual error off-screen): ``` $ tool deploy error: missing required argument <service> usage: tool deploy <service> [flags] Run 'tool deploy --help' for details. ``` - Keep `--help` accurate by construction: generate docs/man/completions from the same parser definitions (rules/04 §3). ## Audit checklist - [ ] One subcommand grammar (noun-verb or verb-noun) used consistently; conventional verb names; ≤2 levels deep. - [ ] Unknown command/flag → suggestion + nonzero exit; never auto-executes the guess. - [ ] Every flag has a long form; kebab-case; `--flag=value` and `--` separator work; repeated flags accumulate. - [ ] No short-flag meaning collisions across subcommands; standard names (`-h`, `-q`, `-v`, `-o`, `-f`, `--json`, `--yes`) mean the standard things. - [ ] ≤2 positional arg types per command; everything else is a flag; no required flags on common-path commands. - [ ] `-` accepted for stdin where files are accepted; no command silently blocks reading TTY stdin. - [ ] No secret accepted via argv; file/stdin/no-echo-prompt paths exist. - [ ] Destructive ops: TTY confirmation (typed name for severe), non-TTY refuses with `--force` hint, `--yes` ≠ `--force`. - [ ] Config precedence is flags > env > project > user > system > defaults, deterministic, and documented; `--show-config` (or equivalent) prints resolved values with sources. - [ ] Config parse errors name file + key and suggest near-misses; unknown keys are errors or loud warnings, not silence. - [ ] All tool-specific env vars carry the `TOOL_` prefix; `NO_COLOR`, proxy vars, `EDITOR`, `PAGER`, `CI` respected. - [ ] `-h/--help` on every subcommand: stdout, exit 0, no side effects, examples present, defaults shown. - [ ] Usage errors print the specific error + short usage + exit 2, not the full help wall. -
02-output-interaction.md 13 KB
# 02 — Output, Exit Codes & Interaction Scope: stdout/stderr contract, machine-readable output, TTY detection, color, exit codes, streaming, progress, prompts, verbosity levels, error messages. ## 1. The stream contract - **stdout = the product. stderr = the process.** Primary output — the data the command exists to produce — goes to stdout. Logs, progress, spinners, warnings, prompts, errors, "Done in 3.2s" — all stderr. - The test: `tool cmd | next-tool` must feed `next-tool` only data; `tool cmd > out.json` must still show the human progress and errors on the terminal (because they're on stderr). ``` # Bad — log line corrupts the JSON consumer: $ tool export | jq . parse error: Invalid literal at line 1: "Connecting to api.example.com..." # Good: $ tool export 2>err.log | jq .name # data flows; chatter captured separately ``` - Success on a pure mutation may print a one-line confirmation to **stderr** (it's status, not output) — or print the created resource's ID to stdout, because that's data scripts will capture. Decide per command; be consistent. - Never write log lines and data interleaved on stdout "because the user sees one terminal". They see one terminal; their pipes don't. ## 2. Machine-readable output - **Every command that lists or reads data gets `--json`.** Output format is API: the moment someone greps your human table, you're locked into its whitespace forever. `--json` gives them a stable contract instead. - JSON mode: pure JSON on stdout, nothing else; one document, or **NDJSON (one object per line) for streams/lists** so consumers can process incrementally and `head`/`grep` keep working. - Field names are versioned API surface: additive changes only; renames go through deprecation (rules/03 §7). - Errors in JSON mode are JSON too (on stderr or as a final object — pick one and document it), with `code`/`message` fields, plus the nonzero exit. - Human table mode: no table borders/box-drawing — they're noise and break `awk`/`grep`. Columns separated by whitespace, one record per line, header row suppressible (`--no-header`) or absent when piped. A `--plain` flag that strips alignment/truncation keeps line-oriented tools viable without JSON. - `-o/--output <format>` (`json|yaml|table|...`) scales better than flag proliferation if you'll ever support more than two formats; `--json` can remain as sugar. - A `--format`/`--template` (Go-template/jsonpath-style) option is a power feature, not a substitute for `--json`. ## 3. Exit codes are API - `0` = success, **only** success. Partial failure is not success: `tool sync` that failed 3 of 10 items must not exit 0 just because it printed warnings. CI reads `$?`, not your prose. - Differentiate at minimum: - `0` success - `1` operational failure (the thing failed) - `2` usage error (bad flags/args — matches widespread Unix convention) - further codes for failure classes scripts must distinguish (e.g. `3` = check found violations, like `grep`'s "no match" `1` vs error `2`) - `130` after SIGINT (128+signal, the shell convention) — most runtimes do this if you re-raise the signal after cleanup rather than `exit(1)` (rules/03 §3). - Document every code in `--help`/man. Undocumented codes are unusable; scripts will treat all nonzero alike and lose the distinction you built. - Never `exit(0)` from a generic top-level error handler. Audit: search for exception/panic handlers that print and fall through to a normal return. - `--help`/`--version` exit 0; unknown flag exits 2. ## 4. TTY detection, color & terminal respect - Detect per stream: `isatty(stdout)` gates color/animation **on stdout**, `isatty(stderr)` gates spinners/progress **on stderr**, `isatty(stdin)` gates prompts. A pipe on stdout must not kill the progress bar on a TTY stderr. - Color policy, in precedence order: 1. `--color`/`--no-color` flag (or `--color=always|never|auto`) 2. `NO_COLOR` env var **set and non-empty** → no color (per no-color.org; empty string does *not* disable). `CLICOLOR_FORCE`/`FORCE_COLOR` non-empty → force color, winning over `NO_COLOR` (per force-color.org, which — like no-color.org — ignores the value; note the Node/chalk divergence: `FORCE_COLOR=0` disables, `1|2|3` set color depth — pick one behavior and document it) 3. auto: color only if the stream is a TTY and `TERM` is set and ≠ `dumb` - `--color=always` must exist — users pipe through `less -R` and CI renderers that handle ANSI. Auto-detection alone strands them. - Color conveys redundant emphasis, never sole meaning (colorblind users, `NO_COLOR` users): "error:" prefix in red, not red-means-error alone. - Pager: output known to be long (help dumps, logs) may auto-page on a TTY via `$PAGER` (default `less -FRX` behavior: quit if one screen). Always bypassable; never page when piped. - Handle `SIGPIPE`/`EPIPE` gracefully: `tool list | head -3` must not print a panic/traceback after `head` exits. Exit silently (conventionally 141). - Don't query or assume terminal width when not a TTY; when a TTY, wrap/truncate to width but never truncate in `--plain`/`--json` modes. ## 5. Streaming & progress - **Stream results as they're produced**; don't buffer the full result set and dump at the end. The user gets feedback, pipes get data early, and `head` terminates work early. Line-buffer stdout when piped (block buffering can hold output for minutes on slow producers). - Anything that can exceed ~2s shows progress on stderr (TTY only): spinner for indeterminate, progress bar with units (`12/87 files, 3.1 MB/s`) when total is known, step log (`[2/5] Building image…`) for phases. - Progress must include enough to act on: current item name (so a hang is diagnosable), rate, ETA when honest. - When not a TTY: replace animation with occasional plain log lines on stderr (e.g. one line per phase) — CI logs full of `\r` spinner frames or a thousand progress-bar redraws are a classic audit finding. - First feedback within ~100ms of invocation (even just validating args or a "Resolving dependencies…" line) — silence reads as a hang. ## 6. Prompts - Prompt **only** when stdin is a TTY. stdin closed or piped ⇒ take the documented default or fail fast naming the flag that supplies the answer. A CLI that hangs in CI awaiting input that will never come is a Critical finding. - **Never require a prompt**: every interactively-gathered value has a flag/env/config path. `--yes`/`-y` accepts benign confirmation defaults; `--no-input` forces "fail instead of prompting" for strict CI. - Prompts go to stderr (stdout may be redirected); read from `/dev/tty` only if you must prompt despite redirected stdin — and prefer not to. - Confirmation default is the safe answer: `Continue? [y/N]` — Enter aborts. - Password input: no echo, and verify the no-echo path actually engages when stdin is a TTY; never read passwords from argv (rules/01 §3). - Interactive convenience must never be the only path: a fancy selector (fuzzy-pick a target) is sugar over `tool deploy <target>`, not a replacement. ## 7. Error messages - Anatomy of an actionable error — **what failed, why, how to fix**: ``` # Bad: $ tool deploy api Error: operation failed # Good: $ tool deploy api error: cannot read config file ./tool.toml cause: line 14: unknown key "tiemout" (did you mean "timeout"?) hint: edit ./tool.toml or regenerate with 'tool config init' ``` Include the concrete noun: full path, URL, resource name, offending value. "Permission denied" without *which file* sends the user to strace. - The fix line is the highest-value text in your tool. If the remedy is a command, print the command, copy-pasteable. - **No stack traces at users by default.** Catch at the top level, print the human message, exit nonzero. Full traceback behind `--debug`/`TOOL_DEBUG=1`, plus a one-liner telling users to use it / where to report bugs. Audit: any user-reachable input that produces a raw traceback is at least Medium. - Errors to stderr, prefixed (`error:`/`warning:`), lowercase-after-prefix, no exclamation marks, no blame ("you provided an invalid…" → "invalid…"). - Expected-failure paths (not found, no match, already exists) are concise one-liners with distinct exit codes — not walls of context meant for bugs. ## 8. Verbosity levels - Three dials, independent of data output: - `-q/--quiet`: suppress non-error stderr chatter; errors still print; exit codes unchanged. (`-qq`/`--silent` to also suppress errors is optional — scripts that only want `$?`.) - default: progress + key status lines on stderr. - `-v/--verbose`, repeatable (`-vv`, `-vvv`): more diagnostic detail on stderr — what's being read, requests made, timing. - `--debug`: everything + internals (stack traces, wire dumps) — for bug reports. May alias `-vvv`. - Verbosity changes **stderr only**. `-v` must never add fields or lines to stdout data, and `-q` must never remove data from stdout — otherwise scripts change behavior based on log level. - Log-style stderr lines in verbose modes carry level prefixes (`debug:`/`info:`) so users can grep; honor `TOOL_LOG=debug`-style env config if the tool embeds a logger. - Redact secrets in every verbosity, including `--debug` wire dumps (`Authorization: Bearer ***`). Verbose modes leaking tokens into CI logs is a Critical finding. ## 9. AI agents as a third user - **Agents are now a consumer class of their own** alongside the human at a TTY and the script in CI — a stricter script: everything above (pure `--json`, structured errors, distinct exit codes, never prompt on non-TTY) applies, plus the constraints below. A well-designed CLI is a peer agent interface to an MCP server, not a lesser one. - Once agents drive the tool, the surface is **frozen, additive-only**: agents act on stale knowledge of your flags for months, so never remove or repurpose — only add (rules/03 §7, with less slack). - **Output costs context window.** Offer `--fields` (or equivalent) to select top-level JSON fields; keep default JSON lean; paginate large lists. - Put the remediation in-band: JSON-mode errors carry `code`/`message` plus a machine-actionable `remediation` (the exact command to run) — an agent can act on that, not on prose (§7's fix line, machine-readable). - Mutations: replace the interactive confirmation with a **confirmation envelope** — the first call returns the concrete plan, a distinct exit code, and the exact re-invocation (e.g. with `--confirm`); executing requires that second call. Never gate a mutation on a TTY prompt an agent cannot answer. - **Validate inputs locally before any network call**: agents hallucinate plausible-looking values (wrong-format IDs, invisible control characters); a precise client-side error beats a round-trip and a vague 404. - Consider suppressing did-you-mean suggestions in machine modes: for an agent, an unknown command should be a hard, unambiguous failure, not a guess it may adopt (rules/01 §1's never-auto-execute rule still holds). ## Audit checklist - [ ] Data on stdout only; logs/progress/prompts/errors on stderr; verified by `tool cmd >out 2>err` inspection. - [ ] `tool list | jq .` works: no ANSI, no banner, no log lines in the stream. - [ ] `--json` (or `-o json`) on every list/read command; NDJSON for streams; JSON-mode errors are machine-readable; fields treated as versioned API. - [ ] Human tables: no borders; `--plain`/`--no-header` or auto-plain when piped. - [ ] Exit codes: 0 only on full success; usage errors = 2; distinct documented codes for failure classes; partial failure ≠ 0; no `exit 0` in catch-all handlers. - [ ] SIGINT exits 130; `tool list | head` causes no EPIPE traceback. - [ ] Color: auto by TTY; `NO_COLOR` (non-empty) and `TERM=dumb` disable; `--color=always|never|auto` supported; color never sole carrier of meaning. - [ ] Output streams incrementally; stdout line-buffered when piped; no end-of-run dumps for long operations. - [ ] >2s operations show progress on stderr (TTY); non-TTY gets sparse plain lines, no `\r` animation spam in CI logs. - [ ] First output within ~100ms; no silent multi-second startup. - [ ] No prompt when stdin is not a TTY: `tool cmd </dev/null` never hangs; `--yes`/`--no-input` paths exist; prompts on stderr; safe default answer. - [ ] Errors name the failing path/value, state the cause, and give a copy-pasteable fix; stack traces only under `--debug`; errors prefixed and on stderr. - [ ] `-q` and `-v`/`-vv` exist; verbosity alters stderr only; stdout data identical at every level. - [ ] No secret/token appears at any verbosity level, including `--debug`. - [ ] Agent-facing surface: JSON output field-selectable (`--fields` or equivalent); JSON-mode errors include a machine-actionable remediation; mutations offer a non-interactive confirm protocol (plan + explicit `--confirm` re-invocation), not a TTY-only prompt. - [ ] Inputs validated locally (format/charset) before network calls; in machine modes unknown commands/flags fail hard rather than fuzzy-matching. -
03-behavior-lifecycle.md 15.2 KB
# 03 — Behavior & Lifecycle Scope: startup latency, idempotency, dry-run, signal handling, crash-only design, network/offline behavior, self-update, telemetry, XDG directories, shell completions, versioning the CLI surface. ## 1. Startup performance - `tool --help` and `tool --version` must feel instant — target well under 100ms perceived. These run constantly (humans exploring, completions, CI sanity checks); a 1.5s help is a tax on every interaction. - Common killers, in audit order: - importing/initializing the world before parsing args (Python tools that `import` heavy deps at module top — defer imports into the subcommand that needs them; compiled langs largely immune) - network calls on startup: update checks, telemetry, auth validation — none may block `--help`/`--version`; do them lazily, async, or in the subcommands that need them - loading/validating full config for commands that don't use it - Measure honestly: `time tool --help` cold and warm; on the JVM/Node, measure on the runtime your users have, not a warmed daemon. - Shell completion functions invoke the binary; slow startup makes every TAB lag the user's shell (§8). ## 2. Idempotency & dry-run - Make commands **re-runnable**: running the same command twice converges to the same state, second run a cheap no-op or explicit "already done" — not an error, not a duplicate. - `tool init` on an initialized dir: report "already initialized", exit 0 (offer `--force` to reset). - `create` where the resource exists with the same spec: succeed (or `already exists` + distinct exit code if callers must distinguish — pick one, document it). Desired-state semantics (`apply`) beat imperative `create` for anything users automate. - **`--dry-run` on every mutating command.** Requirements: - exercises the *real* code path (plan, validate, resolve) and skips only the write — a dry-run that takes a separate code branch lies; - prints the concrete plan: which files/resources, created/changed/deleted; - exits 0 if the run would proceed, nonzero if it would fail validation — making `--dry-run` a usable CI preflight; - performs **zero** writes, including logs-on-server side effects. - Operations on many items: report per-item outcome and end with a summary (`8 updated, 1 skipped, 1 failed`); overall exit nonzero if any failed (rules/02 §3); support resume rather than redo (§4). ## 2a. `--self-test` for tools whose output is a verdict A linter, a health checker, a scanner, a policy gate, a `doctor` command — anything whose output is a **verdict** rather than an artifact — is an instrument, and an instrument that cannot fail returns a plausible verdict on whatever it is handed (`sota-code-security` rules/15 §2). From the outside a user cannot tell: *"0 problems found"* and *"0 checks ran"* print the same, and the second is the ordinary result of a bad path, an empty glob, an over-narrow filter, or a missing toolchain. - **Ship a `--self-test`** (or `doctor --self-test`) that injects each check's declared known-bad and asserts *that named check* reports it. Exit nonzero when a check has no known-bad, when a mutation is not caught, **or when a probe fails for an unrelated reason** — a non-zero exit from the wrong cause is a false pass, not a catch (`sota-code-security` rules/12 §1b). - **Print the denominator on every ordinary run**, not only under `--self-test`: `checked 128 files, 3 problems`. A count of zero problems is only meaningful beside a count of what was examined, and users read the summary line, not the config. - **Skips are output, not silence.** A check that could not run (missing toolchain, no network, unsupported platform) prints `skipped: <reason>` and is counted separately from `passed`. Folding skips into passes is how a tool reports green on a machine where it did almost nothing. - **Keep `--self-test` offline and side-effect-free.** It is what a user runs when they already suspect the tool, so it must not need the network, must not write to the paths it checks, and must work on an installed copy — not just in the source tree (`--dry-run`'s "exercise the real path" rule, §2, applies to it too). ## 3. Signals: Ctrl-C is sacred - First SIGINT: stop accepting new work, cancel in-flight operations, run bounded cleanup (seconds, not minutes), restore terminal state, exit. Second SIGINT: exit immediately, skipping cleanup — the user has spoken. ``` ^C interrupted — rolling back partial upload (ctrl-c again to force quit) ``` - Exit status after SIGINT: re-raise the signal after cleanup (`signal(SIGINT, SIG_DFL); raise(SIGINT)`) so the shell sees death-by-signal (status 130) — `exit(1)` makes shell job control and script `trap`s misread what happened. - **Always restore the terminal**: raw mode off, cursor visible, colors reset, alternate screen exited. A TUI/prompt that leaves the shell invisible-cursor or no-echo on Ctrl-C is a High finding. Use defer/finally/atexit paths that run on signal, and test it: hit Ctrl-C inside every prompt and progress bar. - SIGINT must work **during network operations**: set cancellation on the request context (Go contexts, Python signal→cancel, Rust select on ctrl_c). A blocking socket read that ignores Ctrl-C for 60s reads as a hang. - Handle SIGTERM like SIGINT-without-prompt (CI and orchestrators send it); SIGHUP at minimum doesn't corrupt state. - Never trap SIGINT just to print "use 'exit' to quit" for batch commands — interactive REPLs may, batch operations may not. ## 4. Crash-only, resumable design - Assume every run can die at any instruction (OOM-kill, power, `kill -9` — no cleanup handler runs). Design so the *next* run recovers, instead of relying on graceful shutdown: - **Write-then-rename**: never truncate-and-rewrite config/state/output in place; write `file.tmp` (same filesystem), fsync, atomic `rename(2)`. Audit any `open(path, "w")` on files the tool also reads. - Long multi-step operations journal progress (state file/manifest) so `tool sync` after an interrupt resumes — or at minimum re-runs safely (§2). Partial downloads: `.partial` suffix + resume or restart cleanly. - Locks: prefer OS-released mechanisms (`flock`) over PID/lock files; if a lock file is unavoidable, store PID + start time and detect staleness — "delete .tool.lock and retry" instructions mean the design failed. - On detected leftover state: say what was found, what you did: `note: resuming interrupted sync from step 3/7 (state: .tool/journal)`. ## 5. Network behavior, offline & self-update - Timeouts on every network call — connect and overall — surfaced as a config default + `--timeout` flag. No infinite-hang defaults. - Retries with capped exponential backoff + jitter for idempotent requests only; say what's happening after the first failure (`warn: retrying (2/3) after timeout: GET https://api…`). - **Fail fast and clearly offline**: a DNS failure should produce `error: cannot reach api.example.com (DNS lookup failed) — check your network or set TOOL_API_URL` in ~seconds, not a 5-minute silent retry storm. Commands that *can* work offline (cached data, local ops) must not perform gratuitous network calls. - Update checks: never block startup; cache the result (`$XDG_CACHE_HOME/tool/`); notify at most once per interval on stderr, TTY only, suppressible (`TOOL_NO_UPDATE_CHECK=1`) and disabled when `CI` is set. - **Self-update (`tool self update`) is opt-in only — never automatic.** Auto-updating a CLI changes behavior under scripts between runs; teams pin versions for a reason (rules/04 §4). Self-update must verify signature/checksum before replacing the binary, use atomic rename, and respect non-writable install locations (brew/apt-managed binaries must refuse and point at the package manager). ## 6. Telemetry & files on disk - Telemetry: **opt-in, or at minimum loudly disclosed on first run with a one-command opt-out** (`tool telemetry off` / `TOOL_TELEMETRY=0`); honor the `DO_NOT_TRACK` convention; document exactly what is collected; never collect argv (it contains paths/names/secrets), file contents, or anything identifying without consent. Telemetry must never block or fail a command, never run when disabled, and respect `CI`. Undisclosed phoning-home is a Critical audit finding. - **XDG base directories — no `$HOME` litter.** Per the freedesktop spec: | Content | Env var | Default | |---|---|---| | config | `$XDG_CONFIG_HOME/tool/` | `~/.config/tool/` | | data (user-created, durable) | `$XDG_DATA_HOME/tool/` | `~/.local/share/tool/` | | state (history, logs, last-run) | `$XDG_STATE_HOME/tool/` | `~/.local/state/tool/` | | cache (safe to delete anytime) | `$XDG_CACHE_HOME/tool/` | `~/.cache/tool/` | | sockets/runtime | `$XDG_RUNTIME_DIR/tool/` | (unset ⇒ fall back to tmp + warn) | Respect the env vars when set (absolute paths only). New tools: don't create `~/.tool/`. Existing tools migrating: read old location if present, write new, say so once. On macOS, honoring XDG is the developer-tool norm even though `~/Library/...` is the platform convention — pick one, document it; on Windows use `%APPDATA%`/`%LOCALAPPDATA%`. - Cache must be safe to `rm -rf` at any time — that's its contract. Don't put the only copy of anything in cache. Secrets/tokens: own file with `0600` perms (verify at write *and* read; warn on loose perms), or the OS keychain. ## 7. The CLI surface is a semver'd API - The compatibility surface scripts depend on: command names, flag names and meanings, defaults, exit codes, env vars, config keys, JSON field names, and stdout format in `--json`/`--plain` modes. Changing any of these is a breaking change ⇒ major version. - **Deprecate, don't remove**: keep the old flag working as an alias; print a one-line stderr warning naming the replacement and the removal version (`warning: --out is deprecated, use --output (removal in v3)`); keep it for ≥1 minor cycle, realistically ≥6 months; only then remove — with a clear error pointing at the replacement, not "unknown flag". - Worse than removal is silent **meaning change**: same flag, different behavior. Never. If semantics must change, new flag name. - Human-readable default output may evolve freely **only if** `--json`/ `--plain` exist as the stable contract — ship them early precisely to buy this freedom (rules/02 §2). - `tool --version` prints `tool X.Y.Z` (+ commit/date as extra tokens or via `--version --json`); parseable, no network, instant. ## 8. Shell completions - Generate completions for **bash, zsh, fish** (PowerShell where relevant) from the same parser definition as `--help` — hand-maintained completion scripts drift. clap (`clap_complete`), cobra (built-in `completion` subcommand), Python (`argcomplete`/click/typer) all support this; use it. - Convention: `tool completion <shell>` prints the script to stdout; docs show the one-liner per shell; packages (brew/deb) install them into the standard directories so users get them for free. - Complete values, not just flag names: subcommands, enum flag values (`--output <TAB>` → `json yaml table`), and — where cheap — dynamic resource names. Dynamic completion calls back into the binary: it must be fast (§1) and **never block on the network or prompt**; degrade to nothing on failure. ## Rehearse a costly command before handing it to a human Before giving a person a command whose **failure** is expensive, run it where failure is free. Expensive means: limited attempts before lockout (card PINs, PUKs, 2FA), destructive, irreversible, rate-limited, paid, or requiring physical presence you cannot repeat cheaply. Find the substitution that exercises the same code path at zero cost — a file-based key instead of a hardware one, a dry-run flag, a throwaway target, `--help` parsed against the **installed** binary rather than remembered (and see `sota-code-security` rules/16 §2.15: a flag that parses is not a feature that works). Run it. *Then* hand over the real command. Field-reported: a YubiKey PIN allows **three** attempts before the applet blocks and needs a factory reset. A `cosign sign-blob` command composed from `--help` failed twice on flags deprecated in v3, each round-trip risking an attempt. Rehearsed against a throwaway file-based key — identical argument parsing and bundle-writing path, zero cost — it failed twice more for free, and the third form worked first time on the real device. **State the cost when you hand it over**: "this has three attempts before lockout" changes how carefully the person reads the line. ## Audit checklist - [ ] **Commands handed to a human are rehearsed where failure is free** when a failure costs an attempt, money, or an irreversible change — and the cost is stated in the handover. High wherever a lockout is possible: the recovery is often a factory reset that destroys unrelated material. - [ ] `time tool --help` ≪ 500ms; no network I/O on `--help`/`--version` (verify: airplane-mode or strace/dtruss spot-check). - [ ] Re-running `init`/`create`/`apply` twice converges; second run no-ops or reports "already" without failing. - [ ] Every mutating command has `--dry-run`; it runs the real plan path, prints concrete changes, writes nothing, and fails (nonzero) on what would fail. - [ ] Verdict-producing tools (lint/scan/check/`doctor`) ship a `--self-test` that injects each check's known-bad and asserts the **named** check caught it; every run prints what it examined, and skips are reported with a reason, never folded into passes. - [ ] Batch operations: per-item results + summary; nonzero exit on partial failure; resumable rather than restart-from-zero. - [ ] First Ctrl-C: prompt cleanup + quick exit, status 130 (signal re-raised); second Ctrl-C: immediate; terminal state (echo, cursor, raw mode) restored — tested inside prompts and progress bars. - [ ] Ctrl-C interrupts in-flight network calls promptly; SIGTERM handled like non-interactive SIGINT. - [ ] State/config writes are write-tmp-fsync-rename; no truncate-in-place; interrupted runs leave resumable or ignorable state, with a note on next run. - [ ] Locks self-clean (flock or staleness detection); no "delete the lock file" support folklore. - [ ] All network calls have connect + overall timeouts and `--timeout`; retries are bounded, jittered, idempotent-only, and announced; offline failure is fast and names the unreachable host. - [ ] Update check (if any): async, cached, stderr, TTY-only, off under `CI`, killable by env var; self-update opt-in, signature-verified, refuses package-manager-owned installs. - [ ] Telemetry opt-in or disclosed-on-first-run with documented scope and one-command opt-out; honors `DO_NOT_TRACK`; never collects argv/contents; no hidden network calls (verify with a proxy/strace if suspicious). - [ ] No new top-level `~/.tool*` litter: config/data/state/cache in XDG locations, env vars respected; cache survives `rm -rf` (tool regenerates); token files 0600 and checked. - [ ] Flags/exit codes/JSON fields never removed or repurposed without deprecation warnings naming the replacement; deprecated aliases still function for a full cycle. - [ ] `tool completion bash|zsh|fish` works and is generated from the parser source; dynamic completion is fast, offline-safe, and silent on failure. -
04-distribution-docs.md 5.9 KB
# 04 — Distribution & Documentation Scope: packaging, install channels, supply-chain hygiene for installs, docs from a single source of truth, man pages, README quickstart, CI pinning. ## 1. Packaging - **Prefer a single static binary** (Go, Rust, Zig; or a properly bundled artifact otherwise). The user-visible contract: download one file, `chmod +x`, it runs — no runtime version conflicts, no `pip install` clobbering system packages, trivially pinnable in CI. - Interpreted ecosystems: ship via the isolating installer users expect — `pipx`/`uv tool install` for Python, `npx`/global install for Node — and say so in the README; never instruct `sudo pip install`. Lock your dependency versions in the published artifact; a CLI that breaks because a transitive dep released is your bug. - Build per-platform artifacts for at minimum: linux amd64/arm64, macOS amd64/arm64 (or universal), windows amd64. Predictable asset names (`tool_1.4.2_linux_arm64.tar.gz`) — CI scripts construct these URLs. - The binary must not assume sibling files at runtime (embed assets); must run from any cwd; must not require root for normal operation. - Don't strip `--version` info from release builds; embed version + commit at build time, and ensure source builds (`go install`, `cargo install`) don't report `dev`/`unknown` if avoidable. ## 2. Install channels & supply-chain hygiene - Offer at least: (a) a package manager (Homebrew tap/core, apt/yum repo, scoop/winget, or language registry) for humans, and (b) **direct versioned binary downloads** for CI (§4). GitHub Releases as the canonical artifact store is the de facto norm. - `curl | sh` installers are popular and acceptable **only** with discipline: - the script is short, readable, and pipes-safe (works when partially downloaded — guard by wrapping everything in a function called at EOF); - supports `TOOL_VERSION=1.4.2` pinning and a target-dir override; never silently `sudo`; - downloads over HTTPS and **verifies the checksum** of the artifact it fetches before installing. - Publish `checksums.txt` (SHA-256 of every artifact) with each release, and sign your releases — current SOTA favors keyless signing (e.g. Sigstore `cosign`) and/or build provenance attestations (SLSA-style, GitHub artifact attestations) over bare GPG keys nobody verifies. Document the one-line verification command next to the download link, or nobody will run it. - Package-manager installs own the binary: in-tool `self update` must detect and defer to them (rules/03 §5). - Don't squat ambiguous names; check registries before naming. The binary name is forever-API too. ## 3. Docs from one source of truth - **`--help` is the source of truth.** Generate man pages, the docs-site CLI reference, and completions from the same parser definitions (clap → `clap_mangen`; cobra → `cobra/doc` for man + markdown; click/typer → sphinx-click etc.). Hand-written copies drift within two releases — drift between `--help` and web docs is a standing audit check. - Ship a man page if your users live where `man` is reflexive (system tools); a web reference is otherwise acceptable — but `tool <cmd> --help` must then be fully self-sufficient, with the docs URL printed at the bottom of help. - Docs structure that works for CLIs: quickstart → task-oriented how-tos → full generated reference → exit codes table → env vars table → config file schema. Exit codes and env vars are the pages scripts authors hunt for and most tools forget. - Every example in docs must be copy-pasteable and CI-tested if feasible (doc-test runners / cram-style tests); stale examples are worse than none. ## 4. README quickstart & CI pinning - README top section, in order, ≤ one screen: one-sentence what-it-is; install one-liner; **a 5-line quickstart that produces a visible result**: ``` $ brew install tool $ tool init $ tool deploy api deployed api → https://api.example.dev (2.1s) ``` If the quickstart needs prose paragraphs of prerequisites, the tool's defaults are wrong (rules/01 §4), not the README. - **Version pinning for CI is a feature, not an afterthought**: - stable per-version download URLs that never change or disappear — old releases stay up; - install paths accept an exact version (`TOOL_VERSION=…` in the script, `tool@1.4` in brew/npm, apt version pins); - provide or bless a GitHub Action / setup script with a `version:` input; - never offer only "latest" — CI on floating latest breaks on your release day, and the bug reports land on you. - Changelog per release, human-written headline per breaking change, with the migration command/flag mapping. `tool` major upgrades deserve a short upgrade guide; deprecation warnings in-product should link to it (rules/03 §7). ## Audit checklist - [ ] Single-file (or properly isolated) install; no runtime dependency on system interpreter state; runs from any cwd without sibling files; no root required. - [ ] Artifacts for linux/macOS (amd64+arm64) and windows with predictable names; old release assets remain downloadable. - [ ] `checksums.txt` (SHA-256) published per release; artifacts signed or provenance-attested; verification command documented where the download link is. - [ ] Install script (if any): function-wrapped against partial download, HTTPS-only, checksum-verifying, version-pinnable, no silent sudo. - [ ] Man page and/or web CLI reference generated from the same definitions as `--help`; spot-check three commands for drift between `--help` and published docs. - [ ] Docs include exit-code table, env-var table, and config schema; examples copy-pasteable and current. - [ ] README: install + working 5-line quickstart on the first screen. - [ ] CI consumers can pin an exact version through every advertised install channel; a "latest"-only channel is not the sole option. - [ ] Changelog exists; breaking changes called out with migration steps; deprecation warnings link to them.
-
-
SKILL.md 8.9 KB
--- name: sota-cli-ux description: >- State-of-the-art CLI and developer-tool UX guidance (2026) covering command and flag design, output and interaction (stdout/stderr, --json, TTY detection, exit codes, prompts), runtime behavior and lifecycle (signals, dry-run, idempotency, XDG paths, completions, telemetry), and distribution (packaging, checksums, docs). Use when designing or building any command line tool, subcommand, TUI, or developer tool — in any framework (argparse, click, typer, clap, cobra, oclif, commander) — AND when auditing an existing CLI for usability, scriptability, and compatibility. Not for shell-script correctness or security — use sota-shell-scripting. Trigger keywords: CLI, command line tool, flags, subcommands, terminal output, TUI, developer tool, argparse, clap, cobra, exit code, shell completion, man page, stdin, stdout. --- # SOTA CLI & Developer-Tool UX ## Purpose Expert-level rules for building and auditing command-line tools: the grammar of commands/flags/args, config layering, human-vs-machine output, TTY-aware interaction, exit codes, signal handling, lifecycle behavior, and distribution. The core thesis: **a CLI has three users — a human at a TTY, a script in CI, and an AI agent driving the tool (a stricter script) — and every design decision must serve all three without flags-gymnastics.** Rules are imperative with rationale and good/bad terminal examples; every rules file ends with an audit checklist. Load only the files relevant to the task via the index below. ## BUILD mode When designing or implementing a CLI: 1. **Design the command grammar before writing code.** Write the `--help` text first: subcommand tree, every flag (short + long), args, examples. If the help is hard to write, the grammar is wrong (`rules/01`). 2. **Defaults carry the common path.** A new user must get a useful result with zero flags. Anything required for the 80% case is a design bug (`rules/01` §3). 3. **Split the streams from day one**: primary output → stdout, everything else (logs, progress, prompts, errors) → stderr. Retrofitting this breaks users' pipes (`rules/02` §1). 4. **Make it pipe-safe by default**: detect TTY; disable color/progress/prompts when piped; honor `NO_COLOR`; ship `--json` with every listing/reading command from v0.1, because output format is API (`rules/02`). 5. **Treat exit codes, flags, env vars, and JSON shapes as a public API**: document them, version them, deprecate — never repurpose or remove without a cycle (`rules/02` §3, `rules/03` §7). 6. **Build the unhappy paths with the happy path**: Ctrl-C cleanup, `--dry-run` on mutating commands, re-runnable/resumable operations, stdin-closed CI behavior, offline behavior (`rules/03`). 7. **Plan distribution early**: single static binary if the ecosystem allows, checksums + signatures, completions and man page generated from the same source as `--help` (`rules/03` §8, `rules/04`). 8. Before declaring done, run every relevant **Audit checklist** against your own tool, including the brutal smoke test: `tool cmd | cat`, `tool cmd > out.txt 2> err.txt`, `tool cmd < /dev/null`, `echo $?`. ## AUDIT mode When reviewing an existing CLI: 1. Identify the surface: parser setup (argparse/clap/cobra/etc.), main/entry point, output and error paths, signal handlers, config loading, install/ release scripts. Load the matching rules files. 2. **Run the tool, don't just read it.** Minimum probe set: - `tool --help`, `tool <sub> -h`, `tool --version`, `tool definitelynotacmd` - `tool list | cat` and `tool list | head -1` (color codes? broken pipe panic?) - `tool list > /dev/null` — does anything still reach the human? (it should, on stderr) - `tool mutate < /dev/null` — hang waiting for a prompt = CI killer - `echo $?` after success, after a usage error, after a real failure - Ctrl-C mid-operation: prompt state restored? partial state cleaned or resumable? 3. Then verify in code what can't be probed: config precedence order, secret handling, temp/state file locations, update/telemetry behavior. ### Severity conventions - **Critical** — corrupts data or destroys trust: destructive op with no confirmation/`--force` and no dry-run; non-zero work reported as exit 0 (or vice versa) so CI lies; prompt hangs forever with stdin closed; secrets echoed to terminal/logs/argv; auto-update or telemetry without disclosure. - **High** — breaks scripting or interrupts users: machine output polluted by ANSI codes/log lines on stdout; no `--json` on listing commands; SIGINT leaves corrupt partial state; undocumented/colliding exit codes; config precedence nondeterministic; breaking flag removal without deprecation. - **Medium** — erodes usability: missing long forms; required flags on common path; no progress on >2s ops; errors without remediation; `$HOME` dotfile litter instead of XDG; no completions; no `-q`/`-v`; >500ms `--help`. - **Low** — polish: help without examples, no suggest-on-typo, inconsistent subcommand naming, missing man page, table borders in output. ### Finding format ``` [SEVERITY] <one-line title> Where: <file:line | command invocation that reproduces it> Rule: <rules-file §section> Issue: <what is wrong, with observed evidence (transcript or code)> Impact: <who breaks: the human at the TTY, the script in CI, an agent driving the tool, or all> Fix: <specific change; exact flag/stream/exit-code where load-bearing> ``` Order findings by severity; one finding per root cause; include the reproducing command line whenever the issue was observed by running the tool. ## Rules index | File | Read this when... | |---|---| | `rules/01-commands-flags-config.md` | Designing/auditing the command surface: subcommand grammar (noun-verb consistency), POSIX/GNU flag conventions, short/long forms, `--` separator, args vs flags vs stdin, defaults, dangerous-op flags, config precedence chain, env var naming, help text quality, suggest-on-typo. | | `rules/02-output-interaction.md` | Anything the tool prints or asks: stdout vs stderr contract, `--json`/`--plain`, TTY detection, color and `NO_COLOR`/`TERM`, exit codes as API, streaming vs buffering, progress indication, prompts and `--yes`/CI behavior, quiet/verbose levels, error message anatomy, `--debug` vs stack traces, agent-facing design (`--fields`, confirmation envelopes, frozen output contracts). | | `rules/03-behavior-lifecycle.md` | Runtime behavior and tool lifecycle: startup latency, idempotent re-runnable commands, `--dry-run`, `--self-test` for tools whose output is a verdict, SIGINT/SIGTERM handling, crash-only resumable design, network-op responsiveness and offline behavior, self-update caution, telemetry consent, XDG base directories, shell completions, semantic versioning of the CLI surface. | | `rules/04-distribution-docs.md` | Shipping and documenting: single-binary packaging, install channels (brew/curl-script/registry) with checksums + signatures, docs generated from the `--help` source of truth, man pages, README quickstart, version pinning for CI. | ## Top-10 non-negotiables 1. **Primary output to stdout, everything else to stderr** — logs, progress, prompts, warnings, errors. `tool x | next` must receive only data. (rules/02 §1) 2. **Pipe-safe by default**: TTY-detect; no color, no spinners, no prompts when piped; honor `NO_COLOR` (set and non-empty) and `TERM=dumb`; `--json` on every command that lists or reads data. (rules/02 §2, §4) 3. **Exit 0 only on success; distinct, documented nonzero codes** for usage error vs operational failure; never `exit 0` after printing an error. (rules/02 §3) 4. **Every flag has a long form**; short forms only for the frequent few; GNU/POSIX syntax (`-f`, `--flag`, `--flag=value`, `--` ends flags). (rules/01 §2) 5. **No required flags on the common path** — good defaults; and the reverse: destructive operations require explicit `--force`/typed confirmation on a TTY and refuse (don't hang) without one. (rules/01 §3, §5) 6. **Every prompt is skippable**: `--yes`/`--no-input` equivalent exists; stdin closed or non-TTY ⇒ use default or fail fast with the flag to pass — never block CI. (rules/02 §6) 7. **Errors say what failed, why, and what to do next** — path, value, and the exact command to run; stack traces only under `--debug`. (rules/02 §7) 8. **Ctrl-C is sacred**: first SIGINT exits promptly with cleanup and exit code 130, second SIGINT force-quits; terminal state always restored; interrupted work is resumable or rolled back. (rules/03 §3) 9. **`--dry-run` on every mutating command**, printing the real plan through the real code path. (rules/03 §2) 10. **The CLI surface is a semver'd API**: flags, exit codes, JSON fields, env vars. Deprecate with warnings for a full cycle; never silently change meaning. Config precedence is exactly flags > env > project config > user config > defaults, documented. (rules/01 §6, rules/03 §7)
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.