cli-builder
Build or refactor agent-facing CLI tools with non-interactive commands, stable --help and --json contracts, idempotent operations, and --dry-run previews. Use for CLI design, agent-friction refactors, output/exit-code debugging, or automation safety. Do not use for GUI/TUI design
Install
npx skills add https://github.com/magnus919/agent-skills/tree/main/cli-builder
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install magnus919-agent-skills@llmmart
git clone https://github.com/magnus919/agent-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole magnus919/agent-skills collection as a plugin from our marketplace. Git is the plain clone.
README
CLI Builder — Design Agent-Friendly CLI Tools
A comprehensive design guide and scaffold for building CLI tools that AI agents can actually use. 10 universal design patterns grounded in real failures from building 15+ agent-facing CLIs.
Why Install This Skill
When your agent loads this skill, it can design, build, and refactor CLI tools that agents can discover and use without human help. The goal is to establish and review explicit contracts for help, output, safety, and repeatable operations:
- Treat
--helpas a contract the agent parses to understand your tool - Establish a documented
--jsoncontract for machine-readable output, when the CLI supports it - Design and verify idempotent operations where repeatability is appropriate, with
--dry-runpreviews for changes - Authentication is lazy — help and dry-run work without credentials
- Errors are structured — different exit codes for different failure modes
What You Get
| Directory | Purpose |
|---|---|
SKILL.md |
Concise workflow for discovery, predictable contracts, verification, and maintenance |
templates/ |
Bash CLI scaffold for local wrappers |
references/ |
Python API clients, advanced patterns, readiness checklist, wrapper example, MCP decisions, and improvement cycle |
Triggers
Load this when building a new CLI tool, refactoring an existing tool that causes agent friction, or debugging why your agent keeps failing to use a CLI properly.
Requirements
Bash, Python 3.8+, jq, and a standard Unix CLI environment.
Quick Start
Start with the setup and first workflow in SKILL.md, then use the linked resources for the specific task you need to complete.
Skill manifest
CLI Builder
Treat a CLI as a contract between the tool and the agent. The contract includes
command names, --help, flags, output schemas, exit codes, stderr, and previews.
Keep this file as the workflow; load the linked references only when their
specialized guidance is needed.
When to Use
- Build a new CLI that an agent will discover and call.
- Refactor prompts, ambiguous commands, parser-hostile output, or false success codes.
- Add or review
--json,--dry-run,--yes, idempotency, or lazy authentication.
One Workflow
1. Discover the real contract
Define one CLI per service (except services sharing vendor authentication). Inspect real non-health read endpoints before coding, verify the actual authentication header and response shape, and capture the command tree and failure cases. Do not invent flags from API documentation alone. For HTTP clients, read the Python API client pattern.
Choose Bash for local wrappers and filesystem pipelines. Choose Python for HTTP, JSON, authentication state, or three-level command trees.
2. Build a predictable surface
Use one consistent verb/resource convention. Every command should be non-interactive
and flag-driven, reject unknown flags, and provide useful subcommand help with
concrete examples. Make normal output human-readable, but support --json with a
stable curated schema, normalized types, deterministic ordering, and no other stdout.
Send diagnostics to stderr with truthful non-zero exit codes.
For any state change, implement --dry-run before data-fetching or mutation, and
require an explicit --yes/--force gate for destructive work. Guard creation,
updates, and deletion so reruns are safe. Authentication must be lazy: --help and
safe dry-runs work without credentials.
Use the advanced patterns for chained dry-runs, third-party JSON, version-dependent behavior, and text matching. Use the Bash scaffold when a local shell wrapper is the right fit.
3. Verify the contract
Run syntax checks, every command's --help, parse every --json response, check
missing arguments and unknown flags, confirm errors are on stderr, exercise dry-run
without credentials, and prove a second identical run is a no-op. Against a live
service, test a real authenticated read and dry-run each mutation. Use the
agent-readiness checklist for the final
review.
4. Wrap and maintain
Keep the entry-point skill concise and put conditional detail in references. A wrapper should explain what the CLI is for, setup, common commands, output meaning, and gotchas, not duplicate every flag. See the skill-wrapper example. After real usage, record failures and prioritize fixes with the improvement cycle.
Core Contracts
--helpis the discoverable schema and includes examples.--jsonis parseable on stdout alone; errors have stable machine-readable codes.--dry-rundescribes exact intended changes and performs no writes or prerequisite lookups.- Mutations are explicitly authorized, idempotent, and report meaningful status.
- Exit status distinguishes success, usage failure, and runtime failure.
When Not to Use
Do not use this skill for one-off interactive human commands, GUI/TUI or web UI design, conversational agent tools, or MCP servers. Read the MCP-vs-CLI decision guide for tool-boundary choices, and route general API contract design to api-design-and-evolution.
References
Files (agent-skills)
-
evals
-
evals.json 7.5 KB
{ "schema_version": 1, "skill_name": "cli-builder", "evals": [ { "id": "design-agent-friendly-cli", "prompt": "We are building a new CLI that lets agents manage our deployment environments. I want it designed for AI agent consumption from the start. What are the design rules, and what should the first version of the command surface look like?", "expected_output": "A CLI design following agent-consumption principles: non-interactive by default with every behavior driven by flags, machine-readable JSON output via a --json flag, an explicit --dry-run preview for any state-changing operation, idempotent commands that can be rerun safely, a clear and stable help surface with progressive discovery, and sensible default output for humans when JSON is not requested. The response sketches the concrete command surface for environment management (list, create, promote, destroy with --dry-run, --json, and confirmation gating on destructive ops), explains why an agent-friendly design omits interactive prompts and colored-only output, and specifies the exit-code and error-output contract an agent relies on.", "assertions": [ "The design is non-interactive and flag-driven with every behavior reachable without prompts", "--json machine-readable output and --dry-run preview are part of the core contract", "Destructive operations require an explicit gate such as a confirmation flag", "The response sketches a concrete command surface for the environment-management use case", "The response specifies exit codes and stable error output that agents can rely on" ] }, { "id": "refactor-interactive-cli", "prompt": "We have an existing CLI that asks 'Continue? (y/n)' before every action, prints tables, and exits 0 even when it fails. Agents keep hanging on the prompt or misreading success. How do I refactor it for agent use without rewriting everything?", "expected_output": "A refactor plan that targets the specific agent-hostile behaviors: replace interactive confirmations with a --yes/--no-confirm flag while keeping the human default, add --json output alongside the human table, fix exit codes so failures are non-zero and errors go to stderr with a stable machine-readable error field, and add a --dry-run that shows what the command would do. The response prioritizes the changes by the failures they fix (prompt removal and exit codes first, JSON second) and shows how to keep backward compatibility for humans, and it includes a verification checklist: run every command with --help, confirm no command blocks on input, confirm exit codes are truthful.", "assertions": [ "Interactive confirmations are replaced by a flag while human defaults are preserved", "Exit codes are made truthful with errors on stderr in a stable format", "--json and --dry-run are added alongside the human output", "Changes are prioritized by the agent failures they fix", "A verification checklist confirms no command blocks and exit codes are truthful" ] }, { "id": "json-output-contract", "prompt": "I am adding --json to our status command. What makes JSON output good for agents? I have seen CLIs that dump raw API responses and call it JSON support. What should I actually do?", "expected_output": "A JSON output design that treats the schema as a contract: a documented, stable, versioned schema with consistent field names and types, values that are normalized (timestamps in ISO-8601, enums spelled consistently, numbers not strings), an object at the top level that always contains the same envelope even for errors, and no stray human text mixed into stdout. The response explains why dumping the upstream API response is a trap (it couples agents to an unstable vendor schema and leaks internal fields), recommends a curated projection of the fields an agent actually needs, and specifies that errors under --json must be structured with a machine-readable code and message rather than only a stack trace. It also covers deterministic ordering and stable IDs so agents can diff outputs.", "assertions": [ "JSON output is defined as a documented, stable schema with normalized types", "The response warns against dumping raw upstream API responses and recommends a curated projection", "Errors under --json are structured with a code and message, not only a trace", "Output is deterministic with stable ordering and IDs so agents can diff", "The envelope is consistent across success and failure cases" ] }, { "id": "dry-run-idempotency", "prompt": "Our cleanup script deletes expired sessions when run, and an agent ran it twice and deleted sessions that were renewed between runs. I want --dry-run and idempotent behavior so this cannot happen again. How should I redesign the command?", "expected_output": "A redesign with a dry-run that shows exactly what the command would change (computed from the current state, listing each session and why it qualifies) and a real run that is idempotent: qualifying sessions are selected and deleted by ID with a guard that re-checks the condition immediately before deletion, so a session renewed between the dry-run and the run is skipped. The response specifies the guard order (select by condition, re-verify per item, delete by ID), the --dry-run exit code and output contract, a --force or confirmation gate for the destructive path, and a rerun test proving the second run reports nothing left to do.", "assertions": [ "--dry-run computes and displays the exact changes from current state", "The destructive run re-verifies each item's condition before deleting by ID, preventing stale deletions", "The response specifies the gate between dry-run and real execution", "Idempotency is proven by a rerun that reports nothing left to do", "The redesign addresses the specific race that caused the double-deletion incident" ] }, { "id": "debugging-agent-cli-failures", "prompt": "An agent keeps failing to use our CLI: sometimes it passes flags the CLI does not have, other times it misreads the output, and occasionally it calls the wrong subcommand entirely. How do I debug this and make the tool easier to use correctly?", "expected_output": "A debugging approach that looks at the tool surface before blaming the agent: the response walks through the failure modes and their tool-side causes — invented flags and wrong subcommands are usually a discoverability problem (help text not surfacing the real command tree, ambiguous names, missing examples), misread output is usually a formatting problem (tables that break parsers, progress bars, no JSON mode, colors obscuring values). The response prescribes concrete fixes: a complete and correct --help with examples for each subcommand, unambiguous naming, stable JSON output with documented fields, and a strict mode that errors on unknown flags instead of silently ignoring them, plus a test harness that replays the agent's failing invocations against the CLI to confirm the fixes.", "assertions": [ "The response maps each failure mode to a tool-side cause rather than blaming the agent", "Discoverability fixes include complete help, examples, and unambiguous naming", "Output readability fixes include JSON mode and removing parser-hostile formatting", "Unknown flags are rejected in strict mode rather than silently ignored", "A replay harness verifies the fixes against the agent's actual failing invocations" ] } ] }
-
-
references
-
advanced-patterns.md 6.8 KB
# Advanced Patterns Edge case patterns that don't apply to every CLI but are essential when they do. ## Morphological Matching for Text-Based Filters When a CLI provides `--category`, `--type`, or similar text matching against section headers, **exact substring matching fails on morphological variants**: | User passes | Header reads | Substring match? | |---|---|---| | `warehouse` | DATA WAREHOUSING | ❌ "warehouse" ≠ substring of "warehousing" | | `model` | DATA MODELING | ❌ "model" ≠ substring of "modeling" | | `format` | STORAGE FORMATS | ❌ "format" ≠ substring of "formats" | **Fix: Match on a shared word stem.** Take the first N characters of the user's filter term and check if that stem appears in the lowercased header: ```python def _header_matches(header: str, category: str, stem_len: int = 5) -> bool: """Check if a category filter matches a section header on shared stem.""" cat_lower = category.lower() header_lower = header.lower() # Exact match first (fast path) if cat_lower in header_lower: return True # Stem match (handles morphological variants) stem = cat_lower[:min(len(cat_lower), stem_len)] return len(stem) >= 3 and stem in header_lower ``` **When to use:** Any CLI with `--category`, `--type`, or free-text filtering against known labels where the headers may use different morphological forms. ## Version-Dependent Imports After Dry-Run When a CLI handler wraps an optional library module, imports must respect the dry-run first check: ```python async def cmd_foo(**kw): url = kw["url"] # ... parse ALL params BEFORE any imports ... if DRY_RUN: emit("[dry-run] Would foo", {"dry_run": True, "url": url}) return # Lazy imports — after dry-run, so --dry-run works without the module from some_library.optional import CoolFeature try: from some_library.newer_module import NewThing except ImportError: NewThing = None # graceful fallback for older versions # ... rest of handler using CoolFeature / NewThing ... ``` **The failure mode:** If the import is at the top of the handler, `--dry-run` crashes with `ModuleNotFoundError` even though it should be safe. **Detection:** Syntax checks don't catch this. Run `--dry-run` against the actual target environment. ## Robust JSON Consumption from Subprocesses When consuming JSON from a CLI tool you don't control, filter stdout lines before parsing: ```python def _parse_json_output(result: subprocess.CompletedProcess) -> list | dict: """Parse subprocess stdout as JSON, filtering out non-JSON noise.""" json_lines = [] for line in result.stdout.splitlines(): stripped = line.strip() if stripped.startswith(("[", "{")): json_lines.append(stripped) if not json_lines: return [] return json.loads("\n".join(json_lines)) ``` This acts as a "JSON line filter" — anything that doesn't start with `[` or `{` is discarded. Safe because: - JSON arrays always start with `[` - JSON objects always start with `{` - Warning/status messages rarely start with either character **When to use:** Consuming JSON from tools you didn't build, or tools with environment-specific logging you can't suppress. ## Dry-Run Short-Circuit for Chained-API Commands The basic dry-run pattern breaks down when a command handler chains multiple API calls where the first call provides context for subsequent calls. Example: ```python def cmd_current(client, args): # First call: fetch station list to get device IDs stations = client.get_stations() # Fails in dry-run — returns [] # Second call: fetch observations for that device obs = client.get_observations(stations[0]["id"]) ``` In dry-run mode, the first call returns an empty list. The handler errors: "No stations found." The dry-run never reaches the preview logic. **Fix: Add a command-level dry-run short-circuit BEFORE any data-fetching calls.** ```python def cmd_current(client, args): # Short-circuit at command level, above all data-fetching if client.dry_run: emit("[dry-run] Would query latest station observations.", {"dry_run": True, "command": "current"}) return # All real logic follows — stations lookup, observations fetch stations = client.get_stations() obs = client.get_observations(stations[0]["id"]) # ... format and emit output ... ``` **Pattern rules:** 1. The short-circuit must emit a meaningful preview of what the command would do 2. It must include all parameters the command received (IDs, flags, etc.) 3. It must return — not fall through — so the chained API calls never execute 4. Every command that chains API calls needs its own short-circuit **Detection:** If a command emits a fatal error (not a dry-run preview) when run with `--dry-run`, it has this problem. ## Container-Key Wrapper Ambiguity When a CLI wraps an API that expects the POST body wrapped in a container key (e.g., `{"dashboard": {...}, "overwrite": true}`), there's ambiguity about what `--file` should contain: the inner resource only, or the full POST body. **Rule:** Accept the inner resource body only. Add the wrapper yourself in the handler. ```python def cmd_create(json_file): with open(json_file) as f: data = json.load(f) # expected: {...}, not {"resource": {...}} body = {"resource": data, "overwrite": True} client._request("POST", "/api/resources", json_data=body) ``` Document this explicitly in `--file` help text: "Path to a JSON file containing just the resource body — the CLI adds the API envelope." ## Library Init Banners in Stdout Some libraries (Crawl4AI, Playwright) print init banners to stdout via bare `print()`, not logging. These appear before any of your code runs and contaminate `--json` output. **Preferred fix: route ALL human output to stderr, redirect stdout globally in `main()`.** ```python import sys _REAL_STDOUT = None def emit(human: str, machine: dict) -> None: if JSON_OUTPUT: print(json.dumps(machine), file=_REAL_STDOUT) else: print(human, file=sys.stderr) def main(): global _REAL_STDOUT # ... parse args, detect JSON mode ... if JSON_OUTPUT: _REAL_STDOUT = sys.stdout sys.stdout = sys.stderr # Library noise → stderr # ... dispatch handlers ... ``` No `with` blocks needed in any handler. Only `emit()` writes to the real stdout. ## Input Sanitization for Embedded Queries When user input gets interpolated into SQL, Cypher, or shell commands, sanitize first: ```bash sanitize() { printf '%s' "$1" | sed -e "s/'//g" -e 's/;/./g' } name=$(sanitize "$RAW_NAME") ``` In Python: ```python def sanitize(value: str) -> str: """Strip characters that break string interpolation.""" for char in ["'", ";", "\\"]: value = value.replace(char, "") return value ``` -
agent-readiness-checklist.md 765 B
# Agent-Readiness Checklist - [ ] No interactive prompts (`read`, `select`, `dialog`) - [ ] All inputs arrive through flags or environment variables - [ ] Every subcommand has `--help` examples - [ ] `--json` output parses as JSON - [ ] Every destructive operation supports `--dry-run` - [ ] `--force` or `--yes` skips confirmation - [ ] Repeating an operation produces a no-op rather than an error - [ ] Commands use a consistent `resource verb` structure - [ ] Errors go to stderr - [ ] JSON mode emits no non-JSON stdout - [ ] `--help` and `--dry-run` work without credentials - [ ] Exit codes distinguish success, usage errors, and runtime failures - [ ] Each read endpoint has a live-server verification - [ ] Each chained API path has a dry-run verification -
improvement-cycle.md 2.5 KB
# CLI Improvement Cycle After a CLI ships, real usage reveals what the tests didn't catch. This cycle captures feedback and prioritizes fixes systematically. ## The Flywheel ``` Traces → Feedback → Triage → Fix → Deploy ↑ │ └──────────────────────────────┘ ``` ## Structured Feedback Schema When an agent session reveals a problem with a CLI tool, capture it as structured data — not a note-to-self. This makes patterns visible across multiple sessions: ```python feedback = { "tool": "my-cli", "trace_id": "<session or run ID>", "theme": "ambiguous_error", # one of: missing_flag, silent_failure, wrong_output, # unparseable_json, confusing_help "command": "my-cli deploy --env staging --tag v1.2", "observed": "Agent ran command with correct flags but got " "non-zero exit with no stderr output", "expected": "Non-zero exit should always include a stderr message " "explaining what went wrong", "frequency": "single_occurrence", # or "recurring" — if recurring, escalate to High priority } ``` Review the feedback log before each new CLI build to identify recurring pain points. ## HALO-Style Prioritization When the feedback log accumulates, triage findings by four tiers: | Priority | Criteria | Action | |----------|----------|--------| | **Blocking** | Tool returns wrong output, errors on valid input, or crashes | Fix immediately, add regression test | | **High** | Agent misuses a flag or pattern across multiple sessions (2+ feedback entries with same theme) | Fix this sprint, update help text | | **Medium** | Missing `--json`, missing help examples, inconsistent naming | Schedule next sprint | | **Low** | Stderr hygiene, edge-case idempotency, non-idiomatic flag names | Defer, log for next version | **Triage rule:** Pattern frequency overrides tier. A "Medium" finding that appears in 3+ sessions is actually High. A "Blocking" finding that only appeared once with a workaround may be Medium. The goal is not to fix everything — it's to have a defensible reason for what you're fixing now vs. deferring. ## Applying the Cycle 1. Collect traces from agent sessions using the tool 2. When a pattern emerges, write a structured feedback entry 3. Before the next development cycle, review the backlog 4. Fix the top priority items 5. Add regression tests for each fix 6. Deploy the updated tool -
mcp-vs-cli.md 3.5 KB
# MCP vs CLI — Discourse Summary & Decision Framework ## The Core Argument CLI-first design for agent tools vs MCP servers — what the debate is actually about, not the noise. ### Key Sources | Source | Key Claim | |--------|-----------| | **Eric Zakariasson** — "Building CLIs for Agents" ([X thread](https://x.com/ericzakariasson/status/2036762680401223946)) | CLIs designed for agents need `--json`, `--dry-run`, examples in help, and non-interactive mode. Most CLIs assume a human. | | **ScaleKit Benchmarks** ([scalekit.com/blog/mcp-vs-cli-use](https://www.scalekit.com/blog/mcp-vs-cli-use)) | 9-32× token savings for CLI over MCP on GitHub automation. 100% reliability (CLI) vs 72% (MCP). The gap is schema injection — 43 tool definitions per turn, agent uses 1-2. | | **Ronnie Rocha** — "Don't Build MCPs, Build CLI Tools" ([ronnierocha.dev](https://ronnierocha.dev/blog/dont-build-mcps-build-cli-tools/)) | MCP was designed for sandboxed agents (IDE plugins, web assistants). Terminal-native agents already have access — they don't need a bridge. MCP tax: context bloat, no composability, no pipes, serialized overhead. | | **Garry Tan** ([X](https://x.com/garrytan/status/2031910564344262988)) | "MCP sucks honestly. It eats too much context window… I vibe coded a CLI wrapper for Playwright tonight in 30 minutes… worked 100x better." | | **Peter Steinberger (steipete)** — MCPorter ([github.com/steipete/mcporter](https://github.com/steipete/mcporter)) | Converts MCP tools to CLI commands. Describes MCP as "a crutch" for environments without terminal access. | ## Token Cost Breakdown The ScaleKit benchmark reveals why CLI wins for agent consumption: | Metric | CLI | MCP | |--------|-----|-----| | Schema overhead per turn | 0 tokens (agent calls `--help` on demand) | 500-2000 tokens (full tool definitions injected every turn) | | Output shape | Agent requests exactly what it needs via flags | Full JSON-RPC response, unfiltered | | Composition | Piped through `jq`, `grep`, `mlr` | Atomic calls only | | Failure mode | Exit code + stderr message | ConnectTimeout to MCP endpoint (36% of MCP failures) | The gap grows with tool count. A CLI aggregate (one binary with subcommands) costs ~100 tokens in `--help` output. An MCP aggregate costs ~43 tool schemas × ~500 tokens each = ~21,500 tokens per turn. ## Decision Framework | Situation | Default | Rationale | |-----------|---------|-----------| | Agent already has a terminal | **CLI** | No bridge needed. Agent pipes output directly. | | Single-user, personal/homelab | **CLI** | Simpler to build and debug. No server to maintain. | | Multi-tenant, end-user OAuth | **MCP** | Auth delegation and credential management handled by the protocol. | | Token-sensitive at scale | **CLI** | 9-32× cheaper per operation. | | Need composability (pipes) | **CLI** | `cmd | grep | jq` is zero-cost composition. | | Need dynamic resource discovery | **MCP** | MCP's `list_resources` + `subscribe` provides real-time schema discovery. | | Enterprise audit/traceability | **MCP** | JSON-RPC has structured request/response logging at the protocol level. | | Internal bespoke API | **Either** — CLI with `--json` is simpler; MCP if governed access is required | **Bottom line:** CLI is the default for most agent-facing tools. MCP wins for governed multi-tenant deployments where credential management and audit trails are the primary value. In practice, many deployments end up hybrid: CLI for high-frequency known tools, MCP for sandboxed or governed integrations. -
python-api-client.md 10.4 KB
# Python API Client Pattern Use this pattern when your CLI wraps an HTTP API. Two key design decisions: 1. **Lazy auth** — credentials checked at request time, not client creation time 2. **Pre-parsed global flags** — `--json` and `--dry-run` work in any position ## Client Class ```python import json, os, sys, warnings import requests from typing import Dict, Any, Optional, List DEFAULT_SERVER = os.getenv("MYTOOL_SERVER", "http://localhost:8080") ENV_API_KEY = os.getenv("MYTOOL_API_KEY", "") class MyToolError(Exception): pass class MyToolClient: """API client with lazy auth, dry-run, and error wrapping.""" def __init__(self, server: str, api_key: str = "", dry_run: bool = False): self.server = server.rstrip("/") self.api_key = api_key or ENV_API_KEY self.dry_run = dry_run self._token: Optional[str] = None self._token_file = os.path.expanduser("~/.mytool_token") # Auto-load saved JWT token if not self.api_key: try: with open(self._token_file) as f: saved = f.read().strip() if saved: self._token = saved except (OSError, IOError): pass # ── Auth Headers ────────────────────────────────────────── def _headers(self) -> Dict[str, str]: """Build auth headers. Priority: JWT token > API key > no auth.""" h: Dict[str, str] = {} if self._token: h["Authorization"] = f"Bearer {self._token}" elif self.api_key: # VERIFY header name against YOUR server. # Common options: X-API-Key, Authorization: Bearer, Authorization: Token h["X-API-Key"] = self.api_key if not any("files" in k for k in ["_files"]): h["Content-Type"] = "application/json" return h # ── Centralized HTTP Request ────────────────────────────── def _request(self, method: str, path: str, params: Optional[Dict] = None, json_data: Any = None, files: Optional[Dict] = None) -> Dict[str, Any]: """Centralized HTTP request with error wrapping. Credentials checked HERE, not in __init__. This lets --help and --dry-run work without any API key configured. """ url = f"{self.server}{path}" headers = self._headers() if files: headers.pop("Content-Type", None) # requests sets multipart boundary # Dry-run: return a safe empty shape matching what the handler expects if self.dry_run: msg = f"[dry-run] {method.upper()} {path}" info = {"dry_run": True, "method": method.upper(), "url": url, "params": params, "json": json_data} print(msg, file=sys.stderr) # Return empty shape that won't crash the handler return {"items": [], "total_count": 0} # Check credentials only on real API calls if not self.api_key and not self._token: raise MyToolError( "No credentials configured.\n" " Set MYTOOL_API_KEY environment variable or login with:\n" " mytool login --username <user> --password <pass>" ) try: resp = requests.request(method=method, url=url, params=params, json=json_data, files=files, headers=headers, timeout=120) except requests.ConnectionError as e: raise MyToolError( f"Cannot connect to {self.server}: {e}\n" f" Is the server running? Set MYTOOL_SERVER or use --server." ) if resp.status_code == 204: return {} try: body = resp.json() except (json.JSONDecodeError, ValueError): body_text = resp.text.strip() if not body_text: return {} body = {"raw": body_text[:500]} if resp.status_code == 401: raise MyToolError( f"Auth failed (401). Check your credentials.\n" f" Server response: {body.get('detail', body.get('message', str(body)))}" ) if resp.status_code >= 400: detail = body.get("detail", body.get("message", str(body))) raise MyToolError(f"API error ({resp.status_code}): {detail}") return body # ── Form-Encoded Login ──────────────────────────────────── def _form_post(self, path: str, data: Dict[str, str]) -> Dict[str, Any]: """For login/oauth endpoints that need form-encoded data, not JSON.""" url = f"{self.server}{path}" if self.dry_run: return {"dry_run": True, "method": "POST", "url": url, "form": data} try: resp = requests.post(url, data=data, timeout=30) except requests.ConnectionError as e: raise MyToolError(f"Cannot connect: {e}") try: body = resp.json() except (json.JSONDecodeError, ValueError): body = {"raw": resp.text[:500]} if resp.status_code == 401: raise MyToolError("Login failed: incorrect credentials") if resp.status_code >= 400: detail = body.get("detail", body.get("message", str(body))) raise MyToolError(f"Login error ({resp.status_code}): {detail}") return body def login(self, username: str, password: str) -> Dict[str, Any]: """Form-based login with token persistence.""" result = self._form_post("/login", {"username": username, "password": password}) if "token" in result: self._token = result["token"] try: with open(self._token_file, "w") as f: f.write(self._token) except OSError: pass return result # ── Endpoint Methods ────────────────────────────────────── def list_items(self, limit: int = 50) -> Dict[str, Any]: return self._request("GET", "/items", params={"limit": limit}) def get_item(self, item_id: str) -> Dict[str, Any]: return self._request("GET", f"/items/{item_id}") def create_item(self, name: str, **kwargs) -> Dict[str, Any]: return self._request("POST", "/items", json_data={"name": name, **kwargs}) def delete_item(self, item_id: str) -> Dict[str, Any]: return self._request("DELETE", f"/items/{item_id}") def upload_file(self, file_path: str) -> Dict[str, Any]: """File upload using multipart — Content-Type set by requests.""" with open(os.path.abspath(file_path), "rb") as f: files = {"file": (os.path.basename(file_path), f)} return self._request("POST", "/items/upload", files=files) ``` ## Argparse Dispatch with Pre-Parsed Global Flags The fundamental argparse pitfall: argparse routes all unrecognized flags after a subcommand name to that subparser. If `--json` is only defined on the main parser, `tool subcommand --json` fails. **Fix: pre-parse global flags from argv before argparse sees them.** ```python def _preparse_global_flags(argv: List[str]) -> tuple[Dict[str, Any], List[str]]: """Strip global flags from argv regardless of position. Returns (globals_dict, filtered_argv) where filtered_argv has only positional args and subcommand-specific flags, ready for argparse. """ GLOBAL_BOOLS = {"--json", "--dry-run", "--force", "--quiet", "--verbose"} GLOBAL_VALUES = {"--server", "--key"} globals_map: Dict[str, Any] = {} filtered: List[str] = [argv[0]] i = 1 while i < len(argv): arg = argv[i] if arg in GLOBAL_BOOLS: globals_map[arg.lstrip("-").replace("-", "_")] = True i += 1 elif arg in GLOBAL_VALUES: key = arg.lstrip("-").replace("-", "_") if i + 1 < len(argv) and not argv[i + 1].startswith("-"): globals_map[key] = argv[i + 1] i += 2 else: globals_map[key] = "" i += 1 elif arg == "--": filtered.extend(argv[i:]) break else: filtered.append(arg) i += 1 return globals_map, filtered def main(): # 1. Strip global flags from anywhere in argv global_flags, filtered_argv = _preparse_global_flags(sys.argv) # 2. Suppress Python warnings in machine mode if global_flags.get("json"): warnings.simplefilter("ignore") # 3. Let argparse handle the filtered args parser = argparse.ArgumentParser(prog="mytool") parser.add_argument("--server", default="") # ... subparsers, etc. args = parser.parse_args(filtered_argv[1:]) # 4. Merge: explicit argparse value > pre-parsed global > env var server = args.server or global_flags.get("server") or os.getenv("MYTOOL_SERVER", DEFAULT_SERVER) dry_run = global_flags.get("dry_run", False) json_mode = global_flags.get("json", False) client = MyToolClient(server=server, dry_run=dry_run) # ... dispatch to subcommand handlers ... ``` This handles `--json` in any position: - `mytool --json subcommand --flag value` (before subcommand) - `mytool subcommand --flag value --json` (after subcommand) - `mytool subcommand --json subsub --name foo` (deep nesting) ## Env Var File-Read Fallback Terminal subprocesses may not inherit environment variables from the parent agent process. Always implement a file-read fallback: ```python def _get_env(key: str, default: str = "") -> str: """Get env var with ~/.mytool.env file-read fallback.""" val = os.getenv(key) if val: return val env_path = os.path.expanduser("~/.mytool.env") if os.path.isfile(env_path): try: with open(env_path) as f: for line in f: line = line.strip() if line.startswith("export "): line = line[len("export "):] if line.startswith(f"{key}="): return line.split("=", 1)[1].strip("\"'") except OSError: pass return default ``` Call this at module level to set defaults even when env vars aren't inherited. -
skill-wrapper-example.md 3.8 KB
# Skill Wrapper Example — `weather-cli` A complete worked example of an agent-skills-compliant wrapper around a hypothetical weather API CLI. This follows the Phase 4 pattern: the SKILL.md triggers discovery, the CLI binary provides execution. ## Directory Structure ``` weather-cli/ ├── weather-cli # CLI binary (built with Phases 1-3) └── SKILL.md # Skill wrapper (Phase 4) ``` ## `SKILL.md` ```yaml --- name: weather-cli description: >- Query current weather, forecasts, and historical data from the OpenWeather API. Use when the user asks about the weather, forecasts, temperature, precipitation, wind, or climate conditions for a location. license: MIT compatibility: Requires weather-cli binary on PATH, OPENWEATHER_API_KEY set in environment or ~/.openweather.env metadata: tags: [weather, climate, api-client, openweather] --- ``` ```markdown # Weather CLI Query weather data from the OpenWeather API — current conditions, 7-day forecasts, and historical records for any location. ## When to Use - User asks "what's the weather in [city]" or "is it going to rain today" - User asks about forecasts, temperature trends, wind, humidity, or pressure - User asks "how hot/cold/windy was it on [date]" - User wants to check weather across multiple locations Do NOT use for: severe weather alerts (use a dedicated alert skill), long-term climate projections, or weather data not available via OpenWeather API. ## Setup Credentials are read from the `OPENWEATHER_API_KEY` environment variable or `~/.openweather.env`. If the agent gets a 401, guide the user to set up an API key at https://openweathermap.org/api and set the env var. Default units are metric. Pass `--units imperial` for Fahrenheit/mph. ## Essential Commands ### current — Current conditions for a location ```bash weather-cli current "Raleigh, NC" # metric, human-readable weather-cli current "London, UK" --json # metric, machine-readable weather-cli current "New York, NY" --units imperial # Fahrenheit/mph ``` Output fields: `temperature`, `feels_like`, `humidity`, `wind_speed`, `conditions` (text description), `pressure`, `visibility`. ### forecast — 7-day forecast ```bash weather-cli forecast "Raleigh, NC" # human table weather-cli forecast "Raleigh, NC" --json --days 3 # 3-day forecast as JSON ``` JSON shape: `[{"date", "high", "low", "conditions", "precip_chance"}, ...]`. The `precip_chance` field is 0-100 (percentage). `conditions` uses OpenWeather's label strings (`Clear`, `Clouds`, `Rain`, etc.). ### history — Historical data for a date ```bash weather-cli history "Raleigh, NC" --date 2026-05-15 weather-cli history "London, UK" --date 2025-12-25 --json ``` Historical data is available for dates up to 5 days before the current date (free tier) or full history (paid plans). The CLI will warn if data is unavailable for the requested range. ## Location Format Accepts city names (`"Raleigh, NC"`), ZIP codes (`"27601"`), or latitude,longitude pairs (`"35.78,-78.64"`). City names with commas should be quoted. For disambiguation, prefer `"City, State/Country"` format over bare city names. ## Known Gotchas - **City name ambiguity:** `"London"` resolves to London, UK. Use `"London, OH"` or `"London, Ontario"` for other cities. - **Units apply per-command:** `--units` is not sticky. Set it on every command or override with `WEATHER_CLI_UNITS=imperial` env var. - **Rate limit:** 60 requests/minute on free tier. Cache repeated location queries rather than fetching the same city twice. - **The `conditions` field uses OpenWeather's English labels** regardless of locale. Always compare against `"Clear"`, `"Clouds"`, `"Rain"`, `"Snow"`, `"Drizzle"`, `"Thunderstorm"`, or `"Atmosphere"` (fog, haze, etc.). ```
-
-
templates
-
bash-cli-scaffold.sh 6 KB
#!/usr/bin/env bash # tool.sh — <description> # Usage: ./tool.sh <command> [OPTIONS] # # Agent-friendly CLI scaffold following the cli-builder skill patterns. # Replace <placeholders> and implement cmd_* functions for your use case. # # Pre-wired patterns: # --json Machine-readable JSON output # --dry-run Preview destructive operations # --force Skip confirmations # --quiet/-q Suppress non-essential output # --verbose/-v Additional diagnostic output to stderr # # Available helpers: # log() stdout, suppressed in --json and --quiet modes # warn() stderr, always visible # die() stderr + exit 1 # info() stderr, visible only with --verbose # emit() dual output: machine string vs human string set -euo pipefail # === Configuration (override via env vars) === TOOL_DB="${TOOL_DB:-/path/to/default.db}" TOOL_SERVER="${TOOL_SERVER:-http://localhost:8080}" # === State (modified by parse_args) === COMMAND="" FORCE=false DRY_RUN=false JSON_OUTPUT=false QUIET=false VERBOSE=false # === Argument Parsing (indexed approach) === # Uses while+shift inside the loop, NOT for arg in "$@" — shifts inside # for-each loops don't affect the iteration variable and cause consumed # flag values to appear as positional arguments. parse_args() { while [[ $# -gt 0 ]]; do case "$1" in --force|--yes|-y) FORCE=true; shift ;; --dry-run|-n) DRY_RUN=true; shift ;; --json) JSON_OUTPUT=true; shift ;; --quiet|-q) QUIET=true; shift ;; --verbose|-v) VERBOSE=true; shift ;; --help|-h) usage "${COMMAND:-}"; exit 0 ;; --) shift; break ;; -*) die "Unknown flag '$1'. Run '$0 --help' for usage." ;; *) if [[ -z "$COMMAND" ]]; then COMMAND="$1" else die "Unexpected argument '$1'. Run '$0 $COMMAND --help' for usage." fi shift ;; esac done [[ -z "$COMMAND" ]] && { usage; exit 1; } } # === Logging Helpers === log() { if [[ "$QUIET" != "true" && "$JSON_OUTPUT" != "true" ]]; then echo "$@" fi } warn() { echo "Warning: $*" >&2; } die() { echo "Error: $*" >&2; exit 1; } info() { [[ "$VERBOSE" == "true" ]] && echo "[info] $*" >&2 || true; } # === Dual Output Helper === # Every command calls emit() exactly once. This is the only path to stdout. emit() { if [[ "$JSON_OUTPUT" == "true" ]]; then echo "$1" # machine-readable JSON string else echo "$2" # human-readable text fi } # === Usage / Help === usage() { local cmd="${1:-}" case "$cmd" in list) cat <<'HELP' Usage: tool.sh list [OPTIONS] List resources. Options: --json Output as JSON --quiet Suppress non-essential output Examples: tool.sh list tool.sh list --json | jq '.[].name' HELP ;; create) cat <<'HELP' Usage: tool.sh create --name <name> [OPTIONS] Create a resource. Options: --name Resource name (required) --dry-run Preview without creating --force Overwrite if exists Examples: tool.sh create --name my-resource tool.sh create --name my-resource --dry-run HELP ;; *) cat <<'HELP' Usage: tool.sh <command> [OPTIONS] Commands: list List resources create Create a resource delete Delete a resource Global flags (work in any position): --force, -y Skip confirmations --dry-run, -n Preview changes --json Machine-readable output --quiet, -q Minimal output --verbose, -v Detailed output to stderr Run 'tool.sh <command> --help' for command-specific options. HELP ;; esac } # === Commands === cmd_list() { info "Listing resources..." if [[ "$DRY_RUN" == "true" ]]; then emit '{"dry_run":true,"command":"list"}' "[dry-run] Would list resources" return 0 fi # TODO: implement list logic # Use log() for progress, emit() for output # Use $JSON_OUTPUT to decide format emit '{"items":[]}' "No resources found" } cmd_create() { local NAME="" # Parse command-specific flags while [[ $# -gt 0 ]]; do case "$1" in --name) NAME="$2"; shift 2 ;; --name=*) NAME="${1#--name=}"; shift ;; --) shift; break ;; *) die "Unknown flag '$1'. Run 'tool.sh create --help' for usage." ;; esac done # Validate required flags [[ -z "$NAME" ]] && die "--name is required" # Sanitize input NAME=$(printf '%s' "$NAME" | sed -e "s/'//g" -e 's/;//g') # Idempotency check if resource_exists "$NAME"; then if [[ "$FORCE" != "true" ]]; then emit "{\"status\":\"exists\",\"name\":\"$NAME\"}" "Resource '$NAME' already exists — no-op" return 0 fi info "Overwriting existing resource '$NAME'" fi # Dry-run if [[ "$DRY_RUN" == "true" ]]; then emit "{\"dry_run\":true,\"name\":\"$NAME\"}" "[dry-run] Would create '$NAME'" return 0 fi # TODO: implement create logic create_resource "$NAME" emit "{\"status\":\"created\",\"name\":\"$NAME\"}" "Created '$NAME'" } cmd_delete() { local NAME="" while [[ $# -gt 0 ]]; do case "$1" in --name) NAME="$2"; shift 2 ;; --name=*) NAME="${1#--name=}"; shift ;; --) shift; break ;; *) die "Unknown flag '$1'. Run 'tool.sh delete --help' for usage." ;; esac done [[ -z "$NAME" ]] && die "--name is required" if ! resource_exists "$NAME"; then emit "{\"status\":\"not_found\",\"name\":\"$NAME\"}" "Resource '$NAME' not found" return 1 fi if [[ "$DRY_RUN" == "true" ]]; then emit "{\"dry_run\":true,\"name\":\"$NAME\"}" "[dry-run] Would delete '$NAME'" return 0 fi # TODO: implement delete logic delete_resource "$NAME" emit "{\"status\":\"deleted\",\"name\":\"$NAME\"}" "Deleted '$NAME'" } # === Stub Functions (implement for your use case) === resource_exists() { return 1; } create_resource() { :; } delete_resource() { :; } # === Main Dispatch === parse_args "$@" case "$COMMAND" in list) cmd_list "$@" ;; create) cmd_create "$@" ;; delete) cmd_delete "$@" ;; *) die "Unknown command '$COMMAND'" ;; esac
-
-
README.md 1.7 KB
# CLI Builder — Design Agent-Friendly CLI Tools A comprehensive design guide and scaffold for building CLI tools that **AI agents can actually use**. 10 universal design patterns grounded in real failures from building 15+ agent-facing CLIs. ## Why Install This Skill When your agent loads this skill, it can **design, build, and refactor CLI tools** that agents can discover and use without human help. The goal is to establish and review explicit contracts for help, output, safety, and repeatable operations: - **Treat `--help` as a contract** the agent parses to understand your tool - **Establish a documented `--json` contract** for machine-readable output, when the CLI supports it - **Design and verify idempotent operations** where repeatability is appropriate, with `--dry-run` previews for changes - **Authentication is lazy** — help and dry-run work without credentials - **Errors are structured** — different exit codes for different failure modes ## What You Get | Directory | Purpose | |-----------|---------| | `SKILL.md` | Concise workflow for discovery, predictable contracts, verification, and maintenance | | `templates/` | Bash CLI scaffold for local wrappers | | `references/` | Python API clients, advanced patterns, readiness checklist, wrapper example, MCP decisions, and improvement cycle | ## Triggers Load this when building a new CLI tool, refactoring an existing tool that causes agent friction, or debugging why your agent keeps failing to use a CLI properly. ## Requirements Bash, Python 3.8+, jq, and a standard Unix CLI environment. ## Quick Start Start with the setup and first workflow in SKILL.md, then use the linked resources for the specific task you need to complete. -
SKILL.md 4.8 KB
--- name: cli-builder description: >- Build or refactor agent-facing CLI tools with non-interactive commands, stable --help and --json contracts, idempotent operations, and --dry-run previews. Use for CLI design, agent-friction refactors, output/exit-code debugging, or automation safety. Do not use for GUI/TUI design, conversational tools, MCP server design, or general API architecture; use api-design-and-evolution for the latter. license: MIT compatibility: Requires bash, Python 3.8+, jq, and standard Unix CLI environment. metadata: tags: cli, agent-tooling, design-patterns, automation sources: https://x.com/ericzakariasson/status/2036762680401223946, https://www.scalekit.com/blog/mcp-vs-cli-use, https://github.com/ComposioHQ/awesome-agent-clis, https://ronnierocha.dev/blog/dont-build-mcps-build-cli-tools --- # CLI Builder Treat a CLI as a contract between the tool and the agent. The contract includes command names, `--help`, flags, output schemas, exit codes, stderr, and previews. Keep this file as the workflow; load the linked references only when their specialized guidance is needed. ## When to Use - Build a new CLI that an agent will discover and call. - Refactor prompts, ambiguous commands, parser-hostile output, or false success codes. - Add or review `--json`, `--dry-run`, `--yes`, idempotency, or lazy authentication. ## One Workflow ### 1. Discover the real contract Define one CLI per service (except services sharing vendor authentication). Inspect real non-health read endpoints before coding, verify the actual authentication header and response shape, and capture the command tree and failure cases. Do not invent flags from API documentation alone. For HTTP clients, read [the Python API client pattern](references/python-api-client.md). Choose Bash for local wrappers and filesystem pipelines. Choose Python for HTTP, JSON, authentication state, or three-level command trees. ### 2. Build a predictable surface Use one consistent verb/resource convention. Every command should be non-interactive and flag-driven, reject unknown flags, and provide useful subcommand help with concrete examples. Make normal output human-readable, but support `--json` with a stable curated schema, normalized types, deterministic ordering, and no other stdout. Send diagnostics to stderr with truthful non-zero exit codes. For any state change, implement `--dry-run` before data-fetching or mutation, and require an explicit `--yes`/`--force` gate for destructive work. Guard creation, updates, and deletion so reruns are safe. Authentication must be lazy: `--help` and safe dry-runs work without credentials. Use the [advanced patterns](references/advanced-patterns.md) for chained dry-runs, third-party JSON, version-dependent behavior, and text matching. Use the [Bash scaffold](templates/bash-cli-scaffold.sh) when a local shell wrapper is the right fit. ### 3. Verify the contract Run syntax checks, every command's `--help`, parse every `--json` response, check missing arguments and unknown flags, confirm errors are on stderr, exercise dry-run without credentials, and prove a second identical run is a no-op. Against a live service, test a real authenticated read and dry-run each mutation. Use the [agent-readiness checklist](references/agent-readiness-checklist.md) for the final review. ### 4. Wrap and maintain Keep the entry-point skill concise and put conditional detail in references. A wrapper should explain what the CLI is for, setup, common commands, output meaning, and gotchas, not duplicate every flag. See the [skill-wrapper example](references/skill-wrapper-example.md). After real usage, record failures and prioritize fixes with the [improvement cycle](references/improvement-cycle.md). ## Core Contracts - `--help` is the discoverable schema and includes examples. - `--json` is parseable on stdout alone; errors have stable machine-readable codes. - `--dry-run` describes exact intended changes and performs no writes or prerequisite lookups. - Mutations are explicitly authorized, idempotent, and report meaningful status. - Exit status distinguishes success, usage failure, and runtime failure. ## When Not to Use Do not use this skill for one-off interactive human commands, GUI/TUI or web UI design, conversational agent tools, or MCP servers. Read [the MCP-vs-CLI decision guide](references/mcp-vs-cli.md) for tool-boundary choices, and route general API contract design to [api-design-and-evolution](../api-design-and-evolution/SKILL.md). ## References - [Python API client](references/python-api-client.md) - [Advanced patterns](references/advanced-patterns.md) - [Agent-readiness checklist](references/agent-readiness-checklist.md) - [Skill wrapper example](references/skill-wrapper-example.md) - [MCP vs CLI](references/mcp-vs-cli.md) - [Improvement cycle](references/improvement-cycle.md) - [Bash scaffold](templates/bash-cli-scaffold.sh)
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.