bash-ops
Defensive Bash scripting for production automation, CI scripts, and agent-facing tools. Triggers on: bash, shell script, defensive bash, bash strict mode, set -euo pipefail, set -Eeuo pipefail, shellcheck, trap, IFS, cleanup trap, mktemp, getopts, argument parsing, exit codes, st
Install
npx skills add https://github.com/0xDarkMatter/claude-mods/tree/main/skills/bash-ops
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install 0xdarkmatter-claude-mods@llmmart
git clone https://github.com/0xDarkMatter/claude-mods.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole 0xdarkmatter/claude-mods collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Bash Operations
Defensive Bash for scripts that run unattended — CI steps, automation, and the
scripts/ a skill ships. The goal: a script that fails loudly on the first
problem, never corrupts state, and emits parseable output.
This is the house standard for any shell script in this repo. The script contract
below is the same one enforced by
docs/SKILL-RESOURCE-PROTOCOL.md §2–§7 —
that protocol governs every skill resource, and its rules are bash rules. Treat
the two as one standard: the resource protocol decides what a skill script must
guarantee (streams, exit codes, help block); this skill teaches how to write
the Bash that delivers it. The canonical reference implementation is
skills/supply-chain-defense/scripts/preinstall-check.sh —
read it whenever you need a worked example of every rule here applied at once.
Bash vs Python — choose before you write
Reach for Python (and the python-cli-ops skill) when a script grows past
~100 lines, needs data structures (nested maps, JSON manipulation beyond a
jq filter), arithmetic beyond integers, or string processing with real parsing.
Bash excels at gluing processes together: launching tools, moving files,
checking conditions, wiring pipelines. The moment you find yourself simulating a
hash-of-hashes or doing float math, stop — that's Python's job. This mirrors
SKILL-RESOURCE-PROTOCOL §3, which expects .sh for shell glue and .py for logic.
Strict mode — the first three lines
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
Each flag earns its place — and each has a sharp edge:
| Flag | Buys you | The edge to know |
|---|---|---|
set -e |
Abort on any unchecked non-zero command | Does not fire inside if/&&/|| conditions, in a function called in such a context, or for the non-last command of a pipe. local x=$(cmd) masks cmd's failure — local returns 0. Split: local x; x=$(cmd). |
set -u |
Error on unset variable expansion | "$@" and "${arr[@]}" on an empty array trip -u in old Bash; use "${arr[@]:-}" or guard with Bash 4.4+. |
set -o pipefail |
A pipe fails if any stage fails, not just the last | Without it, grep x file | head hides a grep error. With it, a head that closes the pipe early can surface 141 (SIGPIPE) — expected, not a bug. |
set -E |
ERR trap inherits into functions/subshells/command-subs |
Pair with a trap … ERR that reports $LINENO. |
IFS=$'\n\t' |
Word-splitting only on newline/tab, never spaces | Filenames with spaces stop splitting into pieces. Unset/space-IFS is the #1 cause of "it worked until a path had a space". |
set -e is the contested one. Use the full set -Eeuo pipefail when every
unchecked failure should abort (most scripts). Drop to set -uo pipefail when the
script deliberately inspects exit codes itself (the resource-protocol exemplars do
this — they branch on registry exit codes, so a non-zero curl must not kill the
run). Decide consciously; don't cargo-cult either way.
→ Full treatment, ERR-trap recipes, and the set -e exemption rules:
references/strict-mode-and-traps.md.
Quoting discipline
Quote every expansion unless you have a specific, commented reason not to.
cp "$src" "$dst" # not cp $src $dst — breaks on spaces/globs
for f in "${files[@]}"; do … # not ${files[@]} — array stays element-safe
rm -- "$path" # -- ends options; $path starting with - is data
[[ -n "$x" ]] # [[ ]] doesn't word-split, but quote for habit
grep -- "$pattern" "$file"
- Unquoted
$varundergoes word splitting (onIFS) then glob expansion. A variable holding*.txtora bbecomes multiple args. This is the canonical footgun. "$@"(quoted) preserves arguments exactly;$@and$*mangle them. Always"$@"to forward args.- Use
--before user/agent-supplied operands so a value like-rfis treated as data, not flags.
Argument parsing — case-based long flags
The resource protocol mandates --help with an EXAMPLES section and rejects
unknown flags with a USAGE error (exit 2). Use a while/case loop — it handles
GNU-style long flags (--json, --dry-run), which getopts cannot:
JSON=0; DRY_RUN=0; ARGS=()
while [[ $# -gt 0 ]]; do
case "$1" in
--json) JSON=1 ;;
--dry-run) DRY_RUN=1 ;;
-h|--help) usage; exit 0 ;;
--) shift; ARGS+=("$@"); break ;; # everything after -- is positional
-*) printf 'ERROR: unknown flag: %s (try --help)\n' "$1" >&2; exit 2 ;;
*) ARGS+=("$1") ;;
esac
shift
done
getopts is fine for short flags only (-v -o file) and is more compact there,
but it has no long-flag support and clusters awkwardly. Prefer the case loop for
anything agent-facing — it matches preinstall-check.sh exactly.
→ Both styles in full, value-taking flags, --flag=value, and validation:
references/argument-parsing.md.
Traps, cleanup, and safe tempfiles
Never leave a tempfile or half-written output behind. Create temp paths with
mktemp, register a cleanup trap immediately after, and write atomically.
tmp="$(mktemp)" || exit 1
cleanup() { rm -f "$tmp"; }
trap cleanup EXIT # fires on normal exit, error, and signals via EXIT
build_output >"$tmp" # write to temp
mv -- "$tmp" "$dst" # atomic rename — reader never sees a partial file
trap - EXIT; rm -f "$tmp" # (optional) disarm after successful move
trap cleanup EXITis the workhorse —EXITfires for normal exit,set -eabort, and (in practice)INT/TERMif you let them propagate. Add explicittrap cleanup INT TERMif you do signal handling yourself.mktemp -dfor a temp directory; clean it withrm -rf -- "$tmpdir".- Atomic write =
tmp+mv(same filesystem). A reader sees either the old file or the complete new one, never a truncated mid-write — exactly the idempotency the resource protocol §6 requires.
→ Signal handling, ERR-trap with line numbers, nested traps:
references/strict-mode-and-traps.md.
The stream-separation + exit-code contract
This is the load-bearing rule for any agent-facing script, lifted directly from
SKILL-RESOURCE-PROTOCOL §4–§5. Claude parses stdout; pollution breaks | jq.
- stdout = the data product only. JSON under
--json, else plain/TSV. - stderr = everything else. Headers, progress, warnings, errors, prompts.
- Semantic exit codes, not just 0/1:
| Code | Meaning |
|---|---|
0 |
success |
2 |
usage (bad/missing args, unknown flag) |
3 |
not found (input absent) |
4 |
validation (input present but malformed) |
5 |
precondition (missing dependency, wrong cwd) |
7 |
unavailable (external resource down — advisory, not a real failure) |
10+ |
domain signal — a non-error "finding" the caller branches on |
Code 10 is the workhorse for verifiers/scanners: "ran fine, found something."
Reserve 7 so a network blip never looks like a content failure. Print human
framing to stderr, the record to stdout:
printf '%s\t%s\n' "$name" "$status" # data → stdout
printf ' [ok] %s checked\n' "$name" >&2 # framing → stderr
→ The shipped assets/script-template.sh bakes this
contract in — copy it as the starting point for any new skill script.
ShellCheck — non-negotiable
Run shellcheck on every script; it catches the
quoting/word-splitting/set -e bugs above mechanically.
shellcheck script.sh # lint
shellcheck -x script.sh # follow sourced files
shfmt -i 2 -ci -w script.sh # format (2-space indent, indent switch-cases)
- Fix warnings; don't blanket-suppress. When a suppression is genuinely correct,
scope it to one line with a reason:
# shellcheck disable=SC2086 # word split intended. - CI gate:
shellcheck **/*.shshould pass clean before merge. bash -n script.shis a free syntax-only check (no execution) — run it in tests.
Common footguns (quick table)
| Footgun | Why it bites | Fix |
|---|---|---|
Unquoted $var |
Word-split + glob expansion | "$var" always |
[ "$a" == "$b" ] |
[ is POSIX test; == non-portable, no && grouping |
[[ "$a" == "$b" ]] in Bash |
var=$(cmd) ; echo $? |
$? is the assignment's status (always 0), not cmd's |
cmd; rc=$? or check inline |
cmd | while read x; do total=$x; done |
while runs in a subshell; $total is lost after the pipe |
while …; do … done < <(cmd) (process substitution) |
local x=$(cmd) under set -e |
local returns 0, masking cmd failure |
local x; x=$(cmd) on two lines |
echo "$x" for arbitrary data |
echo mangles -n, -e, backslashes |
printf '%s\n' "$x" |
for f in $(ls) |
Splits on whitespace, breaks on spaces/newlines | for f in * or while IFS= read -r f |
pipefail + head shows 141 |
Downstream closes pipe early (SIGPIPE) | Expected; tolerate 141 from truncating consumers |
→ Each footgun with a reproducer and the underlying mechanism:
references/footguns.md.
Bash version notes (attribute features correctly)
Bash is stable, but several common idioms are version-gated. macOS still ships
Bash 3.2 (2007, GPLv2); Linux/CI is usually Bash 5.x. If a script must run on
stock macOS, avoid the 4.x+ features below or guard with ((BASH_VERSINFO[0]>=4)).
| Feature | Introduced | Notes |
|---|---|---|
mapfile / readarray |
Bash 4.0 | Read lines into an array. mapfile -t arr < file. -d '' (null-delimited) needs 4.4. |
Associative arrays (declare -A) |
Bash 4.0 | Hash maps. Unavailable on macOS stock 3.2. |
${var,,} / ${var^^} (case conversion) |
Bash 4.0 | Lowercase/uppercase expansion. |
&>> append-both-streams, |& |
Bash 4.0 | cmd |& filter = cmd 2>&1 | filter. |
${var@Q} (quote operator) |
Bash 4.4 | Produces a re-input-safe quoted form. Also @U @L @E. |
wait -n (any child) |
Bash 4.3 | Useful for bounded parallelism. |
local -n (nameref) |
Bash 4.3 | Pass array/var by reference into a function. |
When in doubt, state the requirement in the first-comment-block (# Requires: bash 4+)
and check at startup: ((BASH_VERSINFO[0] >= 4)) || { echo "needs bash 4+" >&2; exit 5; }.
Checklist before shipping a skill script
-
#!/usr/bin/env bash+ first-comment-block contract (desc, Usage, Exit, Examples) -
set -Eeuo pipefail(or a deliberateset -uo pipefail) +IFS=$'\n\t' - All expansions quoted;
"$@"to forward args;--before operands -
casearg loop;--helpexits 0 with EXAMPLES; unknown flag → exit 2 -
trap cleanup EXIT+mktemp; atomictmp+mvwrites - stdout data-only, stderr for framing; semantic exit codes (§5)
-
shellcheckclean;bash -npasses;chmod +x - Version-gated features guarded or documented
Files (claude-mods)
-
assets
-
script-template.sh 4 KB
#!/usr/bin/env bash # Starter scaffold for an agent-facing skill script. <one-line, ends with a period.> # # Usage: script-template.sh [OPTIONS] <input>... # Input: one or more inputs as positionals; flags select behaviour # Output: stdout = data product only (TSV, or JSON under --json) # Stderr: headers, progress, warnings, errors # Exit: 0 ok, 2 usage, 3 not-found, 4 validation, 5 missing-dep, # 7 unavailable, 10 <domain signal — document it here> # # Examples: # script-template.sh input.txt # script-template.sh --json a.txt b.txt | jq '.data[]' # script-template.sh --out result.tsv --quiet input.txt # # Requires: bash 4+ (uses mapfile-style idioms); shellcheck-clean. set -Eeuo pipefail IFS=$'\n\t' # --- semantic exit codes (SKILL-RESOURCE-PROTOCOL §5) --- readonly EXIT_OK=0 EXIT_USAGE=2 EXIT_NOTFOUND=3 EXIT_VALIDATION=4 readonly EXIT_MISSING_DEP=5 EXIT_UNAVAILABLE=7 EXIT_FINDING=10 readonly SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")" # --- help (stdout, exit 0, EXAMPLES mandatory) --- usage() { cat <<EOF Usage: ${SCRIPT_NAME} [OPTIONS] <input>... Options: --json emit a JSON envelope to stdout (needs jq) --out FILE write output to FILE atomically (default: stdout) -q, --quiet suppress progress framing on stderr -h, --help show this help and exit EXAMPLES: ${SCRIPT_NAME} input.txt ${SCRIPT_NAME} --json a.txt b.txt | jq '.data[]' ${SCRIPT_NAME} --out result.tsv -q input.txt EOF } # --- framing helpers: human text ALWAYS to stderr, never stdout --- log() { [[ "$QUIET" -eq 1 ]] && return 0; printf '%s\n' "$*" >&2; } die() { printf 'ERROR: %s\n' "$*" >&2; exit "${2:-1}"; } # --- cleanup trap + atomic write scaffolding --- TMPFILE="" cleanup() { local rc=$? [[ -n "$TMPFILE" && -e "$TMPFILE" ]] && rm -f -- "$TMPFILE" exit "$rc" } trap cleanup EXIT trap 'die "interrupted" 130' INT TERM # --- argument parsing: case loop, long flags, hard usage errors --- JSON=0; QUIET=0; OUT=""; ARGS=() while [[ $# -gt 0 ]]; do case "$1" in --json) JSON=1 ;; -q|--quiet) QUIET=1 ;; --out) OUT="${2:?--out needs a value}"; shift ;; --out=*) OUT="${1#*=}" ;; -h|--help) usage; exit "$EXIT_OK" ;; --) shift; ARGS+=("$@"); break ;; -*) die "unknown flag: $1 (try --help)" "$EXIT_USAGE" ;; *) ARGS+=("$1") ;; esac shift done # --- validation (per §5/§6) --- [[ ${#ARGS[@]} -ge 1 ]] || die "need at least one input (try --help)" "$EXIT_USAGE" if [[ "$JSON" -eq 1 ]]; then command -v jq >/dev/null 2>&1 || die "jq required for --json" "$EXIT_MISSING_DEP" fi for in_f in "${ARGS[@]}"; do [[ -e "$in_f" ]] || die "input not found: $in_f" "$EXIT_NOTFOUND" done # --- work: write the DATA PRODUCT to a buffer, framing to stderr --- log "=== ${SCRIPT_NAME}: processing ${#ARGS[@]} input(s) ===" emit_records() { # Replace this with real logic. Data → stdout, one TSV record per line. local f for f in "${ARGS[@]}"; do local lines lines=$(wc -l < "$f" 2>/dev/null || echo 0) printf '%s\t%s\n' "$f" "$lines" # DATA → stdout log " [ok] ${f} (${lines} lines)" # framing → stderr done } if [[ "$JSON" -eq 1 ]]; then # Build the §4 success envelope. Booleans true/false, empty lists [], ISO-8601 Z. records="$(emit_records | jq -R -s -c 'split("\n") | map(select(length>0) | split("\t") | {file: .[0], lines: (.[1]|tonumber)})')" payload="$(jq -cn --argjson d "$records" \ '{data: $d, meta: {count: ($d|length), schema: "claude-mods.bash-ops.script-template/v1"}}')" else payload="$(emit_records)" fi # --- output: atomic write when --out, else stdout --- if [[ -n "$OUT" ]]; then TMPFILE="$(mktemp -- "${OUT}.XXXXXX")" || die "mktemp failed" "$EXIT_UNAVAILABLE" printf '%s\n' "$payload" > "$TMPFILE" mv -- "$TMPFILE" "$OUT" # atomic rename — reader never sees a partial file TMPFILE="" # disarm cleanup; the file is now $OUT log "wrote ${OUT}" else printf '%s\n' "$payload" # DATA → stdout fi exit "$EXIT_OK"
-
-
references
-
argument-parsing.md 4.3 KB
# Argument Parsing — deep dive Robust, agent-facing argument handling. The resource protocol requires `--help` with EXAMPLES (exit 0 to stdout) and a hard USAGE error (exit 2) on unknown flags or extra positionals. Two idioms cover everything: the **`case` loop** (preferred, long-flag capable) and **`getopts`** (short-flags only, more compact). ## The `case` loop (preferred) Handles long flags (`--json`), short flags, value-taking flags, `--flag=value`, and the `--` end-of-options sentinel. This is what `preinstall-check.sh` uses. ```bash usage() { cat <<'EOF' Usage: tool [OPTIONS] <input>... --json emit JSON to stdout --out FILE write result to FILE (default: stdout) --retries N retry N times (default: 3) -q, --quiet suppress progress on stderr -h, --help show this help EXAMPLES: tool input.txt tool --json --out result.json a.txt b.txt tool --retries 5 -q input.txt EOF } JSON=0; QUIET=0; OUT=""; RETRIES=3; ARGS=() while [[ $# -gt 0 ]]; do case "$1" in --json) JSON=1 ;; -q|--quiet) QUIET=1 ;; --out) OUT="${2:?--out needs a value}"; shift ;; --out=*) OUT="${1#*=}" ;; --retries) RETRIES="${2:?--retries needs a value}"; shift ;; --retries=*) RETRIES="${1#*=}" ;; -h|--help) usage; exit 0 ;; --) shift; ARGS+=("$@"); break ;; -*) printf 'ERROR: unknown flag: %s (try --help)\n' "$1" >&2; exit 2 ;; *) ARGS+=("$1") ;; esac shift done # validation [[ ${#ARGS[@]} -ge 1 ]] || { printf 'ERROR: need at least one input (try --help)\n' >&2; exit 2; } [[ "$RETRIES" =~ ^[0-9]+$ ]] || { printf 'ERROR: --retries must be an integer\n' >&2; exit 2; } ``` Key points: - **Value-taking flags** consume `$2` then `shift` an extra time. `${2:?msg}` aborts with `msg` if the value is missing — clean for `set -u` scripts. - **`--flag=value` form** handled by a parallel `--flag=*` arm using `${1#*=}` (strip up to the first `=`). - **`--` sentinel** stops option parsing: everything after is positional, even if it starts with `-`. Essential when an input filename might be `-weird`. - **Unknown flag** (`-*`) is a hard exit 2 — never silently ignore (protocol §6). - **Positionals** accumulate into an array so spaces survive; consume them with `"${ARGS[@]}"`. - Validate *after* parsing: required count, integer ranges, file existence (exit 3 for a missing input file per §5). ## `getopts` (short flags only) POSIX-portable, compact, but **no long-flag support** and no `--flag=value`. Good for a small script with only single-letter options. ```bash JSON=0; OUT=""; verbose=0 while getopts ':jo:vh' opt; do case "$opt" in j) JSON=1 ;; o) OUT="$OPTARG" ;; # ':' after o means it takes a value v) verbose=1 ;; h) usage; exit 0 ;; :) printf 'ERROR: -%s needs a value\n' "$OPTARG" >&2; exit 2 ;; \?) printf 'ERROR: unknown flag: -%s\n' "$OPTARG" >&2; exit 2 ;; esac done shift $((OPTIND - 1)) # drop parsed options; "$@" is now positionals ``` - The **leading `:`** in `':jo:vh'` enables *silent* error mode, letting you handle `:` (missing value) and `\?` (unknown) yourself with proper exit-2 messages. - A letter followed by `:` (here `o:`) takes a value in `$OPTARG`. - `shift $((OPTIND - 1))` discards the consumed options so `"$@"` holds positionals. - Limitations that push you to the `case` loop: `getopts` cannot do `--json`, cannot do `--out=x`, and clustering long flags is impossible. For anything an agent invokes by long name, use the `case` loop. ## Choosing between them | Need | Use | |---|---| | Any long flag (`--json`, `--dry-run`) | `case` loop | | `--flag=value` form | `case` loop | | Agent-facing skill script | `case` loop (matches the exemplar) | | Tiny script, only `-x -y -z` short flags, max portability | `getopts` | ## The `--help` contract (both styles) - Writes to **stdout** and exits **0** (it is requested output, not an error). - Includes a usage line, every option, and an **EXAMPLES** block — the protocol makes EXAMPLES mandatory because it is what makes the tool discoverable when the agent runs `--help`. - A common compact trick (used by `preinstall-check.sh`) is to render help from the first-comment-block itself: `sed -n '2,30p' "$0" | sed 's/^# \{0,1\}//'` — single source of truth for the contract and the help text. -
footguns.md 6.3 KB
# Bash Footguns — reproducers and mechanisms Each entry: what breaks, *why* at the shell-mechanics level, and the fix. These are the bugs `shellcheck` and `set -Eeuo pipefail` exist to catch. ## 1. Unquoted expansion → word splitting + globbing ```bash file="my report.txt" rm $file # runs: rm my report.txt → tries to remove TWO files rm "$file" # correct ``` **Mechanism.** After parameter expansion, an *unquoted* result undergoes (a) word splitting on `IFS` (default: space/tab/newline) then (b) pathname expansion (globbing). `*.bak` in a variable expands to matching files; `a b` splits to two args. Quoting suppresses both. This is the single most common shell bug. Fix: quote everything — `"$file"`, `"${arr[@]}"`. Set `IFS=$'\n\t'` to drop space from the split set as defence-in-depth. ## 2. `[ ]` vs `[[ ]]` ```bash [ -n $x ] # if $x is empty/unset → [ -n ] → true (wrong!) [ "$a" == "$b" ] # == is non-POSIX in [; && doesn't work; word-splits [[ -n $x ]] # safe: [[ ]] does not word-split its operands [[ "$a" == "$b" && -f c ]] # &&, ==, < all work inside [[ ]] ``` **Mechanism.** `[` is the external/builtin `test` command — its operands are subject to normal word splitting, so an unquoted empty variable vanishes and changes the expression's arity. `[[ ]]` is a Bash *keyword*: it parses operands without word splitting or globbing, supports `&&`/`||`/`<`/`==` with pattern matching and `=~` for regex. Use `[[ ]]` in Bash always; reserve `[ ]` for strict POSIX `sh` scripts. ## 3. `$?` after assignment is always 0 ```bash output=$(might_fail) echo $? # prints 0 — the ASSIGNMENT succeeded, not might_fail ``` **Mechanism.** A simple assignment's exit status is that of the *last command substitution*, but a plain `var=$(cmd)` reports the assignment builtin's status, which is 0 unless the substitution itself errors fatally. Capture inline: ```bash if ! output=$(might_fail); then echo "failed" >&2; fi # or output=$(might_fail); rc=$? # rc only reliable when not 'local'/'declare' ``` ## 4. Pipe into `while read` loses variables (subshell) ```bash count=0 printf 'a\nb\nc\n' | while read -r line; do count=$((count+1)); done echo "$count" # prints 0 — the while ran in a subshell ``` **Mechanism.** Each stage of a pipeline runs in its **own subshell**. The `while`'s variable mutations happen in a child process and evaporate when it exits. Fixes: ```bash # process substitution — while runs in the current shell count=0 while read -r line; do count=$((count+1)); done < <(printf 'a\nb\nc\n') echo "$count" # 3 # or a here-string / file redirect while read -r line; do …; done <<< "$data" ``` (Bash's `shopt -s lastpipe` runs the last pipe stage in the current shell, but only non-interactively and with job control off — process substitution is more portable.) ## 5. `local x=$(cmd)` masks failure under `set -e` ```bash set -e get() { local v=$(false); echo "reached"; } # 'reached' prints — failure hidden get() { local v; v=$(false); echo "nope"; } # aborts at v=$(false) ``` **Mechanism.** `local`/`declare`/`export` are commands; their exit status is the builtin's (0 on a valid declaration), which overrides the substitution's failing status. `set -e` sees 0 and continues. Always split declaration and assignment when the command's success matters. ## 6. `echo` is not portable for data ```bash echo "-n" # may print nothing (treated as a flag) or "-n" echo "a\tb" # prints literal \t or a tab depending on shell/xpg_echo printf '%s\n' "$x" # correct, deterministic ``` **Mechanism.** `echo`'s handling of `-n`, `-e`, and backslash escapes is unspecified across shells and `shopt xpg_echo`. For any variable data, use `printf '%s\n'`. Reserve `echo` for fixed, escape-free literals. ## 7. `for f in $(ls)` / parsing `ls` ```bash for f in $(ls *.txt); do … # breaks on spaces, newlines, globs in names for f in *.txt; do # correct: glob directly, no ls [[ -e "$f" ]] || continue # handle "no matches" (glob stays literal) … done ``` **Mechanism.** `$(ls)` produces a single string that word-splits on whitespace — a filename with a space becomes two loop iterations. Globbing (`*.txt`) yields each match as one word safely. Guard the no-match case (a non-matching glob expands to itself unless `shopt -s nullglob`). ## 8. Reading a file line-by-line ```bash while IFS= read -r line; do printf 'got: %s\n' "$line" done < "$file" # trailing line with no newline: while IFS= read -r line || [[ -n "$line" ]]; do …; done < "$file" ``` **Mechanism.** `IFS=` (empty) stops leading/trailing whitespace trimming; `-r` stops backslash interpretation. Without both, indentation and backslashes are mangled. The `|| [[ -n "$line" ]]` clause catches a final line lacking a newline (`read` returns non-zero but still sets `line`). ## 9. `pipefail` + early-closing consumer = exit 141 ```bash set -o pipefail generate_huge_stream | head -5 # head exits after 5 lines → SIGPIPE to generator echo $? # 141 (128 + 13) even though nothing is wrong ``` **Mechanism.** When `head` closes the read end, the producer gets SIGPIPE (signal 13) and dies with `128+13=141`; `pipefail` propagates that as the pipeline status. Tolerate it: `{ generate_huge_stream || [[ $? -eq 141 ]]; } | head -5`. ## 10. Empty array under `set -u` (older Bash) ```bash set -u arr=() printf '%s\n' "${arr[@]}" # Bash <4.4: "unbound variable" error printf '%s\n' "${arr[@]:-}" # safe everywhere ``` **Mechanism.** Pre-4.4 Bash treated `"${arr[@]}"` of an empty array as referencing an unset variable under `-u`. Use `"${arr[@]:-}"`, or test `((${#arr[@]}))` first. Fixed in Bash 4.4 for `@`/`*`, but the guard keeps scripts portable to macOS 3.2 and old CI images. ## 11. `cd` without checking, then destructive op ```bash cd "$dir" && rm -rf -- ./* # if cd fails, rm never runs (good) cd "$dir"; rm -rf -- ./* # if cd fails (under no -e), rm runs in WRONG dir ``` **Mechanism.** A failed `cd` (typo, missing dir) leaves you in the *current* directory. A following unconditional `rm -rf ./*` then wipes the wrong tree. Always `cd … || exit 1`, or chain with `&&`, or run under `set -e`. Add `--` and `./` prefixes so a path beginning with `-` is data, not flags. -
strict-mode-and-traps.md 5.4 KB
# Strict Mode and Traps — deep dive The error-handling backbone of a defensive Bash script. The SKILL.md body covers the three-line preamble; this file covers the edge cases that bite in production. ## The four flags, precisely ### `set -e` (errexit) — and where it silently does nothing `set -e` aborts the script when a command returns non-zero. It is the most useful and the most misunderstood flag because of where it **does not** fire: 1. **Conditions.** A command in an `if`, `while`, `until`, `&&`, `||`, or negated with `!` is a *tested* command — its failure is expected, so `-e` ignores it. ```bash if ! grep -q foo file; then … # grep failing here does NOT abort ``` 2. **Inside functions called in a condition.** If `f` is invoked as `if f; then`, `-e` is disabled *for the entire body of `f`* (POSIX behaviour). A failing command deep inside `f` won't abort. This surprises everyone once. 3. **Non-final pipe stages.** Without `pipefail`, only the last command's status counts: `false | true` succeeds. Add `set -o pipefail`. 4. **`local`/`declare` masking.** `local x=$(cmd)` — the *assignment builtin* returns 0 even if `cmd` failed. Split the declaration from the assignment: ```bash local x; x=$(cmd) # now $? reflects cmd, and -e can fire ``` 5. **Command substitution in an unchecked statement.** `echo "$(false)"` does not abort under older Bash because the outer `echo` succeeds. Assign first if the substitution's success matters. **When to drop `-e`.** Scripts that *inspect* exit codes themselves (a checker that branches on whether `curl` got a 404 vs a network error) must NOT let `-e` kill the run on the first non-zero. Those use `set -uo pipefail` and check `$?` explicitly — this is exactly what `preinstall-check.sh` does, and why the resource protocol §2 says "use `-e` only when every failure is fatal." ### `set -u` (nounset) Expanding an unset variable becomes a fatal error instead of an empty string — catches typos (`$fil` vs `$file`) and missing arguments. - `"$1"` when no `$1` was passed → aborts. Guard: `"${1:-default}"` or `[[ $# -ge 1 ]] || { usage; exit 2; }`. - **Empty-array gotcha:** in Bash before 4.4, `"${arr[@]}"` on an empty array trips `-u`. Use `"${arr[@]:-}"` or test `[[ ${#arr[@]} -gt 0 ]]` first. (Bash 4.4+ fixed this for `@`/`*`.) ### `set -o pipefail` A pipeline's exit status becomes the rightmost **non-zero** status (or 0 if all succeed). Without it, a failing producer is invisible: ```bash set -o pipefail data="$(curl -fsS "$url" | jq '.x')" # now a curl failure fails the assignment ``` **SIGPIPE / 141 caveat:** when a consumer closes the pipe early (`producer | head -1`), the producer is killed by SIGPIPE and reports `141`. With `pipefail` this surfaces as a pipeline failure even though nothing is wrong. Tolerate it for truncating consumers: `{ big_producer || [[ $? -eq 141 ]]; } | head`. ### `set -E` (errtrace) Makes an `ERR` trap inherit into shell functions, command substitutions, and subshells. Without `-E`, your nice line-number-reporting `ERR` trap silently fails to fire inside functions. Always pair `-E` with an `ERR` trap. ## ERR trap with context ```bash set -Eeuo pipefail err() { local rc=$? printf 'ERROR: rc=%d at %s:%d in %s()\n' \ "$rc" "${BASH_SOURCE[1]:-?}" "${BASH_LINENO[0]:-?}" "${FUNCNAME[1]:-main}" >&2 exit "$rc" } trap err ERR ``` `BASH_LINENO`/`BASH_SOURCE`/`FUNCNAME` are parallel stack arrays — index `[0]`/`[1]` walk up the call stack. This turns "it failed somewhere" into "it failed at deploy.sh:42 in push_image()". ## EXIT trap — cleanup that always runs ```bash tmpdir="$(mktemp -d)" cleanup() { local rc=$? # capture BEFORE any command in cleanup changes $? rm -rf -- "$tmpdir" exit "$rc" # preserve the original exit code } trap cleanup EXIT ``` - `EXIT` fires on normal exit, `set -e` abort, `exit N`, and — if not separately trapped — after the default signal handlers. It is the single most reliable place to release resources. - **Capture `$?` as the first line** of the handler. Any command inside `cleanup` overwrites `$?`, so grab it before `rm` et al. - A `trap … EXIT` set in a subshell only covers that subshell. ## Signal handling ```bash interrupted=0 on_int() { interrupted=1; printf '\ninterrupted, cleaning up…\n' >&2; } trap on_int INT TERM ``` - `INT` (Ctrl-C), `TERM` (`kill`), `HUP` (terminal closed) are the common ones. - After handling a signal, `EXIT` still runs — so put resource release in the EXIT handler and use signal handlers only for *additional* behaviour (a message, a flag). - To re-raise a signal with the correct exit code (`128 + signum`), reset and resend: `trap - INT; kill -INT $$`. - `trap '' INT` ignores a signal; `trap - INT` restores the default. ## Idempotency and atomic writes The resource protocol §6 requires re-running with the same inputs to be safe. - Never write the destination directly. Write `"$dst.tmp"` (or a `mktemp` file on the same filesystem) and `mv -- "$tmp" "$dst"`. `mv` within one filesystem is atomic — a concurrent reader sees old-or-new, never a partial file. (`mv` across filesystems falls back to copy+unlink and is *not* atomic — keep the temp beside the destination.) - Guard creation with `mkdir -p` (idempotent) rather than `mkdir` (fails if exists). - For "create only if absent" use `set -C` (noclobber) + `>` or `mkdir` as a lock.
-
-
scripts
-
.gitkeep 0 B · in bundle
-
-
SKILL.md 11.9 KB
--- name: bash-ops description: "Defensive Bash scripting for production automation, CI scripts, and agent-facing tools. Triggers on: bash, shell script, defensive bash, bash strict mode, set -euo pipefail, set -Eeuo pipefail, shellcheck, trap, IFS, cleanup trap, mktemp, getopts, argument parsing, exit codes, stream separation, stdout stderr, quoting, word splitting, pipefail, subshell, CI script, shell footgun, bats, shfmt, POSIX, portable shell." when_to_use: "Use when writing or reviewing any Bash/shell script — especially skill scripts, CI steps, and automation that must fail safely. Covers strict mode, quoting, argument parsing, traps/cleanup, safe tempfiles, the stream-separation + exit-code contract, and shellcheck." license: MIT allowed-tools: "Read Write Edit Bash" metadata: author: claude-mods related-skills: cli-ops, ci-cd-ops --- # Bash Operations Defensive Bash for scripts that run unattended — CI steps, automation, and the `scripts/` a skill ships. The goal: a script that **fails loudly on the first problem, never corrupts state, and emits parseable output**. This is the house standard for any shell script in this repo. The script contract below is the same one enforced by [`docs/SKILL-RESOURCE-PROTOCOL.md`](../../docs/SKILL-RESOURCE-PROTOCOL.md) §2–§7 — that protocol governs every skill resource, and its rules *are* bash rules. Treat the two as one standard: the resource protocol decides **what** a skill script must guarantee (streams, exit codes, help block); this skill teaches **how** to write the Bash that delivers it. The canonical reference implementation is [`skills/supply-chain-defense/scripts/preinstall-check.sh`](../supply-chain-defense/scripts/preinstall-check.sh) — read it whenever you need a worked example of every rule here applied at once. ## Bash vs Python — choose before you write Reach for Python (and the `python-cli-ops` skill) when a script grows past **~100 lines**, needs **data structures** (nested maps, JSON manipulation beyond a `jq` filter), arithmetic beyond integers, or string processing with real parsing. Bash excels at **gluing processes together**: launching tools, moving files, checking conditions, wiring pipelines. The moment you find yourself simulating a hash-of-hashes or doing float math, stop — that's Python's job. This mirrors SKILL-RESOURCE-PROTOCOL §3, which expects `.sh` for shell glue and `.py` for logic. ## Strict mode — the first three lines ```bash #!/usr/bin/env bash set -Eeuo pipefail IFS=$'\n\t' ``` Each flag earns its place — and each has a sharp edge: | Flag | Buys you | The edge to know | |---|---|---| | `set -e` | Abort on any unchecked non-zero command | Does **not** fire inside `if`/`&&`/`||` conditions, in a function called in such a context, or for the *non-last* command of a pipe. `local x=$(cmd)` masks `cmd`'s failure — `local` returns 0. Split: `local x; x=$(cmd)`. | | `set -u` | Error on unset variable expansion | `"$@"` and `"${arr[@]}"` on an empty array trip `-u` in old Bash; use `"${arr[@]:-}"` or guard with Bash 4.4+. | | `set -o pipefail` | A pipe fails if **any** stage fails, not just the last | Without it, `grep x file | head` hides a `grep` error. With it, a `head` that closes the pipe early can surface `141` (SIGPIPE) — expected, not a bug. | | `set -E` | `ERR` trap inherits into functions/subshells/command-subs | Pair with a `trap … ERR` that reports `$LINENO`. | | `IFS=$'\n\t'` | Word-splitting only on newline/tab, never spaces | Filenames with spaces stop splitting into pieces. Unset/space-IFS is the #1 cause of "it worked until a path had a space". | `set -e` is the contested one. Use the full `set -Eeuo pipefail` when **every** unchecked failure should abort (most scripts). Drop to `set -uo pipefail` when the script deliberately inspects exit codes itself (the resource-protocol exemplars do this — they branch on registry exit codes, so a non-zero `curl` must not kill the run). Decide consciously; don't cargo-cult either way. → Full treatment, ERR-trap recipes, and the `set -e` exemption rules: [`references/strict-mode-and-traps.md`](references/strict-mode-and-traps.md). ## Quoting discipline **Quote every expansion** unless you have a specific, commented reason not to. ```bash cp "$src" "$dst" # not cp $src $dst — breaks on spaces/globs for f in "${files[@]}"; do … # not ${files[@]} — array stays element-safe rm -- "$path" # -- ends options; $path starting with - is data [[ -n "$x" ]] # [[ ]] doesn't word-split, but quote for habit grep -- "$pattern" "$file" ``` - Unquoted `$var` undergoes **word splitting** (on `IFS`) then **glob expansion**. A variable holding `*.txt` or `a b` becomes multiple args. This is the canonical footgun. - `"$@"` (quoted) preserves arguments exactly; `$@` and `$*` mangle them. Always `"$@"` to forward args. - Use `--` before user/agent-supplied operands so a value like `-rf` is treated as data, not flags. ## Argument parsing — case-based long flags The resource protocol mandates `--help` with an EXAMPLES section and rejects unknown flags with a USAGE error (exit 2). Use a `while`/`case` loop — it handles GNU-style long flags (`--json`, `--dry-run`), which `getopts` cannot: ```bash JSON=0; DRY_RUN=0; ARGS=() while [[ $# -gt 0 ]]; do case "$1" in --json) JSON=1 ;; --dry-run) DRY_RUN=1 ;; -h|--help) usage; exit 0 ;; --) shift; ARGS+=("$@"); break ;; # everything after -- is positional -*) printf 'ERROR: unknown flag: %s (try --help)\n' "$1" >&2; exit 2 ;; *) ARGS+=("$1") ;; esac shift done ``` `getopts` is fine for **short flags only** (`-v -o file`) and is more compact there, but it has no long-flag support and clusters awkwardly. Prefer the `case` loop for anything agent-facing — it matches `preinstall-check.sh` exactly. → Both styles in full, value-taking flags, `--flag=value`, and validation: [`references/argument-parsing.md`](references/argument-parsing.md). ## Traps, cleanup, and safe tempfiles Never leave a tempfile or half-written output behind. Create temp paths with `mktemp`, register a cleanup `trap` **immediately after**, and write atomically. ```bash tmp="$(mktemp)" || exit 1 cleanup() { rm -f "$tmp"; } trap cleanup EXIT # fires on normal exit, error, and signals via EXIT build_output >"$tmp" # write to temp mv -- "$tmp" "$dst" # atomic rename — reader never sees a partial file trap - EXIT; rm -f "$tmp" # (optional) disarm after successful move ``` - `trap cleanup EXIT` is the workhorse — `EXIT` fires for normal exit, `set -e` abort, and (in practice) `INT`/`TERM` if you let them propagate. Add explicit `trap cleanup INT TERM` if you do signal handling yourself. - `mktemp -d` for a temp **directory**; clean it with `rm -rf -- "$tmpdir"`. - **Atomic write = `tmp` + `mv`** (same filesystem). A reader sees either the old file or the complete new one, never a truncated mid-write — exactly the idempotency the resource protocol §6 requires. → Signal handling, ERR-trap with line numbers, nested traps: [`references/strict-mode-and-traps.md`](references/strict-mode-and-traps.md). ## The stream-separation + exit-code contract This is the load-bearing rule for any agent-facing script, lifted directly from SKILL-RESOURCE-PROTOCOL §4–§5. Claude parses stdout; pollution breaks `| jq`. - **stdout = the data product only.** JSON under `--json`, else plain/TSV. - **stderr = everything else.** Headers, progress, warnings, errors, prompts. - **Semantic exit codes**, not just 0/1: | Code | Meaning | |---|---| | `0` | success | | `2` | usage (bad/missing args, unknown flag) | | `3` | not found (input absent) | | `4` | validation (input present but malformed) | | `5` | precondition (missing dependency, wrong cwd) | | `7` | unavailable (external resource down — *advisory*, not a real failure) | | `10`+ | domain signal — a non-error "finding" the caller branches on | Code `10` is the workhorse for verifiers/scanners: "ran fine, found something." Reserve `7` so a network blip never looks like a content failure. Print human framing to stderr, the record to stdout: ```bash printf '%s\t%s\n' "$name" "$status" # data → stdout printf ' [ok] %s checked\n' "$name" >&2 # framing → stderr ``` → The shipped [`assets/script-template.sh`](assets/script-template.sh) bakes this contract in — copy it as the starting point for any new skill script. ## ShellCheck — non-negotiable Run [`shellcheck`](https://www.shellcheck.net/) on every script; it catches the quoting/word-splitting/`set -e` bugs above mechanically. ```bash shellcheck script.sh # lint shellcheck -x script.sh # follow sourced files shfmt -i 2 -ci -w script.sh # format (2-space indent, indent switch-cases) ``` - Fix warnings; don't blanket-suppress. When a suppression is genuinely correct, scope it to one line with a reason: `# shellcheck disable=SC2086 # word split intended`. - CI gate: `shellcheck **/*.sh` should pass clean before merge. - `bash -n script.sh` is a free syntax-only check (no execution) — run it in tests. ## Common footguns (quick table) | Footgun | Why it bites | Fix | |---|---|---| | Unquoted `$var` | Word-split + glob expansion | `"$var"` always | | `[ "$a" == "$b" ]` | `[` is POSIX `test`; `==` non-portable, no `&&` grouping | `[[ "$a" == "$b" ]]` in Bash | | `var=$(cmd) ; echo $?` | `$?` is the assignment's status (always 0), not `cmd`'s | `cmd; rc=$?` or check inline | | `cmd | while read x; do total=$x; done` | `while` runs in a **subshell**; `$total` is lost after the pipe | `while …; do … done < <(cmd)` (process substitution) | | `local x=$(cmd)` under `set -e` | `local` returns 0, masking `cmd` failure | `local x; x=$(cmd)` on two lines | | `echo "$x"` for arbitrary data | `echo` mangles `-n`, `-e`, backslashes | `printf '%s\n' "$x"` | | `for f in $(ls)` | Splits on whitespace, breaks on spaces/newlines | `for f in *` or `while IFS= read -r f` | | `pipefail` + `head` shows `141` | Downstream closes pipe early (SIGPIPE) | Expected; tolerate `141` from truncating consumers | → Each footgun with a reproducer and the underlying mechanism: [`references/footguns.md`](references/footguns.md). ## Bash version notes (attribute features correctly) Bash is stable, but several common idioms are **version-gated**. macOS still ships Bash **3.2** (2007, GPLv2); Linux/CI is usually Bash 5.x. If a script must run on stock macOS, avoid the 4.x+ features below or guard with `((BASH_VERSINFO[0]>=4))`. | Feature | Introduced | Notes | |---|---|---| | `mapfile` / `readarray` | Bash **4.0** | Read lines into an array. `mapfile -t arr < file`. `-d ''` (null-delimited) needs **4.4**. | | Associative arrays (`declare -A`) | Bash **4.0** | Hash maps. Unavailable on macOS stock 3.2. | | `${var,,}` / `${var^^}` (case conversion) | Bash **4.0** | Lowercase/uppercase expansion. | | `&>>` append-both-streams, `|&` | Bash **4.0** | `cmd |& filter` = `cmd 2>&1 | filter`. | | `${var@Q}` (quote operator) | Bash **4.4** | Produces a re-input-safe quoted form. Also `@U @L @E`. | | `wait -n` (any child) | Bash **4.3** | Useful for bounded parallelism. | | `local -n` (nameref) | Bash **4.3** | Pass array/var by reference into a function. | When in doubt, state the requirement in the first-comment-block (`# Requires: bash 4+`) and check at startup: `((BASH_VERSINFO[0] >= 4)) || { echo "needs bash 4+" >&2; exit 5; }`. ## Checklist before shipping a skill script - [ ] `#!/usr/bin/env bash` + first-comment-block contract (desc, Usage, Exit, Examples) - [ ] `set -Eeuo pipefail` (or a *deliberate* `set -uo pipefail`) + `IFS=$'\n\t'` - [ ] All expansions quoted; `"$@"` to forward args; `--` before operands - [ ] `case` arg loop; `--help` exits 0 with EXAMPLES; unknown flag → exit 2 - [ ] `trap cleanup EXIT` + `mktemp`; atomic `tmp`+`mv` writes - [ ] stdout data-only, stderr for framing; semantic exit codes (§5) - [ ] `shellcheck` clean; `bash -n` passes; `chmod +x` - [ ] Version-gated features guarded or documented
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.