skill-audit
Use when a skill isn't triggering as expected, before adding a new skill (to check for a name collision), during periodic cleanup of an accumulated skills directory, or when asked to review skill quality — checks frontmatter validity, description quality against Skill Discovery O
Install
npx skills add https://github.com/Rtur2003/Claude-Code-Promts-Skills/tree/main/.claude/skills/skill-audit
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install rtur2003-claude-code-promts-skills@llmmart
git clone https://github.com/Rtur2003/Claude-Code-Promts-Skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole rtur2003/claude-code-promts-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Skill Audit
Overview
Skills accumulate. Nobody deletes them, most developers hit 8-12 installed skills before the per-session token cost of loading every description starts to outweigh the benefit, and a skill installed at both the personal and project scope with the same name fails silently — the higher-precedence one wins and the other never fires, with no error anywhere. This script checks the mechanical properties that predict a skill working or silently failing: valid frontmatter, a description that leads with the trigger (not a workflow summary), a body under the token-efficiency guideline, and no name collision across scopes.
Core principle: A skill that never triggers and a skill that doesn't exist have the same effect on the user, but only one of them is visible in a directory listing. This script finds the invisible kind.
Usage
python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py [skills_dir ...]
With no arguments, checks .claude/skills/ (project) and ~/.claude/skills/ (personal) if they exist. Pass explicit paths to also check a plugin's skills/ directory (plugin skill dirs aren't auto-discovered, since a plugin's install location varies). Exit code 0 = clean, 1 = findings, 2 = usage error.
python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py .claude/skills ~/.claude/skills ./my-plugin/skills
What it checks
| Check | Why it matters |
|---|---|
| Frontmatter parses | A SKILL.md without valid ----delimited YAML at the top is invisible to Claude Code — not degraded, just never loaded |
name field matches directory name |
The command comes from the directory name for personal/project skills; a mismatched name field is almost always a copy-paste leftover from another skill |
| Description opens with a trigger phrase | Per Skill Discovery Optimization: "Route the task and adopt it as your instructions" (what it does) trains the model to act on the summary instead of reading the body; "Use when the user names a task" (when to fire) doesn't |
| Description doesn't summarize the workflow | A description that describes the process ("dispatches a subagent per task, reviews between each") gives the model a shortcut that has been observed to cause it to skip steps the actual skill body specifies |
| Description length | Over 1024 characters fails the Agent Skills spec limit |
| Body word count | Past ~2000 words (~500 lines), token cost per invocation is high enough that a reference file loaded on demand almost always beats keeping everything inline |
| Cross-scope name collisions | Two SKILL.md files with the same name (or same directory name) at different scopes — one always wins, the other never triggers, and nothing tells you which |
What it does not check
Whether the skill's instructions are correct, whether it actually gets invoked on the prompts it should (that requires running real scenarios — see the with-skill/without-skill baseline in agent-skills-prompt.md), or whether supporting files referenced from the body actually exist and are useful. This script catches the mechanical failure modes; it is not a substitute for testing trigger accuracy.
Remember
A clean audit means the skill is structurally capable of working. It says nothing about whether it triggers on the right prompts or produces the right output — that's a separate, behavioral test, not a static one.
Files (claude-code-promts-skills)
-
scripts
-
audit.py 4.9 KB
#!/usr/bin/env python3 """Deterministic audit of a Claude Code skills directory: frontmatter validity, description quality heuristics (SDO), body size, and name collisions across scopes. Usage: audit.py [skills_dir ...] With no args, checks the conventional locations that exist: .claude/skills/, ~/.claude/skills/, and any --plugin-dir path is NOT auto-discovered (pass plugin skill dirs explicitly as extra args). Exit code: 0 = no issues, 1 = issues found, 2 = usage error. """ import sys import os import re import glob FRONTMATTER_RE = re.compile(r'\A---\n(.*?)\n---\n', re.DOTALL) WORKFLOW_WORDS = [ 'first', 'then', 'next', 'finally', 'step 1', 'step one', 'dispatches', 'runs', 'writes then', 'reviews then', ] def parse_frontmatter(text): m = FRONTMATTER_RE.match(text) if not m: return None fm = {} for line in m.group(1).splitlines(): if ':' in line and not line.strip().startswith('#'): key, _, val = line.partition(':') fm[key.strip()] = val.strip().strip('"\'') return fm def find_skill_dirs(roots): found = [] for root in roots: if not os.path.isdir(root): continue for entry in sorted(os.listdir(root)): skill_md = os.path.join(root, entry, 'SKILL.md') if os.path.isfile(skill_md): found.append(skill_md) return found def check_description_quality(name, description): issues = [] if not description: issues.append(f"{name}: missing 'description' field") return issues if len(description) > 1024: issues.append(f"{name}: description is {len(description)} chars, over the 1024-char spec limit") lower = description.lower() if not re.match(r'^use (when|for|this|before|after|during|as|to)\b', lower): issues.append(f"{name}: description doesn't open with a trigger phrase ('Use when...', 'Use before...', etc.) — SDO best practice leads with the trigger, not what the skill does") hits = [w for w in WORKFLOW_WORDS if w in lower] if hits: issues.append(f"{name}: description contains workflow-summary language ({', '.join(hits)}) — a description that summarizes process invites the model to act on the summary instead of reading the skill body") if description.strip().startswith('I ') or ' I ' in description[:20]: issues.append(f"{name}: description appears first-person; write in third person (it's injected into the system prompt)") return issues def main(): args = sys.argv[1:] if not args: home = os.path.expanduser('~') roots = [os.path.join('.claude', 'skills'), os.path.join(home, '.claude', 'skills')] else: roots = args skill_files = find_skill_dirs(roots) if not skill_files: print(f"No SKILL.md files found under: {', '.join(roots)}") return 0 issues = [] names_seen = {} total_words = {} for path in skill_files: with open(path, encoding='utf-8') as f: text = f.read() fm = parse_frontmatter(text) dirname = os.path.basename(os.path.dirname(path)) if fm is None: issues.append(f"{path}: no valid YAML frontmatter (must start with '---' on line 1)") continue name = fm.get('name', dirname) if 'name' in fm and fm['name'] != dirname: issues.append(f"{path}: frontmatter name '{fm['name']}' differs from directory name '{dirname}' — the command comes from the directory name for personal/project skills, this is likely a copy-paste leftover") if not re.match(r'^[a-zA-Z0-9-]+$', name): issues.append(f"{path}: name '{name}' contains characters other than letters/numbers/hyphens") issues.extend(check_description_quality(path, fm.get('description', ''))) body = text[FRONTMATTER_RE.match(text).end():] if FRONTMATTER_RE.match(text) else text word_count = len(body.split()) total_words[path] = word_count if word_count > 2000: issues.append(f"{path}: body is ~{word_count} words (roughly {word_count // 4} tokens) — well over the ~500-line / ~2000-token guideline; move detail to a reference file loaded on demand") key = name.lower() names_seen.setdefault(key, []).append(path) for name, paths in names_seen.items(): if len(paths) > 1: issues.append(f"name collision '{name}': defined in {len(paths)} places: {', '.join(paths)} — the one with higher scope precedence silently wins; the others never trigger") print(f"Scanned {len(skill_files)} skill(s) under: {', '.join(r for r in roots if os.path.isdir(r))}") print() if issues: print(f"FINDINGS ({len(issues)}):") for i in issues: print(f" - {i}") return 1 print("clean: frontmatter valid, descriptions follow SDO conventions, no name collisions, no oversized bodies") return 0 if __name__ == '__main__': sys.exit(main())
-
-
SKILL.md 3.8 KB
--- name: skill-audit description: Use when a skill isn't triggering as expected, before adding a new skill (to check for a name collision), during periodic cleanup of an accumulated skills directory, or when asked to review skill quality — checks frontmatter validity, description quality against Skill Discovery Optimization conventions, body size, and cross-scope name collisions. --- # Skill Audit ## Overview Skills accumulate. Nobody deletes them, most developers hit 8-12 installed skills before the per-session token cost of loading every description starts to outweigh the benefit, and a skill installed at both the personal and project scope with the same name fails silently — the higher-precedence one wins and the other never fires, with no error anywhere. This script checks the mechanical properties that predict a skill working or silently failing: valid frontmatter, a description that leads with the trigger (not a workflow summary), a body under the token-efficiency guideline, and no name collision across scopes. **Core principle:** A skill that never triggers and a skill that doesn't exist have the same effect on the user, but only one of them is visible in a directory listing. This script finds the invisible kind. ## Usage ```bash python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py [skills_dir ...] ``` With no arguments, checks `.claude/skills/` (project) and `~/.claude/skills/` (personal) if they exist. Pass explicit paths to also check a plugin's `skills/` directory (plugin skill dirs aren't auto-discovered, since a plugin's install location varies). Exit code 0 = clean, 1 = findings, 2 = usage error. ```bash python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py .claude/skills ~/.claude/skills ./my-plugin/skills ``` ## What it checks | Check | Why it matters | |---|---| | Frontmatter parses | A `SKILL.md` without valid `---`-delimited YAML at the top is invisible to Claude Code — not degraded, just never loaded | | `name` field matches directory name | The command comes from the directory name for personal/project skills; a mismatched `name` field is almost always a copy-paste leftover from another skill | | Description opens with a trigger phrase | Per Skill Discovery Optimization: "Route the task and adopt it as your instructions" (what it does) trains the model to act on the summary instead of reading the body; "Use when the user names a task" (when to fire) doesn't | | Description doesn't summarize the workflow | A description that describes the *process* ("dispatches a subagent per task, reviews between each") gives the model a shortcut that has been observed to cause it to skip steps the actual skill body specifies | | Description length | Over 1024 characters fails the Agent Skills spec limit | | Body word count | Past ~2000 words (~500 lines), token cost per invocation is high enough that a reference file loaded on demand almost always beats keeping everything inline | | Cross-scope name collisions | Two `SKILL.md` files with the same `name` (or same directory name) at different scopes — one always wins, the other never triggers, and nothing tells you which | ## What it does not check Whether the skill's *instructions* are correct, whether it actually gets invoked on the prompts it should (that requires running real scenarios — see the with-skill/without-skill baseline in [`agent-skills-prompt.md`](../../../prompts/english/agents/agent-skills-prompt.md)), or whether supporting files referenced from the body actually exist and are useful. This script catches the mechanical failure modes; it is not a substitute for testing trigger accuracy. ## Remember > A clean audit means the skill is *structurally* capable of working. It says nothing about whether it triggers on the right prompts or produces the right output — that's a separate, behavioral test, not a static one.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.