Claude Skill

skill-audit

Use when a skill isn't triggering as expected, before adding a new skill (to check for a name collision), during periodic cleanup of an accumulated skills directory, or when asked to review skill quality — checks frontmatter validity, description quality against Skill Discovery O

LLM Mart · 0 points · 6 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download rtur2003-claude-code-promts-skills-.claude_skills_skill-audit-37c1edc.zip · 4 KB
Part of rtur2003/claude-code-promts-skills — 5 skills

Install

skills CLI npx skills add https://github.com/Rtur2003/Claude-Code-Promts-Skills/tree/main/.claude/skills/skill-audit
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install rtur2003-claude-code-promts-skills@llmmart
Git git clone https://github.com/Rtur2003/Claude-Code-Promts-Skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole rtur2003/claude-code-promts-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Skill Audit

Overview

Skills accumulate. Nobody deletes them, most developers hit 8-12 installed skills before the per-session token cost of loading every description starts to outweigh the benefit, and a skill installed at both the personal and project scope with the same name fails silently — the higher-precedence one wins and the other never fires, with no error anywhere. This script checks the mechanical properties that predict a skill working or silently failing: valid frontmatter, a description that leads with the trigger (not a workflow summary), a body under the token-efficiency guideline, and no name collision across scopes.

Core principle: A skill that never triggers and a skill that doesn't exist have the same effect on the user, but only one of them is visible in a directory listing. This script finds the invisible kind.

Usage

python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py [skills_dir ...]

With no arguments, checks .claude/skills/ (project) and ~/.claude/skills/ (personal) if they exist. Pass explicit paths to also check a plugin's skills/ directory (plugin skill dirs aren't auto-discovered, since a plugin's install location varies). Exit code 0 = clean, 1 = findings, 2 = usage error.

python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py .claude/skills ~/.claude/skills ./my-plugin/skills

What it checks

Check Why it matters
Frontmatter parses A SKILL.md without valid ----delimited YAML at the top is invisible to Claude Code — not degraded, just never loaded
name field matches directory name The command comes from the directory name for personal/project skills; a mismatched name field is almost always a copy-paste leftover from another skill
Description opens with a trigger phrase Per Skill Discovery Optimization: "Route the task and adopt it as your instructions" (what it does) trains the model to act on the summary instead of reading the body; "Use when the user names a task" (when to fire) doesn't
Description doesn't summarize the workflow A description that describes the process ("dispatches a subagent per task, reviews between each") gives the model a shortcut that has been observed to cause it to skip steps the actual skill body specifies
Description length Over 1024 characters fails the Agent Skills spec limit
Body word count Past ~2000 words (~500 lines), token cost per invocation is high enough that a reference file loaded on demand almost always beats keeping everything inline
Cross-scope name collisions Two SKILL.md files with the same name (or same directory name) at different scopes — one always wins, the other never triggers, and nothing tells you which

What it does not check

Whether the skill's instructions are correct, whether it actually gets invoked on the prompts it should (that requires running real scenarios — see the with-skill/without-skill baseline in agent-skills-prompt.md), or whether supporting files referenced from the body actually exist and are useful. This script catches the mechanical failure modes; it is not a substitute for testing trigger accuracy.

Remember

A clean audit means the skill is structurally capable of working. It says nothing about whether it triggers on the right prompts or produces the right output — that's a separate, behavioral test, not a static one.

Files (claude-code-promts-skills)
  • scripts
    • audit.py 4.9 KB
      #!/usr/bin/env python3
      """Deterministic audit of a Claude Code skills directory: frontmatter validity,
      description quality heuristics (SDO), body size, and name collisions across scopes.
      
      Usage: audit.py [skills_dir ...]
        With no args, checks the conventional locations that exist:
          .claude/skills/, ~/.claude/skills/, and any --plugin-dir path is NOT auto-discovered
          (pass plugin skill dirs explicitly as extra args).
      Exit code: 0 = no issues, 1 = issues found, 2 = usage error.
      """
      import sys
      import os
      import re
      import glob
      
      FRONTMATTER_RE = re.compile(r'\A---\n(.*?)\n---\n', re.DOTALL)
      WORKFLOW_WORDS = [
          'first', 'then', 'next', 'finally', 'step 1', 'step one',
          'dispatches', 'runs', 'writes then', 'reviews then',
      ]
      
      
      def parse_frontmatter(text):
          m = FRONTMATTER_RE.match(text)
          if not m:
              return None
          fm = {}
          for line in m.group(1).splitlines():
              if ':' in line and not line.strip().startswith('#'):
                  key, _, val = line.partition(':')
                  fm[key.strip()] = val.strip().strip('"\'')
          return fm
      
      
      def find_skill_dirs(roots):
          found = []
          for root in roots:
              if not os.path.isdir(root):
                  continue
              for entry in sorted(os.listdir(root)):
                  skill_md = os.path.join(root, entry, 'SKILL.md')
                  if os.path.isfile(skill_md):
                      found.append(skill_md)
          return found
      
      
      def check_description_quality(name, description):
          issues = []
          if not description:
              issues.append(f"{name}: missing 'description' field")
              return issues
          if len(description) > 1024:
              issues.append(f"{name}: description is {len(description)} chars, over the 1024-char spec limit")
          lower = description.lower()
          if not re.match(r'^use (when|for|this|before|after|during|as|to)\b', lower):
              issues.append(f"{name}: description doesn't open with a trigger phrase ('Use when...', 'Use before...', etc.) — SDO best practice leads with the trigger, not what the skill does")
          hits = [w for w in WORKFLOW_WORDS if w in lower]
          if hits:
              issues.append(f"{name}: description contains workflow-summary language ({', '.join(hits)}) — a description that summarizes process invites the model to act on the summary instead of reading the skill body")
          if description.strip().startswith('I ') or ' I ' in description[:20]:
              issues.append(f"{name}: description appears first-person; write in third person (it's injected into the system prompt)")
          return issues
      
      
      def main():
          args = sys.argv[1:]
          if not args:
              home = os.path.expanduser('~')
              roots = [os.path.join('.claude', 'skills'), os.path.join(home, '.claude', 'skills')]
          else:
              roots = args
      
          skill_files = find_skill_dirs(roots)
          if not skill_files:
              print(f"No SKILL.md files found under: {', '.join(roots)}")
              return 0
      
          issues = []
          names_seen = {}
          total_words = {}
      
          for path in skill_files:
              with open(path, encoding='utf-8') as f:
                  text = f.read()
      
              fm = parse_frontmatter(text)
              dirname = os.path.basename(os.path.dirname(path))
      
              if fm is None:
                  issues.append(f"{path}: no valid YAML frontmatter (must start with '---' on line 1)")
                  continue
      
              name = fm.get('name', dirname)
              if 'name' in fm and fm['name'] != dirname:
                  issues.append(f"{path}: frontmatter name '{fm['name']}' differs from directory name '{dirname}' — the command comes from the directory name for personal/project skills, this is likely a copy-paste leftover")
      
              if not re.match(r'^[a-zA-Z0-9-]+$', name):
                  issues.append(f"{path}: name '{name}' contains characters other than letters/numbers/hyphens")
      
              issues.extend(check_description_quality(path, fm.get('description', '')))
      
              body = text[FRONTMATTER_RE.match(text).end():] if FRONTMATTER_RE.match(text) else text
              word_count = len(body.split())
              total_words[path] = word_count
              if word_count > 2000:
                  issues.append(f"{path}: body is ~{word_count} words (roughly {word_count // 4} tokens) — well over the ~500-line / ~2000-token guideline; move detail to a reference file loaded on demand")
      
              key = name.lower()
              names_seen.setdefault(key, []).append(path)
      
          for name, paths in names_seen.items():
              if len(paths) > 1:
                  issues.append(f"name collision '{name}': defined in {len(paths)} places: {', '.join(paths)} — the one with higher scope precedence silently wins; the others never trigger")
      
          print(f"Scanned {len(skill_files)} skill(s) under: {', '.join(r for r in roots if os.path.isdir(r))}")
          print()
      
          if issues:
              print(f"FINDINGS ({len(issues)}):")
              for i in issues:
                  print(f"  - {i}")
              return 1
      
          print("clean: frontmatter valid, descriptions follow SDO conventions, no name collisions, no oversized bodies")
          return 0
      
      
      if __name__ == '__main__':
          sys.exit(main())
      
  • SKILL.md 3.8 KB
    ---
    name: skill-audit
    description: Use when a skill isn't triggering as expected, before adding a new skill (to check for a name collision), during periodic cleanup of an accumulated skills directory, or when asked to review skill quality — checks frontmatter validity, description quality against Skill Discovery Optimization conventions, body size, and cross-scope name collisions.
    ---
    
    # Skill Audit
    
    ## Overview
    
    Skills accumulate. Nobody deletes them, most developers hit 8-12 installed skills before the per-session token cost of loading every description starts to outweigh the benefit, and a skill installed at both the personal and project scope with the same name fails silently — the higher-precedence one wins and the other never fires, with no error anywhere. This script checks the mechanical properties that predict a skill working or silently failing: valid frontmatter, a description that leads with the trigger (not a workflow summary), a body under the token-efficiency guideline, and no name collision across scopes.
    
    **Core principle:** A skill that never triggers and a skill that doesn't exist have the same effect on the user, but only one of them is visible in a directory listing. This script finds the invisible kind.
    
    ## Usage
    
    ```bash
    python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py [skills_dir ...]
    ```
    
    With no arguments, checks `.claude/skills/` (project) and `~/.claude/skills/` (personal) if they exist. Pass explicit paths to also check a plugin's `skills/` directory (plugin skill dirs aren't auto-discovered, since a plugin's install location varies). Exit code 0 = clean, 1 = findings, 2 = usage error.
    
    ```bash
    python3 ${CLAUDE_SKILL_DIR}/scripts/audit.py .claude/skills ~/.claude/skills ./my-plugin/skills
    ```
    
    ## What it checks
    
    | Check | Why it matters |
    |---|---|
    | Frontmatter parses | A `SKILL.md` without valid `---`-delimited YAML at the top is invisible to Claude Code — not degraded, just never loaded |
    | `name` field matches directory name | The command comes from the directory name for personal/project skills; a mismatched `name` field is almost always a copy-paste leftover from another skill |
    | Description opens with a trigger phrase | Per Skill Discovery Optimization: "Route the task and adopt it as your instructions" (what it does) trains the model to act on the summary instead of reading the body; "Use when the user names a task" (when to fire) doesn't |
    | Description doesn't summarize the workflow | A description that describes the *process* ("dispatches a subagent per task, reviews between each") gives the model a shortcut that has been observed to cause it to skip steps the actual skill body specifies |
    | Description length | Over 1024 characters fails the Agent Skills spec limit |
    | Body word count | Past ~2000 words (~500 lines), token cost per invocation is high enough that a reference file loaded on demand almost always beats keeping everything inline |
    | Cross-scope name collisions | Two `SKILL.md` files with the same `name` (or same directory name) at different scopes — one always wins, the other never triggers, and nothing tells you which |
    
    ## What it does not check
    
    Whether the skill's *instructions* are correct, whether it actually gets invoked on the prompts it should (that requires running real scenarios — see the with-skill/without-skill baseline in [`agent-skills-prompt.md`](../../../prompts/english/agents/agent-skills-prompt.md)), or whether supporting files referenced from the body actually exist and are useful. This script catches the mechanical failure modes; it is not a substitute for testing trigger accuracy.
    
    ## Remember
    
    > A clean audit means the skill is *structurally* capable of working. It says nothing about whether it triggers on the right prompts or produces the right output — that's a separate, behavioral test, not a static one.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related