agent-skills-creator
Creates and improves portable Agent Skills with a validator, routing scenarios, and evidence-based keep, cut, merge, or retire decisions. Use when asked to "write a skill", "update all skills", "audit my SKILL.md", "remove redundant instructions", or fix skill triggering. For AGE
Install
npx skills add https://github.com/mblode/agent-skills/tree/main/skills/agent-skills-creator
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install mblode-agent-skills@llmmart
git clone https://github.com/mblode/agent-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole mblode/agent-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Agent Skills Creator
Create and improve skills in the Agent Skills open format: full lifecycle from pattern selection through validation and README update.
- IS: creating new agent skills and auditing or rewriting existing ones: SKILL.md, references, rules folders, scripts, evaluations.
- IS NOT: AGENTS.md/CLAUDE.md instruction files (use
agents-md) or general documentation quality (usedocs-writing).
Choose a Mode
- New skill → Creation Workflow below.
- Audit, improve, or rewrite an existing skill →
references/improving-existing-skills.md, which scores eleven audit dimensions, runs an ordered rewrite, then reuses Steps 5-8 for validation and shipping. - Simplify a skill or update a collection → same reference, plus
references/capability-delta.mdfor retention decisions and the collection ledger. - Skill never triggers, triggers on the wrong prompts, or its description is being trimmed in a crowded listing → the Routing Evals section of
references/evaluation-and-iteration.md, then the description. Body edits do not fix routing. - Skill stops after a first draft and hands back for review when nobody asked → "Say Where the Work Ends" in
references/authoring-tips.md; the fix is the scope of done and the review checkpoints, not the steps.
Reference Files
| File | Read When |
|---|---|
references/capability-delta.md |
Removing generic coaching, evaluating model upgrades, or auditing a whole collection |
references/authoring-tips.md |
Default when writing or cutting body content: judgement over rules, constraint calibration, permission and completion scope, degrees of freedom, content patterns, descriptions |
references/skill-patterns.md |
Choosing a structural pattern |
references/format-specification.md |
Directory layout, spec versus Claude Code-only frontmatter, body substitutions, loading semantics, which host reads the skill from where and what each can do, naming |
references/rules-folder-structure.md |
Building a rules-based audit/lint skill |
references/improving-existing-skills.md |
Auditing, scoring, simplifying, or rewriting an existing skill |
references/executable-code.md |
Skill includes scripts, injects live context with !, depends on packages, or invokes MCP tools |
references/evaluation-and-iteration.md |
Writing evals/evals.json, routing tests, ablating constraints, testing across models |
references/adopt-adapt-author.md |
A public skill already covers this ground, or deciding whether to vendor, adapt, or replace a third-party skill |
The validator is the single local statement of mechanical gates. From the repository root:
skills/agent-skills-creator/scripts/validate.sh skills/<name> # one skill
skills/agent-skills-creator/scripts/validate.sh --all # every skill in the repo
Creation Workflow
Copy this checklist to track progress:
Skill creation progress:
- [ ] Step 1: Choose a pattern
- [ ] Step 2: Create directory and frontmatter
- [ ] Step 3: Write SKILL.md body
- [ ] Step 4: Add reference or rule files
- [ ] Step 5: Validate
- [ ] Step 6: Update README.md (and docs/skills.mdx where the repo has one)
- [ ] Step 7: Smoke-test installation
- [ ] Step 8: Evaluate and iterate
Step 1: Choose a pattern
Simple/hub, workflow, rules-based, or mixed. references/skill-patterns.md has the shapes, the in-repo example for each, and the problem-to-pattern affinity table.
Step 2: Create directory and frontmatter
Create skills/<name>/SKILL.md with name and description, the --- on line 1. Write the description as a model trigger, not a human summary: what it does, what it covers, then "Use when..." with the phrases users actually say, and the key use case first because the listing trims descriptions from the tail when it runs over budget. Keep it as short as it can be while still saying when it applies: every installed description sits in every session, and one that over-claims loads a skill that does not help. validate.sh enforces the limits, so write for routing and let the script police the constraints.
Decide where the skill will run before adding any other field. Use portable fields for this collection; put genuine runtime prerequisites in compatibility. Host-specific fields require current host documentation and a deliberately host-specific package. State authorization boundaries in the body when a workflow deploys, sends, or spends, and grant the safe loops just as explicitly; the body carries both sides of that envelope. The format reference distinguishes these contracts.
Step 3: Write SKILL.md body
references/authoring-tips.md carries the judgement. Apply:
- Open with an IS/IS-NOT pair when adjacent skills exist or scope creep is likely ("Open with Boundaries")
- Add only context the agent lacks ("Don't State the Obvious"); use consistent terminology
- Phrase guidance as an outcome, reserving absolutes for safety, data loss, format contracts, and observed failures ("Judgement Over Rules")
- Check nothing here contradicts the harness, a sibling skill, or the repo AGENTS.md; route instead of restate ("Don't Fight the Harness or a Sibling")
- Keep the opinions that make the skill worth invoking; cut only what the target agent already does unprompted ("Cut Constraints, Keep Opinions"). On current frontier models over-prescription is not merely wasted tokens: instructions carried forward from older models are often too prescriptive and lower output quality, so the constraint cut is correctness work
- Match degrees of freedom to fragility: prose for open-ended work, exact commands for fragile or destructive ops ("Degrees of Freedom")
- Reach for named content patterns: template for fixed output, examples only where style is the deliverable, conditional for decision points
- Write the permission side of the envelope, not only the restriction: grant a known-safe loop with the reason it is safe, and keep confirmation for what deploys, sends, spends, or writes outside the working tree ("Grant Permission, Don't Just Restrict")
- State the workflow dependencies, what the finished state includes, and the completion evidence; scope and evidence are different sentences ("Say Where the Work Ends"). Keep a stop-for-review step only where the decision is genuinely the user's; current frontier models honor a mandatory pause literally and end the task there. Add a checklist only when it helps track a long or resumable task
- Put the deliverable, routing, and task-specific constraints first; loading and compaction behavior depend on the host. A skill with several workflows makes SKILL.md a router (mode choice, shared contract, one pointer per workflow) and keeps each workflow's body in its own reference ("Use the File System for Progressive Disclosure")
- Build a Gotchas section from observed failures: the highest-signal content in any skill
Step 4: Add reference or rule files
- Workflow/mixed: a
references/folder, each file linked from SKILL.md with a "Read when" condition - Rules-based: a
rules/folder;references/rules-folder-structure.mdcovers_sections.md,_template.md, and file naming - Simple/hub: track files alongside SKILL.md, linked from a tracks table
Prefer a reference that is code. An existing implementation or a test suite pins a contract better than prose describing it ("Reference-as-Spec"). Split by loading condition, not line count: two topics read at different moments are two files.
Advanced, all covered in references/executable-code.md and references/format-specification.md: scripts/ for executables the agent composes, resolved relative to the installed SKILL.md; Claude Code can substitute ${CLAUDE_SKILL_DIR}; !`command` injection for data the skill always needs at invocation (a diff, PR comments); hooks frontmatter for a PreToolUse gate that should exist only while the skill is active; config.json for setup context that would otherwise be re-asked every session.
Step 5: Validate
skills/agent-skills-creator/scripts/validate.sh skills/<name>
Output separates format requirements from house conventions, including recommendations adopted as local gates. Fix every FAIL. Run skills-ref validate <dir> for an independent metadata and naming check. Neither check establishes behavioral quality or compatibility with every host.
Step 6: Update README.md
Add a bullet under the matching category heading, and bump the skill count near the top of the README:
- **[<skill-name>](./skills/<skill-name>/SKILL.md)**: <one-line description>
Categories: Architecture, Design, Writing, Quality, Shipping, Authoring. validate.sh verifies the bullet and the count. A repo that also ships docs/skills.mdx needs the same bullet there; the check is conditional on that file existing, so it stays silent in a repo without one.
Step 7: Smoke-test
When installation behavior changed, install the edited local source into a disposable target using the installer's documented local-source options. Verify the loaded SKILL.md and a bundled reference or script against the working copy. Do not overwrite a global installation to smoke-test an edit, and do not install the remote default branch as evidence for unpushed changes.
Step 8: Evaluate and iterate
references/evaluation-and-iteration.md. Write 2-3 scenarios in evals/evals.json, add assertions after the first run, and measure with-skill against without-skill in fresh sessions. Test routing separately with should-trigger and near-miss prompts. Test on each target model, and ablate any rule you suspect is dead weight: delete it, rerun the scenarios, keep it only if one regresses.
Gotchas
- The installed copy under
~/.agents/skills/<name>/is a copy, not a link to your repo. Editing the repo changes nothing in a running session, and the stale copy loads silently, so a skill can be several commits behind while appearing correct. Verify that an evaluation loads the edited local source. - A reference-chain failure calls for moving the load condition into SKILL.md, not disguising the same dependency with different wording.
toc-over-100-lineswants## Contentsinside the first 20 lines of any reference over 100 lines. A TOC further down does not count, and the file fails while looking fine.readme-skill-countcompares the README's stated count againstfind skills -maxdepth 2 -name SKILL.md. Adding a reference file to an existing skill does not change it; only adding or removing a skill does.--agentonskills addis variadic and space-separated (--agent codex cursor); it consumes arguments until the next one starting with-. A comma-separated list is validated element-wise and rejected whole as one invalid name. Do not conclude from an empty~/.codex/skillsthat the install failed: any agent whoseskillsDiris.agents/skillsis treated as universal and installed to~/.agents/skills/, which those agents read directly.- A description that omits "Use when" fails
description-triggersoutright, but a description that has the phrase and the wrong trigger words fails nothing and simply never routes. The validator cannot see this; only a routing eval can. description-lengthpasses anything under 1024 characters, and a collection of descriptions that each pass can still overrun the host's listing budget together, at which point the host shortens every one of them and the "Use when" clauses go first. Judge a description in the listing next to its siblings, not alone.- A step reading "stop here and present the result for review" ends the task on current frontier models even when the user asked for the whole job and is not watching. The step reads as a safety rule, so it is rarely questioned during a constraint cut, and the skill ships stopping one implementation short.
Anti-patterns
- Usage examples standing in for an expressive interface; name the parameters and enums instead
- A rule stated in SKILL.md and again in a script's
--help, a tool description, or a rule file - Dumping the full specification into the SKILL.md body instead of a reference file
- Time-sensitive content ("before August 2025, use..."), including model names as the reason a rule exists
- A
context: forkskill whose body is guidelines rather than a task; the subagent gets conventions and no prompt, and returns nothing - Vague names (
helper,utils,tools,documents,data) that give the model nothing to route on - Magic numbers in scripts with no justifying comment
- Shipping without testing across the capability tiers and effort levels the skill will actually run under; what reads well to a frontier model may underspecify a small fast one, and a step the model only volunteers at high effort is a step the workflow does not really have
Related Skills
agents-mdfor auditing AGENTS.md/CLAUDE.md instruction filesdocs-writingfor documentation quality rules
Maintenance only: evals/evals.json holds the behavioural scenarios and routing prompts for anyone changing this skill. It never loads during a user task, which is the baseline Phase A of references/improving-existing-skills.md asks for.
Files (agent-skills)
-
evals
-
evals.json 7.6 KB
{ "skill_name": "agent-skills-creator", "evals": [ { "id": 1, "prompt": "Improve our deploy-checklist skill. It's 380 lines and people say it's too long. Here's the folder.", "expected_output": "A concrete keep, cut, merge, or retire decision, preserving deployment contracts and team preferences while removing generic coaching. Reports available baseline and revised evidence separately from unrun scenarios, with validator results.", "files": [], "assertions": [ "Records a baseline revision and existing validator result before editing", "Separates measured behavioral evidence from static judgement and unrun scenarios", "Preserves repo-specific release contracts while deleting generic reminders", "Considers retirement or merging when no distinct payload remains", "Runs validate.sh and reports its result" ] }, { "id": 2, "prompt": "Write a skill that reviews screenshots of our checkout for visual regressions and then generates a corrected mockup image for the designer.", "expected_output": "A skill whose body hands the screenshot to the model directly rather than instructing a describe-then-reason step, and which routes the mockup generation to a named image-generation tool with an explicit path for when that tool is absent, because image input is universal across current frontier models and image output is not.", "files": [], "assertions": [ "Does not instruct the model to describe or transcribe the screenshot as an intermediate step", "Names a specific image-generation tool for the mockup and states what the skill does when it is unavailable", "Does not assume the invoking model can emit an image directly", "States the shell or repo precondition if any step depends on one, so a browser host fails at step one rather than improvising", "Description opens in third person with a Use when clause and quoted trigger phrases" ] }, { "id": 3, "prompt": "My skill works great in Claude Code but does nothing useful when a teammate runs it from Cursor, and it broke the claude.ai upload. Here it is.", "expected_output": "A diagnosis that separates the two failures by their shared cause: Claude Code-only frontmatter and ${CLAUDE_*} substitutions degrade silently on Cursor and hard-error on a claude.ai upload from the same lines, plus a check of whether the body assumes a shell or a repo that a browser host cannot supply, and a fix that keeps the six spec fields and states preconditions in the body.", "files": [], "assertions": [ "Identifies the Claude Code-only frontmatter fields present and explains that they are ignored on Cursor and rejected on upload", "Finds any ${CLAUDE_*} substitution in the body and explains it is left literal on other agents", "Checks whether the body's steps assume bash or a working tree and adds a stated precondition where they do", "Does not recommend removing the skill from Cursor or maintaining two copies", "Runs validate.sh and reports frontmatter-portable as PASS or explains the remaining SKIP" ] }, { "id": 4, "prompt": "Update all skills for new frontier models. The vendor announcement says coding and judgement improved. No cross-model evaluation runner is connected.", "expected_output": "Updates retention guidance first, audits every skill with a ledger, removes generic coaching, retains task contracts, and explicitly marks cross-model behavior untested.", "files": [], "assertions": [ "Does not infer training contents or passing model evaluations from the announcement", "Records a concrete disposition for every skill", "Does not delete an operational security contract on benchmark reputation alone", "Distinguishes authored scenarios from executed tests" ] }, { "id": 5, "prompt": "We have 38 skills installed and Codex has started truncating the descriptions. It keeps loading the database skill for anything that mentions a model. Fix the routing.", "expected_output": "Reads every description as one listing in the order the host shows it, shortens descriptions instead of adding trigger words, replaces domain-noun category lists with the moment of use, gives colliding siblings a For X use Y edge, and verifies with should-trigger and near-miss prompts run against the full listing. Bodies are untouched.", "files": [], "assertions": [ "Concatenates or reads all installed descriptions together before editing any single one", "Reduces total description length rather than lengthening the misrouting skill's description", "Rewrites the database skill's trigger from a category list to a specific moment of use such as adding or changing a migration", "Adds a disambiguating clause to both members of any colliding pair, not just one", "Runs or authors a routing eval with near-miss prompts and reports the result against the whole listing", "Does not edit SKILL.md bodies to fix a routing problem" ] }, { "id": 6, "prompt": "Our feature-implementation skill used to run a task to completion. Since we upgraded the model it does the first cut, writes a summary, and asks us to review before it will run the tests or fix anything. Here's the skill.", "expected_output": "Locates the stop-for-review step and any blanket confirmation language, decides for each whether the decision is genuinely the user's, removes the ones that are not, and replaces them with a stated scope of done, completion evidence, and an explicit grant for the local test loop with the reason it is safe. Does not add step-by-step instructions to compensate.", "files": [], "assertions": [ "Identifies the specific step or sentence that instructs the model to stop for review after the first implementation", "Distinguishes checkpoints that guard a user decision (deploy, spend, destructive action) from a generic review pause and keeps only the former", "Adds a scope-of-done sentence covering implementation, running the affected tests, and fixing what the change broke", "Grants the local test loop explicitly with the reason it is safe rather than only removing the restriction", "Does not add prescriptive step lists or additional reminders to run tests", "Runs validate.sh and reports its result" ] } ], "routing": { "should_trigger": [ "How do I write a skill for our release process?", "Improve this skill, it never seems to change what the agent does.", "Audit my SKILL.md against the format spec.", "Why doesn't my skill trigger when I ask about deployments?", "Rightsize this skill, it's gotten bloated.", "Which frontmatter fields can I use if this has to work on claude.ai too?", "Should I install the popular react-best-practices skill or write our own?", "Codex is cutting our skill descriptions short and picking the wrong one, can you tighten them?", "Our skill stops after the first draft and asks for review, make it finish the job." ], "near_miss": [ "Rewrite our CLAUDE.md so Codex and Cursor read the same instructions. (agents-md owns instruction files)", "Audit our docs folder for Diataxis compliance. (docs-writing owns documentation quality)", "Write the README for this skills repo. (readme-creator)", "Configure a PreToolUse hook in settings.json. (update-config; a hook inside a skill's frontmatter is this skill's, a repo-level hook is not)" ] } }
-
-
references
-
adopt-adapt-author.md 4.2 KB
# Adopt, Adapt, or Author Use when a public skill already covers the ground you were about to write, or when deciding whether to vendor, fork, or replace a third-party skill you installed earlier. The default answer keeps moving toward "author". A current frontier model already carries general craft: WCAG thresholds, React hygiene, REST conventions, the standard library. A fetched skill that teaches those buys tokens and reconciliation, not capability. What no model carries is what *this* project decided. Weight the sort accordingly: adopt less than the skill's star count suggests, author more. ## The Sort Classify the candidate before you read it closely. The kind decides the action. | Kind | What it is | Action | |------|-----------|--------| | Process | A method that holds whatever the product is: how to audit a surface, how to decide whether a thing should animate at all | Adopt. Change little. | | Craft | A rule that composes with anything: a contrast threshold, a budget, a measurement | Take the rules, override the numbers with this project's numbers | | Taste | What this product looks like, sounds like, and refuses to do | Author. No fetched file knows the constraints. | Most candidates are mixed. Sort each *section*, not each file: a design skill is usually process in its audit steps and taste in its token values, and taking the whole file imports someone else's taste along with their method. A frontier model will apply plausible defaults for anything you leave unstated. That is why taste has to be written down and why craft numbers have to be explicit: the failure is not a gap, it is a confident wrong answer that reads as intentional. ## Why Not Just Install Both Two skills that each answer "what should this look like" do not compose. They arrive in the same context and the model reconciles them before it can act, usually by following whichever it read last or whichever was more specific about the wrong thing. This is the reason behind the IS / IS NOT boundary opener, not a separate rule. A boundary is only enforceable if exactly one skill owns each question. Installing an overlapping skill breaks the boundary from outside the file, where the audit will not catch it. Before adopting, check the candidate against every sibling already installed. If a sibling's description would also fire on the same prompt, one of them has to change or go. ## The Vendoring Cost Rule Vendoring verbatim is worth its cost only while the file still diffs usefully against upstream and still works. - **Diffs usefully.** You can pull upstream fixes and read what changed. Once you have edited it heavily, every diff is noise and you are maintaining a fork while pretending you are not. - **Still works.** A skill lifted out of a family of nine references its siblings. Take two, and both point at files that are not there. The model follows the pointer, finds nothing, and improvises. When either fails, rewrite it as your own file and record where it came from. A rewrite you own beats a vendored file you have already diverged from. Check for sibling dependencies *before* installing, not after: grep the candidate for relative paths and skill names, and confirm each one resolves in your bundle. ## Record the Rejection Write down what you rejected and why, next to what you adopted. A rejection is worth more later than an adoption. The next agent, or you in six months, finds the same well-starred skill and has no way to know it was already evaluated and turned down. Without the record it gets installed, collides with the skill that replaced it, and the boundary work is redone. Two forms, both fine: - **Machine-readable, for vendored files.** A `skills-lock.json` beside the bundle, pinning each vendored skill's upstream source and a content hash, so drift is detectable rather than assumed. - **Prose, for derivations.** A short `## Sources` section in the skill that names what it drew on, what it took, and what it left. Put it in the file, not only in the commit message: the next repo is cloned from a copy, not from the history, so anything that lives only in a commit message does not travel. Record the reason, not just the verdict. "Rejected: teaches craft the model already has" and "Rejected: depends on six siblings we did not take" lead to different decisions next time. -
authoring-tips.md 20.1 KB
# Authoring Tips What to put in a skill and how hard to say it. The mechanical rules live in `scripts/validate.sh`; this file is the judgement the script cannot make. ## Contents - Don't State the Obvious - Don't Instruct Behavior the Model Already Has - Judgement Over Rules - Don't Fight the Harness or a Sibling - Cut Constraints, Keep Opinions - Grant Permission, Don't Just Restrict - Say Where the Work Ends - Open with Boundaries (IS/IS-NOT) - Build a Gotchas Section - Use the File System for Progressive Disclosure - Comprehensive Reference Folders - Reference-as-Spec - Degrees of Freedom - Provide a Default, Not a Menu - Design the Interface, Not the Examples - Common Content Patterns - The Description Field Is For the Model - Think Through the Setup - Memory and Storing Data - Standing Instructions, Not One-Time Steps - Store Scripts and Generate Code - On-Demand Hooks - Composing Skills ## Don't State the Obvious The agent brings general knowledge, but needs the project facts and procedures that affect this task. Ground new guidance in actual artifacts, corrections, or observed failures. - Omit anything Claude would do correctly unsupervised - General coding advice ("use descriptive variable names") is noise - Standard conventions (2-space indentation, semicolons) are known - Target where your org deviates from defaults or Claude consistently errs **Test:** for each line, ask "Would removing this cause a mistake?" If not, cut it. ## Don't Instruct Behavior the Model Already Has The sharpest case of the section above, and the one that costs most, because these instructions do not sit there inertly: they compound with behavior the model already performs and push it past useful. Generic reminders such as "double-check your answer" or "run the tests before you finish" do not define a task-specific check. Current frontier models test and verify unprompted, so the reminder buys nothing on the first run and produces redundant runs after it. Remove them in a static simplification pass; claim unchanged output quality only after a behavioral comparison. Preserve the exact observation or command that establishes the task's completion. An external check is not this. "Run the test suite and quote the output" and "watch the check fail, then pass" produce evidence the model cannot generate by reasoning, so they stay. The line between the two is whether the instruction names which check and what result counts. Three related levers skew long by default and are worth setting deliberately: - **Calibrate the length of artifacts the skill writes to disk.** Reports, plans, briefs, and docs run long, and a fixed output template invites filling every section instead of dropping the ones the task does not need. Say that length follows the work, not the template. This is a real opinion, unlike "write concisely", which the model already believes it is doing. - **Write the quiet form of commands the skill tells Claude to run.** `npm test`, `git log`, and a full build dump hundreds of lines that get re-sent every remaining turn. Prefer `--reporter=dot`, `--quiet`, or `tail`. - **Leave orchestration to the host.** Specify independent work and expected results where delegation adds value. Do not prescribe a fixed agent count, a model override, or repeated review rounds for every task. **Test:** would the model do this unprompted? If yes, the instruction is at best inert and at worst additive. Same test as "Cut Constraints, Keep Opinions" below, pointed at the model's behavior rather than at its knowledge. ## Judgement Over Rules Absolutes earn their place in a narrow set of cases: safety, data loss, format contracts, and rules Claude has been observed to break. Everywhere else, state the outcome and let surrounding context pick the path. The reason is asymmetric cost. A rule that is wrong on one prompt in ten still gets followed on that prompt, and the model cannot tell that this is the case where you would have wanted it to pivot. Guidance that names the goal survives the exception; a prohibition does not. **Rule:** "Default to writing no comments. Never write multi-line comment blocks." **Outcome:** "Write code that reads like the surrounding code: match its comment density, naming, and idiom." The second is shorter, has no exception list to maintain, and gets a densely commented file right without being told about it. ## Don't Fight the Harness or a Sibling Before adding a directive, check whether the harness already does it, a sibling skill owns it, or the repo AGENTS.md states it. Overlapping instructions in one context ("leave documentation as appropriate" against "DO NOT add comments") make the model reconcile before it can act, and reconciliation is paid on every invocation. Route instead of restate: name the sibling in the IS-NOT line, and resolve precedence in advance where a clash is likely. `tidy` states that repository conventions override its defaults when they conflict. The model is told who wins rather than left to arbitrate. ## Cut Constraints, Keep Opinions The counterweight to the two sections above, and the one most often misapplied. A skill's value *is* the opinion it encodes: your team's taste, your product's constraints, the thing Claude does not do by default. Deleting that leaves a skill that adds nothing. The deletion target is guardrails duplicating the model's own judgement, not strong wording. **Keep:** a specific banned-word list, a dark-first deck rule, a house punctuation style. Claude does not share these by default. **Cut:** "write clear prose", "handle errors properly", "use good naming". Claude does this unprompted. The test is "would Claude do this anyway", never "is this strongly worded". ## Grant Permission, Don't Just Restrict A skill's authority envelope has two sides and most skills write only one. "Ask before applying the migration" is a restriction. "The local suite runs against disposable fixtures and cannot reach production, so run it, fix the failures your change caused, and rerun without checking in" is a permission, and it is the side that usually goes unwritten. The cost of the missing half is not neutral. An agent with no stated permission pauses at every step that looks consequential, which turns a five-minute loop into five round trips and strands the workflow half-done when nobody answers. Grant the permission with the reason it is safe, not just the verb. The reason is what lets the agent extend the grant to the next case that matches and withhold it from the one that does not; "run the tests" does not survive contact with a suite that suddenly has a staging credential in it, and "the fixtures are disposable and there is no production access" does. Keep restricting what earns it: anything that deploys, sends, spends, or writes outside the working tree. Blanket caution written defensively against a weaker model ("confirm before every file edit", "never run a command without approval") now buys nothing and costs a turn each time it fires. ## Say Where the Work Ends Name the full scope of done in the same breath as the task, because the verb does not carry it. "Implement the change" and "implement the change, run the affected tests, and fix what the change broke" stop in different places, and an agent reading the first one is not wrong to hand back a passing-looking diff it never ran. This is not the same as completion evidence. Evidence says how anyone knows the work finished; scope says what finishing includes. A skill wants both, and they are usually one sentence apart. For work that spans passes, say what each pass covers and what ends the loop. Prefer a condition ("repeat until `validate.sh` reports no FAIL") over a count, unless the count is the actual contract. If the skill wants exploration past a first working version, say what to explore and where it stops; "keep going" without a stop is as unactionable as no scope at all. Audit every stop-for-review step the same way. Current frontier models honor "present the plan and wait" or "stop after the first implementation for review" literally, and the workflow ends there whether or not anyone is reviewing. Keep a checkpoint only where the decision is genuinely the user's (a destructive step, a spend, a design fork the skill cannot resolve); everywhere else, state the scope of done and let the workflow run to it. A checkpoint carried over from a model that needed reining in is the mirror image of blanket caution, and it costs the same turn. ## Open with Boundaries (IS/IS-NOT) When sibling skills exist or scope creep is likely, open the body (right after the H1 intro) with a bold IS/IS-NOT pair to prevent wrong-skill routing and scope creep. ```markdown - **IS:** producing a self-contained brief another agent can execute without clarification. - **IS NOT:** doing the task itself, or planning work you will execute in this session. ``` Name the sibling to route to in the IS-NOT line ("use `agents-md`"). Skip it when a skill has no neighbors and unmistakable scope; it would just restate the description. ## Build a Gotchas Section The highest-signal content in any skill. Build from real failure points Claude hits. - Place near the end of SKILL.md ("Gotchas" or "Anti-patterns"), as short scannable bullets, not paragraphs - Ground each in an observed failure, not a hypothetical - Name the concrete command, value, or path and the consequence of getting it wrong; a warning without a consequence reads as optional - Update over time as new failure modes appear **Good:** "Don't use the brand domain for tenant subdomains; reputation damage from one tenant affects all" **Bad:** "Be careful with domain naming" (too vague, no reason given) ## Use the File System for Progressive Disclosure A skill is a folder, not one file: treat the file system as context engineering. List the files and Claude loads them when relevant. - `references/`: deep-dive docs loaded on demand - `scripts/`: executable utilities Claude composes - `assets/`: template files to copy and adapt - `examples/`: usage examples and snippets - `rules/`: categorized rule files for audit/lint skills SKILL.md is a map to that tree, not a repository of everything the domain knows. A reference nothing ever loads is dead weight, and a SKILL.md that inlines what a reference should hold is paid for on every invocation: reading a skill spends context, brings compaction closer, and adds guidance that may not apply to this task. For a skill with several workflows, make SKILL.md a router: the mode choice, the shared contract, and a pointer per workflow, with each workflow's body in its own file so the model reads the one it needs and none of the others. ## Comprehensive Reference Folders For broad domains (a design system, a full CLI surface, a style guide), many small focused files beat a few monoliths. Full treatment, the `index.md` map, and the 40-file design-system example are in the comprehensive-reference variant in `skill-patterns.md`. ## Reference-as-Spec The highest-fidelity reference is code. An existing implementation, a test suite, or a vendored library in another language communicates a contract better than prose describing the same contract, because it cannot be vague and it cannot drift from itself. When a skill needs Claude to match a contract, point at the code and interrogate only the deviations. `planning` does this with its reference-as-spec probe: it asks whether existing code, a library, or a site already does this the way the user wants, then treats those semantics as the spec. Prefer, in order: the code itself, a test suite that pins its behavior, then prose. ## Degrees of Freedom Match specificity to task fragility. Over-constraining open work makes the skill brittle; under-constraining fragile work loses determinism. Narrow bridge with cliffs: hand over exact steps. Open field: point a direction. - **High freedom** (multiple valid approaches, context picks the path): prose. "Review the code for bugs, readability, and adherence to project conventions." - **Medium freedom** (preferred pattern, variation acceptable): pseudocode or a parameterized signature. - **Low freedom** (fragile, consistency-critical, or destructive): the exact command. ```bash python scripts/migrate.py --verify --backup ``` Prescriptive for: format contracts, safety constraints, naming conventions, API schemas, migrations. Flexible for: implementation approach, code structure, tool selection. **Railroading:** "Use exactly this signature: `async function fetchUser(id: string): Promise<User>`" **Flexible:** "Fetch functions return typed promises and accept string IDs" ## Provide a Default, Not a Menu When several tools or libraries could work, pick one and show it; listing every option forces Claude to choose with no basis and bloats the skill. Add an escape hatch only for the known exception. **Bad:** "You can use pypdf, or pdfplumber, or PyMuPDF, or pdf2image." **Good:** "Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image with pytesseract instead." ## Design the Interface, Not the Examples For anything with a callable surface (a `scripts/` utility, `config.json`, rule filenames), spend the tokens on the interface rather than on usage examples. Expressive parameter names and enums communicate intent while leaving the exploration space open; a worked example narrows it to the case you happened to write. One line of behavioral instruction attached to an interface outperforms a usage block. A status enum of `pending`, `in_progress`, `completed` plus "keep exactly one item in_progress" fully defines a todo tool with no example at all. In this skill, `validate.sh --all` needs no example because the flag says what it does. ## Common Content Patterns Three patterns recur. Name them explicitly when reaching for one. ### Template pattern A fixed or flexible output format for consistent results. **Strict** when the format is a contract ("ALWAYS use this exact template"); **flexible** when a starting point ("sensible default; adjust sections as needed"). ```markdown # [Title] ## Executive summary [One paragraph] ## Key findings - Finding 1 ``` ### Examples pattern Examples narrow the exploration space, which is exactly what you want when output style *is* the deliverable: commit messages, copy, changelog entries. Give 2-3 input/output pairs there, because style is faster to show than to describe. Do not reach for it to teach an interface or to cover input Claude must adapt to. Narrowing then works against you: the model pattern-matches your example instead of reading the situation. See "Design the Interface, Not the Examples". ### Conditional workflow pattern Route through decision points instead of listing every path upfront. ```markdown Determine modification type: - Creating new content? → Follow "Creation workflow" below - Editing existing content? → Follow "Editing workflow" below ``` Push large branches into separate reference files so SKILL.md stays scannable. ## The Description Field Is For the Model At session start, Claude scans every description to decide relevance. It is a trigger description, not a human summary. - Optimize for the words users say when they need the skill: action verbs and domain nouns the model routes on - Add quoted user phrases: `"how do I..."`, `"build a..."`, `"fix my..."` - Structure: `[Does what] for/using [domain]. [Covers what]. Use when [specific trigger phrases]. For [adjacent job] use [sibling].` - Front-load. Hosts have different listing budgets and truncation policies. The 1024-character spec limit is a ceiling, not a target; a description that states its key use case in the first sentence survives trimming, one that saves the triggers for the end does not. - Undertriggering is the common failure, so lean pushy: name the contexts where the skill applies even when the user did not ask for it by name. Use near-miss evaluations to tighten scope when it overtriggers; invocation-control fields are host-specific. - Lean pushy on situations, not on domain nouns. A trailing category list ("use when working with databases, queries, models, or persistence") looks like broad coverage and behaves like noise: it matches every adjacent prompt and starves the sibling that should have won. The same words spent on the moment of use ("use when adding or changing a migration, or reviewing its rollout") route more often and collide less. - As short as it can be while still saying when it applies. Every installed description is in every session's context, and a host with too many of them shortens all of them to fit, so a long description costs its neighbors as well as itself. Pushy means naming the moment of use, not adding clauses. - Read the description against its siblings as one listing, the way the model sees it. Two descriptions that could both claim a prompt, or one that over-emphasizes its own applicability, load a skill that does not help the task, and neither problem is visible from inside a single file. **Weak:** "Provides architecture guidance for multi-tenant platforms" **Strong:** "Provides architecture guidance for multi-tenant platforms on Cloudflare or Vercel. Use when defining domain strategy, tenant identification, isolation, routing, or asking 'how do I support multiple tenants' or 'build a white-label platform'." ## Think Through the Setup Some skills need user-specific context first. Store it in a `config.json` in the skill directory rather than re-asking every session: Step 1 checks for the config, gathers missing values via AskUserQuestion, and later steps consume it. A reinstall overwrites the folder, so this is for values cheap to re-ask, not for state the skill accumulates. ## Memory and Storing Data Follow the host's memory policy and the user's authorization for persistent personal facts. Do not prescribe automatic memory writes or a universal memory owner. Store durable task artifacts outside the installed skill directory, where updates may replace files. Use a project location or user-selected path. Host-specific storage substitutions require explicit support; portable skills should resolve a concrete path before writing. ## Standing Instructions, Not One-Time Steps Put enduring constraints and routing before optional detail. Skill reloads, context retention, and compaction differ by host; do not encode a universal token budget or assume a reference remains available forever. ## Store Scripts and Generate Code Scripts let Claude spend turns on composition, not reconstructing boilerplate. Ship executables (`.sh`, `.py`, `.ts`) as helper functions to compose, and let Claude generate the wrappers. A data skill shipping `fetch_events()`, `fetch_users()`, and `run_query()` turns each analysis into a few lines of glue. Resolve scripts relative to the installed skill directory. Use `${CLAUDE_SKILL_DIR}` only in a host that documents that substitution. For paths and permissions, `!` context injection, error handling, constants, plan-validate-execute, runtime, and package dependencies, see the executable-code reference listed in SKILL.md. ## On-Demand Hooks The `hooks` frontmatter field registers hooks when the skill is invoked and keeps them for the rest of the session (or until the first successful run, with `once: true`). Use it for opinionated safety or observation that should not always run. - PreToolUse: validate or block tool calls (e.g. block `rm -rf` in a prod skill) - PostToolUse: observe and log tool results **Shapes this fits:** a `/careful` skill that blocks destructive commands via a PreToolUse matcher on Bash; a `/freeze` skill that blocks Edit and Write outside one directory during debugging; an `/observe` skill that logs every Bash command to an audit trail. The field is Claude Code-only; see the format reference before adding it to a skill that travels. ## Composing Skills Composition is name-based; no built-in dependency management. Reference another skill by name and the model invokes it if installed. Document it in a "Related skills" section ("After this workflow, run `skill-name`") and keep each skill on one concern, not duplicating another's. For a self-contained task the skill hands off every time, `context: fork` with an `agent:` runs the body itself as the subagent prompt with no conversation history; the body must then be a task, not conventions. For a job several skills share, an `agents/` prompt with a cheaper `model:` is the reusable form. -
capability-delta.md 5.3 KB
# Capability Delta Use when simplifying a skill, responding to a model upgrade, or auditing a collection. ## Retention test A skill supplies something the task, tools, repository, and host do not already supply. Classify each section before expanding it: | Content | Default action | Evidence to preserve | |---|---|---| | Generic competence: reason carefully, write clear prose, inspect code, fix mistakes | Delete | None unless a specific regression justifies a targeted instruction | | Host behavior: tool syntax, permissions, progress updates, memory, delegation | Delete duplication; scope necessary adapters to the host | Actual tool interface or host documentation | | Public domain knowledge | Cut tutorial prose; retain a compact rubric when an explicit audit needs repeatable coverage | Rule applicability, detection method, false positives | | Team taste or product policy | Keep in one authoritative location | User preference or repository convention | | Operational contract or observed failure | Keep the minimum reproducible guidance | Command, schema, failure consequence, or regression case | | Volatile facts: quotas, prices, SDK APIs | Resolve from current official sources when used | Source and date; no undated snapshot presented as live | A skill in a shared repository is read by several contributors' agents running different models, so it cannot be tuned to one of them. Step lists calibrated to the gaps of the weakest target overconstrain the strongest; a stated outcome and a stated scope of done are actionable at every tier and are what travels. A model upgrade moves two things at once, in opposite directions, and an audit checks both. Guidance written to rein in an older model (step-by-step recipes, blanket confirmation, a stop for review after the first draft) now overconstrains and makes the newer model stop early or defer decisions it could take. At the same time the newer model is more tentative about how far to take a task, so the scope of done and the permission for known-safe loops need to be stated where they were previously left implicit. Cutting the first without adding the second leaves a skill that stops sooner than the one it replaced. Strong model performance is a reason to revisit instructions, not proof that removing a particular contract preserves behavior. Do not infer what was in a vendor's post-training from an announcement. Label a static deletion judgement as such; reserve measured claims for actual runs. ## Keep, cut, merge, retire Keep a skill with a distinct trigger and useful payload. Cut generic explanations inside it. Merge when the remaining payload shares an existing skill's trigger and output contract. Retire when nothing unique remains; record the replacement or native capability and remove routing pointers, README entries, and obsolete fixtures together. A domain checklist can remain useful even when every rule is familiar: the user requested consistent coverage. Prefer applicability and detection recipes over lectures explaining the concept. Never delete a shipped application's security, accessibility, or data-integrity requirement merely because the model knows its name. ## Collection workflow 1. Pull safely and record the baseline revision, dirty paths, skill inventory, and validator result. 2. Update the creator's retention criteria first. Audit each skill's entry point, references, scripts, routing neighbors, and evaluation coverage against those criteria. 3. Audit the descriptions as one listing before editing any body. Concatenate every description in the order the host lists them and read the result as the model does: total length against the host's listing budget, pairs that could claim the same prompt, and any description that over-emphasizes its own applicability. Shorten before adding; a listing that runs over budget is trimmed from every tail at once, so one long description degrades routing for the whole collection. 4. Keep a collection ledger with one row per skill: unique payload, concrete change or retention reason, and verification status. For large rule sets, record which categories were sampled and expand inspection when a sample fails. 5. Fill gaps with a concrete contract, tool, or regression scenario. Do not add a new skill simply to cover a topic a frontier agent already handles. 6. Validate every changed skill and the collection; check stale paths after deletions. Update descriptions and README entries to match final behavior. 7. Report static checks separately from behavior runs. If target models or a runner are unavailable, ship reviewable edits and scenarios with that limitation explicit. Do not fabricate scores or call authored assertions passing tests. ## Behavioral comparison Use identical task inputs, repository state, tool access, and effort settings in fresh contexts for no-skill, previous-skill, and revised-skill arms. Record model identifier, host, date, loaded files, output artifact, assertion evidence, and failures. Compare task success, preference conformance, tool calls, latency, and context cost. Repeat borderline results before a destructive retirement decision. User-named target models define the matrix. Unavailable models stay untested; a different model cannot stand in for them. Preserve contract assertions even when both arms pass, since a later edit can regress them. Revisit the retained rule when the task or host changes; no rule has permanent tenure. -
evaluation-and-iteration.md 9.2 KB
# Evaluation and Iteration Build evals before docs: they reveal real gaps, not imagined ones. Two things fail independently and are measured separately: whether the skill triggers on the prompts it should (routing), and whether the output is right once it has (quality). ## Contents - Build Evaluations First - Routing Evals - Ablate Constraints - Test Across Models - Iterate with Two Claudes - Observe How Claude Navigates - Re-Evaluating After a Rewrite - Measuring Adoption ## Build Evaluations First Write 2-3 scenarios before expanding SKILL.md, else content chases imaginary problems. Expand the set after the first round shows where it is thin. Process: 1. Run the task **without** the skill in a fresh session; note failures 2. Convert each failure to a scenario 3. Measure baseline (no skill) vs. treatment (with skill) on each 4. Iterate until treatment beats baseline by more than it costs in tokens and time Store scenarios in `evals/evals.json` inside the skill folder. Use this repository scenario format, adapting it to the chosen runner when needed: ```json { "skill_name": "pdf-processing", "evals": [ { "id": 1, "prompt": "Extract all text from reports/q3.pdf and save it to output.txt", "expected_output": "output.txt containing the text of every page in reading order", "files": ["evals/files/q3.pdf"], "assertions": [ "output.txt exists and is non-empty", "Text from the last page is present", "No page is skipped or duplicated" ] } ] } ``` Write `prompt` the way a user types it, with real paths and context; vary formality across cases and include one boundary case. Add `assertions` after the first run, not before: you do not know what "good" looks like until you have seen an output. Keep them objective (countable, checkable); style and feel are for human review. Keep contract assertions even when both configurations pass. They guard against regressions; use differentiating assertions to measure added value. Each run needs a clean context, or authoring residue masks gaps in the written instructions. A subagent per case gives that in Claude Code; otherwise use a separate session. Disable the skill for the baseline with `skillOverrides` (`"off"`) rather than deleting it. `/plugin install skill-creator@claude-plugins-official` automates the loop: isolated runs, assertion grading with evidence, a with-versus-without benchmark, blind A/B between two versions, and description tuning. The `evals/` folder loads only when someone is changing the skill, never during a user task, and SKILL.md should say so where it lists the folder. ## Routing Evals Seeing a skill trigger says Claude found it, not that it does the job; never triggering says the description failed, whatever the body holds. Test routing on its own with two prompt sets: - **Should-trigger:** 8-10 prompts a user would type when they need this skill, phrased differently each time and none quoting the description verbatim - **Near-miss:** 8-10 prompts that look adjacent but belong to a named sibling skill, or to no skill A near-miss that routes here is a boundary problem: sharpen the IS-NOT line and the "For X use `sibling`" clause in both descriptions. A should-trigger that misses is a vocabulary problem: add the words the prompt used, and take out a clause of equal weight so the description does not grow. Run both sets against the whole installed listing, not one skill in isolation: the failure the model actually sees is two descriptions claiming the same prompt, or a listing so long the host has trimmed the trigger clause off the end. `skill-creator`'s description-tuning mode generates both sets, measures the hit rate, and proposes edits; a hand-kept JSONL of `{"prompt", "expected"}` pairs does the same job across a whole bundle. ## Ablate Constraints Evals tell you what to add. Ablation tells you what to remove, and it is the only honest way to run the constraint cut in `improving-existing-skills.md`. For a rule you suspect is carrying no weight: delete it, rerun the scenarios, and keep it only if one regresses. Large system prompts have been cut by most of their length this way with no measurable eval loss, because most of the text guarded against failures the current model no longer makes. - Ablate one rule at a time, or you learn nothing about which one mattered - A rule kept without an ablation is a guess, and guesses accumulate into the bloat you are trying to cut - A rule whose absence regresses a scenario has evidence for retention; link the scenario and revisit when the task, host, or model changes - Opinions are not ablatable this way: a house style has no failing scenario, it is the preference the skill exists to encode An opinion still has a dead state, and ablation cannot see it. A constraint dies when the model stops needing it, which shows up as an ablation that does not regress. An opinion dies when the model stops *following* it, which shows up as nothing at all: removing it regresses no scenario, and the model would not have produced it unprompted either, so both of the tests in `improving-existing-skills.md` vote to keep a line that is changing nothing. The test for an opinion is conformance, not regression: run the scenario with the skill and check whether the output actually took the position the opinion states. An opinion the model overrides, waters down, or silently ignores is dead weight exactly like a dead constraint, and the fix is usually placement or phrasing rather than deletion. Move ignored preferences closer to the decision they govern and retest; placement does not guarantee conformance. ## Test Across Models Skills augment the model, so the same body lands differently on each one. Guidance written for a frontier model may underspecify a small fast model; guidance written for a small one clutters a frontier model and, on the newest frontier models, measurably lowers output quality. More than one vendor's migration guidance now says the same two things: prompts carried forward from prior models are too prescriptive, and boundaries written to stop an older model make the newer one stop early. That makes the constraint cut a correctness pass, not tidying, and it makes "does the workflow run to completion" a scenario to run on the newest model, not only the smallest. Two axes decide the test matrix, and most skills only think about the first: - **Capability tier.** A small fast model asks: enough guidance and explicit steps? A frontier model asks: does this over-explain, or re-teach something it already does? Test the floor and the ceiling of the tiers the skill may run under, not the one you author on. - **Effort level.** Where the host exposes reasoning effort, include the settings used in deployment. The same body is read at the terse end (fewer, more consolidated tool calls, less preamble) and at the exhaustive end. A workflow that only completes because the model volunteered an unstated step is a workflow that breaks at low effort. Run scenarios at supported deployment settings and make required contracts explicit. A skill that travels across vendors adds a third question: does anything in the body assume one harness's tools, paths, or permission model? That is a portability bug, and it surfaces on another vendor's agent long before it surfaces in an eval. ## Iterate with Two Claudes **Claude A** authors and refines; **Claude B** runs tasks in a fresh session with the skill loaded. 1. Give B a real task 2. Watch where B struggles, skips a rule, or surprises you 3. Report the specific observation to A ("B forgot to filter test accounts") 4. A suggests targeted edits: stronger language, reordering, new section 5. Apply and retest Improve from observed behavior, not assumptions or memory of what Claude "should" need. Give A the failed assertions, the human feedback, and the transcript together; the fix should generalize past the failing case, not patch it. ## Observe How Claude Navigates Watch real sessions for: - **Unexpected exploration:** files read in an unplanned order; structure may be wrong - **Missed connections:** a reference isn't followed; make links more prominent - **Overreliance on one section:** same file read every time; move it into SKILL.md - **Ignored content:** a never-accessed reference; delete it or signal it better in SKILL.md - **Wasted work in transcripts:** unrequested validation, intermediate files nobody uses; the instruction that caused it is a removal candidate - **Repeated helper scripts:** every run writes the same parser or chart builder; bundle it in `scripts/` `name` and `description` drive triggering. If the skill isn't invoked when expected, fix the description's triggers before body content. ## Re-Evaluating After a Rewrite After improving a skill (see `improving-existing-skills.md`), rerun evals before shipping, with the pre-edit snapshot as the baseline instead of no-skill. Better audit dimensions but worse evals is a regression: dimensions measure form, evals measure behavior. ## Measuring Adoption Log invocations with a PreToolUse hook and compare actual usage against the trigger rate you expected. Undertriggering is a description problem, not a body problem: fix the "Use when" phrases before touching content. Across an org the same log finds promotion candidates for a shared library. -
executable-code.md 7.3 KB
# Executable Code in Skills For skills that include scripts, depend on packages, inject live context, or invoke MCP tools. Patterns that keep scripts reliable and cheap to execute. ## Contents - Execute vs. Read as Reference - Paths and Permissions - Dynamic Context Injection - Solve, Don't Punt - No Voodoo Constants - Plan-Validate-Execute - Runtime Environment - Package Dependencies - MCP Tool References - Visual Analysis ## Execute vs. Read as Reference State execution intent in SKILL.md. Otherwise Claude reconstructs the script's logic instead of running it, wasting tokens and diverging from canonical behavior. - **Execute:** "Run `scripts/analyze_form.py input.pdf > fields.json`" - **Reference:** "See `scripts/analyze_form.py` for the field-extraction algorithm" Execute deterministic work; read-as-reference only when Claude must adapt the algorithm to novel input. ## Paths and Permissions The session shell's working directory moves whenever Claude runs `cd`, so a bare `scripts/x.sh` resolves against wherever the shell happens to be. Resolve `scripts/x.sh` against the installed SKILL.md directory. Claude Code can substitute `${CLAUDE_SKILL_DIR}`; other hosts need the actual resolved path. Run this repository's validator from its root with the explicit skill argument. Pair the path with an `allowed-tools` rule when the script should run without a permission prompt. The variable is substituted in both places, so the rule matches the exact command the body issues: ```yaml allowed-tools: Bash(${CLAUDE_SKILL_DIR}/scripts/render.sh *) ``` The grant lasts the invoking turn and clears on the next user message. Keep it narrow: a skill checked into a repo can grant itself broad access, and it applies even in an untrusted folder. ## Dynamic Context Injection `` !`command` `` at the start of a line (or after whitespace) runs before Claude sees the skill and is replaced by the command's output, stdout and stderr merged. A ```` ```! ```` fenced block does the same for several lines. Use it for data the skill always needs on invocation: `git diff HEAD`, `gh pr view --comments`, a version check. Claude then starts with the facts inlined instead of spending a turn fetching them. Failure rules that catch people: - A non-zero exit aborts the whole invocation; Claude never sees the skill. Search commands get a pass on exit 1, nothing else does. Append `|| true` to any check that exits non-zero on findings. - Injected commands never prompt. A permission check that would ask, or deny, aborts the invocation. Pre-approve with `allowed-tools`; a matching deny rule still wins. - Each command runs under the Bash tool's 2-minute timeout, and large output arrives as a file path plus preview. - It runs only in Claude Code on the local machine. Skills synced from claude.ai, Cowork, the Skills API, and claude.ai chat replace or skip it. A skill that must travel fetches the data in its body instead. Substitution runs once; command output is not rescanned for more placeholders. ## Solve, Don't Punt Handle recoverable errors in the script, not defer them to Claude. Punting wastes a turn and is non-deterministic. A script that returns `open(path).read()` hands Claude a `FileNotFoundError` to interpret; one that catches it, creates the default, and says so keeps the turn deterministic. Return a sensible default, or fail with a specific actionable message. Never raise raw exceptions to Claude. ## No Voodoo Constants Every magic number needs a comment explaining why. If the author can't justify it, neither can Claude, and an unexplained constant is one nobody dares change. ```python # HTTP requests typically complete under 30s; extra margin for slow connections REQUEST_TIMEOUT = 30 ``` `TIMEOUT = 47` is the failure mode: a number chosen once for a reason now lost. ## Plan-Validate-Execute For batch or destructive operations, split into three phases so errors surface before changes apply. 1. **Plan:** Claude writes an intermediate file describing the operation (e.g., `changes.json` listing every field and value) 2. **Validate:** a script checks the plan against the target (schema, conflicts, missing fields) and produces actionable errors 3. **Execute:** a second script applies the plan once validation passes Use for multi-record edits, schema migrations, form filling, anywhere a dry run helps. Validation scripts name specific problems: "Field `signature_date` not in form. Available: customer_name, order_total, signed_date." ## Runtime Environment Executable skills require filesystem and code-execution tools. Discover those capabilities from the host; an API or browser alone does not imply a shell. - Only the frontmatter (`name`, `description`) is pre-loaded at session start - SKILL.md is read when a trigger matches; reference files are read on demand - Scripts can be **executed** via bash without their source entering the context window; only output counts, and that output is re-sent every later turn, so prefer quiet flags (`--reporter=dot`, `--quiet`, `tail`) - Large reference files and datasets are free until accessed - Name files descriptively (`form-validation-rules.md`, not `doc2.md`) so Claude can guess content from the path Bundle comprehensive resources (docs, examples, datasets); they cost nothing until read. ## Package Dependencies List required packages explicitly in SKILL.md and use `compatibility` for prerequisites that determine whether the workflow can run. Check available runtimes, network access, and installation policy in the active environment; a vendor name alone does not establish those capabilities. Prefer the standard library; when third-party packages are required, name them and show the install command once in SKILL.md. ## MCP Tool References Reference MCP tools by their fully qualified name: `ServerName:tool_name`. Unqualified names cause "tool not found" errors when multiple servers expose similarly named tools. - `BigQuery:bigquery_schema`, not `bigquery_schema` - `GitHub:create_issue`, not `create_issue` - `Linear:list_issues`, not `list_issues` Use the qualified form in instructions and examples. If a server name changes, update every reference at once. ## Visual Analysis When the host supports image input, hand over the picture rather than a description of it. For layout-heavy formats this beats parsing the structured source, because position and grouping are the information. - PDF forms → render pages to images, read field positions off the render - Charts and diagrams → read the image, not the plotting source - Web pages → screenshot and inspect the rendered layout Provide a script that produces the image (`pdf_to_images.py`), then hand the output over directly. Keep the script focused on conversion. Two failure modes follow from treating vision as something to route around: - **Instructing a transcription step.** "Describe the chart, then reason about the description" throws away the thing that made the image worth producing, and the description becomes a lossy intermediate nobody can audit. Ask for the conclusion from the image. - **Assuming image output.** Image input and generation are separate host capabilities. Several current frontier models take images in and return only text. A skill whose deliverable is a generated image routes to an image-generation tool and says which one, and states what it does when that tool is absent. A skill that only needs to *see* something has no such dependency. -
format-specification.md 4.2 KB
# Format and Host Compatibility Use the [Agent Skills specification](https://agentskills.io/specification) as the source of truth for portable packages and the [creator best practices](https://agentskills.io/skill-creation/best-practices) for authoring guidance. Check the current host documentation when a task needs extensions. The repository validator encodes mechanical limits; its `format` checks enforce the portable contract and its `house` checks enforce local conventions, including recommendations adopted as gates. ## Portable package A skill needs `SKILL.md` with YAML frontmatter and a Markdown body. Supporting directories are optional; the spec permits additional files and directories. This repository uses: ```text skill-name/ ├── SKILL.md ├── references/ # documentation loaded for a named condition ├── scripts/ # executable helpers with documented dependencies ├── assets/ # templates and static resources ├── rules/ # local audit pattern, not a spec requirement └── evals/evals.json # local evaluation fixtures, not a spec requirement ``` Root track files, rule schemas, README bullets, naming preferences for reference files, and evaluation JSON are collection conventions. They must not be presented as restrictions imposed by the open format. ## Frontmatter `name` and `description` are required strings. The optional portable fields are `license`, `compatibility`, `metadata`, and `allowed-tools`. The validator checks their types and limits. Put custom properties inside `metadata`, with string keys and values, rather than inventing top-level fields. Use `compatibility` only when a real prerequisite affects execution: a Git checkout, required CLI, network access, browser automation, or a particular host. A capability requirement is usually more useful than a vendor name. `allowed-tools` is an experimental portable field. Support and permission behavior depend on the host. It is not a portable sandbox, and listing a tool does not establish that the host provides it. ## Host extensions Fields such as `disable-model-invocation`, `context`, and `hooks` are host extensions, not portable fields. A host may support, reject, or ignore them. For a host-specific package, consult that host's current schema and document the dependency. This collection's portable validator rejects extra top-level fields. Do not add a host extension automatically because a workflow has side effects. State the action scope in the body, follow the user's authorization, and rely on the host's permission system. A skill cannot create tool access or authorization by declaring it in frontmatter. Claude Code substitutions such as `${CLAUDE_SKILL_DIR}`, `$ARGUMENTS`, and shell injection syntax require documented host support. For portable commands, resolve the installed SKILL.md directory and use its absolute path for bundled scripts. Do not assume the shell's current directory is the skill directory. ## Loading and references Put the task, completion evidence, and essential constraints in SKILL.md. Give each optional reference a direct path and a condition for loading it. Keep references focused so the agent can read only the material needed now. Avoid chains that require loading one document merely to discover another. Skills do not provide a universal context budget, reload policy, compaction behavior, installation directory, or dependency manager. Resolve those from the active host and installer. A sibling skill named in prose may be absent; give a fallback or report the specific missing dependency when it is essential. ## Validation and execution Run this collection's validator after edits. For an independent format check, use `skills-ref validate <skill-directory>` from the [official reference implementation](https://github.com/agentskills/agentskills/tree/main/skills-ref), recording the revision used. Its scope is metadata and naming, not workflow correctness or host compatibility. An installation smoke-test must load the edited local source in a disposable target. Installing an unchanged remote branch does not test local edits. Real task evaluations check routing, output quality, tool use, and failure handling separately from format compliance. -
improving-existing-skills.md 10.4 KB
# Improving Existing Skills Audit-then-rewrite protocol for a shipped skill. Use when asked to improve, audit, rewrite, review, simplify, or rightsize one. ## Contents - Relationship to the Creation Workflow - Should This Skill Still Exist - Audit Dimensions - Rewrite Procedure - Structure Normalization Decision Table - Large Rule-Set Scoping - Validation ## Relationship to the Creation Workflow Improvement replaces Steps 1-4 of the Creation Workflow with audit and rewrite phases, then reuses Steps 5-8 (validate, README, smoke-test, evaluate). Never skip validation: an unvalidated rewrite is a regression risk, not an improvement. Copy this checklist to track progress: ```text Skill improvement progress: - [ ] Phase A: Read everything (SKILL.md, every linked file, repo AGENTS.md, README entry), capture available eval results or mark the baseline unrun, decide the skill should still exist - [ ] Phase B: Score the eleven audit dimensions (before) - [ ] Phase C: Rewrite in the ordered procedure - [ ] Phase D: Validate (scripts/validate.sh + re-run the same evals + re-score) - [ ] Phase E: Re-score dimensions (after), update README one-liner, ship ``` ### Phase A: Read everything first - Read SKILL.md fully, then every linked file (references, tracks, rules layers). - Rules-based skills: read `_sections.md`, `_template.md`, and 2+ sample rules per category. - Read the repo AGENTS.md and the skill's README entry; source of truth for install commands and conventions. - `ls -R` the folder; `validate.sh` reports orphan files, so run it here to seed the audit. - Capture a behavioral baseline when a runner and target model are available. Otherwise record that limitation and author concrete scenarios before broad rewrites. Scenarios are specifications, not executed evidence. Do not edit during Phase A; mid-edit findings cause inconsistent half-rewrites. ## Should This Skill Still Exist Answer this before scoring anything, because the eleven dimensions all assume the answer is yes. A skill's entire value is the delta between the model with it and the model without it. Only one side of that subtraction is in this repo. The other side moves on its own: every constraint was written against a failure, and failures get fixed upstream, so a skill loses value with no edit, no bug report, and no signal that anything changed. `authoring-tips.md` applies this reasoning line by line under "Don't Instruct Behavior the Model Already Has"; it applies to whole skills too, and nothing else in this protocol asks the question. So run the without-skill arm of the eval, not just the with-skill arm. Three outcomes: - **The delta is real.** Proceed to the dimensions. - **The delta is small and concentrated.** The skill has become one or two paragraphs wearing a bundle. Cut it to those, or fold them into a sibling that already routes on the same prompts, and delete the folder. - **The delta is gone.** Retire it. Removing a skill that no longer changes behavior is a better outcome than rewriting it, and it is the one this protocol otherwise has no path to: every other branch here terminates in a rewrite. Record the reason next to the removal the way `adopt-adapt-author.md` records a rejection, or the same skill gets proposed again next quarter. Retirement needs the README bullet and count updated, so it reuses Step 6 of the Creation Workflow exactly as a rewrite does. ## Audit Dimensions Use these dimensions to locate substantive gaps. Score 1-5 when an audit score is requested; otherwise record concrete findings and their resolution. These are the judgement calls, so none of them is scriptable; everything mechanical is already a check. | # | Dimension | What 5/5 looks like | |---|-----------|---------------------| | 1 | Trigger coverage | Third-person description; "Use when..." with quoted user phrases; disambiguated from siblings; no longer than it takes to say when it applies | | 2 | Boundary clarity | IS/IS-NOT opener present and accurate where sibling skills exist | | 3 | Structure conformity | Pattern matches content; files in pattern-correct folders | | 4 | Signal density | Every line passes "would removing this cause Claude to make a mistake?"; one term per concept | | 5 | Gotchas quality | Each gotcha names a concrete command/value and consequence; from observed failures | | 6 | Freshness | No stale commands, paths, version pins, or model names; frontmatter fields valid for every place the skill is meant to run | | 7 | Progressive disclosure | Every reference earns its load condition and adds value SKILL.md does not already carry | | 8 | Workflow integrity | Dependencies clear; the scope of done stated upfront and the terminal step names observable completion evidence; every stop-for-review step guards a decision that is genuinely the user's | | 9 | Cross-skill coherence | Related Skills accurate; no trigger overlap with sibling descriptions | | 10 | Content patterns | Template, examples, and conditional patterns used where they fit; examples confined to style-sensitive output | | 11 | Constraint calibration | Absolutes confined to safety, data loss, and format contracts; other guidance phrased as an outcome; known-safe loops granted explicitly rather than left to per-step confirmation; no directive duplicating or contradicting the harness, a sibling skill, or a script's own interface | ## Rewrite Procedure Execute in order: correctness, then triggers, then structure, then deletion, then polish. Reordering causes rework, for example density-cutting a section you later move. 1. **Stale fixes.** Anything contradicting repo AGENTS.md or reality (install commands, paths, rule counts, CLI flags, frontmatter fields the target runtime rejects). Bugs; fix before stylistic work. 2. **Description.** Third-person opener of what it does, capability summary, "Use when..." triggers with quoted user phrases, key use case first, and no longer than that takes. Read it in the listing next to its siblings: if two descriptions could route the same prompt, both need an edge ("For X, use `other-skill`"), and a description that claims a whole domain rather than a moment of use gets cut to the moment. Check with a should-trigger and near-miss prompt set, not by rereading. 3. **Boundary opener.** Add or repair the IS/IS-NOT pair after the H1. 4. **Structure.** Apply the decision table below. After a move, update every link and grep all SKILL.md repo-wide for the old path. 5. **Signal-density cut.** Delete lines Claude would do anyway; dedupe SKILL.md/reference overlap; merge near-duplicate sections. 6. **Constraint cut.** Same pass over the same text, different target. Convert absolutes to outcome phrasing, delete rules the current model honors unsupervised, and delete anything an interface, sibling, or the harness already states. Blanket caution ("confirm before each edit", "ask before running anything") is the usual find here: replace it with a restriction on what deploys, sends, or spends, plus an explicit grant for the loop that is safe and the reason it is safe. The other usual find is a review checkpoint ("stop after the first implementation and present it", "wait for approval of the plan") that the current model honors literally and ends the task at. Keep it where the decision is the user's; otherwise delete it and make sure the scope of done in step 8 covers what the checkpoint was implicitly guarding. **Stop condition:** an opinion particular to this repo, team, or product is the skill's payload. Never cut it for being opinionated, only for being wrong or already the model's default. The test is "would Claude do this unprompted", not "is this strongly worded". A skill stripped of its opinions validates clean and helps nobody. 7. **Gotchas.** Rewrite vague warnings into concrete-failure format (command/value plus consequence); delete hypotheticals nobody has observed. 8. **Workflow integrity.** Long workflows benefit from progress tracking. State what the finished state includes before the first step, and have the final step name the command result or artifact that establishes completion. ## Structure Normalization Decision Table | Situation | Action | |-----------|--------| | Supporting .md files at skill root, skill is simple/hub with a tracks table | Keep: sanctioned hub track files | | Supporting .md files at skill root, any other pattern | Move to `references/`, update all links | | Multiple rules folders (`rules-arch/` + `rules-ax/` in `ax-audit`), SKILL.md dispatches to each layer explicitly | Keep: sanctioned layered design | | Multiple rules folders, no explicit dispatch | Consolidate into one `rules/` folder | | `agents/` folder with subagent prompts dispatched from SKILL.md | Keep: sanctioned | | A bundle file whose stated load condition is "do not load in normal use" (launcher metadata for external runners, e.g. `agents/openai.yaml`) | Keep: the condition is the point. State it in the reference table so nobody re-litigates it per skill | | A folder whose only load condition is "when changing this skill" (`evals/evals.json`, fixtures) | Keep, but say so explicitly: it never loads during a user task, so it is not dead weight and not progressive disclosure either | | File in the folder but never linked from SKILL.md | Link it with a read-when condition, or delete it | After any rename or move: `grep -rn "<old-path>" <repo>/skills/*/SKILL.md` must return nothing. ## Large Rule-Set Scoping For rules-based skills with 30+ rule files, don't rewrite every rule. Drift concentrates in SKILL.md, `_sections.md`, and `_template.md`: rewrite those fully, then let `validate.sh` handle the mechanical sweep over the rule files (frontmatter, prefix-to-section match, count reconciliation). Sample-read roughly 10% of rules per category and deep-rewrite only those that fail on substance. Rewriting a correct rule can only stay equal or get worse. ## Validation 1. `scripts/validate.sh skills/<name>` passes clean. Every mechanical constraint is a check there, so a clean run replaces reading a checklist. 2. Re-score the eleven dimensions; report before/after with files moved and anything deferred. 3. Rerun the evaluations from Phase A and diff against that baseline. Better dimension scores with worse eval results is a regression: dimensions measure form, evals measure behavior. Without the Phase A run there is nothing to diff against, and "the evals pass" says only that the rewrite is not catastrophic. 4. When install behavior changed, smoke-test the edited local source in a disposable target. Installing the remote default branch does not test unpushed edits. -
rules-folder-structure.md 2.3 KB
# Rules Folder Structure For rules-based skills (audits, lints, checklists), create a `rules/` folder: a section map, a rule template, one file per rule. ## `rules/_sections.md` Category map with impact levels: ```markdown # Sections This file defines all sections, their ordering, impact levels, and descriptions. The section ID (in parentheses) is the filename prefix used to group rules. --- ## 1. Category Name (prefix) **Impact:** CRITICAL | HIGH | MEDIUM-HIGH | MEDIUM | LOW-MEDIUM **Description:** One sentence explaining why this category matters. ``` ## `rules/_template.md` Per-rule file template: ```markdown --- title: Rule Title Here impact: MEDIUM tags: tag1, tag2 --- ## Rule Title Here Brief explanation of the rule and why it matters. **Incorrect (description of what's wrong):** [code block with bad example] **Correct (description of what's right):** [code block with good example] ``` ## Individual rule files - Named `<prefix>-<slug>.md`; prefix matches the section ID - One rule per file - Each follows `_template.md` ## Tier-based audit variant Audit skills may use a richer rule schema when `SKILL.md` explains it and `_sections.md` reconciles counts and categories. Common fields: `defaultTier`, `category`, `surfaces`, plus rule-specific override fields. Do not force `title`/`impact`/`tags` onto these skills if their template and dispatch instructions document the alternate schema. ## SKILL.md priority table Map categories to prefixes and rule counts: ```markdown | Priority | Category | Impact | Prefix | Rules | |----------|----------|--------|--------|-------| | 1 | Category Name | CRITICAL | `prefix-` | N | ``` ## Large rule sets At 30+ rules, maintenance shifts from writing rules to keeping the set consistent: - Every rule has frontmatter that follows the documented schema, plus the example or detection structure required by that schema - Every filename prefix matches a `_sections.md` section - Rule counts reconcile everywhere (description, priority table, prose): `ls rules/ | grep -v '^_' | wc -l` ## Multi-layer variant A skill may carry more than one rules folder (`ax-audit` runs `rules-arch/` for architecture and `rules-ax/` for trust) **only when** SKILL.md dispatches to each layer with its own loading condition and its own count. Otherwise consolidate into a single `rules/` folder. -
skill-patterns.md 4.3 KB
# Skill Patterns Four structural patterns. Pick one by what the skill has to do, then copy the shape of the named in-repo example rather than a generic skeleton: a real skill shows how the pattern actually holds up. ## Contents - Picking a pattern - Simple/hub - Workflow - Rules-based - Mixed - Cross-cutting: failure decision tables ## Picking a Pattern | Pattern | Use when | In-repo example | |---------|----------|-----------------| | Simple/hub | Dispatching to 2-5 focused files by track or mode | `ui-design` | | Workflow | A multi-step process with progressive reference loading | `agents-md`, `pr-reviewer` | | Rules-based | Auditing or linting against categorized rules | `typography-audit`, `docs-writing` | | Mixed | Workflow steps with conditional or platform-specific references | `multi-tenant-architecture` | Decision guide: auditing against a checklist is rules-based; guiding a process is workflow; dispatching by context is simple/hub. Unsure means workflow, the most flexible. The problem a skill solves suggests its pattern. These are recommendations, not requirements: a runbook could be rules-based with categorized diagnostic checks. | Problem the skill solves | Typical pattern | |--------------------------|-----------------| | Library, API, or CLI reference | Simple/hub or Workflow | | Product verification with tools | Workflow | | Data fetching and analysis | Workflow or Mixed | | Business process automation | Workflow | | Code scaffolding and templates | Workflow | | Code quality and review | Rules-based or Workflow | | CI/CD and deployment | Workflow | | Runbooks | Workflow or Mixed | | Infrastructure operations | Workflow | ## Simple/Hub Dispatch to focused files by track. SKILL.md is the tracks table plus whatever every track shares (boundary, output contract, gotchas); the substance lives in the track files, and a hub that starts teaching a track inline has stopped being a hub. ``` skills/<name>/ SKILL.md <track-1>.md <track-2>.md ``` Root-level track files are exclusive to this pattern; every other pattern keeps supporting files in `references/` or a rules folder. **Comprehensive-reference variant** (canonical home for this guidance): for broad domains, the hub dispatches into a folder of small files, e.g. `design-guidelines/` with 40 files (`buttons.md`, `colors.md`, `forms.md`), each 50-200 lines, mapped from an `index.md`. One concern per file, named after it; each stands alone with no cross-file reading order; files may run to roughly 450 lines when single-topic and TOC'd. Claude loads what the task needs instead of a 2000-line reference. ## Workflow A sequential process with references loaded per step. SKILL.md holds the steps and what every step needs; anything a single step needs goes in that step's reference. ``` skills/<name>/ SKILL.md references/<detail>.md ``` Load-bearing parts, in the order they matter: a "Read when" table mapping each reference to its trigger condition, a copyable progress checklist, numbered steps, and a final step that produces evidence. `agents-md` is the reference implementation. ## Rules-Based An audit or lint against categorized rules. SKILL.md is the dispatch and the output contract; the rules folder is the content. ``` skills/<name>/ SKILL.md rules/ _sections.md (categories: prefix, impact, why it matters) _template.md (per-rule schema) <prefix>-<slug>.md (one per rule) ``` SKILL.md carries a priority table (category, impact, prefix, rule count) so a truncated audit still surfaces the worst findings first, plus an output contract fixing the finding format. Rule counts in the description and the table must reconcile with the folder, which `validate.sh` checks. Folder mechanics are in `rules-folder-structure.md`. ## Mixed Workflow steps where one branch of references applies and the rest do not: platform-specific, framework-specific, or context-specific. The workflow determines context first, then loads only the matching reference. `multi-tenant-architecture` dispatches on Cloudflare versus Vercel this way. ## Cross-Cutting: Failure Decision Tables A compact symptom, cause, and recovery table helps with observed operational failures. Keep commands or error signatures that distinguish branches. Do not script rebuttals to a user who changes scope, declines an interview, or already authorized the next step.
-
-
scripts
-
validate.sh 20.4 KB
#!/usr/bin/env bash # # Validates a skill against the Agent Skills format and this repo's house style. # Every mechanical authoring rule is stated here and nowhere else; prose in # SKILL.md and references/ carries only judgement a script cannot make. # # validate.sh skills/<name> one skill # validate.sh --all every skill in the repo # validate.sh --format tsv --policy private --repo /path/to/private-repo # # Output groups checks under "format" (the Agent Skills spec) and "house style" # (this repo's taste). Exits 1 if any check FAILs. SKIP means a check does not # apply here, always with a reason. # # Portable to bash 3.2 (macOS default): no associative arrays, no mapfile. set -uo pipefail POLICY=public FORMAT=text # Bare mktemp, not `-t skillvalidate`: GNU mktemp rejects a -t template with no # X's, so the named form ran on macOS and died on every Linux checkout. RESULTS="$(mktemp)" trap 'rm -f "$RESULTS"' EXIT # record <PASS|FAIL|SKIP> <format|house> <check-name> <detail> record() { detail="$4" detail="${detail//$'\t'/ }" detail="${detail//$'\n'/ }" detail="${detail//$'\r'/ }" printf '%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "$detail" >>"$RESULTS" } # check <format|house> <check-name> <detail-on-failure> <count> # A count of 0 passes; anything else fails. Keeps call sites to one line. check() { if [ "$4" -eq 0 ]; then record PASS "$1" "$2" ""; else record FAIL "$1" "$2" "$3"; fi } validate_skill() { skill_dir="${1%/}" name="$(basename "$skill_dir")" md="$skill_dir/SKILL.md" if [ ! -f "$md" ]; then record FAIL format skill-md-present "$skill_dir has no SKILL.md" return fi # --- frontmatter: ruby owns YAML parsing and emits its own result lines --- # The magic comment keeps \p{L} valid when the shell has no UTF-8 locale # (LANG unset in CI or a sandbox makes -e scripts US-ASCII and ruby rejects # the property), so the check reports the real result instead of "ruby failed". SKILL_VALIDATION_POLICY="$POLICY" ruby -E UTF-8 -ryaml -e '# encoding: utf-8 path, folder = ARGV src = File.read(path) m = src.match(/\A---\n(.*?)\n---\n/m) # Detail explains a failure, so suppress it on PASS to avoid lines that read # as their own contradiction ("PASS name-reserved ... contains a reserved word"). def out(status, name, detail) puts [status, "format", name, status == "PASS" ? "" : detail].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") end unless m out("FAIL", "frontmatter-present", "no --- delimited YAML block at the top") exit end out("PASS", "frontmatter-present", "") y = begin YAML.safe_load(m[1]) || {} rescue => e out("FAIL", "frontmatter-parses", e.message.split("\n").first.to_s) exit end unless y.is_a?(Hash) out("FAIL", "frontmatter-mapping", "frontmatter must be a mapping") exit end fields = %w[name description license compatibility metadata allowed-tools] extra = y.keys - fields out(extra.empty? ? "PASS" : "FAIL", "frontmatter-portable", "unsupported fields: #{extra.join(", ")}") %w[name description license compatibility allowed-tools].each do |field| next unless y.key?(field) value = y[field] out(value.is_a?(String) ? "PASS" : "FAIL", "#{field}-type", "#{field} must be a string") end if y.key?("compatibility") value = y["compatibility"] valid = value.is_a?(String) && !value.strip.empty? && value.length <= 500 out(valid ? "PASS" : "FAIL", "compatibility-length", "compatibility must contain 1-500 characters") end if y.key?("metadata") value = y["metadata"] valid = value.is_a?(Hash) && value.all? { |k, v| k.is_a?(String) && v.is_a?(String) } out(valid ? "PASS" : "FAIL", "metadata-types", "metadata must map strings to strings") end n = y["name"].is_a?(String) ? y["name"] : "" d = y["description"].is_a?(String) ? y["description"] : "" def house(status, name, detail) return if ENV["SKILL_VALIDATION_POLICY"] == "private" puts [status, "house", name, status == "PASS" ? "" : detail].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") end if n.strip.empty? then out("FAIL", "name-present", "missing name") else out("PASS", "name-present", "") out(n.length <= 64 ? "PASS" : "FAIL", "name-length", "#{n.length} chars, max 64") out((n == n.downcase && n =~ /\A[\p{L}\p{N}]+(-[\p{L}\p{N}]+)*\z/) ? "PASS" : "FAIL", "name-charset", "#{n.inspect} must be lowercase alphanumeric, single hyphens, no leading or trailing hyphen") house(n =~ /anthropic|claude/ ? "FAIL" : "PASS", "name-reserved", "#{n.inspect} contains a reserved word") out(n == folder ? "PASS" : "FAIL", "name-matches-folder", "name #{n.inspect} vs folder #{folder.inspect}") end if d.strip.empty? then out("FAIL", "description-present", "missing description") else out("PASS", "description-present", "") out(d.length <= 1024 ? "PASS" : "FAIL", "description-length", "#{d.length} chars, max 1024") house(d =~ /<[a-zA-Z\/]/ ? "FAIL" : "PASS", "description-no-xml", "contains an XML-ish tag") house(d =~ /Use when/ ? "PASS" : "FAIL", "description-triggers", "no \"Use when\" trigger phrase") house(d =~ /\A\s*(I |Use this|This skill)/ ? "FAIL" : "PASS", "description-third-person", "opens in first person or as a human summary, not third-person capability") end ' "$md" "$name" >>"$RESULTS" 2>/dev/null || record FAIL format frontmatter-parses "ruby failed on $md" if [ "$POLICY" = public ]; then # --- body --- lines=$(wc -l <"$md" | tr -d ' ') if [ "$lines" -lt 500 ]; then record PASS house body-under-500-lines "" else record FAIL house body-under-500-lines "$lines lines, split into references/" fi # Windows separators, only on lines that look like paths, to avoid flagging # regex or LaTeX inside fenced blocks. n=$(grep -cE '[A-Za-z0-9_.-]+\\[A-Za-z0-9_.-]+\.(md|json|sh|py|ts)' "$md" 2>/dev/null || true) check house forward-slashes "backslash in a file path" "${n:-0}" # --- references --- # Reachability covers every markdown file in the skill, not just references/. # Hub skills index subfolders (guidelines/, direction/) from a root track file, # so a file counts as reachable if SKILL.md or any root .md names it. Without # this, deleting a file and its only link passes silently. # NUL-delimited throughout: a filename with whitespace otherwise word-splits, # which made this check pass and the kebab check miss the very name it exists # to catch. Process substitution, not a pipe, so $orphans survives the loop. orphans="" while IFS= read -r -d '' f; do base="$(basename "$f")" named=0 # Grep each root .md itself. `find -exec grep {} +` hits ARG_MAX (and # sandboxes that deny sysconf(_SC_ARG_MAX)), which made every file look # unreachable even when SKILL.md named it. -F so a dot is literal. for root_md in "$skill_dir"/*.md; do [ -f "$root_md" ] || continue if grep -qF -- "$base" "$root_md"; then named=1 break fi done [ "$named" -eq 1 ] || orphans="$orphans $base" done < <(find "$skill_dir" -name '*.md' -not -name 'SKILL.md' -not -path '*/rules*' -print0 2>/dev/null) check house all-md-reachable "unreachable from SKILL.md or a root track file:$orphans" \ "$(printf '%s' "$orphans" | wc -w | tr -d ' ')" if [ -d "$skill_dir/references" ]; then # A chain is a reference telling you to load another reference. A prose # cross-mention without an imperative is a pointer, not a chain. chains="" for f in "$skill_dir"/references/*.md; do [ -f "$f" ] || continue self="$(basename "$f")" for g in "$skill_dir"/references/*.md; do other="$(basename "$g")" [ "$self" = "$other" ] && continue grep -qiE "\b(load|read)\b[^.]*$other" "$f" && chains="$chains $self->$other" done done check house no-reference-chains "imperative chain:$chains" "$(printf '%s' "$chains" | wc -w | tr -d ' ')" else record SKIP house references-linked "no references/ folder" fi # TOC on long references and track files. Rule files are exempt: _template.md # fixes their schema, so they are single-topic by construction and a TOC there # is noise, not navigation. no_toc="" while IFS= read -r -d '' f; do fl=$(wc -l <"$f" | tr -d ' ') [ "${fl:-0}" -le 100 ] && continue head -20 "$f" | grep -qiE '^#{2,3} (contents|table of contents)' || no_toc="$no_toc $(basename "$f"):${fl}L" done < <(find "$skill_dir" -name '*.md' -not -name 'SKILL.md' -not -path '*/rules*' -print0 2>/dev/null) check house toc-over-100-lines "over 100 lines with no Contents heading:$no_toc" \ "$(printf '%s' "$no_toc" | wc -w | tr -d ' ')" # bash 3.2 mis-parses `case` inside $( ), so build this in the current shell. bad_names="" while IFS= read -r -d '' f; do b="$(basename "$f" .md)" if [ "$b" = "SKILL" ] || [ "$b" = "_sections" ] || [ "$b" = "_template" ]; then continue; fi printf '%s' "$b" | grep -qE '^[a-z0-9]+(-[a-z0-9]+)*$' || bad_names="$bad_names ${b// /<space>}" done < <(find "$skill_dir" -name '*.md' -print0 2>/dev/null) check house kebab-case-filenames "not kebab-case:$bad_names" "$(printf '%s' "$bad_names" | wc -w | tr -d ' ')" # Root-level track files belong to the simple/hub pattern only. root_md=$(find "$skill_dir" -maxdepth 1 -name '*.md' -not -name 'SKILL.md' 2>/dev/null | wc -l | tr -d ' ') if [ "$root_md" -eq 0 ]; then record PASS house root-md-hub-only "" elif grep -qiE '^#{2,3} (modes|tracks)' "$md"; then record PASS house root-md-hub-only "" else record FAIL house root-md-hub-only "$root_md root .md files but no Modes or Tracks table in SKILL.md" fi # --- rules folders --- rules_total=0 found_rules=0 for rd in "$skill_dir"/rules*; do [ -d "$rd" ] || continue found_rules=1 rname="$(basename "$rd")" [ -f "$rd/_sections.md" ] || record FAIL house rules-sections-present "$rname/_sections.md missing" [ -f "$rd/_template.md" ] || record FAIL house rules-template-present "$rname/_template.md missing" prefixes="$(grep -oE '\(([a-z0-9-]+)\)' "$rd/_sections.md" 2>/dev/null | tr -d '()' | sort -u)" bad_prefix="" no_fm="" for rf in "$rd"/*.md; do [ -f "$rf" ] || continue b="$(basename "$rf" .md)" case "$b" in _*) continue ;; esac rules_total=$((rules_total + 1)) head -1 "$rf" | grep -q '^---$' || no_fm="$no_fm $b" matched=0 for p in $prefixes; do case "$b" in "$p"-*) matched=1; break ;; esac done [ "$matched" -eq 0 ] && bad_prefix="$bad_prefix $b" done check house rules-prefix-in-sections "prefix not in $rname/_sections.md:$bad_prefix" \ "$(printf '%s' "$bad_prefix" | wc -w | tr -d ' ')" check house rules-frontmatter "no frontmatter:$no_fm" "$(printf '%s' "$no_fm" | wc -w | tr -d ' ')" done # Counts stated in the SKILL.md BODY (priority tables, gotchas) drift too, and # the description check below misses them entirely when the description states # no count. A body count is valid only if it equals the grand total, one # folder's total, or one prefix's total. Anything else is stale. if [ "$found_rules" -eq 1 ]; then valid=" $rules_total " for rd in "$skill_dir"/rules*; do [ -d "$rd" ] || continue valid="$valid $(find "$rd" -maxdepth 1 -name '*.md' -not -name '_*' | wc -l | tr -d ' ') " for p in $(find "$rd" -maxdepth 1 -name '*.md' -not -name '_*' -exec basename {} \; | sed 's/-.*//' | sort -u); do valid="$valid $(find "$rd" -maxdepth 1 -name "$p-*.md" | wc -l | tr -d ' ') " done done stale="" for n in $(grep -oE '[0-9]+ rules' "$md" | grep -oE '^[0-9]+' | sort -u); do case "$valid" in *" $n "*) ;; *) stale="$stale $n" ;; esac done check house rules-body-counts "SKILL.md states a rule count matching no folder, prefix, or total:$stale" \ "$(printf '%s' "$stale" | wc -w | tr -d ' ')" fi if [ "$found_rules" -eq 1 ]; then stated=$(ruby -E UTF-8 -ryaml -e ' src = File.read(ARGV[0]) m = src.match(/\A---\n(.*?)\n---\n/m) d = m ? (YAML.safe_load(m[1])["description"].to_s rescue "") : "" puts(d[/(\d+)\s+rules/, 1] || "") ' "$md" 2>/dev/null) if [ -z "$stated" ]; then record SKIP house rules-count-reconciles "description states no rule count" elif [ "$stated" = "$rules_total" ]; then record PASS house rules-count-reconciles "" else record FAIL house rules-count-reconciles "description says $stated, folders hold $rules_total" fi else record SKIP house rules-count-reconciles "no rules folder" fi # --- hygiene --- aux="$(find "$skill_dir" -maxdepth 1 \( -name 'README.md' -o -name 'CHANGELOG.md' -o -name 'INSTALLATION_GUIDE.md' \) -exec basename {} \; 2>/dev/null | tr '\n' ' ')" check house no-auxiliary-docs "auxiliary docs in the skill folder: $aux" "$(printf '%s' "$aux" | wc -w | tr -d ' ')" # Only a real command line counts. Prose warning against cp -R is correct content. n=$(grep -rhE '^[[:space:]]*cp -R.*\.claude/skills' "$skill_dir" 2>/dev/null | wc -l | tr -d ' ') check house install-not-cp-r "install instructions use cp -R into ~/.claude/skills" "$n" # Frontmatter is only read when --- is the first byte of the file. A BOM or a # leading blank line loads the body with empty metadata and no error. if head -c 3 "$md" | grep -q -- '---'; then record PASS format frontmatter-first-line "" else record FAIL format frontmatter-first-line "SKILL.md does not open with --- on line 1 (BOM or leading blank line)" fi n=$(grep -rlE '(/Users/|/home/)[a-z]' "$skill_dir" 2>/dev/null | wc -l | tr -d ' ') check house no-absolute-paths "hardcoded home path; resolve against the installed skill or project directory" "$n" n=$(grep -rnE 'MCP' "$skill_dir" 2>/dev/null | grep -E '`[a-z][a-z0-9]*_[a-z0-9_]+`' | grep -vE 'mcp__|[A-Za-z]:[a-z]' | wc -l | tr -d ' ') check house mcp-tools-qualified "MCP tool named without a Server:tool prefix" "$n" fi # Authored evaluation scenarios are data, not executed tests. Validate their # local contract so malformed or empty cases cannot masquerade as coverage. if [ -f "$skill_dir/evals/evals.json" ]; then ruby -rjson -e ' path, name = ARGV def fail_case(message) puts ["FAIL", "house", "eval-scenarios", message].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") exit end begin data = JSON.parse(File.read(path)) rescue JSON::ParserError => e fail_case("invalid JSON: #{e.message.lines.first.strip}") end fail_case("skill_name differs from folder") unless data.is_a?(Hash) && data["skill_name"] == name cases = data["evals"] fail_case("evals must be a nonempty array") unless cases.is_a?(Array) && !cases.empty? ids = [] cases.each do |item| fail_case("case must be an object") unless item.is_a?(Hash) id = item["id"] fail_case("case id must be a unique positive integer") unless id.is_a?(Integer) && id > 0 && !ids.include?(id) ids << id %w[prompt expected_output].each do |field| value = item[field] fail_case("case #{id}: #{field} must be nonempty text") unless value.is_a?(String) && !value.strip.empty? end %w[files assertions].each do |field| value = item[field] fail_case("case #{id}: #{field} must be an array of nonempty strings") unless value.is_a?(Array) && value.all? { |v| v.is_a?(String) && !v.strip.empty? } end fail_case("case #{id}: missing assertions") if item["assertions"].empty? end puts ["PASS", "house", "eval-scenarios", "#{cases.length} authored cases; behavior not executed"].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") ' "$skill_dir/evals/evals.json" "$name" >>"$RESULTS" 2>/dev/null || record FAIL house eval-scenarios "evaluation validation failed" else record SKIP house eval-scenarios "no evals/evals.json; behavior coverage not established" fi if [ "$POLICY" = public ]; then # --- house style --- n=$(find "$skill_dir" -name '*.md' -exec perl -CSD -ne 'print if /\x{2014}/' {} + 2>/dev/null | wc -l | tr -d ' ') check house no-em-dashes "em dash present, restructure with commas, colons, or periods" "$n" fi # --- repo integration --- repo_root="$(cd "$skill_dir/../.." 2>/dev/null && pwd)" if [ -f "$repo_root/README.md" ] && [ -d "$repo_root/skills" ]; then { grep -Fq "skills/$name/SKILL.md" "$repo_root/README.md" || { [ "$POLICY" = private ] && grep -Fq "| \`$name\` |" "$repo_root/README.md"; }; } \ && record PASS house readme-bullet "" \ || record FAIL house readme-bullet "no bullet in README.md" if [ "$POLICY" = public ] && [ -f "$repo_root/docs/skills.mdx" ]; then grep -q "skills/$name/SKILL.md" "$repo_root/docs/skills.mdx" \ && record PASS house docs-bullet "" \ || record FAIL house docs-bullet "no bullet in docs/skills.mdx" fi # Count folders holding a SKILL.md, not raw ls: git leaves empty directories # behind when a skill is moved out, and those must not inflate the count. actual=$(find "$repo_root/skills" -maxdepth 2 -name SKILL.md | wc -l | tr -d ' ') stated=$(grep -oE '^([0-9]+) skills' "$repo_root/README.md" | head -1 | grep -oE '[0-9]+') if [ -z "$stated" ]; then record SKIP house readme-skill-count "README states no skill count" elif [ "$stated" = "$actual" ]; then record PASS house readme-skill-count "" else record FAIL house readme-skill-count "README says $stated, skills/ holds $actual" fi else record SKIP house readme-bullet "not inside the skills repo" record SKIP house readme-skill-count "not inside the skills repo" fi } # --- entry point --- root="" target="" all=0 while [ "$#" -gt 0 ]; do case "$1" in --format|--policy|--repo) [ "$#" -ge 2 ] || { echo "Missing value for $1" >&2; exit 2; } case "$1" in --format) FORMAT="$2";; --policy) POLICY="$2";; --repo) root="$2"; all=1;; esac shift 2 ;; --all) all=1; shift ;; -*) echo "Unknown option: $1" >&2; exit 2 ;; *) [ -z "$target" ] || { echo "Only one skill path is accepted" >&2; exit 2; }; target="$1"; shift ;; esac done case "$POLICY" in public|private) ;; *) echo "Policy must be public or private" >&2; exit 2;; esac case "$FORMAT" in text|tsv) ;; *) echo "Format must be text or tsv" >&2; exit 2;; esac if { [ "$all" -eq 1 ] && [ -n "$target" ]; } || { [ "$all" -eq 0 ] && [ -z "$target" ]; }; then echo "Usage: validate.sh [--format text|tsv] [--policy public|private] SKILL_PATH | --all | --repo REPO" >&2 exit 2 fi emit() { [ -s "$RESULTS" ] || return 0 if [ "$FORMAT" = tsv ]; then awk -v skill="$name" -F'\t' 'BEGIN { OFS="\t" } { detail=$4; for(i=5;i<=NF;i++) detail=detail " " $i; gsub(/\r/, " ",detail); print skill,$1,$2,$3,detail }' "$RESULTS" else printf '\n=== %s\n' "$name" awk -F'\t' '{ printf " %-5s %-11s %-26s %s\n", $1, $2, $3, $4 }' "$RESULTS" printf ' %s FAIL, %s PASS, %s SKIP\n' "$(grep -c '^FAIL' "$RESULTS" || true)" "$(grep -c '^PASS' "$RESULTS" || true)" "$(grep -c '^SKIP' "$RESULTS" || true)" fi if grep -q '^FAIL' "$RESULTS"; then any_fail=1; fi } any_fail=0 if [ "$all" -eq 1 ]; then if [ -z "$root" ]; then here="$(cd "$(dirname "$0")" && pwd)" root="$(cd "$here/../../.." && pwd)" fi [ -d "$root/skills" ] || { echo "Missing skills directory: $root/skills" >&2; exit 2; } count=0 for d in "$root"/skills/*/; do [ -d "$d" ] || continue [ -n "$(find "$d" -type f -print -quit)" ] || continue count=$((count + 1)) : >"$RESULTS" validate_skill "$d" emit done : >"$RESULTS" name=_catalogue ruby -e ' root, policy = ARGV path = File.join(root, "README.md") unless File.file?(path) puts ["FAIL", "house", "readme-bullet", "README.md missing"].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") exit end text = File.read(path) names = text.scan(%r{skills/([a-z0-9-]+)/SKILL\.md}).flatten names += text.scan(/^\| `([a-z0-9-]+)` \|/).flatten if policy == "private" names.uniq.each do |name| unless File.file?(File.join(root, "skills", name, "SKILL.md")) puts ["FAIL", "house", "readme-bullet", "README lists missing skill #{name}"].map { |value| value.to_s.gsub(/[\t\r\n]/, " ") }.join("\t") end end ' "$root" "$POLICY" >>"$RESULTS" || record FAIL house readme-bullet "catalogue validation failed" emit [ "$count" -gt 0 ] || { echo "No skills found in $root" >&2; exit 2; } else validate_skill "$target" emit fi exit "$any_fail"
-
-
SKILL.md 13.4 KB
--- name: agent-skills-creator description: Creates and improves portable Agent Skills with a validator, routing scenarios, and evidence-based keep, cut, merge, or retire decisions. Use when asked to "write a skill", "update all skills", "audit my SKILL.md", "remove redundant instructions", or fix skill triggering. For AGENTS.md or CLAUDE.md use agents-md. compatibility: Repository validation requires Bash, Ruby with YAML and JSON, Perl, and standard Unix utilities. --- # Agent Skills Creator Create and improve skills in the Agent Skills open format: full lifecycle from pattern selection through validation and README update. - **IS:** creating new agent skills and auditing or rewriting existing ones: SKILL.md, references, rules folders, scripts, evaluations. - **IS NOT:** AGENTS.md/CLAUDE.md instruction files (use `agents-md`) or general documentation quality (use `docs-writing`). ## Choose a Mode - New skill → Creation Workflow below. - Audit, improve, or rewrite an existing skill → `references/improving-existing-skills.md`, which scores eleven audit dimensions, runs an ordered rewrite, then reuses Steps 5-8 for validation and shipping. - Simplify a skill or update a collection → same reference, plus `references/capability-delta.md` for retention decisions and the collection ledger. - Skill never triggers, triggers on the wrong prompts, or its description is being trimmed in a crowded listing → the Routing Evals section of `references/evaluation-and-iteration.md`, then the description. Body edits do not fix routing. - Skill stops after a first draft and hands back for review when nobody asked → "Say Where the Work Ends" in `references/authoring-tips.md`; the fix is the scope of done and the review checkpoints, not the steps. ## Reference Files | File | Read When | |------|-----------| | `references/capability-delta.md` | Removing generic coaching, evaluating model upgrades, or auditing a whole collection | | `references/authoring-tips.md` | Default when writing or cutting body content: judgement over rules, constraint calibration, permission and completion scope, degrees of freedom, content patterns, descriptions | | `references/skill-patterns.md` | Choosing a structural pattern | | `references/format-specification.md` | Directory layout, spec versus Claude Code-only frontmatter, body substitutions, loading semantics, which host reads the skill from where and what each can do, naming | | `references/rules-folder-structure.md` | Building a rules-based audit/lint skill | | `references/improving-existing-skills.md` | Auditing, scoring, simplifying, or rewriting an existing skill | | `references/executable-code.md` | Skill includes scripts, injects live context with `!`, depends on packages, or invokes MCP tools | | `references/evaluation-and-iteration.md` | Writing `evals/evals.json`, routing tests, ablating constraints, testing across models | | `references/adopt-adapt-author.md` | A public skill already covers this ground, or deciding whether to vendor, adapt, or replace a third-party skill | The validator is the single local statement of mechanical gates. From the repository root: ```bash skills/agent-skills-creator/scripts/validate.sh skills/<name> # one skill skills/agent-skills-creator/scripts/validate.sh --all # every skill in the repo ``` ## Creation Workflow Copy this checklist to track progress: ```text Skill creation progress: - [ ] Step 1: Choose a pattern - [ ] Step 2: Create directory and frontmatter - [ ] Step 3: Write SKILL.md body - [ ] Step 4: Add reference or rule files - [ ] Step 5: Validate - [ ] Step 6: Update README.md (and docs/skills.mdx where the repo has one) - [ ] Step 7: Smoke-test installation - [ ] Step 8: Evaluate and iterate ``` ### Step 1: Choose a pattern Simple/hub, workflow, rules-based, or mixed. `references/skill-patterns.md` has the shapes, the in-repo example for each, and the problem-to-pattern affinity table. ### Step 2: Create directory and frontmatter Create `skills/<name>/SKILL.md` with `name` and `description`, the `---` on line 1. Write the description as a model trigger, not a human summary: what it does, what it covers, then "Use when..." with the phrases users actually say, and the key use case first because the listing trims descriptions from the tail when it runs over budget. Keep it as short as it can be while still saying when it applies: every installed description sits in every session, and one that over-claims loads a skill that does not help. `validate.sh` enforces the limits, so write for routing and let the script police the constraints. Decide where the skill will run before adding any other field. Use portable fields for this collection; put genuine runtime prerequisites in `compatibility`. Host-specific fields require current host documentation and a deliberately host-specific package. State authorization boundaries in the body when a workflow deploys, sends, or spends, and grant the safe loops just as explicitly; the body carries both sides of that envelope. The format reference distinguishes these contracts. ### Step 3: Write SKILL.md body `references/authoring-tips.md` carries the judgement. Apply: - Open with an IS/IS-NOT pair when adjacent skills exist or scope creep is likely ("Open with Boundaries") - Add only context the agent lacks ("Don't State the Obvious"); use consistent terminology - Phrase guidance as an outcome, reserving absolutes for safety, data loss, format contracts, and observed failures ("Judgement Over Rules") - Check nothing here contradicts the harness, a sibling skill, or the repo AGENTS.md; route instead of restate ("Don't Fight the Harness or a Sibling") - Keep the opinions that make the skill worth invoking; cut only what the target agent already does unprompted ("Cut Constraints, Keep Opinions"). On current frontier models over-prescription is not merely wasted tokens: instructions carried forward from older models are often too prescriptive and lower output quality, so the constraint cut is correctness work - Match degrees of freedom to fragility: prose for open-ended work, exact commands for fragile or destructive ops ("Degrees of Freedom") - Reach for named content patterns: template for fixed output, examples only where style is the deliverable, conditional for decision points - Write the permission side of the envelope, not only the restriction: grant a known-safe loop with the reason it is safe, and keep confirmation for what deploys, sends, spends, or writes outside the working tree ("Grant Permission, Don't Just Restrict") - State the workflow dependencies, what the finished state includes, and the completion evidence; scope and evidence are different sentences ("Say Where the Work Ends"). Keep a stop-for-review step only where the decision is genuinely the user's; current frontier models honor a mandatory pause literally and end the task there. Add a checklist only when it helps track a long or resumable task - Put the deliverable, routing, and task-specific constraints first; loading and compaction behavior depend on the host. A skill with several workflows makes SKILL.md a router (mode choice, shared contract, one pointer per workflow) and keeps each workflow's body in its own reference ("Use the File System for Progressive Disclosure") - Build a Gotchas section from observed failures: the highest-signal content in any skill ### Step 4: Add reference or rule files - **Workflow/mixed**: a `references/` folder, each file linked from SKILL.md with a "Read when" condition - **Rules-based**: a `rules/` folder; `references/rules-folder-structure.md` covers `_sections.md`, `_template.md`, and file naming - **Simple/hub**: track files alongside SKILL.md, linked from a tracks table Prefer a reference that is code. An existing implementation or a test suite pins a contract better than prose describing it ("Reference-as-Spec"). Split by loading condition, not line count: two topics read at different moments are two files. Advanced, all covered in `references/executable-code.md` and `references/format-specification.md`: `scripts/` for executables the agent composes, resolved relative to the installed SKILL.md; Claude Code can substitute `${CLAUDE_SKILL_DIR}`; `` !`command` `` injection for data the skill always needs at invocation (a diff, PR comments); `hooks` frontmatter for a PreToolUse gate that should exist only while the skill is active; `config.json` for setup context that would otherwise be re-asked every session. ### Step 5: Validate ```bash skills/agent-skills-creator/scripts/validate.sh skills/<name> ``` Output separates **format** requirements from **house** conventions, including recommendations adopted as local gates. Fix every FAIL. Run `skills-ref validate <dir>` for an independent metadata and naming check. Neither check establishes behavioral quality or compatibility with every host. ### Step 6: Update README.md Add a bullet under the matching category heading, and bump the skill count near the top of the README: ```markdown - **[<skill-name>](./skills/<skill-name>/SKILL.md)**: <one-line description> ``` Categories: Architecture, Design, Writing, Quality, Shipping, Authoring. `validate.sh` verifies the bullet and the count. A repo that also ships `docs/skills.mdx` needs the same bullet there; the check is conditional on that file existing, so it stays silent in a repo without one. ### Step 7: Smoke-test When installation behavior changed, install the edited local source into a disposable target using the installer's documented local-source options. Verify the loaded SKILL.md and a bundled reference or script against the working copy. Do not overwrite a global installation to smoke-test an edit, and do not install the remote default branch as evidence for unpushed changes. ### Step 8: Evaluate and iterate `references/evaluation-and-iteration.md`. Write 2-3 scenarios in `evals/evals.json`, add assertions after the first run, and measure with-skill against without-skill in fresh sessions. Test routing separately with should-trigger and near-miss prompts. Test on each target model, and ablate any rule you suspect is dead weight: delete it, rerun the scenarios, keep it only if one regresses. ## Gotchas - The installed copy under `~/.agents/skills/<name>/` is a copy, not a link to your repo. Editing the repo changes nothing in a running session, and the stale copy loads silently, so a skill can be several commits behind while appearing correct. Verify that an evaluation loads the edited local source. - A reference-chain failure calls for moving the load condition into SKILL.md, not disguising the same dependency with different wording. - `toc-over-100-lines` wants `## Contents` inside the first 20 lines of any reference over 100 lines. A TOC further down does not count, and the file fails while looking fine. - `readme-skill-count` compares the README's stated count against `find skills -maxdepth 2 -name SKILL.md`. Adding a reference file to an existing skill does not change it; only adding or removing a skill does. - `--agent` on `skills add` is variadic and space-separated (`--agent codex cursor`); it consumes arguments until the next one starting with `-`. A comma-separated list is validated element-wise and rejected whole as one invalid name. Do not conclude from an empty `~/.codex/skills` that the install failed: any agent whose `skillsDir` is `.agents/skills` is treated as universal and installed to `~/.agents/skills/`, which those agents read directly. - A description that omits "Use when" fails `description-triggers` outright, but a description that has the phrase and the wrong trigger words fails nothing and simply never routes. The validator cannot see this; only a routing eval can. - `description-length` passes anything under 1024 characters, and a collection of descriptions that each pass can still overrun the host's listing budget together, at which point the host shortens every one of them and the "Use when" clauses go first. Judge a description in the listing next to its siblings, not alone. - A step reading "stop here and present the result for review" ends the task on current frontier models even when the user asked for the whole job and is not watching. The step reads as a safety rule, so it is rarely questioned during a constraint cut, and the skill ships stopping one implementation short. ## Anti-patterns - Usage examples standing in for an expressive interface; name the parameters and enums instead - A rule stated in SKILL.md and again in a script's `--help`, a tool description, or a rule file - Dumping the full specification into the SKILL.md body instead of a reference file - Time-sensitive content ("before August 2025, use..."), including model names as the reason a rule exists - A `context: fork` skill whose body is guidelines rather than a task; the subagent gets conventions and no prompt, and returns nothing - Vague names (`helper`, `utils`, `tools`, `documents`, `data`) that give the model nothing to route on - Magic numbers in scripts with no justifying comment - Shipping without testing across the capability tiers and effort levels the skill will actually run under; what reads well to a frontier model may underspecify a small fast one, and a step the model only volunteers at high effort is a step the workflow does not really have ## Related Skills - `agents-md` for auditing AGENTS.md/CLAUDE.md instruction files - `docs-writing` for documentation quality rules Maintenance only: `evals/evals.json` holds the behavioural scenarios and routing prompts for anyone changing this skill. It never loads during a user task, which is the baseline Phase A of `references/improving-existing-skills.md` asks for.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.