agent-skills
Use this skill when creating, reviewing, or editing Agent Skills-format skills, or when implementing skill discovery and loading in an agent client. It covers directory structure, SKILL.md metadata, progressive disclosure, evals, and repository conventions. Do not use this skill
Install
npx skills add https://github.com/magnus919/agent-skills/tree/main/agent-skills
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install magnus919-agent-skills@llmmart
git clone https://github.com/magnus919/agent-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole magnus919/agent-skills collection as a plugin from our marketplace. Git is the plain clone.
README
Agent Skills — The Standard Itself
This skill is the meta-skill: it documents the Agent Skills open format itself. Every other skill in this repository follows the conventions defined here.
Why Install This Skill
When your agent loads this skill, it gains the ability to create, review, and edit valid Agent Skills-format skills. That means:
- Your agent can build new skills on demand. Describe a workflow you want encoded, and your agent can scaffold a valid SKILL.md with correct frontmatter, progressive disclosure structure, and agent-friendly descriptions.
- Existing skills stay valid. When your agent edits a skill, it checks against the format standard — no broken frontmatter, no missing required fields.
- Consistency across your skill library. Every skill in your repository follows the same conventions, making them loadable by any Agent Skills-compatible harness.
What You Get
| Directory | Purpose |
|---|---|
SKILL.md |
A create-and-review workflow, the core format rules, and a completion checklist |
references/specification.md |
A bundled snapshot of the official format specification |
references/best-practices.md |
Guidance for useful, well-scoped instructions |
references/optimizing-descriptions.md |
How to design and test reliable automatic triggers |
references/using-scripts.md |
How to bundle safe, agent-friendly executable helpers |
references/vetting-third-party-skills.md |
How to vet a skill from a registry, colleague, or LLM before running it |
references/evaluating-skills.md |
A practical evaluation loop for testing skill quality |
references/client-implementation.md |
Guidance for products that discover and load skills |
Quick Start
Ask your agent to create or review a skill, then load this skill first. For a structural check, run:
skills-ref validate path/to/skill
If the reference validator is not installed, your agent should report that clearly and complete the equivalent manual checks.
Triggers
- You're creating a new skill and need to know the correct format
- You're reviewing an existing skill for format compliance
- You want your agent to generate a valid SKILL.md scaffold
- You're setting up conventions for a shared skill repository
Requirements
The documentation itself has no runtime dependencies. Optional validation uses the skills-ref reference library; install it in an isolated environment when you want to run its CLI.
Skill manifest
Agent Skills Standard Reference
This skill documents the Agent Skills open format — a standardized way to give AI agents new capabilities and expertise. Follow this workflow when creating or editing skills in this repository.
Authoritative source: agentskills.io/specification. The bundled specification is a working snapshot; check the authoritative source when currentness matters.
Directory Structure
A skill is a directory containing, at minimum, a SKILL.md file:
skill-name/
├── SKILL.md # Required: metadata + instructions
├── evals/ # Required for new skills in this repository
├── scripts/ # Optional: executable code
├── references/ # Optional: documentation
├── assets/ # Optional: templates, resources
└── ... # Any additional files or directories
Required Workflow
Create or edit a skill
- Read the specification before changing
SKILL.mdmetadata or directory structure. - Ground instructions in real domain knowledge, project artifacts, and observed failure modes. Read best practices when designing or materially revising instructions.
- Keep the skill a coherent, triggerable unit. Put only essential instructions in
SKILL.md; put conditional detail in focused reference files and state exactly when to read each one. - Use a precise
descriptionthat says what the skill does, when it applies, and when it does not apply. For skills with meaningful overlap, name the nearest alternative or prerequisite in a## When not to usesection. Test the boundary with at least three should-trigger prompts and two should-not-trigger near-misses; keep these harness-specific trigger checks separate from portable output-quality evals. Read optimizing descriptions for trigger design. - For every new skill, create
evals/evals.jsonwith at least five representative output-quality cases. Each case needs a realistic prompt, an expected outcome, and observable assertions. Include edge cases that exercise risky or ambiguous behavior. Read evaluating skills for the eval format and iteration workflow. - When bundling executable code, read using scripts. Document prerequisites and non-interactive invocation in the skill.
- Before handoff, run the validation checks in this skill and correct every finding.
Review a skill
- Validate the required frontmatter, field constraints, parent-directory/name match, and YAML syntax against the specification.
- Check that the description has both positive and negative trigger boundaries, the workflow is actionable, and resource references are conditional and reachable.
- Check that any script has documented dependencies, safe non-interactive inputs, clear errors, and structured output where useful.
- For this repository, also verify its required human-facing
README.md: title, Why Install This Skill, What You Get, Quick Start (unless genuinely reference-only), Triggers, and Requirements. Keep it human-facing, concise, and free of agent-only instructions. - Verify every new skill has
evals/evals.jsonwith at least five output-quality cases, and check that their expected outcomes and assertions test meaningful behavior. For an existing skill without evals, flag the gap but do not block the review solely for that legacy absence. Read evaluating skills before declaring the work complete.
Client implementation work
When implementing skill discovery, activation, or context management in an agent product, read client implementation guidance. Do not apply client conventions such as search paths as universal format requirements.
SKILL.md Format
The SKILL.md file must contain YAML frontmatter followed by Markdown body content.
Frontmatter Fields
| Field | Required | Constraints |
|---|---|---|
name |
Yes | Max 64 chars. Lowercase letters, numbers, and hyphens only. Must not start or end with a hyphen. Must match the parent directory name. |
description |
Yes | Max 1024 chars. Non-empty. Describes what the skill does and when to use it. |
license |
No | License name or reference to a bundled license file. |
compatibility |
No | Max 500 chars. Indicates environment requirements. |
metadata |
No | Arbitrary key-value mapping. |
allowed-tools |
No | Space-separated string of pre-approved tools. (Experimental) |
name field rules
- 1–64 characters
- Only lowercase unicode alphanumeric (
a-z,0-9) and hyphens (-) - Must not start or end with a hyphen
- Must not contain consecutive hyphens (
--) - Must match the parent directory name
description field rules
- 1–1024 characters
- Should describe both what the skill does and when to use it
- Should include specific keywords that help agents identify relevant tasks
compatibility field rules
- If present, it must be 1–500 characters
- Include it only for concrete environment requirements, such as a required product, system package, network access, or runtime
metadata field rules
- Must be a map of string keys to string values
- Use reasonably unique keys to avoid collisions with other clients or tools
Body Content
The Markdown body has no format restrictions beyond being helpful to the agent. Recommended sections:
- Step-by-step instructions — the procedure the agent should follow
- Examples of inputs and outputs — what data looks like going in and coming out
- Common edge cases — situations the agent might not handle correctly without guidance
- Gotchas — environment-specific facts that defy reasonable assumptions
Keep SKILL.md under 500 lines and 5000 tokens. Move detailed reference material to separate files in references/.
Progressive Disclosure
Agents load skills in three stages:
- Metadata (~100 tokens):
nameanddescriptionloaded at startup for all skills - Instructions (< 5000 tokens recommended): Full
SKILL.mdloaded when activated - Resources (as needed): Files in
scripts/,references/,assets/loaded on demand
Supporting Directories
scripts/
Executable code agents can run. Scripts should:
- Be self-contained or clearly document dependencies
- Include helpful error messages
- Handle edge cases gracefully
- Use relative paths from the skill root (e.g.,
scripts/extract.py) - Be explicit about intent: say "run this script" or "read this as reference" — never leave the agent to guess whether a bundled file is executable or illustrative
references/
Additional documentation loaded on demand. Keep individual files focused — agents load these when instructed, so smaller files save context.
- Every file under
references/must be at or under 60,000 characters. When a reference grows past the cap, split it into focused files (e.g.,references/<topic>-a.md,references/<topic>-b.md) and updateSKILL.mdso each new file is reachable and the load-on-demand instructions name the right file.
assets/
Static resources: templates, images, data files, schemas.
evals/
Portable output-quality cases for the skill. New skills in this repository must include evals/evals.json with at least five cases; trigger-only checks belong in the harness-specific test set instead of this file.
File References
Use relative paths from the skill root when referencing other files:
See [the specification](references/specification.md) for details.
Run a bundled script:
scripts/<script-name>
Keep file references one level deep from SKILL.md. Avoid deeply nested reference chains.
Writing Effective Descriptions
The description field is the primary mechanism for automatic skill selection. Clients can also support explicit activation. Follow these principles:
- Use imperative phrasing. "Use this skill when..." rather than "This skill does..."
- Focus on user intent, not implementation. Describe what the user is trying to achieve.
- Err on the side of being pushy. Explicitly list contexts where the skill applies.
- Keep it concise. A few sentences to a short paragraph is right.
Best Practices
Start from real expertise
Feed domain-specific context into skill creation. Skills grounded in real project artifacts (runbooks, API specs, code review comments, actual failure cases) outperform ones synthesized from generic knowledge.
Spend context wisely
Focus on what the agent wouldn't know without the skill: project-specific conventions, domain-specific procedures, non-obvious edge cases. Don't explain general concepts the agent already knows.
Calibrate control
- Give freedom when multiple approaches are valid — describe why, not just what
- Be prescriptive when operations are fragile or a specific sequence must be followed
- Match prescriptiveness to fragility: for a step that must be exactly right every time, prefer a deterministic script in
scripts/over instructions the agent improvises on each run. A test only catches what you already thought to check; a script takes the guess out of the loop entirely - Provide defaults, not menus — pick one approach, mention alternatives briefly
- Favor procedures over declarations — teach how to approach a class of problems, not what to produce for one instance
Design coherent units
Scope skills like functions: one coherent unit of work that composes well with other skills. Too narrow → multiple skills needed for one task. Too broad → hard to activate precisely.
Use gotchas sections
The highest-value content is often environment-specific corrections — things the agent will get wrong unless told otherwise. When an agent makes a mistake, add the correction to the gotchas section.
Provide output templates
When output needs a specific format, provide a template inline or in assets/. Agents pattern-match well against concrete structures.
Adopting Third-Party Skills
Skills are executable capability: when a skill activates, its instructions and scripts run with the agent's permissions — shell access, file system, and credentials. Before running a skill you did not author (from a registry, a colleague, or an LLM generation), read vetting third-party skills and treat it like any other dependency: inspect provenance, read the body and every script, and check what the skill reaches out to.
Validation
Use the skills-ref reference library to validate skills:
skills-ref validate ./my-skill
This checks that SKILL.md frontmatter is valid and follows all naming conventions.
For this repository, also run the bundled whole-repository checker:
ruby scripts/validate-skills.rb
It checks canonical top-level and bundle skills for frontmatter, supported fields, line limits, local links, and required README sections. Vendored profile skills under agent-council/profiles/skills/ are intentionally excluded because they follow the source repository's conventions.
If skills-ref is unavailable, do not claim a successful validator run. Perform and report the equivalent structural checks manually, or install and run the reference validator when the task permits it.
Files (agent-skills)
-
evals
-
evals.json 10.1 KB
{ "schema_version": 1, "skill_name": "agent-skills", "evals": [ { "id": "skill-creation-structure", "prompt": "I want to create a new skill called 'pdf-tools' that teaches an agent how to merge, split, and extract text from PDFs. What is the required directory structure and what must the SKILL.md contain to be format-compliant?", "expected_output": "A format-compliant skill scaffold: a directory named pdf-tools containing SKILL.md with YAML frontmatter and a markdown body, a human-facing README.md, and for a new skill an evals/evals.json with at least five output-quality cases. The response specifies the frontmatter contract: name must match the directory name in lowercase hyphenated form, description must be 1-1024 characters, start with an imperative verb, and define both positive triggers and a negative boundary (when not to use this skill), and the body must stay under 500 lines with supporting material in references/ loaded on demand. It also explains why progressive disclosure matters: core instructions in SKILL.md, detail in references/scripts, so activation cost stays low.", "assertions": [ "The scaffold names the required structure: SKILL.md, README.md, and evals/evals.json for new skills", "Frontmatter rules are specified: name matches directory, imperative-verb description with negative boundary", "The 500-line body limit and progressive disclosure structure are stated", "File references are relative from the skill root", "The rationale for progressive disclosure (low activation cost) is explained" ] }, { "id": "skill-review-compliance", "prompt": "I need to review a skill in our repository before merging it. The skill has a SKILL.md, a README.md, and a scripts folder, but I am not sure it is format-compliant. What checklist should I run, and what are the most common compliance failures?", "expected_output": "A review checklist covering the format requirements: name matches the directory, description is 1-1024 characters starting with an imperative verb and defining a negative boundary, body under 500 lines, valid frontmatter with only allowed fields, README.md present with the required human-facing sections, file references resolve to real relative paths, and for new skills an eval manifest with at least five cases using the canonical assertions field. The response lists the most common failures: descriptions that do not start with an imperative verb, missing negative boundaries, frontmatter fields beyond the allowed set, stale or broken relative links, references to skills that do not exist in the catalog, and evals that use a non-canonical alias instead of the canonical assertions field. It prescribes running the repository's validation scripts to catch what the checklist misses.", "assertions": [ "The checklist covers frontmatter, description rules, body limits, README sections, references, and evals", "Names a description that does not start with an imperative verb as a common failure", "Names a description without a negative trigger boundary as a common failure", "Names broken relative references and dead routing as common failures", "The canonical assertions field versus the non-canonical alias is called out", "The review verifies relative references resolve", "Running the repository validators is prescribed as the final check" ] }, { "id": "progressive-disclosure-fix", "prompt": "A skill I wrote has a 900-line SKILL.md because I put everything in the main file. Agents loading it consume enormous context even when they only need one section. How do I restructure it for progressive disclosure?", "expected_output": "A restructuring plan that moves the SKILL.md to a lean core: keep the description-driven triggers, the operating instructions, and the routing that tells the agent when to load which reference, then move the detailed material into references/ files organized by concern, templates/ for fillable documents, and scripts/ for executable tooling, each referenced from SKILL.md with the load condition. The response explains the progressive disclosure contract: metadata at startup, instructions on activation, resources on demand, and that the SKILL.md should state when to read each supporting file rather than embedding it. It also covers the practical audit: after the split, the body must be under 500 lines, every reference must resolve, and nothing the core workflow depends on may be left only in a reference the agent is not told to load.", "assertions": [ "The plan moves detail into references, templates, and scripts while keeping SKILL.md as the lean core", "The response applies the load-on-demand contract: instructions on activation, resources on demand", "SKILL.md states when to load each supporting file rather than embedding it", "The post-split body is verified under 500 lines with all references resolving", "Core workflow dependencies are not stranded in unlinked references" ] }, { "id": "evals-manifest-authoring", "prompt": "I am adding an eval manifest to an existing skill that has no evals. What must the manifest contain to pass the repository's v1 validation, and what makes the cases actually useful for judging output quality?", "expected_output": "An eval manifest written to the v1 contract: schema_version 1, skill_name matching the skill directory, and an evals array of at least five cases, each with a stable lowercase-hyphenated id, a realistic prompt, an expected_output describing the correct behavior, and a list of assertions that are observable claims about the output, using the canonical assertions field and never a non-canonical alias. The response explains what makes cases useful: prompts that reflect real activation scenarios for the skill, assertions that are specific enough to fail meaningfully rather than vague quality platitudes, stable IDs that durable evidence references can rely on, and coverage across the skill's main behaviors. It also notes the trigger-only probes belong in a separate harness-specific test set, not in evals.json.", "assertions": [ "The manifest uses schema_version 1, matching skill_name, and at least five cases", "Each case has a stable lowercase-hyphenated id", "Each case has a realistic prompt", "Each case has an expected_output describing correct behavior", "Each case has observable assertions about the output", "The assertions field is canonical and non-canonical aliases are never used", "Cases reflect real activation scenarios and are specific enough to fail meaningfully", "Trigger-only probes are excluded from evals.json as harness-specific" ] }, { "id": "client-discovery-loading", "prompt": "I am building an agent client that needs to discover and load skills from a directory of Agent Skills-format skills. How should discovery and loading work so the client respects the format's cost model?", "expected_output": "A discovery and loading design implementing the three-stage model: at startup the client reads only each skill's name and description from frontmatter as metadata, when a user request matches a description the client loads the full SKILL.md, and supporting files under references/, templates/, and scripts/ are loaded on demand when the skill's instructions say to. The response explains why loading everything at startup defeats the format's purpose (context cost grows with catalog size) and how routing works: the description is the trigger surface, so description quality directly determines whether the right skill activates. It covers the failure modes: skills whose descriptions do not overlap the user's phrasing fail to trigger, and a client that reads beyond frontmatter during discovery pays the cost the format was designed to avoid.", "assertions": [ "Discovery reads only name and description frontmatter at startup", "Full SKILL.md is loaded only on trigger match and supporting files on demand", "The response explains the context-cost rationale for staged loading", "Description quality is tied to routing correctness", "The failure modes of eager loading and weak trigger overlap are covered" ] }, { "id": "third-party-vetting", "prompt": "I found a skill on a public registry that looks useful: it fetches weather data and sends me a notification. Before I install it, how do I tell whether it is safe to run?", "expected_output": "A dependency-style vetting procedure performed before first run: check provenance (who published it, when, license, maintainer history), read the full SKILL.md body and compare it to the description for hidden or deceptive instructions (obfuscation, ignore-previous-instructions patterns, requests to reveal credentials), read every script for network calls to unknown hosts, filesystem access outside the skill directory (dotfiles, SSH keys, credential stores), hardcoded or echoed secrets, and dynamic behavior like curl-pipe-bash or remote imports, then run the first execution sandboxed with no real credentials and read-only access. The response explains that an open format standard says nothing about safety and that skills run with the agent's own permissions (shell, filesystem, credentials), citing that audits of public skill registries have found critical issues in a substantial fraction of skills.", "assertions": [ "Checks publisher provenance before first run", "Reads the complete SKILL.md body before first run", "Reviews every bundled script for side effects before first run", "Concrete red flags are named: unknown network hosts, out-of-directory filesystem access, credential exposure, obfuscation, curl-pipe-bash", "First run is sandboxed with no real credentials and read-only access", "The response states that format compliance or registry publication is not a safety guarantee", "Skills running with the agent's permissions (shell, filesystem, credentials) is called out" ] } ] }
-
-
references
-
best-practices.md 15.7 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Best practices for skill creators > How to write skills that are well-scoped and calibrated to the task. ## Start from real expertise A common pitfall in skill creation is asking an LLM to generate a skill without providing domain-specific context — relying solely on the LLM's general training knowledge. The result is vague, generic procedures ("handle errors appropriately," "follow best practices for authentication") rather than the specific API patterns, edge cases, and project conventions that make a skill valuable. Effective skills are grounded in real expertise. The key is feeding domain-specific context into the creation process. ### Extract from a hands-on task Complete a real task in conversation with an agent, providing context, corrections, and preferences along the way. Then extract the reusable pattern into a skill. Pay attention to: * **Steps that worked** — the sequence of actions that led to success * **Corrections you made** — places where you steered the agent's approach (e.g., "use library X instead of Y," "check for edge case Z") * **Input/output formats** — what the data looked like going in and coming out * **Context you provided** — project-specific facts, conventions, or constraints the agent didn't already know ### Synthesize from existing project artifacts When you have a body of existing knowledge, you can feed it into an LLM and ask it to synthesize a skill. A data-pipeline skill synthesized from your team's actual incident reports and runbooks will outperform one synthesized from a generic "data engineering best practices" article, because it captures *your* schemas, failure modes, and recovery procedures. The key is project-specific material, not generic references. Good source material includes: * Internal documentation, runbooks, and style guides * API specifications, schemas, and configuration files * Code review comments and issue trackers (captures recurring concerns and reviewer expectations) * Version control history, especially patches and fixes (reveals patterns through what actually changed) * Real-world failure cases and their resolutions ## Refine with real execution The first draft of a skill usually needs refinement. Run the skill against real tasks, then feed the results — all of them, not just failures — back into the creation process. Ask: what triggered false positives? What was missed? What could be cut? Even a single pass of execute-then-revise noticeably improves quality, and complex domains often benefit from several. <Tip> Read agent execution traces, not just final outputs. If the agent wastes time on unproductive steps, common causes include instructions that are too vague (the agent tries several approaches before finding one that works), instructions that don't apply to the current task (the agent follows them anyway), or too many options presented without a clear default. </Tip> For a more structured approach to iteration, including test cases, assertions, and grading, see [Evaluating skill output quality](/skill-creation/evaluating-skills). ## Spending context wisely Once a skill activates, its full `SKILL.md` body loads into the agent's context window alongside conversation history, system context, and other active skills. Every token in your skill competes for the agent's attention with everything else in that window. ### Add what the agent lacks, omit what it knows Focus on what the agent *wouldn't* know without your skill: project-specific conventions, domain-specific procedures, non-obvious edge cases, and the particular tools or APIs to use. You don't need to explain what a PDF is, how HTTP works, or what a database migration does. ````markdown theme={null} <!-- Too verbose — the agent already knows what PDFs are --> ## Extract PDF text PDF (Portable Document Format) files are a common file format that contains text, images, and other content. To extract text from a PDF, you'll need to use a library. pdfplumber is recommended because it handles most cases well. <!-- Better — jumps straight to what the agent wouldn't know on its own --> ## Extract PDF text Use pdfplumber for text extraction. For scanned documents, fall back to pdf2image with pytesseract. ```python import pdfplumber with pdfplumber.open("file.pdf") as pdf: text = pdf.pages[0].extract_text() ``` ```` Ask yourself about each piece of content: "Would the agent get this wrong without this instruction?" If the answer is no, cut it. If you're unsure, test it. And if the agent already handles the entire task well without the skill, the skill may not be adding value. See [Evaluating skill output quality](/skill-creation/evaluating-skills) for how to test this systematically. ### Design coherent units Deciding what a skill should cover is like deciding what a function should do: you want it to encapsulate a coherent unit of work that composes well with other skills. Skills scoped too narrowly force multiple skills to load for a single task, risking overhead and conflicting instructions. Skills scoped too broadly become hard to activate precisely. A skill for querying a database and formatting the results may be one coherent unit, while a skill that also covers database administration is probably trying to do too much. ### Aim for moderate detail Overly comprehensive skills can hurt more than they help — the agent struggles to extract what's relevant and may pursue unproductive paths triggered by instructions that don't apply to the current task. Concise, stepwise guidance with a working example tends to outperform exhaustive documentation. When you find yourself covering every edge case, consider whether most are better handled by the agent's own judgment. ### Structure large skills with progressive disclosure The [specification](/specification#progressive-disclosure) recommends keeping `SKILL.md` under 500 lines and 5,000 tokens — just the core instructions the agent needs on every run. When a skill legitimately needs more content, move detailed reference material to separate files in `references/` or similar directories. The key is telling the agent *when* to load each file. "Read `references/api-errors.md` if the API returns a non-200 status code" is more useful than a generic "see references/ for details." This lets the agent load context on demand rather than up front, which is how [progressive disclosure](/specification#progressive-disclosure) is designed to work. The repository validator also caps individual reference files at **60,000 characters**. When a reference grows past the cap, split it into focused parts — for example, break an oversized `references/<topic>.md` into `references/<topic>-a.md` and `references/<topic>-b.md`, keeping file references one level deep, then update the skill's index/routing table and any load-on-demand instructions so each new file stays reachable. The cap is enforced by `scripts/validate-references.rb`, which runs via `scripts/validate-skills.rb` in CI. ## Calibrating control Not every part of a skill needs the same level of prescriptiveness. Match the specificity of your instructions to the fragility of the task. ### Match specificity to fragility **Give the agent freedom** when multiple approaches are valid and the task tolerates variation. For flexible instructions, explaining *why* can be more effective than rigid directives — an agent that understands the purpose behind an instruction makes better context-dependent decisions. A code review skill can describe what to look for without prescribing exact steps: ```markdown theme={null} ## Code review process 1. Check all database queries for SQL injection (use parameterized queries) 2. Verify authentication checks on every endpoint 3. Look for race conditions in concurrent code paths 4. Confirm error messages don't leak internal details ``` **Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed: ````markdown theme={null} ## Database migration Run exactly this sequence: ```bash python scripts/migrate.py --verify --backup ``` Do not modify the command or add additional flags. ```` Most skills have a mix. Calibrate each part independently. ### Provide defaults, not menus When multiple tools or approaches could work, pick a default and mention alternatives briefly rather than presenting them as equal options. ````markdown theme={null} <!-- Too many options --> You can use pypdf, pdfplumber, PyMuPDF, or pdf2image... <!-- Clear default with escape hatch --> Use pdfplumber for text extraction: ```python import pdfplumber ``` For scanned PDFs requiring OCR, use pdf2image with pytesseract instead. ```` ### Favor procedures over declarations A skill should teach the agent *how to approach* a class of problems, not *what to produce* for a specific instance. Compare: ```markdown theme={null} <!-- Specific answer — only useful for this exact task --> Join the `orders` table to `customers` on `customer_id`, filter where `region = 'EMEA'`, and sum the `amount` column. <!-- Reusable method — works for any analytical query --> 1. Read the schema from `references/schema.yaml` to find relevant tables 2. Join tables using the `_id` foreign key convention 3. Apply any filters from the user's request as WHERE clauses 4. Aggregate numeric columns as needed and format as a markdown table ``` This doesn't mean skills can't include specific details — output format templates (see [Templates for output format](#templates-for-output-format)), constraints like "never output PII," and tool-specific instructions are all valuable. The point is that the *approach* should generalize even when individual details are specific. ## Patterns for effective instructions These are reusable techniques for structuring skill content. Not every skill needs all of them — use the ones that fit your task. ### Gotchas sections The highest-value content in many skills is a list of gotchas — environment-specific facts that defy reasonable assumptions. These aren't general advice ("handle errors appropriately") but concrete corrections to mistakes the agent will make without being told otherwise: ```markdown theme={null} ## Gotchas - The `users` table uses soft deletes. Queries must include `WHERE deleted_at IS NULL` or results will include deactivated accounts. - The user ID is `user_id` in the database, `uid` in the auth service, and `accountId` in the billing API. All three refer to the same value. - The `/health` endpoint returns 200 as long as the web server is running, even if the database connection is down. Use `/ready` to check full service health. ``` Keep gotchas in `SKILL.md` where the agent reads them before encountering the situation. A separate reference file works if you tell the agent when to load it, but for non-obvious issues, the agent may not recognize the trigger. <Tip> When an agent makes a mistake you have to correct, add the correction to the gotchas section. This is one of the most direct ways to improve a skill iteratively (see [Refine with real execution](#refine-with-real-execution)). </Tip> ### Use-when sections Not every instruction in a skill applies to every task it handles. Mark conditional sections with a `**Applicability:**` line at the start of the section so the agent can skip what does not apply. Keep applicability conditions short and binary. ```markdown ## Database migration **Applicability:** Only when the task involves schema changes or data migration. ``` ### Templates for output format When you need the agent to produce output in a specific format, provide a template. This is more reliable than describing the format in prose, because agents pattern-match well against concrete structures. Short templates can live inline in `SKILL.md`; for longer templates, or templates only needed in certain cases, store them in `assets/` and reference them from `SKILL.md` so they only load when needed. ````markdown theme={null} ## Report structure Use this template, adapting sections as needed for the specific analysis: ```markdown # [Analysis Title] ## Executive summary [One-paragraph overview of key findings] ## Key findings - Finding 1 with supporting data - Finding 2 with supporting data ## Recommendations 1. Specific actionable recommendation 2. Specific actionable recommendation ``` ```` ### Checklists for multi-step workflows An explicit checklist helps the agent track progress and avoid skipping steps, especially when steps have dependencies or validation gates. ```markdown theme={null} ## Form processing workflow Progress: - [ ] Step 1: Analyze the form (run `scripts/analyze_form.py`) - [ ] Step 2: Create field mapping (edit `fields.json`) - [ ] Step 3: Validate mapping (run `scripts/validate_fields.py`) - [ ] Step 4: Fill the form (run `scripts/fill_form.py`) - [ ] Step 5: Verify output (run `scripts/verify_output.py`) ``` ### Exit conditions Define when the skill is complete so the agent stops following instructions once the work is done. Use observable criteria or a bounded escalation. ```markdown ## Exit criteria This skill is complete when: - The requested artifact exists - Validation passes - The result has been presented to the user ``` ### Validation loops Instruct the agent to validate its own work before moving on. The pattern is: do the work, run a validator (a script, a reference checklist, or a self-check), fix any issues, and repeat until validation passes. ```markdown theme={null} ## Editing workflow 1. Make your edits 2. Run validation: `python scripts/validate.py output/` 3. If validation fails: - Review the error message - Fix the issues - Run validation again 4. Only proceed when validation passes ``` A reference document can also serve as the "validator" — instruct the agent to check its work against the reference before finalizing. ### Plan-validate-execute For batch or destructive operations, have the agent create an intermediate plan in a structured format, validate it against a source of truth, and only then execute. ```markdown theme={null} ## PDF form filling 1. Extract form fields: `python scripts/analyze_form.py input.pdf` → `form_fields.json` (lists every field name, type, and whether it's required) 2. Create `field_values.json` mapping each field name to its intended value 3. Validate: `python scripts/validate_fields.py form_fields.json field_values.json` (checks that every field name exists in the form, types are compatible, and required fields aren't missing) 4. If validation fails, revise `field_values.json` and re-validate 5. Fill the form: `python scripts/fill_form.py input.pdf field_values.json output.pdf` ``` The key ingredient is step 3: a validation script that checks the plan (`field_values.json`) against the source of truth (`form_fields.json`). Errors like "Field 'signature\_date' not found — available fields: customer\_name, order\_total, signature\_date\_signed" give the agent enough information to self-correct. ### Bundling reusable scripts When [iterating on a skill](/skill-creation/evaluating-skills), compare the agent's execution traces across test cases. If you notice the agent independently reinventing the same logic each run — building charts, parsing a specific format, validating output — that's a signal to write a tested script once and bundle it in `scripts/`. For more on designing and bundling scripts, see [Using scripts in skills](/skill-creation/using-scripts). ## Next steps Once you have a working skill, two guides can help you refine it further: * **[Evaluating skill output quality](/skill-creation/evaluating-skills)** — Set up test cases, grade results, and iterate systematically. * **[Optimizing skill descriptions](/skill-creation/optimizing-descriptions)** — Test and improve your skill's `description` field so it triggers on the right prompts. -
client-implementation.md 20.4 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # How to add skills support to your agent > A guide for adding Agent Skills support to an AI agent or development tool. This guide walks through how to add Agent Skills support to an AI agent or development tool. It covers the full lifecycle: discovering skills, telling the model about them, loading their content into context, and keeping that content effective over time. The core integration is the same regardless of your agent's architecture. The implementation details vary based on two factors: * **Where do skills live?** A locally-running agent can scan the user's filesystem for skill directories. A cloud-hosted or sandboxed agent will need an alternative discovery mechanism — an API, a remote registry, or bundled assets. * **How does the model access skill content?** If the model has file-reading capabilities, it can read `SKILL.md` files directly. Otherwise, you'll provide a dedicated tool or inject skill content into the prompt programmatically. The guide notes where these differences matter. You don't need to support every scenario — follow the path that fits your agent. **Prerequisites**: Familiarity with the [Agent Skills specification](/specification), which defines the `SKILL.md` file format, frontmatter fields, and directory conventions. ## The core principle: progressive disclosure Every skills-compatible agent follows the same three-tier loading strategy: | Tier | What's loaded | When | Token cost | | --------------- | --------------------------- | ------------------------------------ | --------------------------- | | 1. Catalog | Name + description | Session start | \~50-100 tokens per skill | | 2. Instructions | Full `SKILL.md` body | When the skill is activated | \<5000 tokens (recommended) | | 3. Resources | Scripts, references, assets | When the instructions reference them | Varies | The model sees the catalog from the start, so it knows what skills are available. When it decides a skill is relevant, it loads the full instructions. If those instructions reference supporting files, the model loads them individually as needed. This keeps the base context small while giving the model access to specialized knowledge on demand. An agent with 20 installed skills doesn't pay the token cost of 20 full instruction sets upfront — only the ones actually used in a given conversation. ## Step 1: Discover skills At session startup, find all available skills and load their metadata. ### Where to scan Which directories you scan depends on your agent's environment. Most locally-running agents scan at least two scopes: * **Project-level** (relative to the working directory): Skills specific to a project or repository. * **User-level** (relative to the home directory): Skills available across all projects for a given user. Other scopes are possible too — for example, organization-wide skills deployed by an admin, or skills bundled with the agent itself. The right set of scopes depends on your agent's deployment model. Within each scope, consider scanning both a **client-specific directory** and the **`.agents/skills/` convention**: | Scope | Path | Purpose | | ------- | ---------------------------------- | ----------------------------- | | Project | `<project>/.<your-client>/skills/` | Your client's native location | | Project | `<project>/.agents/skills/` | Cross-client interoperability | | User | `~/.<your-client>/skills/` | Your client's native location | | User | `~/.agents/skills/` | Cross-client interoperability | The `.agents/skills/` paths have emerged as a widely-adopted convention for cross-client skill sharing. While the Agent Skills specification does not mandate where skill directories live (it only defines what goes inside them), scanning `.agents/skills/` means skills installed by other compliant clients are automatically visible to yours, and vice versa. <Note> Some implementations also scan `.claude/skills/` (both project-level and user-level) for pragmatic compatibility, since many existing skills are installed there. Other additional locations include ancestor directories up to the git root (useful for monorepos), [XDG](https://specifications.freedesktop.org/basedir-spec/latest/) config directories, and user-configured paths. </Note> ### What to scan for Within each skills directory, look for **subdirectories containing a file named exactly `SKILL.md`**: ``` ~/.agents/skills/ ├── pdf-processing/ │ ├── SKILL.md ← discovered │ └── scripts/ │ └── extract.py ├── data-analysis/ │ └── SKILL.md ← discovered └── README.md ← ignored (not a skill directory) ``` Practical scanning rules: * Skip directories that won't contain skills, such as `.git/` and `node_modules/` * Optionally respect `.gitignore` to avoid scanning build artifacts * Set reasonable bounds (e.g., max depth of 4-6 levels, max 2000 directories) to prevent runaway scanning in large directory trees ### Handling name collisions When two skills share the same `name`, apply a deterministic precedence rule. The universal convention across existing implementations: **project-level skills override user-level skills.** Within the same scope (e.g., two skills named `code-review` found under both `<project>/.agents/skills/` and `<project>/.<your-client>/skills/`), either first-found or last-found is acceptable — pick one and be consistent. Log a warning when a collision occurs so the user knows a skill was shadowed. ### Trust considerations Project-level skills come from the repository being worked on, which may be untrusted (e.g., a freshly cloned open-source project). Consider gating project-level skill loading on a trust check — only load them if the user has marked the project folder as trusted. This prevents untrusted repositories from silently injecting instructions into the agent's context. ### Cloud-hosted and sandboxed agents If your agent runs in a container or on a remote server, it won't have access to the user's local filesystem. Discovery needs to work differently depending on the skill scope: * **Project-level skills** are often the easiest case. If the agent operates on a cloned repository (even inside a sandbox), project-level skills travel with the code and can be scanned from the repo's directory tree. * **User-level and organization-level skills** don't exist in the sandbox. You'll need to provision them from an external source — for example, cloning a configuration repository, accepting skill URLs or packages through your agent's settings, or letting users upload skill directories through a web UI. * **Built-in skills** can be packaged as static assets within the agent's deployment artifact, making them available in every session without external fetching. Once skills are available to the agent, the rest of the lifecycle — parsing, disclosure, activation — works the same. ## Step 2: Parse `SKILL.md` files For each discovered `SKILL.md`, extract the metadata and body content. ### Frontmatter extraction A `SKILL.md` file has two parts: YAML frontmatter between `---` delimiters, and a markdown body after the closing delimiter. To parse: 1. Find the opening `---` at the start of the file and the closing `---` after it. 2. Parse the YAML block between them. Extract `name` and `description` (required), plus any optional fields. 3. Everything after the closing `---`, trimmed, is the skill's body content. See the [specification](/specification) for the full set of frontmatter fields and their constraints. ### Handling malformed YAML Skill files authored for other clients may contain technically invalid YAML that their parsers happen to accept. The most common issue is unquoted values containing colons: ```yaml theme={null} # Technically invalid YAML — the colon breaks parsing description: Use this skill when: the user asks about PDFs ``` Consider a fallback that wraps such values in quotes or converts them to YAML block scalars before retrying. This improves cross-client compatibility at minimal cost. ### Lenient validation Warn on issues but still load the skill when possible: * Name doesn't match the parent directory name → warn, load anyway * Name exceeds 64 characters → warn, load anyway * Description is missing or empty → skip the skill (a description is essential for disclosure), log the error * YAML is completely unparseable → skip the skill, log the error Record diagnostics so they can be surfaced to the user (in a debug command, log file, or UI), but don't block skill loading on cosmetic issues. <Note> The [specification](/specification) defines strict constraints on the `name` field (matching the parent directory, character set, max length). The lenient approach above deliberately relaxes these to improve compatibility with skills authored for other clients. </Note> ### What to store At minimum, each skill record needs three fields: | Field | Description | | ------------- | ------------------------------------ | | `name` | From frontmatter | | `description` | From frontmatter | | `location` | Absolute path to the `SKILL.md` file | Store these in an in-memory map keyed by `name` for fast lookup during activation. You can also store the **body** (the markdown content after the frontmatter) at discovery time, or read it from `location` at activation time. Storing it makes activation faster; reading it at activation time uses less memory in aggregate and picks up changes to skill files between activations. The skill's **base directory** (the parent directory of `location`) is needed later to resolve relative paths and enumerate bundled resources — derive it from `location` when needed. ## Step 3: Disclose available skills to the model Tell the model what skills exist without loading their full content. This is [tier 1 of progressive disclosure](#the-core-principle-progressive-disclosure). ### Building the skill catalog For each discovered skill, include `name`, `description`, and optionally `location` (the path to the `SKILL.md` file) in whatever structured format suits your stack — XML, JSON, or a bulleted list all work: ```xml theme={null} <available_skills> <skill> <name>pdf-processing</name> <description>Extract PDF text, fill forms, merge files. Use when handling PDFs.</description> <location>/home/user/.agents/skills/pdf-processing/SKILL.md</location> </skill> <skill> <name>data-analysis</name> <description>Analyze datasets, generate charts, and create summary reports.</description> <location>/home/user/project/.agents/skills/data-analysis/SKILL.md</location> </skill> </available_skills> ``` The `location` field serves two purposes: it enables file-read activation (see [Step 4](#step-4-activate-skills)), and it gives the model a base path for resolving relative references in the skill body (like `scripts/evaluate.py`). If your dedicated activation tool provides the skill directory path in its result (see [Structured wrapping](#structured-wrapping) in Step 4), you can omit `location` from the catalog. Otherwise, include it. Each skill adds roughly 50-100 tokens to the catalog. Even with dozens of skills installed, the catalog remains compact. ### Where to place the catalog Two approaches are common: **System prompt section**: Add the catalog as a labeled section in the system prompt, preceded by brief instructions on how to use skills. This is the simplest approach and works with any model that has access to a file-reading tool. **Tool description**: Embed the catalog in the description of a dedicated skill-activation tool (see [Step 4](#step-4-activate-skills)). This keeps the system prompt clean and naturally couples discovery with activation. Both work. System prompt placement is simpler and more broadly compatible; tool description embedding is cleaner when you have a dedicated activation tool. ### Behavioral instructions Include a short instruction block alongside the catalog telling the model how and when to use skills. The wording depends on which activation mechanism you support (see [Step 4](#step-4-activate-skills)): **If the model activates skills by reading files:** ``` The following skills provide specialized instructions for specific tasks. When a task matches a skill's description, use your file-read tool to load the SKILL.md at the listed location before proceeding. When a skill references relative paths, resolve them against the skill's directory (the parent of SKILL.md) and use absolute paths in tool calls. ``` **If the model activates skills via a dedicated tool:** ``` The following skills provide specialized instructions for specific tasks. When a task matches a skill's description, call the activate_skill tool with the skill's name to load its full instructions. ``` Keep these instructions concise. The goal is to tell the model that skills exist and how to load them — the skill content itself provides the detailed instructions once loaded. ### Filtering Some skills should be excluded from the catalog. Common reasons: * The user has disabled the skill in settings * A permission system denies access to the skill * The skill has opted out of model-driven activation (e.g., via a `disable-model-invocation` flag) **Hide filtered skills entirely** from the catalog rather than listing them and blocking at activation time. This prevents the model from wasting turns attempting to load skills it can't use. ### When no skills are available If no skills are discovered, omit the catalog and behavioral instructions entirely. Don't show an empty `<available_skills/>` block or register a skill tool with no valid options — this would confuse the model. ## Step 4: Activate skills When the model or user selects a skill, deliver the full instructions into the conversation context. This is [tier 2 of progressive disclosure](#the-core-principle-progressive-disclosure). ### Model-driven activation Most implementations rely on the model's own judgment as the activation mechanism, rather than implementing harness-side trigger matching or keyword detection. The model reads the catalog (from [Step 3](#step-3-disclose-available-skills-to-the-model)), decides a skill is relevant to the current task, and loads it. Two implementation patterns: **File-read activation**: The model calls its standard file-read tool with the `SKILL.md` path from the catalog. No special infrastructure needed — the agent's existing file-reading capability is sufficient. The model receives the file content as a tool result. This is the simplest approach when the model has file access. **Dedicated tool activation**: Register a tool (e.g., `activate_skill`) that takes a skill name and returns the content. This is required when the model can't read files directly, and optional (but useful) even when it can. Advantages over raw file reads: * Control what content is returned — e.g., strip YAML frontmatter or preserve it (see [What the model receives](#what-the-model-receives) below) * Wrap content in structured tags for identification during context management * List bundled resources (e.g., `references/*`) alongside the instructions * Enforce permissions or prompt for user consent * Track activation for analytics <Tip> If you use a dedicated activation tool, constrain the `name` parameter to the set of valid skill names (e.g., as an enum in the tool schema). This prevents the model from hallucinating nonexistent skill names. If no skills are available, don't register the tool at all. </Tip> ### User-explicit activation Users should also be able to activate skills directly, without waiting for the model to decide. The most common pattern is a **slash command or mention syntax** (`/skill-name` or `$skill-name`) that the harness intercepts. The specific syntax is up to you — the key idea is that the harness handles the lookup and injection, so the model receives skill content without needing to take an activation action itself. An autocomplete widget (listing available skills as the user types) can also make this discoverable. ### What the model receives When a skill is activated, the model receives the skill's instructions. Two options for what exactly that content looks like: **Full file**: The model sees the entire `SKILL.md` including YAML frontmatter. This is the natural outcome with file-read activation, where the model reads the raw file. It's also a valid choice for dedicated tools. The frontmatter may contain fields useful at activation time — for example, [`compatibility`](/specification#compatibility-field) notes environment requirements that could inform how the model executes the skill's instructions. **Body only (frontmatter stripped)**: The harness parses and removes the YAML frontmatter, returning only the markdown instructions. Among existing implementations with dedicated activation tools, most take this approach — stripping the frontmatter after extracting `name` and `description` during discovery. Both approaches work in practice. ### Structured wrapping If you use a dedicated activation tool, consider wrapping skill content in identifying tags. For example: ```xml theme={null} <skill_content name="pdf-processing"> # PDF Processing ## When to use this skill Use this skill when the user needs to work with PDF files... [rest of SKILL.md body] Skill directory: /home/user/.agents/skills/pdf-processing Relative paths in this skill are relative to the skill directory. <skill_resources> <file>scripts/extract.py</file> <file>scripts/merge.py</file> <file>references/pdf-spec-summary.md</file> </skill_resources> </skill_content> ``` This has practical benefits: * The model can clearly distinguish skill instructions from other conversation content * The harness can identify skill content during context compaction ([Step 5](#step-5-manage-skill-context-over-time)) * Bundled resources are surfaced to the model without being eagerly loaded ### Listing bundled resources When a dedicated activation tool returns skill content, it can also enumerate supporting files (scripts, references, assets) in the skill directory — but it should **not eagerly read them**. The model loads specific files on demand using its file-read tools when the skill's instructions reference them. For large skill directories, consider capping the listing and noting that it may be incomplete. ### Permission allowlisting If your agent has a permission system that gates file access, **allowlist skill directories** so the model can read bundled resources without triggering user confirmation prompts. Without this, every reference to a bundled script or reference file results in a permission dialog, breaking the flow for skills that include resources beyond the `SKILL.md` itself. ## Step 5: Manage skill context over time Once skill instructions are in the conversation context, keep them effective for the duration of the session. ### Protect skill content from context compaction If your agent truncates or summarizes older messages when the context window fills up, **exempt skill content from pruning**. Skill instructions are durable behavioral guidance — losing them mid-conversation silently degrades the agent's performance without any visible error. The model continues operating but without the specialized instructions the skill provided. Common approaches: * Flag skill tool outputs as protected so the pruning algorithm skips them * Use the [structured tags](#structured-wrapping) from Step 4 to identify skill content and preserve it during compaction ### Deduplicate activations Consider tracking which skills have been activated in the current session. If the model (or user) attempts to load a skill that's already in context, you can skip the re-injection to avoid the same instructions appearing multiple times in the conversation. ### Subagent delegation (optional) This is an advanced pattern only supported by some clients. Instead of injecting skill instructions into the main conversation, the skill is run in a **separate subagent session**. The subagent receives the skill instructions, performs the task, and returns a summary of its work to the main conversation. This pattern is useful when a skill's workflow is complex enough to benefit from a dedicated, focused session. -
evaluating-skills.md 16.6 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Evaluating skill output quality > How to test whether your skill produces good outputs using eval-driven iteration. You wrote a skill, tried it on a prompt, and it seemed to work. But does it work reliably — across varied prompts, in edge cases, better than no skill at all? Running structured evaluations (evals) answers these questions and gives you a feedback loop for improving the skill systematically. ## Designing test cases A test case has three parts: * **Prompt**: a realistic user message — the kind of thing someone would actually type. * **Expected output**: a human-readable description of what success looks like. * **Input files** (optional): files the skill needs to work with. Store test cases in `evals/evals.json` inside your skill directory: ```json evals/evals.json theme={null} { "skill_name": "csv-analyzer", "evals": [ { "id": 1, "prompt": "I have a CSV of monthly sales data in data/sales_2025.csv. Can you find the top 3 months by revenue and make a bar chart?", "expected_output": "A bar chart image showing the top 3 months by revenue, with labeled axes and values.", "files": ["evals/files/sales_2025.csv"] }, { "id": 2, "prompt": "there's a csv in my downloads called customers.csv, some rows have missing emails — can you clean it up and tell me how many were missing?", "expected_output": "A cleaned CSV with missing emails handled, plus a count of how many were missing.", "files": ["evals/files/customers.csv"] } ] } ``` **Tips for writing good test prompts:** * **Start with 2-3 test cases.** Don't over-invest before you've seen your first round of results. You can expand the set later. * **Vary the prompts.** Use different phrasings, levels of detail, and formality. Some prompts should be casual ("hey can you clean up this csv"), others precise ("Parse the CSV at data/input.csv, drop rows where column B is null, and write the result to data/output.csv"). * **Cover edge cases.** Include at least one prompt that tests a boundary condition — a malformed input, an unusual request, or a case where the skill's instructions might be ambiguous. * **Use realistic context.** Real users mention file paths, column names, and personal context. Prompts like "process this data" are too vague to test anything useful. Don't worry about defining specific pass/fail checks yet — just the prompts and expected outputs. You'll add detailed checks (called assertions) after you see what the first run produces. ## Running evals The core pattern is to run each test case twice: once **with the skill** and once **without it** (or with a previous version). This gives you a baseline to compare against. ### Workspace structure Organize eval results in a workspace directory alongside your skill directory. Each pass through the full eval loop gets its own `iteration-N/` directory. Within that, each test case gets an eval directory with `with_skill/` and `without_skill/` subdirectories: ``` csv-analyzer/ ├── SKILL.md └── evals/ └── evals.json csv-analyzer-workspace/ └── iteration-1/ ├── eval-top-months-chart/ │ ├── with_skill/ │ │ ├── outputs/ # Files produced by the run │ │ ├── timing.json # Tokens and duration │ │ └── grading.json # Assertion results │ └── without_skill/ │ ├── outputs/ │ ├── timing.json │ └── grading.json ├── eval-clean-missing-emails/ │ ├── with_skill/ │ │ ├── outputs/ │ │ ├── timing.json │ │ └── grading.json │ └── without_skill/ │ ├── outputs/ │ ├── timing.json │ └── grading.json └── benchmark.json # Aggregated statistics ``` The main file you author by hand is `evals/evals.json`. The other JSON files (`grading.json`, `timing.json`, `benchmark.json`) are produced during the eval process — by the agent, by scripts, or by you. ### Spawning runs Each eval run should start with a clean context — no leftover state from previous runs or from the skill development process. This ensures the agent follows only what the `SKILL.md` tells it. In environments that support subagents (Claude Code, for example), this isolation comes naturally: each child task starts fresh. Without subagents, use a separate session for each run. For each run, provide: * The skill path (or no skill for the baseline) * The test prompt * Any input files * The output directory Here's an example of the instructions you'd give the agent for a single with-skill run: ``` Execute this task: - Skill path: /path/to/csv-analyzer - Task: I have a CSV of monthly sales data in data/sales_2025.csv. Can you find the top 3 months by revenue and make a bar chart? - Input files: evals/files/sales_2025.csv - Save outputs to: csv-analyzer-workspace/iteration-1/eval-top-months-chart/with_skill/outputs/ ``` For the baseline, use the same prompt but without the skill path, saving to `without_skill/outputs/`. When improving an existing skill, use the previous version as your baseline. Snapshot it before editing (`cp -r <skill-path> <workspace>/skill-snapshot/`), point the baseline run at the snapshot, and save to `old_skill/outputs/` instead of `without_skill/`. ### Capturing timing data Timing data lets you compare how much time and tokens the skill costs relative to the baseline — a skill that dramatically improves output quality but triples token usage is a different trade-off than one that's both better and cheaper. When each run completes, record the token count and duration: ```json timing.json theme={null} { "total_tokens": 84852, "duration_ms": 23332 } ``` <Tip> In Claude Code, when a subagent task finishes, the [task completion notification](https://platform.claude.com/docs/en/agent-sdk/typescript#sdk-task-notification-message) includes `total_tokens` and `duration_ms`. Save these values immediately — they aren't persisted anywhere else. </Tip> ## Writing assertions Assertions are verifiable statements about what the output should contain or achieve. Add them after you see your first round of outputs — you often don't know what "good" looks like until the skill has run. Good assertions: * `"The output file is valid JSON"` — programmatically verifiable. * `"The bar chart has labeled axes"` — specific and observable. * `"The report includes at least 3 recommendations"` — countable. Weak assertions: * `"The output is good"` — too vague to grade. * `"The output uses exactly the phrase 'Total Revenue: $X'"` — too brittle; correct output with different wording would fail. Not everything needs an assertion. Some qualities — writing style, visual design, whether the output "feels right" — are hard to decompose into pass/fail checks. These are better caught during [human review](#reviewing-results-with-a-human). Reserve assertions for things that can be checked objectively. Add assertions to each test case in `evals/evals.json`: ```json evals/evals.json highlight={9-14} theme={null} { "skill_name": "csv-analyzer", "evals": [ { "id": 1, "prompt": "I have a CSV of monthly sales data in data/sales_2025.csv. Can you find the top 3 months by revenue and make a bar chart?", "expected_output": "A bar chart image showing the top 3 months by revenue, with labeled axes and values.", "files": ["evals/files/sales_2025.csv"], "assertions": [ "The output includes a bar chart image file", "The chart shows exactly 3 months", "Both axes are labeled", "The chart title or caption mentions revenue" ] } ] } ``` ## Grading outputs Grading means evaluating each assertion against the actual outputs and recording **PASS** or **FAIL** with specific evidence. The evidence should quote or reference the output, not just state an opinion. The simplest approach is to give the outputs and assertions to an LLM and ask it to evaluate each one. For assertions that can be checked by code (valid JSON, correct row count, file exists with expected dimensions), use a verification script — scripts are more reliable than LLM judgment for mechanical checks and reusable across iterations. ```json grading.json theme={null} { "assertion_results": [ { "text": "The output includes a bar chart image file", "passed": true, "evidence": "Found chart.png (45KB) in outputs directory" }, { "text": "The chart shows exactly 3 months", "passed": true, "evidence": "Chart displays bars for March, July, and November" }, { "text": "Both axes are labeled", "passed": false, "evidence": "Y-axis is labeled 'Revenue ($)' but X-axis has no label" }, { "text": "The chart title or caption mentions revenue", "passed": true, "evidence": "Chart title reads 'Top 3 Months by Revenue'" } ], "summary": { "passed": 3, "failed": 1, "total": 4, "pass_rate": 0.75 } } ``` ### Grading principles * **Require concrete evidence for a PASS.** Don't give the benefit of the doubt. If an assertion says "includes a summary" and the output has a section titled "Summary" with one vague sentence, that's a FAIL — the label is there but the substance isn't. * **Review the assertions themselves, not just the results.** While grading, notice when assertions are too easy (always pass regardless of skill quality), too hard (always fail even when the output is good), or unverifiable (can't be checked from the output alone). Fix these for the next iteration. <Tip> For comparing two skill versions, try **blind comparison**: present both outputs to an LLM judge without revealing which came from which version. The judge scores holistic qualities — organization, formatting, usability, polish — on its own rubric, free from bias about which version "should" be better. This complements assertion grading: two outputs might both pass all assertions but differ significantly in overall quality. </Tip> ## Aggregating results Once every run in the iteration is graded, compute summary statistics per configuration and save them to `benchmark.json` alongside the eval directories (e.g., `csv-analyzer-workspace/iteration-1/benchmark.json`): ```json benchmark.json theme={null} { "run_summary": { "with_skill": { "pass_rate": { "mean": 0.83, "stddev": 0.06 }, "time_seconds": { "mean": 45.0, "stddev": 12.0 }, "tokens": { "mean": 3800, "stddev": 400 } }, "without_skill": { "pass_rate": { "mean": 0.33, "stddev": 0.10 }, "time_seconds": { "mean": 32.0, "stddev": 8.0 }, "tokens": { "mean": 2100, "stddev": 300 } }, "delta": { "pass_rate": 0.50, "time_seconds": 13.0, "tokens": 1700 } } } ``` The `delta` tells you what the skill costs (more time, more tokens) and what it buys (higher pass rate). A skill that adds 13 seconds but improves pass rate by 50 percentage points is probably worth it. A skill that doubles token usage for a 2-point improvement might not be. <Note> Standard deviation (`stddev`) is only meaningful with multiple runs per eval. In early iterations with just 2-3 test cases and single runs, focus on the raw pass counts and the delta — the statistical measures become useful as you expand the test set and run each eval multiple times. </Note> ## Analyzing patterns Aggregate statistics can hide important patterns. After computing the benchmarks: * **Remove or replace assertions that always pass in both configurations.** These don't tell you anything useful — the model handles them fine without the skill. They inflate the with-skill pass rate without reflecting actual skill value. * **Investigate assertions that always fail in both configurations.** Either the assertion is broken (asking for something the model can't do), the test case is too hard, or the assertion is checking for the wrong thing. Fix these before the next iteration. * **Study assertions that pass with the skill but fail without.** This is where the skill is clearly adding value. Understand *why* — which instructions or scripts made the difference? * **Tighten instructions when results are inconsistent across runs.** If the same eval passes sometimes and fails others (reflected as high `stddev` in the benchmark), the eval may be flaky (sensitive to model randomness), or the skill's instructions may be ambiguous enough that the model interprets them differently each time. Add examples or more specific guidance to reduce ambiguity. * **Check time and token outliers.** If one eval takes 3x longer than the others, read its execution transcript (the full log of what the model did during the run) to find the bottleneck. ## Reviewing results with a human Assertion grading and pattern analysis catch a lot, but they only check what you thought to write assertions for. A human reviewer brings a fresh perspective — catching issues you didn't anticipate, noticing when the output is technically correct but misses the point, or spotting problems that are hard to express as pass/fail checks. For each test case, review the actual outputs alongside the grades. Record specific feedback for each test case and save it in the workspace (e.g., as a `feedback.json` alongside the eval directories): ```json feedback.json theme={null} { "eval-top-months-chart": "The chart is missing axis labels and the months are in alphabetical order instead of chronological.", "eval-clean-missing-emails": "" } ``` "The chart is missing axis labels" is actionable; "looks bad" is not. Empty feedback means the output looked fine — that test case passed your review. During the [iteration step](#iterating-on-the-skill), focus your improvements on the test cases where you had specific complaints. ## Iterating on the skill After grading and reviewing, you have three sources of signal: * **Failed assertions** point to specific gaps — a missing step, an unclear instruction, or a case the skill doesn't handle. * **Human feedback** points to broader quality issues — the approach was wrong, the output was poorly structured, or the skill produced a technically correct but unhelpful result. * **Execution transcripts** reveal *why* things went wrong. If the agent ignored an instruction, the instruction may be ambiguous. If the agent spent time on unproductive steps, those instructions may need to be simplified or removed. The most effective way to turn these signals into skill improvements is to give all three — along with the current `SKILL.md` — to an LLM and ask it to propose changes. The LLM can synthesize patterns across failed assertions, reviewer complaints, and transcript behavior that would be tedious to connect manually. When prompting the LLM, include these guidelines: * **Generalize from feedback.** The skill will be used across many different prompts, not just the test cases. Fixes should address underlying issues broadly rather than adding narrow patches for specific examples. * **Keep the skill lean.** Fewer, better instructions often outperform exhaustive rules. If transcripts show wasted work (unnecessary validation, unneeded intermediate outputs), remove those instructions. If pass rates plateau despite adding more rules, the skill may be over-constrained — try removing instructions and see if results hold or improve. * **Explain the why.** Reasoning-based instructions ("Do X because Y tends to cause Z") work better than rigid directives ("ALWAYS do X, NEVER do Y"). Models follow instructions more reliably when they understand the purpose. * **Bundle repeated work.** If every test run independently wrote a similar helper script (a chart builder, a data parser), that's a signal to bundle the script into the skill's `scripts/` directory. See [Using scripts](/skill-creation/using-scripts) for how to do this. ### The loop 1. Give the eval signals and current `SKILL.md` to an LLM and ask it to propose improvements. 2. Review and apply the changes. 3. Rerun all test cases in a new `iteration-<N+1>/` directory. 4. Grade and aggregate the new results. 5. Review with a human. Repeat. Stop when you're satisfied with the results, feedback is consistently empty, or you're no longer seeing meaningful improvement between iterations. <Tip> The [`skill-creator`](https://github.com/anthropics/skills/tree/main/skills/skill-creator) Skill automates much of this workflow — running evals, grading assertions, aggregating benchmarks, and presenting results for human review. </Tip> -
home.md 23.3 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Agent Skills Overview > A standardized way to give AI agents new capabilities and expertise. export const LogoCarousel = ({clients}) => { const [shuffled, setShuffled] = useState(clients); useEffect(() => { const shuffle = items => { const copy = [...items]; for (let i = copy.length - 1; i > 0; i--) { const j = Math.floor(Math.random() * (i + 1)); [copy[i], copy[j]] = [copy[j], copy[i]]; } return copy; }; setShuffled(shuffle(clients)); }, []); const doubled = [...shuffled, ...shuffled]; const GAP_PX = 48; const PX_PER_SECOND = 40; const cycleWidth = shuffled.reduce((sum, client) => sum + 150 * (client.scale || 1) + GAP_PX, 0); const cycleDuration = cycleWidth / PX_PER_SECOND; const Logo = ({client}) => <a href={client.url} className="block no-underline border-none w-full h-full"> <img className="block dark:hidden object-contain w-full h-full !my-0" src={client.lightSrc} alt={client.name} noZoom /> <img className="hidden dark:block object-contain w-full h-full !my-0" src={client.darkSrc} alt={client.name} noZoom /> </a>; return <div className="logo-carousel"> <div className="logo-carousel-track" style={{ animation: `logo-scroll ${cycleDuration}s linear infinite` }}> {doubled.map((client, i) => <div key={`${client.name}-${i}`} style={{ width: 150 * (client.scale || 1), maxWidth: "100%" }}> <Logo client={client} /> </div>)} </div> </div>; }; export const clients = [{ name: "Junie", description: "Junie is an LLM-agnostic coding agent built for real-world development. It is built on top of the IntelliJ Platform, so it understands your project the same way your editor does.", url: "https://junie.jetbrains.com/", lightSrc: "/images/logos/junie/junie-logo-on-white.svg", darkSrc: "/images/logos/junie/junie-logo-on-dark.svg", instructionsUrl: "https://junie.jetbrains.com/docs/agent-skills.html" }, { name: "Gemini CLI", description: "Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal.", url: "https://geminicli.com", lightSrc: "/images/logos/gemini-cli/gemini-cli-logo_light.svg", darkSrc: "/images/logos/gemini-cli/gemini-cli-logo_dark.svg", instructionsUrl: "https://geminicli.com/docs/cli/skills/", sourceCodeUrl: "https://github.com/google-gemini/gemini-cli" }, { name: "Autohand Code CLI", description: "Autohand Code CLI is an autonomous LLM-powered coding agent that lives in your terminal. It uses the ReAct (Reason + Act) pattern to understand your codebase, plan changes, and execute them with your approval.", url: "https://autohand.ai/", lightSrc: "/images/logos/autohand/autohand-light.svg", darkSrc: "/images/logos/autohand/autohand-dark.svg", scale: 0.8, instructionsUrl: "https://autohand.ai/docs/working-with-autohand-code/agent-skills.html", sourceCodeUrl: "https://github.com/autohandai/code-cli" }, { name: "OpenCode", description: "OpenCode is an open source agent that helps you write code in your terminal, IDE, or desktop.", url: "https://opencode.ai/", lightSrc: "/images/logos/opencode/opencode-wordmark-light.svg", darkSrc: "/images/logos/opencode/opencode-wordmark-dark.svg", instructionsUrl: "https://opencode.ai/docs/skills/", sourceCodeUrl: "https://github.com/sst/opencode" }, { name: "OpenHands", description: "OpenHands is the open platform for cloud coding agents. Scale from one to thousands of agents — open source, model-agnostic, and enterprise-ready.", url: "https://openhands.dev/", lightSrc: "/images/logos/openhands/openhands-logo-light.svg", darkSrc: "/images/logos/openhands/openhands-logo-dark.svg", instructionsUrl: "https://docs.openhands.dev/overview/skills", sourceCodeUrl: "https://github.com/OpenHands/OpenHands" }, { name: "Mux", description: "Mux makes it easy to run parallel coding agents, each with its own isolated workspace, right from your browser or desktop. Mux is open source and LLM provider-agnostic.", url: "https://mux.coder.com/", lightSrc: "/images/logos/mux/mux-editor-light.svg", darkSrc: "/images/logos/mux/mux-editor-dark.svg", scale: 0.8, instructionsUrl: "https://mux.coder.com/agent-skills", sourceCodeUrl: "https://github.com/coder/mux" }, { name: "Cursor", description: "Cursor is an AI editor and coding agent. Use it to understand your codebase, plan and build features, fix bugs, review changes, and work with the tools you already use.", url: "https://cursor.com/", lightSrc: "/images/logos/cursor/LOCKUP_HORIZONTAL_2D_LIGHT.svg", darkSrc: "/images/logos/cursor/LOCKUP_HORIZONTAL_2D_DARK.svg", instructionsUrl: "https://cursor.com/docs/context/skills" }, { name: "Amp", description: "Amp is the frontier coding agent that lets you wield the full power of leading models.", url: "https://ampcode.com/", lightSrc: "/images/logos/amp/amp-logo-light.svg", darkSrc: "/images/logos/amp/amp-logo-dark.svg", scale: 0.8, instructionsUrl: "https://ampcode.com/manual#agent-skills" }, { name: "Letta", description: "Letta is the platform for building stateful agents: AI with advanced memory that can learn and self-improve over time.", url: "https://www.letta.com/", lightSrc: "/images/logos/letta/Letta-logo-RGB_OffBlackonTransparent.svg", darkSrc: "/images/logos/letta/Letta-logo-RGB_GreyonTransparent.svg", instructionsUrl: "https://docs.letta.com/letta-code/skills/", sourceCodeUrl: "https://github.com/letta-ai/letta" }, { name: "Firebender", description: "Firebender is the first Android-native coding agent that writes features, tests them in the emulator, and fixes issues automatically.", url: "https://firebender.com/", lightSrc: "/images/logos/firebender/firebender-wordmark-light.svg", darkSrc: "/images/logos/firebender/firebender-wordmark-dark.svg", instructionsUrl: "https://docs.firebender.com/multi-agent/skills" }, { name: "Goose", description: "Goose is an open source, extensible AI agent that goes beyond code suggestions — install, execute, edit, and test with any LLM.", url: "https://block.github.io/goose/", lightSrc: "/images/logos/goose/goose-logo-black.png", darkSrc: "/images/logos/goose/goose-logo-white.png", instructionsUrl: "https://block.github.io/goose/docs/guides/context-engineering/using-skills/", sourceCodeUrl: "https://github.com/block/goose" }, { name: "GitHub Copilot", description: "GitHub Copilot works alongside you directly in your editor, suggesting whole lines or entire functions for you.", url: "https://github.com/", lightSrc: "/images/logos/github/GitHub_Lockup_Dark.svg", darkSrc: "/images/logos/github/GitHub_Lockup_Light.svg", instructionsUrl: "https://docs.github.com/en/copilot/concepts/agents/about-agent-skills", sourceCodeUrl: "https://github.com/microsoft/vscode-copilot-chat" }, { name: "VS Code", description: "Visual Studio Code combines the simplicity of a code editor with what developers need for their core edit-build-debug cycle.", url: "https://code.visualstudio.com/", lightSrc: "/images/logos/vscode/vscode.svg", darkSrc: "/images/logos/vscode/vscode-alt.svg", instructionsUrl: "https://code.visualstudio.com/docs/copilot/customization/agent-skills", sourceCodeUrl: "https://github.com/microsoft/vscode" }, { name: "Claude Code", description: "Claude Code is an agentic coding tool that reads your codebase, edits files, runs commands, and integrates with your development tools. Available in your terminal, IDE, desktop app, and browser.", url: "https://claude.ai/code", lightSrc: "/images/logos/claude-code/Claude-Code-logo-Slate.svg", darkSrc: "/images/logos/claude-code/Claude-Code-logo-Ivory.svg", instructionsUrl: "https://code.claude.com/docs/en/skills" }, { name: "Claude", description: "Claude is Anthropic's AI, built for problem solvers. Tackle complex challenges, analyze data, write code, and think through your hardest work.", url: "https://claude.ai/", lightSrc: "/images/logos/claude-ai/Claude-logo-Slate.svg", darkSrc: "/images/logos/claude-ai/Claude-logo-Ivory.svg", instructionsUrl: "https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview" }, { name: "OpenAI Codex", description: "Codex is OpenAI's coding agent for software development.", url: "https://developers.openai.com/codex", lightSrc: "/images/logos/oai-codex/OAI_Codex-Lockup_400px.svg", darkSrc: "/images/logos/oai-codex/OAI_Codex-Lockup_400px_Darkmode.svg", instructionsUrl: "https://developers.openai.com/codex/skills/", sourceCodeUrl: "https://github.com/openai/codex" }, { name: "Piebald", description: "Piebald is a desktop & web app that makes it easier than ever to do agentic development, while at the same time giving you complete control over the configuration, context, and flow.", url: "https://piebald.ai", lightSrc: "/images/logos/piebald/Piebald_wordmark_light.svg", darkSrc: "/images/logos/piebald/Piebald_wordmark_dark.svg" }, { name: "Factory", description: "Factory is an AI-native software development platform that works everywhere you do. From IDE to CI/CD — delegate complete tasks like refactors, incident response, and migrations to Droids without changing your tools, models, or workflow.", url: "https://factory.ai/", lightSrc: "/images/logos/factory/factory-logo-light.svg", darkSrc: "/images/logos/factory/factory-logo-dark.svg", instructionsUrl: "https://docs.factory.ai/cli/configuration/skills" }, { name: "pi", description: "Pi is a minimal terminal coding harness. Adapt pi to your workflows, not the other way around.", url: "https://shittycodingagent.ai/", lightSrc: "/images/logos/pi/pi-logo-light.svg", darkSrc: "/images/logos/pi/pi-logo-dark.svg", scale: 0.55, instructionsUrl: "https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/skills.md", sourceCodeUrl: "https://github.com/badlogic/pi-mono" }, { name: "Databricks Genie Code", description: "Genie Code is an autonomous AI partner purpose-built for data work in Databricks.", url: "https://databricks.com/", lightSrc: "/images/logos/databricks/databricks-logo-light.svg", darkSrc: "/images/logos/databricks/databricks-logo-dark.svg", instructionsUrl: "https://docs.databricks.com/aws/en/assistant/skills" }, { name: "Agentman", description: "Agentman is an agentic healthcare platform. It automates revenue cycle workflows using AI agents without sacrificing control. Every action is testable, traceable, and auditable.", url: "https://agentman.ai/", lightSrc: "/images/logos/agentman/agentman-wordmark-light.svg", darkSrc: "/images/logos/agentman/agentman-wordmark-dark.svg", instructionsUrl: "https://agentman.ai/agentskills" }, { name: "TRAE", description: "Trae is an adaptive AI IDE that transforms how you work, collaborating with you to run faster.", url: "https://trae.ai/", lightSrc: "/images/logos/trae/trae-logo-lightmode.svg", darkSrc: "/images/logos/trae/trae-logo-darkmode.svg", instructionsUrl: "https://www.trae.ai/blog/trae_tutorial_0115", sourceCodeUrl: "https://github.com/bytedance/trae-agent" }, { name: "Spring AI", description: "Spring AI aims to streamline the development of applications that incorporate artificial intelligence functionality without unnecessary complexity.", url: "https://docs.spring.io/spring-ai/reference", lightSrc: "/images/logos/spring-ai/spring-ai-logo-light.svg", darkSrc: "/images/logos/spring-ai/spring-ai-logo-dark.svg", instructionsUrl: "https://spring.io/blog/2026/01/13/spring-ai-generic-agent-skills/", sourceCodeUrl: "https://github.com/spring-projects/spring-ai" }, { name: "Roo Code", description: "Roo Code puts an entire AI dev team right in your editor, outpacing closed tools with deep project-wide context, multi-step agentic coding, and unmatched developer-centric flexibility.", url: "https://roocode.com", lightSrc: "/images/logos/roo-code/roo-code-logo-black.svg", darkSrc: "/images/logos/roo-code/roo-code-logo-white.svg", instructionsUrl: "https://docs.roocode.com/features/skills", sourceCodeUrl: "https://github.com/RooCodeInc/Roo-Code" }, { name: "Mistral AI Vibe", description: "Mistral Vibe is a command-line coding assistant powered by Mistral's models. It provides a conversational interface to your codebase, allowing you to use natural language to explore, modify, and interact with your projects through a powerful set of tools.", url: "https://github.com/mistralai/mistral-vibe", lightSrc: "/images/logos/mistral-vibe/vibe-logo_black.svg", darkSrc: "/images/logos/mistral-vibe/vibe-logo_white.svg", scale: 0.55, instructionsUrl: "https://github.com/mistralai/mistral-vibe", sourceCodeUrl: "https://github.com/mistralai/mistral-vibe" }, { name: "Command Code", description: "Command Code is a coding agent that continuously learns your coding taste. Our meta neuro-symbolic AI model taste-1 with continuous reinforcement learning combines LLMs with your coding taste.", url: "https://commandcode.ai/", lightSrc: "/images/logos/command-code/command-code-logo-for-light.svg", darkSrc: "/images/logos/command-code/command-code-logo-for-dark.svg", scale: 1.33, instructionsUrl: "https://commandcode.ai/docs/skills" }, { name: "Ona", description: "Ona is a platform for background agents. Run a team of AI software engineers in the cloud. Orchestrated, governed, secured at the kernel.", url: "https://ona.com", lightSrc: "/images/logos/ona/ona-wordmark-light.svg", darkSrc: "/images/logos/ona/ona-wordmark-dark.svg", scale: 0.8, instructionsUrl: "https://ona.com/docs/ona/agents-md#skills-for-repository-specific-workflows" }, { name: "VT Code", description: "VT Code is an open-source coding agent with LLM-native code understanding and robust shell safety. Supports multiple LLM providers with automatic failover and efficient context management.", url: "https://github.com/vinhnx/vtcode", lightSrc: "/images/logos/vtcode/vt_code_light.svg", darkSrc: "/images/logos/vtcode/vt_code_dark.svg", instructionsUrl: "https://github.com/vinhnx/vtcode/blob/main/docs/skills/SKILLS_GUIDE.md", sourceCodeUrl: "https://github.com/vinhnx/VTCode" }, { name: "Qodo", description: "Qodo is an agentic code integrity platform for reviewing, testing, and writing code, integrating AI across development workflows to strengthen code quality at every stage.", url: "https://www.qodo.ai/", lightSrc: "/images/logos/qodo/qodo-logo-light.png", darkSrc: "/images/logos/qodo/qodo-logo-dark.svg", instructionsUrl: "https://www.qodo.ai/blog/how-i-use-qodos-agent-skills-to-auto-fix-issues-in-pull-requests/" }, { name: "Laravel Boost", description: "Laravel Boost accelerates AI-assisted development by providing the essential guidelines and agent skills that help AI agents write high-quality Laravel applications that adhere to Laravel best practices.", url: "https://github.com/laravel/boost", lightSrc: "/images/logos/laravel-boost/boost-light-mode.svg", darkSrc: "/images/logos/laravel-boost/boost-dark-mode.svg", instructionsUrl: "https://laravel.com/docs/12.x/boost#agent-skills", sourceCodeUrl: "https://github.com/laravel/boost" }, { name: "Emdash", description: "Emdash is a provider-agnostic desktop app that lets you run multiple coding agents in parallel, each isolated in its own git worktree, either locally or over SSH on a remote machine.", url: "https://emdash.sh", lightSrc: "/images/logos/emdash/emdash-logo-light.svg", darkSrc: "/images/logos/emdash/emdash-logo-dark.svg", instructionsUrl: "https://docs.emdash.sh/skills", sourceCodeUrl: "https://github.com/generalaction/emdash" }, { name: "Snowflake Cortex Code", description: "Cortex Code is an AI-driven intelligent agent integrated into the Snowflake platform, optimized for complex data engineering, analytics, machine learning, and agent-building tasks.", url: "https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code", lightSrc: "/images/logos/snowflake/snowflake-logo-light.svg", darkSrc: "/images/logos/snowflake/snowflake-logo-dark.svg", instructionsUrl: "https://docs.snowflake.com/en/user-guide/cortex-code/extensibility#extensibility-skills" }, { name: "Kiro", description: "Kiro helps you do your best work by bringing structure to AI coding with spec-driven development.", url: "https://kiro.dev/", lightSrc: "/images/logos/kiro/kiro-logo-light.svg", darkSrc: "/images/logos/kiro/kiro-logo-dark.svg", instructionsUrl: "https://kiro.dev/docs/skills/" }, { name: "Workshop", description: "Workshop is a cross-platform AI coding agent for building full applications. It supports multi-LLM models, sub-agents, custom agents, and skills — available as a desktop app, web app, and CLI.", url: "https://workshop.ai/", lightSrc: "/images/logos/workshop/workshop-logo-light.svg", darkSrc: "/images/logos/workshop/workshop-logo-dark.svg", instructionsUrl: "https://docs.workshop.ai/core-concepts/working-with-the-agent#create-your-own-agents" }, { name: "Google AI Edge Gallery", description: "Google AI Edge Gallery is the premier destination for running the world's most powerful open-source Large Language Models (LLMs) on your mobile device", url: "https://github.com/google-ai-edge/gallery", lightSrc: "/images/logos/google-ai-edge-gallery/google-ai-edge-gallery-light.svg", darkSrc: "/images/logos/google-ai-edge-gallery/google-ai-edge-gallery-dark.svg", scale: 0.45, instructionsUrl: "https://github.com/google-ai-edge/gallery/tree/main/skills", sourceCodeUrl: "https://github.com/google-ai-edge/gallery" }, { name: "nanobot", description: "nanobot is an ultra-lightweight, open-source personal AI agent. It runs across multiple platforms — terminal, Telegram, Discord, Slack, WeChat, and more — with built-in MCP support and a skills system for extensibility.", url: "https://nanobot.wiki/", lightSrc: "/images/logos/nanobot/nanobot-logo-light.png", darkSrc: "/images/logos/nanobot/nanobot-logo-dark.png", instructionsUrl: "https://nanobot.wiki/docs/0.1.5/use-nanobot/skills", sourceCodeUrl: "https://github.com/HKUDS/nanobot" }, { name: "fast-agent", description: "fast-agent is a simple, extendable way to interact with LLMs. Excellent for Coding, Evals, ACPX and Skills development.", url: "https://fast-agent.ai/", lightSrc: "/images/logos/fast-agent/fast-agent-light.svg", darkSrc: "/images/logos/fast-agent/fast-agent-dark.svg", scale: 1.33, instructionsUrl: "https://fast-agent.ai/agents/skills/", sourceCodeUrl: "https://github.com/evalstate/fast-agent" }, { name: "bub", description: "Bub is a lightweight, hook-first Python framework for channel-native agents that live alongside people.", url: "https://bub.build/", lightSrc: "/images/logos/bub/bub-light.svg", darkSrc: "/images/logos/bub/bub-dark.svg", instructionsUrl: "https://bub.build/docs/build/skills/", sourceCodeUrl: "https://github.com/bubbuild/bub" }, { name: "Tabnine", description: "Tabnine is an AI engineering platform that combines code assistants, agentic workflows, and enterprise context to help development teams build, review, and maintain software with context-aware AI—while keeping code private, secure, and fully under your control.", url: "https://www.tabnine.com/", lightSrc: "/images/logos/tabnine/tabnine-logo-light.svg", darkSrc: "/images/logos/tabnine/tabnine-logo-dark.svg", instructionsUrl: "https://docs.tabnine.com/main/getting-started/tabnine-cli/features/agent-skills" }, { name: "Vita", description: "Vita provides autonomous digital workers with virtual desktops to execute complex workflows. It learns your intent to automate end-to-end tasks like creating and posting engaging social media content.", url: "https://www.vita-ai.net", lightSrc: "/images/logos/vita/logo-horizontal-light.svg", darkSrc: "/images/logos/vita/logo-horizontal-dark.svg", instructionsUrl: "https://www.vita-ai.net/docs/features/agent-skills" }, { name: "Superconductor", description: "Superconductor is a multiplayer workspace for your team and coding agents. Run many agents in the cloud, build together in shared sessions, and ship faster with live app previews and guided code review.", url: "https://superconductor.com/", lightSrc: "/images/logos/superconductor/superconductor-wordmark-light.svg", darkSrc: "/images/logos/superconductor/superconductor-wordmark-dark.svg", scale: 1.33, instructionsUrl: "https://superconductor.com/docs/project/mcp-and-skills" }]; ## What are Agent Skills? Agent Skills are a lightweight, open format for extending AI agent capabilities with specialized knowledge and workflows. At its core, a skill is a folder containing a `SKILL.md` file. This file includes metadata (`name` and `description`, at minimum) and instructions that tell an agent how to perform a specific task. Skills can also bundle scripts, reference materials, templates, and other resources. ``` my-skill/ ├── SKILL.md # Required: metadata + instructions ├── scripts/ # Optional: executable code ├── references/ # Optional: documentation ├── assets/ # Optional: templates, resources └── ... # Any additional files or directories ``` ## Why Agent Skills? Agents are increasingly capable, but often don't have the context they need to do real work reliably. Skills solve this by packaging procedural knowledge and company-, team-, and user-specific context into portable, version-controlled folders that agents load on demand. This gives agents: * **Domain expertise**: Capture specialized knowledge — from legal review processes to data analysis pipelines to presentation formatting — as reusable instructions and resources. * **Repeatable workflows**: Turn multi-step tasks into consistent, auditable procedures. * **Cross-product reuse**: Build a skill once and use it across any skills-compatible agent. ## How do Agent Skills work? Agents load skills through **progressive disclosure**, in three stages: 1. **Discovery**: At startup, agents load only the name and description of each available skill, just enough to know when it might be relevant. 2. **Activation**: When a task matches a skill's description, the agent reads the full `SKILL.md` instructions into context. 3. **Execution**: The agent follows the instructions, optionally executing bundled code or loading referenced files as needed. Full instructions load only when a task calls for them, so agents can keep many skills on hand with only a small context footprint. ## Where can I use Agent Skills? Agent Skills are supported by a large number of AI tools and agentic clients — see the [Client Showcase](/clients) to explore some of them! <LogoCarousel clients={clients} /> ## Open development The Agent Skills format was originally developed by [Anthropic](https://www.anthropic.com/), released as an open standard, and has been adopted by a growing number of agent products. The standard is open to contributions from the broader ecosystem. Come join the discussion on [GitHub](https://github.com/agentskills/agentskills) or [Discord](https://discord.gg/MKPE9g8aUy)! ## Get started with Agent Skills <CardGroup cols={2}> <Card title="Quickstart" icon="rocket" href="/skill-creation/quickstart"> Create your first Agent Skill and see it in action. </Card> <Card title="Specification" icon="file-code" href="/specification"> The complete format specification for Agent Skills. </Card> </CardGroup> -
optimizing-descriptions.md 13 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Optimizing skill descriptions > How to improve your skill's description so it triggers reliably on relevant prompts. A skill only helps if it gets activated. The `description` field in your `SKILL.md` frontmatter is the primary mechanism agents use to decide whether to load a skill for a given task. An under-specified description means the skill won't trigger when it should; an over-broad description means it triggers when it shouldn't. This guide covers how to systematically test and improve your skill's description for triggering accuracy. ## How skill triggering works Agents use [progressive disclosure](/specification#progressive-disclosure) to manage context. At startup, they load only the `name` and `description` of each available skill — just enough to decide when a skill might be relevant. When a user's task matches a description, the agent reads the full `SKILL.md` into context and follows its instructions. This means the description carries the entire burden of triggering. If the description doesn't convey when the skill is useful, the agent won't know to reach for it. One important nuance: agents typically only consult skills for tasks that require knowledge or capabilities beyond what they can handle alone. A simple, one-step request like "read this PDF" may not trigger a PDF skill even if the description matches perfectly, because the agent can handle it with basic tools. Tasks that involve specialized knowledge — an unfamiliar API, a domain-specific workflow, or an uncommon format — are where a well-written description can make the difference. ## Writing effective descriptions Before testing, it helps to know what a good description looks like. A few principles: * **Use imperative phrasing.** Frame the description as an instruction to the agent: "Use this skill when..." rather than "This skill does..." The agent is deciding whether to act, so tell it when to act. * **Focus on user intent, not implementation.** Describe what the user is trying to achieve, not the skill's internal mechanics. The agent matches against what the user asked for. * **Err on the side of being pushy.** Explicitly list contexts where the skill applies, including cases where the user doesn't name the domain directly: "even if they don't explicitly mention 'CSV' or 'analysis.'" * **Keep it concise.** A few sentences to a short paragraph is usually right — long enough to cover the skill's scope, short enough that it doesn't bloat the agent's context across many skills. The [specification](/specification#description-field) enforces a hard limit of 1024 characters. ## Designing trigger eval queries To test triggering, you need a set of eval queries — realistic user prompts labeled with whether they should or shouldn't trigger your skill. ```json eval_queries.json theme={null} [ { "query": "I've got a spreadsheet in ~/data/q4_results.xlsx with revenue in col C and expenses in col D — can you add a profit margin column and highlight anything under 10%?", "should_trigger": true }, { "query": "whats the quickest way to convert this json file to yaml", "should_trigger": false } ] ``` Aim for about 20 queries: 8-10 that should trigger and 8-10 that shouldn't. ### Should-trigger queries These test whether the description captures the skill's scope. Vary them along several axes: * **Phrasing**: some formal, some casual, some with typos or abbreviations. * **Explicitness**: some name the skill's domain directly ("analyze this CSV"), others describe the need without naming it ("my boss wants a chart from this data file"). * **Detail**: mix terse prompts with context-heavy ones — a short "analyze my sales CSV and make a chart" alongside a longer message with file paths, column names, and backstory. * **Complexity**: vary the number of steps and decision points. Include single-step tasks alongside multi-step workflows to test whether the agent can discern the skill is relevant when the task it addresses is buried in a larger chain. The most useful should-trigger queries are ones where the skill would help but the connection isn't obvious from the query alone. These are the cases where description wording makes the difference — if the query already asks for exactly what the skill does, any reasonable description would trigger. ### Should-not-trigger queries The most valuable negative test cases are **near-misses** — queries that share keywords or concepts with your skill but actually need something different. These test whether the description is precise, not just broad. For a CSV analysis skill, weak negative examples would be: * `"Write a fibonacci function"` — obviously irrelevant, tests nothing. * `"What's the weather today?"` — no keyword overlap, too easy. Strong negative examples: * `"I need to update the formulas in my Excel budget spreadsheet"` — shares "spreadsheet" and "data" concepts, but needs Excel editing, not CSV analysis. * `"can you write a python script that reads a csv and uploads each row to our postgres database"` — involves CSV, but the task is database ETL, not analysis. ### Tips for realism Real user prompts contain context that generic test queries lack. Include: * File paths (`~/Downloads/report_final_v2.xlsx`) * Personal context (`"my manager asked me to..."`) * Specific details (column names, company names, data values) * Casual language, abbreviations, and occasional typos ## Testing whether a description triggers The basic approach: run each query through your agent with the skill installed and observe whether the agent invokes it. Make sure the skill is registered and discoverable by your agent — how this works varies by client (e.g., a skills directory, a configuration file, or a CLI flag). Most agent clients provide some form of observability — execution logs, tool call histories, or verbose output — that lets you see which skills were consulted during a run. Check your client's documentation for details. The skill triggered if the agent loaded your skill's `SKILL.md`; it didn't trigger if the agent proceeded without consulting it. A query "passes" if: * `should_trigger` is `true` and the skill was invoked, or * `should_trigger` is `false` and the skill was not invoked. ### Running multiple times Model behavior is nondeterministic — the same query might trigger the skill on one run but not the next. Run each query multiple times (3 is a reasonable starting point) and compute a **trigger rate**: the fraction of runs where the skill was invoked. A should-trigger query passes if its trigger rate is above a threshold (0.5 is a reasonable default). A should-not-trigger query passes if its trigger rate is below that threshold. With 20 queries at 3 runs each, that's 60 invocations. You'll want to script this. Here's the general structure — replace the `claude` invocation and detection logic in `check_triggered` with whatever your agent client provides: ```bash theme={null} #!/bin/bash QUERIES_FILE="${1:?Usage: $0 <queries.json>}" SKILL_NAME="my-skill" RUNS=3 # This example uses Claude Code's JSON output to check for Skill tool calls. # Replace this function with detection logic for your agent client. # Should return 0 (success) if the skill was invoked, 1 otherwise. check_triggered() { local query="$1" claude -p "$query" --output-format json 2>/dev/null \ | jq -e --arg skill "$SKILL_NAME" \ 'any(.messages[].content[]; .type == "tool_use" and .name == "Skill" and .input.skill == $skill)' \ > /dev/null 2>&1 } count=$(jq length "$QUERIES_FILE") for i in $(seq 0 $((count - 1))); do query=$(jq -r ".[$i].query" "$QUERIES_FILE") should_trigger=$(jq -r ".[$i].should_trigger" "$QUERIES_FILE") triggers=0 for run in $(seq 1 $RUNS); do check_triggered "$query" && triggers=$((triggers + 1)) done jq -n \ --arg query "$query" \ --argjson should_trigger "$should_trigger" \ --argjson triggers "$triggers" \ --argjson runs "$RUNS" \ '{query: $query, should_trigger: $should_trigger, triggers: $triggers, runs: $runs, trigger_rate: ($triggers / $runs)}' done | jq -s '.' ``` <Tip> If your agent client supports it, you can stop a run early once the outcome is clear — the agent either consulted the skill or started working without it. This can significantly reduce the time and cost of running the full eval set. </Tip> ## Avoiding overfitting with train/validation splits If you optimize the description against all your queries, you risk overfitting — crafting a description that works for these specific phrasings but fails on new ones. The solution is to split your query set: * **Train set (\~60%)**: the queries you use to identify failures and guide improvements. * **Validation set (\~40%)**: queries you set aside and only use to check whether improvements generalize. Make sure both sets contain a proportional mix of should-trigger and should-not-trigger queries — don't accidentally put all the positives in one set. Shuffle randomly and keep the split fixed across iterations so you're comparing apples to apples. If you're using a script like the one [above](#running-multiple-times), you can split your queries into two files — `train_queries.json` and `validation_queries.json` — and run the script against each one separately. ## The optimization loop 1. **Evaluate** the current description on both *train and validation sets*. The train results guide your changes; the validation results tell you whether those changes are generalizing. 2. **Identify failures** in the *train set*: which should-trigger queries didn't trigger? Which should-not-trigger queries did? * Only use train set failures to guide your changes — whether you're revising the description yourself or prompting an LLM, keep validation set results out of the process. 3. **Revise the description.** Focus on generalizing: * If should-trigger queries are failing, the description may be too narrow. Broaden the scope or add context about when the skill is useful. * If should-not-trigger queries are false-triggering, the description may be too broad. Add specificity about what the skill does *not* do, or clarify the boundary between this skill and adjacent capabilities. * Avoid adding specific keywords from failed queries — that's overfitting. Instead, find the general category or concept those queries represent and address that. * If you're stuck after several iterations, try a structurally different approach to the description rather than incremental tweaks. A different framing or sentence structure may break through where refinement can't. * Check that the description stays under the 1024-character limit — descriptions tend to grow during optimization. 4. **Repeat** steps 1-3 until all *train set* queries pass or you stop seeing meaningful improvement. 5. **Select the best iteration** by its validation pass rate — the fraction of queries in the *validation set* that passed. Note that the best description may not be the last one you produced; an earlier iteration might have a higher validation pass rate than later ones that overfit to the train set. Five iterations is usually enough. If performance isn't improving, the issue may be with the queries (too easy, too hard, or poorly labeled) rather than the description. <Tip> The [`skill-creator`](https://github.com/anthropics/skills/tree/main/skills/skill-creator) Skill automates this loop end-to-end: it splits the eval set, evaluates trigger rates in parallel, proposes description improvements using Claude, and generates a live HTML report you can watch as it runs. </Tip> ## Applying the result Once you've selected the best description: 1. Update the `description` field in your `SKILL.md` frontmatter. 2. Verify the description is under the [1024-character limit](/specification#description-field). 3. Verify the description triggers as expected. Try a few prompts manually as a quick sanity check. For a more rigorous test, write 5-10 fresh queries (a mix of should-trigger and should-not-trigger) and run them through the eval script — since these queries were never part of the optimization process, they give you an honest check on whether the description generalizes. Before and after: ```yaml theme={null} # Before description: Process CSV files. # After description: > Analyze CSV and tabular data files — compute summary statistics, add derived columns, generate charts, and clean messy data. Use this skill when the user has a CSV, TSV, or Excel file and wants to explore, transform, or visualize the data, even if they don't explicitly mention "CSV" or "analysis." ``` The improved description is more specific about what the skill does (summary stats, derived columns, charts, cleaning) and broader about when it applies (CSV, TSV, Excel; even without explicit keywords). ## Next steps Once your skill triggers reliably, you'll want to evaluate whether it produces good outputs. See [Evaluating skill output quality](/skill-creation/evaluating-skills) for how to set up test cases, grade results, and iterate. -
quickstart.md 3.9 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Quickstart > Create your first Agent Skill and see it work in VS Code. In this tutorial, you'll create a skill that gives an agent the capability to roll dice using a random number generator. ## Prerequisites * [VS Code](https://code.visualstudio.com/) with [GitHub Copilot](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot) <Note> This tutorial uses VS Code, but Agent Skills are an open format. The same skill works in any compatible agent, including Claude Code and OpenAI Codex. </Note> ## Create the skill A skill is a folder containing a `SKILL.md` file. VS Code looks for skills in `.agents/skills/` by default. Create `.agents/skills/roll-dice/SKILL.md` in your project: ````markdown .agents/skills/roll-dice/SKILL.md theme={null} --- name: roll-dice description: Roll dice using a random number generator. Use when asked to roll a die (d6, d20, etc.), roll dice, or generate a random dice roll. --- To roll a die, use the following command that generates a random number from 1 to the given number of sides: ```bash echo $((RANDOM % <sides> + 1)) ``` ```powershell Get-Random -Minimum 1 -Maximum (<sides> + 1) ``` Replace `<sides>` with the number of sides on the die (e.g., 6 for a standard die, 20 for a d20). ```` That's it — one file, under 20 lines. Here's what each part does: * **`name`** — A short identifier for the skill. Must match the folder name. * **`description`** — Tells the agent when to use this skill. This is how the agent decides whether to activate it. * **The body** — Instructions the agent follows when the skill activates. Here, the agent is instructed to generate a random number using a terminal command, substituting the number of sides from the user's request. ## Try it out 1. Open your project in VS Code. 2. Open the Copilot Chat panel. 3. Select **Agent** mode from the mode dropdown at the bottom of the chat panel. 4. Type `/skills` to confirm that `roll-dice` appears in the list. If it doesn't, check that the file is at `.agents/skills/roll-dice/SKILL.md` relative to your project root. 5. Ask: **"Roll a d20"** The agent should activate the `roll-dice` skill. It may ask for permission to run a terminal command — allow it. It will run the command and return a random number between 1 and 20. <Note> Tool-use reliability varies across models — some follow skill instructions and run commands consistently, while others may attempt to answer on their own. If the agent responds without running a terminal command, try selecting a different model from the model dropdown. </Note> ## How it works Here's what happened behind the scenes: 1. **Discovery** — When the chat session started, the agent scanned default skill directories and found your skill. It read only the `name` and `description`, just enough to know when the skill might be relevant. 2. **Activation** — When you asked about rolling dice, the agent matched your question to the skill's description and loaded the full `SKILL.md` body into context. 3. **Execution** — The agent followed the instructions in the body, adapting the terminal command to the number of sides in your request. This process uses **progressive disclosure** to let the agent access many skills without loading all their instructions up front. ## Next steps You've created a working Agent Skill. From here: * **[Best practices](/skill-creation/best-practices)** — How to write skills that are well-scoped and effective. * **[Optimizing skill descriptions](/skill-creation/optimizing-descriptions)** — Test and improve your skill's description so it activates on the right prompts. * **[Specification](/specification)** — The complete format reference for `SKILL.md` files. * **[Example skills](https://github.com/anthropics/skills)** — Browse real-world skills on GitHub. -
specification.md 7.9 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Specification > The complete format specification for Agent Skills. ## Directory structure A skill is a directory containing, at minimum, a `SKILL.md` file: ``` skill-name/ ├── SKILL.md # Required: metadata + instructions ├── scripts/ # Optional: executable code ├── references/ # Optional: documentation ├── assets/ # Optional: templates, resources └── ... # Any additional files or directories ``` ## `SKILL.md` format The `SKILL.md` file must contain YAML frontmatter followed by Markdown content. ### Frontmatter | Field | Required | Constraints | | --------------- | -------- | ----------------------------------------------------------------------------------------------------------------- | | `name` | Yes | Max 64 characters. Lowercase letters, numbers, and hyphens only. Must not start or end with a hyphen. | | `description` | Yes | Max 1024 characters. Non-empty. Describes what the skill does and when to use it. | | `license` | No | License name or reference to a bundled license file. | | `compatibility` | No | Max 500 characters. Indicates environment requirements (intended product, system packages, network access, etc.). | | `metadata` | No | Arbitrary key-value mapping for additional metadata. | | `allowed-tools` | No | Space-separated string of pre-approved tools the skill may use. (Experimental) | <Card> **Minimal example:** ```markdown SKILL.md theme={null} --- name: skill-name description: A description of what this skill does and when to use it. --- ``` **Example with optional fields:** ```markdown SKILL.md theme={null} --- name: pdf-processing description: Extract PDF text, fill forms, merge files. Use when handling PDFs. license: Apache-2.0 metadata: author: example-org version: "1.0" --- ``` </Card> #### `name` field The required `name` field: * Must be 1-64 characters * May only contain unicode lowercase alphanumeric characters (`a-z`, `0-9`) and hyphens (`-`) * Must not start or end with a hyphen (`-`) * Must not contain consecutive hyphens (`--`) * Must match the parent directory name <Card> **Valid examples:** ```yaml theme={null} name: pdf-processing ``` ```yaml theme={null} name: data-analysis ``` ```yaml theme={null} name: code-review ``` **Invalid examples:** ```yaml theme={null} name: PDF-Processing # uppercase not allowed ``` ```yaml theme={null} name: -pdf # cannot start with hyphen ``` ```yaml theme={null} name: pdf--processing # consecutive hyphens not allowed ``` </Card> #### `description` field The required `description` field: * Must be 1-1024 characters * Should describe both what the skill does and when to use it * Should include specific keywords that help agents identify relevant tasks <Card> **Good example:** ```yaml theme={null} description: Extracts text and tables from PDF files, fills PDF forms, and merges multiple PDFs. Use when working with PDF documents or when the user mentions PDFs, forms, or document extraction. ``` **Poor example:** ```yaml theme={null} description: Helps with PDFs. ``` </Card> #### `license` field The optional `license` field: * Specifies the license applied to the skill * We recommend keeping it short (either the name of a license or the name of a bundled license file) <Card> **Example:** ```yaml theme={null} license: Proprietary. LICENSE.txt has complete terms ``` </Card> #### `compatibility` field The optional `compatibility` field: * Must be 1-500 characters if provided * Should only be included if your skill has specific environment requirements * Can indicate intended product, required system packages, network access needs, etc. <Card> **Examples:** ```yaml theme={null} compatibility: Designed for Claude Code (or similar products) ``` ```yaml theme={null} compatibility: Requires git, docker, jq, and access to the internet ``` ```yaml theme={null} compatibility: Requires Python 3.14+ and uv ``` </Card> <Note> Most skills do not need the `compatibility` field. </Note> #### `metadata` field The optional `metadata` field: * A map from string keys to string values * Clients can use this to store additional properties not defined by the Agent Skills spec * We recommend making your key names reasonably unique to avoid accidental conflicts <Card> **Example:** ```yaml theme={null} metadata: author: example-org version: "1.0" ``` </Card> #### `allowed-tools` field The optional `allowed-tools` field: * A space-separated string of tools that are pre-approved to run * Experimental. Support for this field may vary between agent implementations <Card> **Example:** ```yaml theme={null} allowed-tools: Bash(git:*) Bash(jq:*) Read ``` </Card> ### Body content The Markdown body after the frontmatter contains the skill instructions. There are no format restrictions. Write whatever helps agents perform the task effectively. Recommended sections: * Step-by-step instructions * Examples of inputs and outputs * Common edge cases * Applicability conditions for instructions that only apply to a subset of tasks Note that the agent will load this entire file once it's decided to activate a skill. Consider splitting longer `SKILL.md` content into referenced files. ## Optional directories ### `scripts/` Contains executable code that agents can run. Scripts should: * Be self-contained or clearly document dependencies * Include helpful error messages * Handle edge cases gracefully Supported languages depend on the agent implementation. Common options include Python, Bash, and JavaScript. ### `references/` Contains additional documentation that agents can read when needed: * `REFERENCE.md` - Detailed technical reference * `FORMS.md` - Form templates or structured data formats * Domain-specific files (`finance.md`, `legal.md`, etc.) Keep individual [reference files](#file-references) focused. Agents load these on demand, so smaller files mean less use of context. ### `assets/` Contains static resources: * Templates (document templates, configuration templates) * Images (diagrams, examples) * Data files (lookup tables, schemas) ## Progressive disclosure Agents load skills *progressively*, pulling in more detail only as a task calls for it. Skills should be structured to take advantage of this: 1. **Metadata** (\~100 tokens): The `name` and `description` fields are loaded at startup for all skills 2. **Instructions** (\< 5000 tokens recommended): The full `SKILL.md` body is loaded when the skill is activated 3. **Resources** (as needed): Files (e.g. those in `scripts/`, `references/`, or `assets/`) are loaded only when required Keep your main `SKILL.md` under 500 lines. Move detailed reference material to separate files. ## File references When referencing other files in your skill, use relative paths from the skill root: ```markdown SKILL.md theme={null} See [the reference guide](references/REFERENCE.md) for details. Run the extraction script: scripts/extract.py ``` Keep file references one level deep from `SKILL.md`. Avoid deeply nested reference chains. ## Validation Use the [skills-ref](https://github.com/agentskills/agentskills/tree/main/skills-ref) reference library to validate your skills: ```bash theme={null} skills-ref validate ./my-skill ``` This checks that your `SKILL.md` frontmatter is valid and follows all naming conventions. -
using-scripts.md 12.6 KB
> ## Documentation Index > Fetch the complete documentation index at: https://agentskills.io/llms.txt > Use this file to discover all available pages before exploring further. # Using scripts in skills > How to run commands and bundle executable scripts in your skills. Skills can instruct agents to run shell commands and bundle reusable scripts in a `scripts/` directory. This guide covers one-off commands, self-contained scripts with their own dependencies, and how to design script interfaces for agentic use. ## One-off commands When an existing package already does what you need, you can reference it directly in your `SKILL.md` instructions without a `scripts/` directory. Many ecosystems provide tools that auto-resolve dependencies at runtime. <Tabs sync={false}> <Tab title="uvx"> [uvx](https://docs.astral.sh/uv/guides/tools/) runs Python packages in isolated environments with aggressive caching. It ships with [uv](https://docs.astral.sh/uv/). ```bash theme={null} uvx ruff@0.8.0 check . uvx black@24.10.0 . ``` * Not bundled with Python — requires a separate install. * Fast. Caches aggressively so repeat runs are near-instant. </Tab> <Tab title="pipx"> [pipx](https://pipx.pypa.io/) runs Python packages in isolated environments. Available via OS package managers (`apt install pipx`, `brew install pipx`). ```bash theme={null} pipx run 'black==24.10.0' . pipx run 'ruff==0.8.0' check . ``` * Not bundled with Python — requires a separate install. * A mature alternative to `uvx`. While `uvx` has become the standard recommendation, `pipx` remains a reliable option with broader OS package manager availability. </Tab> <Tab title="npx"> [npx](https://docs.npmjs.com/cli/commands/npx) runs npm packages, downloading them on demand. It ships with npm (which ships with Node.js). ```bash theme={null} npx eslint@9 --fix . npx create-vite@6 my-app ``` * Bundled with Node.js — no extra install needed. * Downloads the package, runs it, and caches it for future use. * **Pin versions for any command an agent will copy-paste** (`npx package@version`, e.g. `npx eslint@9.0.0`) so the command behaves the same over time — never leave a copy-pasteable `npx` invocation unpinned. </Tab> <Tab title="bunx"> [bunx](https://bun.sh/docs/cli/bunx) is Bun's equivalent of `npx`. It ships with [Bun](https://bun.sh/). ```bash theme={null} bunx eslint@9 --fix . bunx create-vite@6 my-app ``` * Drop-in replacement for `npx` in Bun-based environments. * Only appropriate when the user's environment has Bun rather than Node.js. </Tab> <Tab title="deno run"> [deno run](https://docs.deno.com/runtime/reference/cli/run/) runs scripts directly from URLs or specifiers. It ships with [Deno](https://deno.com/). ```bash theme={null} deno run npm:create-vite@6 my-app deno run --allow-read npm:eslint@9 -- --fix . ``` * Permission flags (`--allow-read`, etc.) are required for filesystem/network access. * Use `--` to separate Deno flags from the tool's own flags. </Tab> <Tab title="go run"> [go run](https://pkg.go.dev/cmd/go#hdr-Compile_and_run_Go_program) compiles and runs Go packages directly. It is built into the `go` command. ```bash theme={null} go run golang.org/x/tools/cmd/goimports@v0.28.0 . go run github.com/golangci/golangci-lint/cmd/golangci-lint@v1.62.0 run ``` * Built into Go — no extra tooling needed. * Pin versions or use `@latest` to make the command explicit. </Tab> </Tabs> **Tips for one-off commands in skills:** * **Pin versions** (e.g., `npx eslint@9.0.0`) so the command behaves the same over time. * **State prerequisites** in your `SKILL.md` (e.g., "Requires Node.js 18+") rather than assuming the agent's environment has them. For runtime-level requirements, use the [`compatibility` frontmatter field](/specification#compatibility-field). * **Move complex commands into scripts.** A one-off command works well when you're invoking a tool with a few flags. When a command grows complex enough that it's hard to get right on the first try, a tested script in `scripts/` is more reliable. ## Referencing scripts from `SKILL.md` Use **relative paths from the skill directory root** to reference bundled files. The agent resolves these paths automatically — no absolute paths needed. List available scripts in your `SKILL.md` so the agent knows they exist: ```markdown SKILL.md theme={null} ## Available scripts - **`scripts/validate.sh`** — Validates configuration files - **`scripts/process.py`** — Processes input data ``` Then instruct the agent to run them: ````markdown SKILL.md theme={null} ## Workflow 1. Run the validation script: ```bash bash scripts/validate.sh "$INPUT_FILE" ``` 2. Process the results: ```bash python3 scripts/process.py --input results.json ``` ```` <Note> The same relative-path convention works in support files like `references/*.md` — script execution paths (in code blocks) are relative to the **skill directory root**, because the agent runs commands from there. </Note> ## Self-contained scripts When you need reusable logic, bundle a script in `scripts/` that declares its own dependencies inline. The agent can run the script with a single command — no separate manifest file or install step required. Several languages support inline dependency declarations: <Tabs sync={false}> <Tab title="Python"> [PEP 723](https://peps.python.org/pep-0723/) defines a standard format for inline script metadata. Declare dependencies in a TOML block inside `# ///` markers: ```python scripts/extract.py theme={null} # /// script # dependencies = [ # "beautifulsoup4", # ] # /// from bs4 import BeautifulSoup html = '<html><body><h1>Welcome</h1><p class="info">This is a test.</p></body></html>' print(BeautifulSoup(html, "html.parser").select_one("p.info").get_text()) ``` Run with [uv](https://docs.astral.sh/uv/) (recommended): ```bash theme={null} uv run scripts/extract.py ``` `uv run` creates an isolated environment, installs the declared dependencies, and runs the script. [pipx](https://pipx.pypa.io/) (`pipx run scripts/extract.py`) also supports PEP 723. * Pin versions with [PEP 508](https://peps.python.org/pep-0508/) specifiers: `"beautifulsoup4>=4.12,<5"`. * Use `requires-python` to constrain the Python version. * Use `uv lock --script` to create a lockfile for full reproducibility. </Tab> <Tab title="Deno"> Deno's `npm:` and `jsr:` import specifiers make every script self-contained by default: ```typescript scripts/extract.ts theme={null} #!/usr/bin/env -S deno run import * as cheerio from "npm:cheerio@1.0.0"; const html = `<html><body><h1>Welcome</h1><p class="info">This is a test.</p></body></html>`; const $ = cheerio.load(html); console.log($("p.info").text()); ``` ```bash theme={null} deno run scripts/extract.ts ``` * Use `npm:` for npm packages, `jsr:` for Deno-native packages. * Version specifiers follow semver: `@1.0.0` (exact), `@^1.0.0` (compatible). * Dependencies are cached globally. Use `--reload` to force re-fetch. * Packages with native addons (node-gyp) may not work — packages that ship pre-built binaries work best. </Tab> <Tab title="Bun"> Bun auto-installs missing packages at runtime when no `node_modules` directory is found. Pin versions directly in the import path: ```typescript scripts/extract.ts theme={null} #!/usr/bin/env bun import * as cheerio from "cheerio@1.0.0"; const html = `<html><body><h1>Welcome</h1><p class="info">This is a test.</p></body></html>`; const $ = cheerio.load(html); console.log($("p.info").text()); ``` ```bash theme={null} bun run scripts/extract.ts ``` * No `package.json` or `node_modules` needed. TypeScript works natively. * Packages are cached globally. First run downloads; subsequent runs are near-instant. * If a `node_modules` directory exists anywhere up the directory tree, auto-install is disabled and Bun falls back to standard Node.js resolution. </Tab> <Tab title="Ruby"> Bundler ships with Ruby since 2.6. Use `bundler/inline` to declare gems directly in the script: ```ruby scripts/extract.rb theme={null} require 'bundler/inline' gemfile do source 'https://rubygems.org' gem 'nokogiri' end html = '<html><body><h1>Welcome</h1><p class="info">This is a test.</p></body></html>' doc = Nokogiri::HTML(html) puts doc.at_css('p.info').text ``` ```bash theme={null} ruby scripts/extract.rb ``` * Pin versions explicitly (`gem 'nokogiri', '~> 1.16'`) — there is no lockfile. * An existing `Gemfile` or `BUNDLE_GEMFILE` env var in the working directory can interfere. </Tab> </Tabs> ## Designing scripts for agentic use When an agent runs your script, it reads stdout and stderr to decide what to do next. A few design choices make scripts dramatically easier for agents to use. ### Avoid interactive prompts This is a hard requirement of the agent execution environment. Agents operate in non-interactive shells — they cannot respond to TTY prompts, password dialogs, or confirmation menus. A script that blocks on interactive input will hang indefinitely. Accept all input via command-line flags, environment variables, or stdin: ``` # Bad: hangs waiting for input $ python scripts/deploy.py Target environment: _ # Good: clear error with guidance $ python scripts/deploy.py Error: --env is required. Options: development, staging, production. Usage: python scripts/deploy.py --env staging --tag v1.2.3 ``` ### Document usage with `--help` `--help` output is the primary way an agent learns your script's interface. Include a brief description, available flags, and usage examples: ``` Usage: scripts/process.py [OPTIONS] INPUT_FILE Process input data and produce a summary report. Options: --format FORMAT Output format: json, csv, table (default: json) --output FILE Write output to FILE instead of stdout --verbose Print progress to stderr Examples: scripts/process.py data.csv scripts/process.py --format csv --output report.csv data.csv ``` Keep it concise — the output enters the agent's context window alongside everything else it's working with. ### Write helpful error messages When an agent gets an error, the message directly shapes its next attempt. An opaque "Error: invalid input" wastes a turn. Instead, say what went wrong, what was expected, and what to try: ``` Error: --format must be one of: json, csv, table. Received: "xml" ``` ### Use structured output Prefer structured formats — JSON, CSV, TSV — over free-form text. Structured formats can be consumed by both the agent and standard tools (`jq`, `cut`, `awk`), making your script composable in pipelines. ``` # Whitespace-aligned — hard to parse programmatically NAME STATUS CREATED my-service running 2025-01-15 # Delimited — unambiguous field boundaries {"name": "my-service", "status": "running", "created": "2025-01-15"} ``` **Separate data from diagnostics:** send structured data to stdout and progress messages, warnings, and other diagnostics to stderr. This lets the agent capture clean, parseable output while still having access to diagnostic information when needed. ### Further considerations * **Idempotency.** Agents may retry commands. "Create if not exists" is safer than "create and fail on duplicate." * **Input constraints.** Reject ambiguous input with a clear error rather than guessing. Use enums and closed sets where possible. * **Dry-run support.** For destructive or stateful operations, a `--dry-run` flag lets the agent preview what will happen. * **Meaningful exit codes.** Use distinct exit codes for different failure types (not found, invalid arguments, auth failure) and document them in your `--help` output so the agent knows what each code means. * **Safe defaults.** Consider whether destructive operations should require explicit confirmation flags (`--confirm`, `--force`) or other safeguards appropriate to the risk level. * **Predictable output size.** Many agent harnesses automatically truncate tool output beyond a threshold (e.g., 10-30K characters), potentially losing critical information. If your script might produce large output, default to a summary or a reasonable limit, and support flags like `--offset` so the agent can request more information when needed. Alternatively, if output is large and not amenable to pagination, require agents to pass an `--output` flag that specifies either an output file or `-` to explicitly opt in to stdout. -
vetting-third-party-skills.md 6.1 KB
# Vetting third-party skills before you run them > How to review a skill you did not author — or one your agent generated for you — before giving it the permissions your agent has. A skill is not a document you read; it is **executable capability**. When an agent activates a skill, the skill's instructions and scripts run with the agent's own permissions: shell access, read/write to the file system, credentials from environment variables and config files, and the ability to send messages. Installing an unvetted skill is like installing a package from a registry you have never heard of, except that no package manager gate exists by default. The risk is not theoretical. Snyk's ToxicSkills audit (published February 2026) scanned 3,984 skills from ClawHub and skills.sh, the largest public corpus of agent skills then known: - **13.4% (534) contained at least one critical-level issue** — malware, prompt injection, or exposed secrets. - **36.82% (1,467) had at least one security flaw at any severity** — hardcoded API keys, insecure credential handling, or dangerous third-party content exposure. - **76 confirmed malicious payloads** were found, with credential theft, backdoor installation, and data exfiltration; 8 were still publicly available at publication. Source: [Snyk — ToxicSkills: Snyk finds malware and prompt injection in 36% of AI agent skills](https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/). The open Agent Skills standard says nothing about whether a given skill is safe; treat "published in a registry" as zero security signal. ## When to use this reference Use it whenever you are about to **run** a skill you did not author and have not previously vetted: - A skill pulled from a public registry or marketplace (skills.sh, ClawHub, or similar). - A skill shared by a colleague, copied from a blog post, or vendored from another repository. - A skill an LLM generated for you in one pass and you have not read. - An updated version of a third-party skill you already run — re-vet on material version changes. It also applies to skills embedded in repositories you clone: project-level skills from an untrusted repository can inject instructions into your agent's context (see `client-implementation.md` for the client-side trust-gating angle). ## The vetting checklist Work through this before the first run. It is a dependency review, not a skim. ### 1. Provenance - Who published it, and when? Is the author identifiable, with a history beyond the skill itself? A registry account that is days old is a red flag. - Does it have a license, a README, a changelog? Does the repository look maintained or abandoned? - How did you obtain it? A direct link from a trusted source you already rely on is different from a search-result download. - Prefer pinned versions and record what you installed, so you can diff later updates. ### 2. The SKILL.md itself Read the whole file — not just the description. The description is marketing; the body is the contract. - Does the body's actual behavior match the description's promise? - Look for hidden or deceptive instructions: base64 or other obfuscation, Unicode smuggling, "ignore previous instructions" patterns, system-message impersonation, instructions to do something unrelated to the stated purpose (e.g., a "weather" skill that tells the agent to read files in `~/.ssh`). - Does it instruct the agent to fetch and follow remote content at runtime? That is indirect prompt injection — the remote author gains control over your agent. - Does it ask the agent to print, echo, or reveal credentials, API keys, or tokens? ### 3. Every script and executable Scripts run with your agent's permissions. Read each one before it ever executes. - **Network calls**: what hosts does it reach out to? Unknown domains, IP-literal URLs, typosquatted package names, `curl | bash` patterns, and password-protected archives are red flags. - **File system access**: does it read or write outside its own skill directory? Pay special attention to dotfiles, SSH keys, shell profiles, credential stores, and config files. - **Credential handling**: hardcoded secrets, keys passed on command lines, instructions to store secrets in plaintext. - **Dynamic behavior**: remote imports, downloads that execute, obfuscated or minified code you cannot read. - **Dependencies**: are they named, pinned, and from known sources? Executables requiring elevated privileges deserve extra scrutiny. ### 4. References and assets Files in `references/` and `assets/` can carry instructions too. Scan them for the same patterns as the SKILL.md: remote fetching, hidden instructions, secrets. ## Safe first-run practice Even after a clean review, run it the way you would run any untrusted dependency the first time: - Run in a sandboxed environment (container, VM, or disposable profile) where the agent has no real credentials. - Run with an empty or minimal environment — no production API keys, no personal tokens. - Prefer read-only access to the file system where the harness supports it. - Watch what it actually does before granting it your real working context. ## If you find a problem - Do **not** run the skill further, and do not "fix and continue" silently. - Report it: the registry or marketplace the skill came from, and the author if contactable. Malicious skills are a supply-chain incident, not a code review finding. - If the skill is part of a dependency chain you already run, treat discovery like any other vulnerability: assess exposure, rotate any credentials the skill could have seen, and record what you know. ## Gotchas - **An open standard is not a safety standard.** Format compliance says nothing about intent. A perfectly schema-valid SKILL.md can be malware. - **Curation is not an audit.** A "top skills" list measures popularity, not safety. - **LLM-generated skills need the same review as stranger's skills.** The model that wrote the skill is not a security reviewer, and generated skills often contain plausible-looking but unverified instructions. - **Re-vet on updates.** A skill you vetted at v1.0 is a new dependency at v2.0. Diff the change before letting it back in.
-
-
README.md 2.5 KB
# Agent Skills — The Standard Itself This skill is the **meta-skill**: it documents the [Agent Skills](https://agentskills.io) open format itself. Every other skill in this repository follows the conventions defined here. ## Why Install This Skill When your agent loads this skill, it gains the ability to **create, review, and edit valid Agent Skills-format skills**. That means: - **Your agent can build new skills on demand.** Describe a workflow you want encoded, and your agent can scaffold a valid SKILL.md with correct frontmatter, progressive disclosure structure, and agent-friendly descriptions. - **Existing skills stay valid.** When your agent edits a skill, it checks against the format standard — no broken frontmatter, no missing required fields. - **Consistency across your skill library.** Every skill in your repository follows the same conventions, making them loadable by any Agent Skills-compatible harness. ## What You Get | Directory | Purpose | |-----------|---------| | `SKILL.md` | A create-and-review workflow, the core format rules, and a completion checklist | | `references/specification.md` | A bundled snapshot of the official format specification | | `references/best-practices.md` | Guidance for useful, well-scoped instructions | | `references/optimizing-descriptions.md` | How to design and test reliable automatic triggers | | `references/using-scripts.md` | How to bundle safe, agent-friendly executable helpers | | `references/vetting-third-party-skills.md` | How to vet a skill from a registry, colleague, or LLM before running it | | `references/evaluating-skills.md` | A practical evaluation loop for testing skill quality | | `references/client-implementation.md` | Guidance for products that discover and load skills | ## Quick Start Ask your agent to create or review a skill, then load this skill first. For a structural check, run: ```bash skills-ref validate path/to/skill ``` If the reference validator is not installed, your agent should report that clearly and complete the equivalent manual checks. ## Triggers - You're creating a new skill and need to know the correct format - You're reviewing an existing skill for format compliance - You want your agent to generate a valid SKILL.md scaffold - You're setting up conventions for a shared skill repository ## Requirements The documentation itself has no runtime dependencies. Optional validation uses the `skills-ref` reference library; install it in an isolated environment when you want to run its CLI. -
SKILL.md 11.9 KB
--- name: agent-skills description: >- Use this skill when creating, reviewing, or editing Agent Skills-format skills, or when implementing skill discovery and loading in an agent client. It covers directory structure, SKILL.md metadata, progressive disclosure, evals, and repository conventions. Do not use this skill for general software work that does not involve the Agent Skills format or lifecycle. license: MIT --- # Agent Skills Standard Reference This skill documents the [Agent Skills](https://agentskills.io) open format — a standardized way to give AI agents new capabilities and expertise. **Follow this workflow when creating or editing skills in this repository.** > Authoritative source: [agentskills.io/specification](https://agentskills.io/specification). The bundled specification is a working snapshot; check the authoritative source when currentness matters. --- ## Directory Structure A skill is a directory containing, at minimum, a `SKILL.md` file: ``` skill-name/ ├── SKILL.md # Required: metadata + instructions ├── evals/ # Required for new skills in this repository ├── scripts/ # Optional: executable code ├── references/ # Optional: documentation ├── assets/ # Optional: templates, resources └── ... # Any additional files or directories ``` ## Required Workflow ### Create or edit a skill 1. Read [the specification](references/specification.md) before changing `SKILL.md` metadata or directory structure. 2. Ground instructions in real domain knowledge, project artifacts, and observed failure modes. Read [best practices](references/best-practices.md) when designing or materially revising instructions. 3. Keep the skill a coherent, triggerable unit. Put only essential instructions in `SKILL.md`; put conditional detail in focused reference files and state exactly when to read each one. 4. Use a precise `description` that says what the skill does, when it applies, and when it does **not** apply. For skills with meaningful overlap, name the nearest alternative or prerequisite in a `## When not to use` section. Test the boundary with at least three should-trigger prompts and two should-not-trigger near-misses; keep these harness-specific trigger checks separate from portable output-quality evals. Read [optimizing descriptions](references/optimizing-descriptions.md) for trigger design. 5. For every new skill, create `evals/evals.json` with at least five representative output-quality cases. Each case needs a realistic prompt, an expected outcome, and observable assertions. Include edge cases that exercise risky or ambiguous behavior. Read [evaluating skills](references/evaluating-skills.md) for the eval format and iteration workflow. 6. When bundling executable code, read [using scripts](references/using-scripts.md). Document prerequisites and non-interactive invocation in the skill. 7. Before handoff, run the validation checks in this skill and correct every finding. ### Review a skill 1. Validate the required frontmatter, field constraints, parent-directory/name match, and YAML syntax against [the specification](references/specification.md). 2. Check that the description has both positive and negative trigger boundaries, the workflow is actionable, and resource references are conditional and reachable. 3. Check that any script has documented dependencies, safe non-interactive inputs, clear errors, and structured output where useful. 4. For this repository, also verify its required human-facing `README.md`: title, **Why Install This Skill**, **What You Get**, **Quick Start** (unless genuinely reference-only), **Triggers**, and **Requirements**. Keep it human-facing, concise, and free of agent-only instructions. 5. Verify every new skill has `evals/evals.json` with at least five output-quality cases, and check that their expected outcomes and assertions test meaningful behavior. For an existing skill without evals, flag the gap but do not block the review solely for that legacy absence. Read [evaluating skills](references/evaluating-skills.md) before declaring the work complete. ### Client implementation work When implementing skill discovery, activation, or context management in an agent product, read [client implementation guidance](references/client-implementation.md). Do not apply client conventions such as search paths as universal format requirements. ## SKILL.md Format The `SKILL.md` file must contain YAML frontmatter followed by Markdown body content. ### Frontmatter Fields | Field | Required | Constraints | |-------|----------|-------------| | `name` | Yes | Max 64 chars. Lowercase letters, numbers, and hyphens only. Must not start or end with a hyphen. **Must match the parent directory name.** | | `description` | Yes | Max 1024 chars. Non-empty. Describes what the skill does and when to use it. | | `license` | No | License name or reference to a bundled license file. | | `compatibility` | No | Max 500 chars. Indicates environment requirements. | | `metadata` | No | Arbitrary key-value mapping. | | `allowed-tools` | No | Space-separated string of pre-approved tools. (Experimental) | #### `name` field rules - 1–64 characters - Only lowercase unicode alphanumeric (`a-z`, `0-9`) and hyphens (`-`) - Must not start or end with a hyphen - Must not contain consecutive hyphens (`--`) - Must match the parent directory name #### `description` field rules - 1–1024 characters - Should describe both **what** the skill does and **when** to use it - Should include specific keywords that help agents identify relevant tasks #### `compatibility` field rules - If present, it must be 1–500 characters - Include it only for concrete environment requirements, such as a required product, system package, network access, or runtime #### `metadata` field rules - Must be a map of string keys to string values - Use reasonably unique keys to avoid collisions with other clients or tools ### Body Content The Markdown body has no format restrictions beyond being helpful to the agent. Recommended sections: - **Step-by-step instructions** — the procedure the agent should follow - **Examples of inputs and outputs** — what data looks like going in and coming out - **Common edge cases** — situations the agent might not handle correctly without guidance - **Gotchas** — environment-specific facts that defy reasonable assumptions Keep `SKILL.md` under **500 lines and 5000 tokens**. Move detailed reference material to separate files in `references/`. ## Progressive Disclosure Agents load skills in three stages: 1. **Metadata** (~100 tokens): `name` and `description` loaded at startup for all skills 2. **Instructions** (< 5000 tokens recommended): Full `SKILL.md` loaded when activated 3. **Resources** (as needed): Files in `scripts/`, `references/`, `assets/` loaded on demand ## Supporting Directories ### `scripts/` Executable code agents can run. Scripts should: - Be self-contained or clearly document dependencies - Include helpful error messages - Handle edge cases gracefully - Use relative paths from the skill root (e.g., `scripts/extract.py`) - Be explicit about intent: say "run this script" or "read this as reference" — never leave the agent to guess whether a bundled file is executable or illustrative ### `references/` Additional documentation loaded on demand. Keep individual files focused — agents load these when instructed, so smaller files save context. - Every file under `references/` must be at or under **60,000 characters**. When a reference grows past the cap, split it into focused files (e.g., `references/<topic>-a.md`, `references/<topic>-b.md`) and update `SKILL.md` so each new file is reachable and the load-on-demand instructions name the right file. ### `assets/` Static resources: templates, images, data files, schemas. ### `evals/` Portable output-quality cases for the skill. New skills in this repository must include `evals/evals.json` with at least five cases; trigger-only checks belong in the harness-specific test set instead of this file. ## File References Use **relative paths from the skill root** when referencing other files: ```markdown See [the specification](references/specification.md) for details. Run a bundled script: scripts/<script-name> ``` Keep file references one level deep from `SKILL.md`. Avoid deeply nested reference chains. ## Writing Effective Descriptions The `description` field is the primary mechanism for automatic skill selection. Clients can also support explicit activation. Follow these principles: - **Use imperative phrasing.** "Use this skill when..." rather than "This skill does..." - **Focus on user intent, not implementation.** Describe what the user is trying to achieve. - **Err on the side of being pushy.** Explicitly list contexts where the skill applies. - **Keep it concise.** A few sentences to a short paragraph is right. ## Best Practices ### Start from real expertise Feed domain-specific context into skill creation. Skills grounded in real project artifacts (runbooks, API specs, code review comments, actual failure cases) outperform ones synthesized from generic knowledge. ### Spend context wisely Focus on what the agent wouldn't know without the skill: project-specific conventions, domain-specific procedures, non-obvious edge cases. Don't explain general concepts the agent already knows. ### Calibrate control - **Give freedom** when multiple approaches are valid — describe *why*, not just *what* - **Be prescriptive** when operations are fragile or a specific sequence must be followed - **Match prescriptiveness to fragility**: for a step that must be exactly right every time, prefer a deterministic script in `scripts/` over instructions the agent improvises on each run. A test only catches what you already thought to check; a script takes the guess out of the loop entirely - **Provide defaults, not menus** — pick one approach, mention alternatives briefly - **Favor procedures over declarations** — teach *how to approach* a class of problems, not *what to produce* for one instance ### Design coherent units Scope skills like functions: one coherent unit of work that composes well with other skills. Too narrow → multiple skills needed for one task. Too broad → hard to activate precisely. ### Use gotchas sections The highest-value content is often environment-specific corrections — things the agent will get wrong unless told otherwise. When an agent makes a mistake, add the correction to the gotchas section. ### Provide output templates When output needs a specific format, provide a template inline or in `assets/`. Agents pattern-match well against concrete structures. ## Adopting Third-Party Skills Skills are executable capability: when a skill activates, its instructions and scripts run with the agent's permissions — shell access, file system, and credentials. Before running a skill you did not author (from a registry, a colleague, or an LLM generation), read [vetting third-party skills](references/vetting-third-party-skills.md) and treat it like any other dependency: inspect provenance, read the body and every script, and check what the skill reaches out to. ## Validation Use the [skills-ref](https://github.com/agentskills/agentskills/tree/main/skills-ref) reference library to validate skills: ```bash skills-ref validate ./my-skill ``` This checks that `SKILL.md` frontmatter is valid and follows all naming conventions. For this repository, also run the bundled whole-repository checker: ```bash ruby scripts/validate-skills.rb ``` It checks canonical top-level and bundle skills for frontmatter, supported fields, line limits, local links, and required README sections. Vendored profile skills under `agent-council/profiles/skills/` are intentionally excluded because they follow the source repository's conventions. If `skills-ref` is unavailable, do not claim a successful validator run. Perform and report the equivalent structural checks manually, or install and run the reference validator when the task permits it.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.