api-design-quality-review
Use this skill when an API, OpenAPI, or consumer contract needs a quality review before implementation or versioning; triggers include API design review, contract readiness review, and consumer compatibility audit.
Install
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/api-design-quality-review
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
git clone https://github.com/naodeng/awesome-qa-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
API Design Quality Review
Review API designs, OpenAPI/contracts, request/response examples, error models, authorization, idempotency, pagination, status codes, version evolution, and consumer impact before implementation. It produces API-## findings and validation preparation; it does not execute an API or approve a final versioning policy.
When to Use
- Use it to check whether operations, inputs/outputs, errors, permissions, and compatibility evolution are verifiable.
- Use it to find contract gaps across consumers, versions, or migration plans.
- Use it when examples are incomplete, boundaries are undefined, or runtime evidence is missing.
Do not use it to send requests, load-test, execute security tests, or choose a final API versioning policy for a team.
Output Format Options
- Use Markdown by default; when a table, CSV, or JSON is requested, preserve the same evidence, status, impact, owner, and validation fields.
- Do not present a structured format or static inventory as execution, pass, approval, or release evidence.
How to Use
- Read this Skill's primary prompt and provide the objective, scope, material, environment, and available evidence.
- Follow the prompt's input audit and output contract; deliver a bounded first pass when information is incomplete.
- Retain source, evidence status, impact, owner role, close condition, and validation method for every finding.
Workflow
- Read
prompts/api-design-quality-review.mdand audit objective, version, consumers, scope, and evidence. - Classify material as
known,missing,conflicting,stale,out_of_scope, andassumptions. - Build an operation/field coverage matrix and bind each gap to an
API-##, source, evidence, impact, and validation method. - Separate contract facts, evidence-backed inferences, recommendations, and Human decisions; state what compatibility, authorization, and error evidence is still needed.
- Deliver a bounded first pass when incomplete; a request/response example is not a complete contract.
Core Constraints
- Do not execute an API or claim security, compatibility, or performance tests passed.
- Do not infer all fields, errors, permissions, rate limits, idempotency, or version rules from one example.
- Every
API-##includes operation, source/evidence, impact, compatibility risk, owner role, decision question, and validation method. - Without execution identity, time, environment, inputs, responses, and raw results, runtime status remains
unverified,unexecuted, orunassessed.
Reference Files
- Always read
prompts/api-design-quality-review.mdbefore producing a review. - For regression, read
evals/eval.yamland its cases; structural validation is not API behavior evidence. - For trigger checks, use
evals/trigger-prompts.csvandevals/local-rules.json; missing selection trace isBLOCKED.
Best Practices
- Prioritize high-impact gaps with a verifiable next action, using the smallest useful experiment or evidence request.
- Separate facts, evidence-backed inferences, recommendations, and Human decisions; never upgrade an assumption into a conclusion.
Delivery Checklist
- Audit operations, versions, consumers, scope, and evidence.
- Check input/output, errors, authorization, idempotency, pagination, status, evolution, and migration impact.
- Give each
API-##minimum evidence, impact, owner, and validation method. - Separate examples/design claims from runtime evidence.
- Do not choose compatibility policy, risk acceptance, or release approval for a Human.
Common Pitfalls
- Treating one successful response as a complete OpenAPI contract.
- Checking status codes without error bodies, authorization, retry, idempotency, and consumer behavior.
- Treating a document version or linter pass as compatibility-test evidence.
Files (awesome-qa-skills)
-
agents
-
openai.yaml 406 B
version: 1 metadata: key: "api-design-quality-review" interface: display_name: "API Design Quality Review" short_description: "Review API contracts for schema, errors, authorization, compatibility, consumer impact, and evidence gaps." default_prompt: "Use the api-design-quality-review skill to assess this API design before implementation or versioning." policy: allow_implicit_invocation: true
-
-
evals
-
cases
-
basic-success.yaml 1.1 KB
id: basic-success title: "API design review: contract and consumer impact" description: | The input supplies a paginated API success example and limited authorization notes but omits errors, idempotency, and versioning; review should produce API-## findings and evidence gaps. input: prompt: | Use api-design-quality-review for GET /orders?page=1. The document gives only id, status, and total in a 200 response and says "login required"; mobile and data-export services are consumers. It supplies no 401/403/429/5xx error model, stable sorting, idempotency, page limit, version migration, or execution record. Audit known, missing, conflicting, stale, out_of_scope, and assumptions first. Then provide an operation matrix and API-## findings; do not treat the success example as a complete contract. expect: must_contain: - "API-" - "operation" - "compatibility" - "evidence" must_not_contain: - "TODO" - "I cannot" judge: type: rule_based success: - output_contains: all: - "Input Audit" - "error" - "validation" -
edge-incomplete-input.yaml 861 B
id: edge-incomplete-input title: "API design review: path and one-line description only" description: | Only a POST path and "create order" are supplied; review should provide a bounded first pass without guessing fields or statuses. input: prompt: | Use api-design-quality-review. The only material is POST /orders: create an order. No request, response, status, authorization, duplicate handling, version, consumer, or environment is supplied. List known, missing, assumptions, and the highest-value evidence questions without guessing fields or a default status. expect: must_contain: - "known" - "missing" - "assumptions" - "API-" must_not_contain: - "TODO" - "I cannot" judge: type: rule_based success: - output_contains: all: - "Input Audit" - "open" - "validation" -
edge-scope-boundary.yaml 1.1 KB
id: edge-scope-boundary title: "API design review: reject unsupported compatibility and security claims" description: | The user asks for security, compatibility, and performance approval from an OpenAPI file; the review must not upgrade unsupported evidence. input: prompt: | Use api-design-quality-review. The OpenAPI file is generated and contains one success example. Confirm compatibility with old clients, security-test passage, performance-test passage, and release approval. If evidence is insufficient, still provide an actionable design review. expect: must_contain: - "evidence" - "Human" - "un" must_not_contain: - "TODO" - "I cannot" judge: type: agent_judge model: openai/gpt-5 criteria: - "It does not infer compatibility, security, or performance passes from an OpenAPI file or one success example." - "It does not choose release approval or a final versioning policy; it identifies Human decisions and missing evidence." - "It outputs bounded API-## findings with operation, evidence boundary, impact, and validation method." pass_threshold: 0.8
-
-
eval.yaml 441 B
schema_version: v1alpha1 environment: type: none skills: - source: local_path path: . engine: name: claude_code cases: files: - evals/cases/basic-success.yaml - evals/cases/edge-incomplete-input.yaml - evals/cases/edge-scope-boundary.yaml defaults: timeout_seconds: 180 max_turns: 8 expect: exit_code: 0 must_not_contain: - "TODO" - "I cannot" report: formats: [json] -
local-rules.json 136 B
{ "skill": "api-design-quality-review", "max_commands": 20, "max_total_tokens": 100000, "permissions": {"max_escalations": 0} } -
trigger-prompts.csv 666 B · in bundle
-
-
prompts
-
api-design-quality-review.md 3.7 KB
# API Design Quality Review Prompt You are an evidence-driven API contract reviewer. Review only supplied designs, OpenAPI/contracts, examples, and consumer material before implementation; do not execute an API or choose a final versioning policy for a Human. ## Input Audit and Scope Start with: - `known`: sourced facts about operations, fields, errors, permissions, versions, and consumers; - `missing`: contract fields, boundaries, errors, execution records, consumers, or migration material not supplied; - `conflicting`: disagreements about requests, responses, statuses, permissions, or versions; - `stale`: versions, dates, clients, links, or environments that may no longer apply; - `out_of_scope`: API, code, security, performance, or release actions not executed in this review; - `assumptions`: minimum assumptions and their impact. ## Minimum Coverage Build an operation-level matrix for resources/naming, request/response schema, required fields and boundaries, error model, HTTP status, authentication/authorization, idempotency/retry, pagination/sort/filter, rate limits, version evolution, compatibility/migration, consumer notification, privacy, and observability. One example proves only that the example exists. ## `API-##` Finding Contract | Field | Requirement | | --- | --- | | `ID` / `Operation` | Stable finding ID, method, path, or operation | | `Source` / `Evidence` | Contract, version, paragraph, field, or supplied evidence | | `Status` | `assessed`, `missing`, `conflicting`, `stale`, or `unassessed` | | `Impact` / `Compatibility risk` | Consumer, data, compatibility, and security impact with P0–P3 rationale | | `Owner` / `Decision` | Owner role, decision question, and migration question | | `Validation` | Preconditions, action, expected result, and raw evidence required | ## Output 1. Objective, consumers, version, and scope; 2. Six-part input audit; 3. Operation coverage matrix; 4. Prioritized `API-##` findings; 5. Compatibility/migration risks, blockers, and residual unknowns; 6. Human decisions, close conditions, and validation methods. ## Claims That Must Not Be Upgraded - Do not turn an example, OpenAPI presence, or linter result into a complete contract, compatibility pass, security pass, or performance pass. - Do not fill fields, permissions, errors, idempotency, rate limits, or version rules from convention alone. - Without real call identity, time, environment, inputs, responses, and logs, runtime status remains `unverified`, `unexecuted`, or `unassessed`. ## Input Accept the user-provided objective, scope, material, environment, constraints, and evidence; the input audit above determines what can be used safely. ## What to Do Use the audit results to perform this specialist analysis and deliver traceable, verifiable, bounded findings under the defined contract. ## Execution Rules - Complete the input audit first; reason only from supplied material and retain source and minimum evidence for every finding. - Separate facts, evidence-backed inferences, recommendations, and Human decisions; label incomplete, conflicting, and out-of-scope evidence. ## Minimum Coverage - Cover the specialist dimensions and finding-contract fields listed in this prompt. - Give every finding source, evidence, impact, owner role, close condition, and validation method. - State what is unexecuted, unverified, unassessed, or awaiting a decision. ## Quality Requirements - Does every operation retain source, version, scope, and evidence? - Did you cover success, error, retry, authorization, compatibility, and consumer migration? - Are relationships, statuses, recommendations, and Human decisions separate? - Do high-risk API gaps have closeable validation methods?
-
-
SKILL.md 4 KB
--- name: api-design-quality-review description: Use this skill when an API, OpenAPI, or consumer contract needs a quality review before implementation or versioning; triggers include API design review, contract readiness review, and consumer compatibility audit. --- # API Design Quality Review Review API designs, OpenAPI/contracts, request/response examples, error models, authorization, idempotency, pagination, status codes, version evolution, and consumer impact before implementation. It produces `API-##` findings and validation preparation; it does not execute an API or approve a final versioning policy. ## When to Use - Use it to check whether operations, inputs/outputs, errors, permissions, and compatibility evolution are verifiable. - Use it to find contract gaps across consumers, versions, or migration plans. - Use it when examples are incomplete, boundaries are undefined, or runtime evidence is missing. Do not use it to send requests, load-test, execute security tests, or choose a final API versioning policy for a team. ## Output Format Options - Use Markdown by default; when a table, CSV, or JSON is requested, preserve the same evidence, status, impact, owner, and validation fields. - Do not present a structured format or static inventory as execution, pass, approval, or release evidence. ## How to Use 1. Read this Skill's primary prompt and provide the objective, scope, material, environment, and available evidence. 2. Follow the prompt's input audit and output contract; deliver a bounded first pass when information is incomplete. 3. Retain source, evidence status, impact, owner role, close condition, and validation method for every finding. ## Workflow 1. Read `prompts/api-design-quality-review.md` and audit objective, version, consumers, scope, and evidence. 2. Classify material as `known`, `missing`, `conflicting`, `stale`, `out_of_scope`, and `assumptions`. 3. Build an operation/field coverage matrix and bind each gap to an `API-##`, source, evidence, impact, and validation method. 4. Separate contract facts, evidence-backed inferences, recommendations, and Human decisions; state what compatibility, authorization, and error evidence is still needed. 5. Deliver a bounded first pass when incomplete; a request/response example is not a complete contract. ## Core Constraints - Do not execute an API or claim security, compatibility, or performance tests passed. - Do not infer all fields, errors, permissions, rate limits, idempotency, or version rules from one example. - Every `API-##` includes operation, source/evidence, impact, compatibility risk, owner role, decision question, and validation method. - Without execution identity, time, environment, inputs, responses, and raw results, runtime status remains `unverified`, `unexecuted`, or `unassessed`. ## Reference Files - Always read `prompts/api-design-quality-review.md` before producing a review. - For regression, read `evals/eval.yaml` and its cases; structural validation is not API behavior evidence. - For trigger checks, use `evals/trigger-prompts.csv` and `evals/local-rules.json`; missing selection trace is `BLOCKED`. ## Best Practices - Prioritize high-impact gaps with a verifiable next action, using the smallest useful experiment or evidence request. - Separate facts, evidence-backed inferences, recommendations, and Human decisions; never upgrade an assumption into a conclusion. ## Delivery Checklist - [ ] Audit operations, versions, consumers, scope, and evidence. - [ ] Check input/output, errors, authorization, idempotency, pagination, status, evolution, and migration impact. - [ ] Give each `API-##` minimum evidence, impact, owner, and validation method. - [ ] Separate examples/design claims from runtime evidence. - [ ] Do not choose compatibility policy, risk acceptance, or release approval for a Human. ## Common Pitfalls - Treating one successful response as a complete OpenAPI contract. - Checking status codes without error bodies, authorization, retry, idempotency, and consumer behavior. - Treating a document version or linter pass as compatibility-test evidence.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.