plugin-review
Review plugin quality with tiered checks and dependency scoping. Use for PR and pre-release audits.
Install
npx skills add https://github.com/athola/claude-night-market/tree/master/plugins/abstract/skills/plugin-review
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install athola-claude-night-market@llmmart
git clone https://github.com/athola/claude-night-market.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole athola/claude-night-market collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Plugin Review
Overview
Tiered quality review of plugins with dependency-aware scoping.
When NOT To Use
- Reviewing one skill rather than a plugin (use
abstract:skills-eval) - Reviewing hooks (use
abstract:hooks-eval) - Tracing
Skill()references across plugins (useabstract:skill-graph-audit)
Tiers
| Tier | Trigger | Scope | Depth | Duration |
|---|---|---|---|---|
| branch | Default | Affected and related | Quick gates | ~2 min |
| pr | Before merge | Affected and related | Standard | ~5 min |
| release | Before version bump | All 17 plugins | Full | ~15 min |
Orchestration
- Detect scope: parse
--tierflag, find affected plugins from git diff, resolve related plugins fromdocs/plugin-dependencies.json - Plan: build check matrix (tier x plugin x role)
- Execute: run checks per tier definition
- Report: per-plugin table, aggregate verdict
Scope Detection
Affected plugins: git diff main --name-only filtered to
plugins/*/.
Related plugins: load docs/plugin-dependencies.json,
look up each affected plugin's reverse index to find
dependents. Mark as "related" (lighter checks).
If --tier release or no git diff available, scope to
all plugins.
Module Loading
- Always: this SKILL.md (orchestration logic)
- branch tier: load
modules/tier-branch.md - pr tier: load
modules/tier-branch.mdthenmodules/tier-pr.md - release tier: load all tier modules plus
modules/tier-release.md - When resolving deps: load
modules/dependency-detection.md
Verdict
| Result | Meaning |
|---|---|
| PASS | All checks green |
| PASS-WITH-WARNINGS | Non-blocking issues |
| FAIL | Blocking issues found |
Output Format
Plugin Review (<tier> tier)
Affected: <list>
Related: <list> (<reason>)
Plugin test lint type reg verdict
<name> PASS PASS PASS PASS PASS
...
Verdict: <PASS|PASS-WITH-WARNINGS|FAIL> (N/N plugins healthy)
PR and release tiers add scorecard sections.
Quality Gate Mode
The --quality-gate flag enables CI/CD integration with
exit codes that distinguish warnings from failures:
0: all quality gates passed1: warnings present but gates passed (non-blocking)2: quality gate failures (blocking)3: critical issues found (blocking)
Use --fail-on warning to treat warnings as blocking.
Configuration
Place a .plugin-review.yaml file in the plugin root
to customize thresholds and focus areas:
plugin_review:
quality_gates:
structure_min: 80
skills_min: 75
hooks_min: 70
tokens_max_total: 50000
bloat_max_percentage: 15
focus_areas:
- skills
- hooks
- tokens
exclude_patterns:
- "*/legacy/*"
- "*/deprecated/*"
severity_overrides:
missing_description: warning
large_file: info
See the /plugin-review command reference for full
usage examples.
Exit Criteria
- The output includes a per-plugin table with columns: test, lint, type, reg, verdict, and an aggregate PASS / PASS-WITH-WARNINGS / FAIL verdict.
- Scope detection identifies affected plugins via
git diff main --name-onlyfiltered toplugins/*/; if git diff is unavailable, the skill falls back to all-plugin scope and states this explicitly. - Any plugin scoring below the
structure_min(default 80) orskills_min(default 75) threshold is listed as a FAIL, not a warning. -
--quality-gatemode returns exit code 2 for blocking failures and exit code 1 for non-blocking warnings, distinguishable by the calling CI step.
Files (claude-night-market)
-
modules
-
dependency-detection.md 1.5 KB
# Dependency Detection How to resolve affected and related plugins for scoped review. ## Affected Plugins Detect from git diff against the base branch: ```bash git diff main --name-only | \ grep '^plugins/' | \ sed 's|^plugins/\([^/]*\)/.*|\1|' | \ sort -u ``` If no diff available (detached HEAD, no main branch), fall back to all plugins. ## Related Plugins Load `docs/plugin-dependencies.json` and for each affected plugin, look up who depends on it: 1. Read the `dependencies` section to find the affected plugin 2. If `dependents` is `["*"]`, ALL other plugins are related 3. Otherwise, `dependents` lists the specific related plugins Example: if `abstract` is affected and has `dependents: ["*"]`, then all 16 other plugins are related. If `leyline` is affected and has `dependents: ["conjure"]`, then only conjure is related. ## Deduplication A plugin that is both affected (has direct changes) AND related (depends on another changed plugin) is treated as affected. The "affected" role always wins. ## Special Cases - **Root-level changes** (pyproject.toml, Makefile, scripts/): These affect the workspace, not individual plugins. At branch tier, skip. At pr/release tier, flag for manual review. - **No plugins changed**: If the diff has no plugin changes, report "No plugin changes detected" and exit with PASS. - **New plugin**: If a plugin directory exists on disk but is not in `plugin-dependencies.json`, flag it and suggest running `scripts/generate_dependency_map.py` to update. -
tier-branch.md 2.3 KB
# Branch Tier Checks The branch tier runs quick quality gates on affected and related plugins. Designed for fast feedback during development. ## Checks for Affected Plugins Run these sequentially for each affected plugin: ### 1. Plugin Structure Validation Run the plugin validator to check plugin.json schema, skill/agent frontmatter, hook event types, and path references against the official Claude Code spec: ```bash python3 plugins/abstract/scripts/validate_plugin.py \ plugins/<plugin> ``` If critical issues found, mark plugin as FAIL. Warnings are non-blocking at branch tier. ### 2. Registration Audit **Only available in night-market repo.** Check script exists first: ```bash # Check if the script exists (night-market only) if [[ -f "plugins/sanctum/scripts/update_plugin_registrations.py" ]]; then python3 plugins/sanctum/scripts/update_plugin_registrations.py \ <plugin-name> --dry-run else echo "Registration audit skipped - not running in night-market" fi ``` Report any missing or stale registrations. If skipped, note in the result table that registration audit is unavailable. ### 3. Test Gate ```bash cd plugins/<plugin> && make test ``` Capture pass/fail and test count. If tests fail, mark plugin as FAIL. ### 4. Lint Gate ```bash cd plugins/<plugin> && make lint ``` Capture pass/fail. If lint fails, mark as WARNING (not blocking at branch tier). ### 5. Typecheck Gate ```bash cd plugins/<plugin> && make typecheck ``` Capture pass/fail. If typecheck fails, mark as WARNING. ### 6. Diff Analysis Run `git diff main -- plugins/<plugin>/` to identify: - New files (commands, skills, hooks, agents) - Deleted files - Modified production code vs test-only changes Flag high-risk patterns: - Hook changes (security surface) - `__init__.py` export changes (API surface) - pyproject.toml dependency changes ## Checks for Related Plugins Run a lighter subset: ### 1. Registration Audit Same as affected plugins. ### 2. Test Gate ```bash cd plugins/<plugin> && make test ``` Tests must pass. This catches side-effect breakage from dependency changes. ## Result Table Build a table with columns: plugin, test, lint, type, reg, verdict. Use `--` for skipped checks on related plugins. ## Verdict Rules - Any test FAIL on affected or related: overall FAIL - Any lint/type FAIL on affected: overall PASS-WITH-WARNINGS - All green: PASS -
tier-pr.md 1.9 KB
# PR Tier Checks The PR tier adds quality scoring on top of branch tier gates. Runs before merge. ## Additional Checks for Affected Plugins After all branch tier checks pass, run these: ### 1. Skills Evaluation Invoke `Skill(abstract:skills-eval)` for each affected plugin. Capture: - Per-skill quality scores - Token efficiency ratings - Compliance status Report any skill scoring below 70 (MINIMUM_QUALITY_THRESHOLD). ### 2. Hooks Evaluation If the plugin has hooks that changed in the diff: Invoke `Skill(abstract:hooks-eval)` targeting only changed hook files. Capture: - Security findings - Performance concerns - Compliance status ### 3. Test Review Invoke `Skill(pensive:test-review)` for each affected plugin. Capture: - Coverage percentage - Anti-pattern findings - Missing edge cases ### 4. Quick Bloat Scan Invoke `Skill(conserve:bloat-detector)` at Tier 1 (quick) for each affected plugin. Capture: - Dead code candidates - Duplicate content - Stale files ### 5. Rules Evaluation If `.claude/rules/` files changed in the diff: Invoke `Skill(abstract:rules-eval)`. Capture: - YAML validity - Pattern specificity - Content quality ## Parallel Execution Group affected plugins into 2-3 clusters and dispatch agents in parallel. Each agent runs the full PR check suite on its assigned plugins. ## Scorecard Output Add a scorecard section after the branch tier table: ``` Scorecard (PR tier) Plugin skills hooks tests bloat grade sanctum 88/100 92/100 93% 90/100 A- memory-palace 85/100 -- 89% 82/100 B+ Top 5 Remediation Actions: 1. [sanctum] skills/commit-messages: missing trigger phrases 2. [memory-palace] hooks/research_interceptor.py: no timeout ... ``` ## Verdict Rules - Any branch-tier FAIL: overall FAIL - Any skill score below 50: overall FAIL - Any security finding (HIGH): overall FAIL - All scores above 70: PASS - Otherwise: PASS-WITH-WARNINGS with details -
tier-release.md 2.9 KB
# Release Tier Checks The release tier runs a full ecosystem audit across all 17 plugins. Requires plan mode for parallel agent dispatch. ## Scope All 17 plugins, regardless of git diff. This is the pre-release validation gate. ## Additional Checks (Beyond PR Tier) ### 1. Architecture Review Invoke `Skill(pensive:architecture-review)` to assess: - ADR compliance across all plugins - Cross-plugin coupling analysis - Module boundary violations - Design pattern adherence ### 2. Unified Review Invoke `Skill(pensive:unified-review)` for multi-domain orchestration covering API, architecture, test, and code quality dimensions. ### 3. Deep Bloat Scan Invoke `Skill(conserve:bloat-detector)` at Tier 2-3 (deep) for full dead code analysis, cross-file duplication detection, and dependency audit. ### 4. Token Efficiency Analysis For each plugin, calculate: - Total skill description budget usage - Per-skill token estimates - Progressive loading compliance - Module size distribution Flag any plugin exceeding its proportional token budget allocation. ### 5. Meta-Evaluation Run `python3 plugins/sanctum/scripts/meta_evaluation.py` to validate that evaluation-related skills (skills-eval, hooks-eval, rules-eval) meet their own quality standards. ### 6. Cross-Plugin Dependency Validation Regenerate the dependency map and compare with the committed version: ```bash python3 scripts/generate_dependency_map.py --stdout | \ diff - docs/plugin-dependencies.json ``` Flag any drift between actual and documented dependencies. ## Agent Dispatch Plan Requires plan mode (4+ agents rule). Suggested grouping: | Agent | Plugins | Focus | |-------|---------|-------| | 1 | abstract, leyline | Foundation layer | | 2 | sanctum, imbue, conserve | Core infrastructure | | 3 | attune, conjure, hookify | Workflow plugins | | 4 | pensive, memory-palace, minister | Domain specialists | | 5 | parseltongue, egregore, spec-kit | Language/pipeline | | 6 | scribe, scry, archetypes | Media/docs/patterns | Each agent runs full PR-tier checks on its plugins plus the release-specific checks. ## Full Ecosystem Report ``` Plugin Review (release tier) - vX.Y.Z Date: YYYY-MM-DD Plugins: 17/17 ECOSYSTEM HEALTH Plugin test lint type skills hooks bloat grade abstract PASS PASS PASS 92 88 90 A attune PASS PASS PASS 85 -- 82 B+ conjure PASS PASS PASS 88 85 87 A- ... ARCHITECTURE ADR compliance: 7/7 Coupling score: 0.06 Boundary violations: 0 TOKEN BUDGET Used: 15,200 / 17,000 chars (89.4%) Largest: attune (3,920 chars, 23.1%) DEPENDENCY MAP Status: current (no drift) Verdict: PASS (17/17 plugins healthy) ``` ## Verdict Rules - Any test FAIL: overall FAIL - Any security finding (HIGH): overall FAIL - ADR violation without justification: overall FAIL - Coupling score > 0.7: FAIL - All plugins grade B or above: PASS - Otherwise: PASS-WITH-WARNINGS
-
-
SKILL.md 3.9 KB
--- name: plugin-review description: 'Review plugin quality with tiered checks and dependency scoping. Use for PR and pre-release audits.' alwaysApply: false category: plugin-management tags: - review - quality - validation - testing - architecture dependencies: - skills-eval - hooks-eval - rules-eval tools: - validate_plugin.py - skill_analyzer.py progressive_loading: true model_hint: standard --- # Plugin Review ## Overview Tiered quality review of plugins with dependency-aware scoping. ## When NOT To Use - Reviewing one skill rather than a plugin (use `abstract:skills-eval`) - Reviewing hooks (use `abstract:hooks-eval`) - Tracing `Skill()` references across plugins (use `abstract:skill-graph-audit`) ## Tiers | Tier | Trigger | Scope | Depth | Duration | |------|---------|-------|-------|----------| | branch | Default | Affected and related | Quick gates | ~2 min | | pr | Before merge | Affected and related | Standard | ~5 min | | release | Before version bump | All 17 plugins | Full | ~15 min | ## Orchestration 1. **Detect scope**: parse `--tier` flag, find affected plugins from git diff, resolve related plugins from `docs/plugin-dependencies.json` 2. **Plan**: build check matrix (tier x plugin x role) 3. **Execute**: run checks per tier definition 4. **Report**: per-plugin table, aggregate verdict ## Scope Detection Affected plugins: `git diff main --name-only` filtered to `plugins/*/`. Related plugins: load `docs/plugin-dependencies.json`, look up each affected plugin's reverse index to find dependents. Mark as "related" (lighter checks). If `--tier release` or no git diff available, scope to all plugins. ## Module Loading - **Always**: this SKILL.md (orchestration logic) - **branch tier**: load `modules/tier-branch.md` - **pr tier**: load `modules/tier-branch.md` then `modules/tier-pr.md` - **release tier**: load all tier modules plus `modules/tier-release.md` - **When resolving deps**: load `modules/dependency-detection.md` ## Verdict | Result | Meaning | |--------|---------| | PASS | All checks green | | PASS-WITH-WARNINGS | Non-blocking issues | | FAIL | Blocking issues found | ## Output Format ``` Plugin Review (<tier> tier) Affected: <list> Related: <list> (<reason>) Plugin test lint type reg verdict <name> PASS PASS PASS PASS PASS ... Verdict: <PASS|PASS-WITH-WARNINGS|FAIL> (N/N plugins healthy) ``` PR and release tiers add scorecard sections. ## Quality Gate Mode The `--quality-gate` flag enables CI/CD integration with exit codes that distinguish warnings from failures: - `0`: all quality gates passed - `1`: warnings present but gates passed (non-blocking) - `2`: quality gate failures (blocking) - `3`: critical issues found (blocking) Use `--fail-on warning` to treat warnings as blocking. ## Configuration Place a `.plugin-review.yaml` file in the plugin root to customize thresholds and focus areas: ```yaml plugin_review: quality_gates: structure_min: 80 skills_min: 75 hooks_min: 70 tokens_max_total: 50000 bloat_max_percentage: 15 focus_areas: - skills - hooks - tokens exclude_patterns: - "*/legacy/*" - "*/deprecated/*" severity_overrides: missing_description: warning large_file: info ``` See the `/plugin-review` command reference for full usage examples. ## Exit Criteria - [ ] The output includes a per-plugin table with columns: test, lint, type, reg, verdict, and an aggregate PASS / PASS-WITH-WARNINGS / FAIL verdict. - [ ] Scope detection identifies affected plugins via `git diff main --name-only` filtered to `plugins/*/`; if git diff is unavailable, the skill falls back to all-plugin scope and states this explicitly. - [ ] Any plugin scoring below the `structure_min` (default 80) or `skills_min` (default 75) threshold is listed as a FAIL, not a warning. - [ ] `--quality-gate` mode returns exit code 2 for blocking failures and exit code 1 for non-blocking warnings, distinguishable by the calling CI step.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.