phx-audit
Project health audit and health check — architecture, performance, tests,
Install
npx skills add https://github.com/oliver-kriska/claude-elixir-phoenix/tree/main/targets/amp/skills/phx-audit
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install oliver-kriska-claude-elixir-phoenix@llmmart
git clone https://github.com/oliver-kriska/claude-elixir-phoenix.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole oliver-kriska/claude-elixir-phoenix collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Project Health Audit
Comprehensive project-wide health assessment across five independent concern tracks.
Usage
phx-audit # Full audit (default)
phx-audit --quick # 2-3 minute pulse check
phx-audit --focus=security # Deep dive single area
phx-audit --focus=performance
phx-audit --since abc123 # Incremental audit since commit
phx-audit --since HEAD~10 # Audit last 10 commits
When to Use
- Quarterly health checks
- Before major releases
- After large refactors
- New team member onboarding (understand codebase health)
Iron Laws
- Complete every selected track before synthesizing — partial results make cross-category scores misleading
- Scope each track to concrete directories and checks — vague project-wide analysis produces generic findings
- Never compare scores across projects — track trends only within the same codebase
- Run quick mode before full mode — catch basic failures before expensive analysis
Portable Audit Workflow
- Create
.claude/audit/reports/and.claude/audit/summaries/. - Run the quick checks below. Stop and report a blocker when the project cannot compile or its test command cannot start.
- Complete five tracks: architecture, performance, security, tests, and dependencies. Native generic workers may run independent tracks in parallel when the runtime provides them; otherwise run every track sequentially in this session. Never require named custom agents.
- Write one evidence-focused report per track under
.claude/audit/reports/. Report issues only, cite paths and lines, and use one summary line for a clean area. - After all selected reports exist, deduplicate findings, identify
cross-category correlations, calculate scores using
references/scoring-methodology.md, and write.claude/audit/summaries/project-health-{date}.md.
If two or more optional workers fail or hit limits, finish the missing tracks sequentially. Never present an incomplete track as audited.
Output Format
Report an executive health score, per-category scores for Architecture, Performance, Security, Tests, and Dependencies, critical issues, top recommendations, and an Immediate/Short-term/Long-term action plan.
Quick Mode (--quick)
Only run essential checks (~2-3 minutes):
Run mix compile --warnings-as-errors, then mix hex.audit && mix deps.audit,
then mix xref graph --format stats, then mix test --trace 2>&1 | tail -20.
Skip: Full security scan, N+1 analysis, test quality metrics, architecture deep dive.
Focus Mode (--focus=area)
Run only the selected concern track with its deeper checks:
| Focus | Extra checks |
|---|---|
security |
Full OWASP review, Sobelow, manual authorization patterns |
performance |
Query plans, N+1 inventory, profiling evidence |
architecture |
Full xref graph, coupling matrix, cohesion |
tests |
Coverage by context, isolation, flaky-test indicators |
deps |
Vulnerabilities, licenses, maintenance status |
Incremental Mode (--since <commit>)
Analyze only changes since a specific commit. Useful for pre-merge checks:
Run git diff --name-only <commit>...HEAD to identify changed files, then run targeted audits on changed files only (skips full project scan).
Combines with other flags: phx-audit --since HEAD~5 --focus=security
Relationship to Other Commands
| Command | Scope | Frequency |
|---|---|---|
phx-review |
Changed files (diff) | Every PR |
phx-audit |
Entire project | Quarterly |
phx-boundaries |
Context structure | On-demand |
phx-verify |
Compile/test pass | Anytime |
References
references/scoring-methodology.md- How scores are calculatedreferences/architecture-checks.md- Detailed architecture criteria
Files (claude-elixir-phoenix)
-
references
-
architecture-checks.md 5.1 KB
# Architecture Checks Reference Detailed criteria for architecture health assessment. ## Context Health Matrix ### What to Check For each context in `lib/{app}/`: | Metric | Healthy Range | Red Flag | |--------|---------------|----------| | Modules per context | 3-15 | >20 or <2 | | Public API functions | 5-30 | >40 | | Schemas per context | 1-5 | >8 | | Fan-out (contexts called) | 1-4 | >6 | | Fan-in (called by contexts) | 1-6 | >10 | ### Commands ```bash # Module count per context for dir in lib/my_app/*/; do echo "$(basename $dir): $(find $dir -name '*.ex' | wc -l) modules" done # Public function count grep -r "^ def " lib/my_app/*.ex --include="*.ex" | wc -l # Schema count grep -rl "use Ecto.Schema" lib/my_app/ | wc -l # Fan-out via xref mix xref graph --format stats ``` ## Coupling Analysis ### Fan-Out (Efferent Coupling) How many other contexts does this context depend on? ```bash # For each context, count outgoing dependencies mix xref graph --label compile-connected --format dot | grep "my_app_accounts ->" ``` | Fan-Out | Assessment | |---------|------------| | 0-2 | Excellent - well isolated | | 3-4 | Good - reasonable dependencies | | 5-6 | Warning - consider splitting | | 7+ | Critical - "god context" | ### Fan-In (Afferent Coupling) How many contexts depend on this context? | Fan-In | Assessment | |--------|------------| | 0 | Dead code? Or utility only | | 1-4 | Good - clear responsibility | | 5-8 | Common utility - ensure stable API | | 9+ | Core abstraction - avoid changes | ## Cohesion Analysis ### Signs of Low Cohesion - Context name is generic ("Utils", "Helpers", "Services") - Functions don't share domain vocabulary - Multiple unrelated schemas in same context - Context has >50 public functions ### Assessment ```bash # Check for generic names ls lib/my_app/ | grep -E "utils|helpers|services|common|shared" # Large API surface for file in lib/my_app/*.ex; do funcs=$(grep -c "^ def " "$file" 2>/dev/null || echo 0) if [ "$funcs" -gt 30 ]; then echo "WARNING: $(basename $file) has $funcs public functions" fi done ``` ## Boundary Violations ### Types of Violations | Violation | Severity | Example | |-----------|----------|---------| | Direct Repo from web | High | `Web.Controller` calls `Repo.all` | | Cross-context schema import | Medium | `Orders` aliases `Accounts.User` | | Direct schema access | Medium | `%Accounts.User{}` outside Accounts | | Context calling _web module | High | Business logic → presentation | ### Detection ```bash # Repo calls outside contexts grep -rn "Repo\." lib/my_app_web/ --include="*.ex" # Cross-context schema aliases grep -rn "alias MyApp\." lib/my_app/ --include="*.ex" | grep -v "alias MyApp\.$(dirname)" ``` ## Circular Dependencies (Compile-Time) Runtime cycles (e.g., from `verified_routes()`) are benign and don't cause recompilation cascades. Only compile-time cycles affect build performance and are scored. ### Detection ```bash mix xref graph --format cycles --label compile ``` ### Assessment | Cycles | Assessment | |--------|------------| | 0 | Excellent | | 1-2 | Warning - analyze and fix | | 3+ | Critical - architectural issue | ### Resolution Patterns 1. **Extract shared module** - Move shared code to new context 2. **Behavior/protocol** - Define interface, implement separately 3. **Event-driven** - Replace direct calls with PubSub 4. **Merge contexts** - If truly coupled, they're one context ## Naming Conventions ### Module Naming | Pattern | Assessment | |---------|------------| | `MyApp.{Domain}.{Entity}` | Correct | | `MyApp.{Domain}Service` | Avoid "Service" suffix | | `MyApp.{Domain}Manager` | Avoid "Manager" suffix | | `MyApp.{Domain}Helper` | Move to domain context | ### Function Naming | Pattern | Assessment | |---------|------------| | `get_user/1` | Correct - raises | | `fetch_user/1` | Correct - returns tuple | | `list_users/0` | Correct | | `find_user/1` | Inconsistent - use get/fetch | | `user/1` | Too vague | ### Context API Surface Well-designed context has: ```elixir defmodule MyApp.Accounts do # Queries (list/get/fetch) def list_users(opts \\ []) def get_user!(id) def fetch_user(id) # Commands (create/update/delete) def create_user(attrs) def update_user(user, attrs) def delete_user(user) # Domain operations def authenticate(email, password) def verify_email(token) end ``` ## Output Format ```markdown ## Architecture Review ### Context Health Matrix | Context | Modules | Public API | Fan-Out | Fan-In | Assessment | |---------|---------|------------|---------|--------|------------| | Accounts | 5 | 12 | 2 | 4 | Healthy | | Orders | 18 | 45 | 8 | 3 | Too Large | | Shared | 2 | 8 | 0 | 12 | Utility OK | ### Boundary Violations | Type | Location | Severity | |------|----------|----------| | Direct Repo | post_controller.ex:45 | High | ### Circular Dependencies - None found ✅ ### Recommendations 1. **Split Orders context** - 18 modules is too large - Extract Fulfillment (shipping, tracking) - Extract Invoicing (billing, receipts) 2. **Fix boundary violation** - Move Repo call to context - `PostController.create/2` should call `Blog.create_post/1` ``` -
scoring-methodology.md 4.3 KB
# Scoring Methodology How health scores are calculated for each category. ## Score Ranges | Score | Grade | Status | Meaning | |-------|-------|--------|---------| | 90-100 | A | Excellent | No critical issues, minor improvements only | | 80-89 | B | Good | Few warnings, solid foundation | | 70-79 | C | Needs Attention | Some issues to address | | 60-69 | D | Needs Work | Multiple issues, prioritize fixing | | <60 | F | Critical | Significant problems, immediate action | ## Category Scoring ### Architecture (100 points) | Criterion | Points | Deductions | |-----------|--------|------------| | Context boundaries respected | 25 | -5 per violation | | Module naming consistency | 15 | -3 per inconsistency | | Fan-out <5 contexts per module | 15 | -5 per over-coupled module | | API surface reasonable (<30 funcs/context) | 15 | -5 per bloated context | | No compile-time circular dependencies | 15 | -10 per cycle | | Folder structure follows conventions | 15 | -5 per deviation | **Commands used:** ```bash mix xref graph --format stats mix xref graph --format cycles --label compile find lib -name "*.ex" -type f | wc -l ``` ### Performance (100 points) | Criterion | Points | Deductions | |-----------|--------|------------| | No N+1 patterns detected | 30 | -5 per N+1 | | Indexes for common queries | 20 | -5 per missing index | | Preloads used appropriately | 15 | -3 per missing preload | | No GenServer bottlenecks | 15 | -10 per bottleneck | | LiveView streams for large lists | 10 | -5 per regular assign list | | Queries avoid SELECT * | 10 | -2 per SELECT * | **Commands used:** ```bash grep -B5 -A5 "Enum.map" lib/ -r --include="*.ex" | grep "Repo\." grep -r "Repo.preload" lib/ --include="*.ex" grep -r "assign(socket" lib/my_app_web/live/ --include="*.ex" ``` ### Security (100 points) | Criterion | Points | Deductions | |-----------|--------|------------| | No sobelow critical issues | 30 | -15 per critical | | No sobelow high issues | 20 | -5 per high | | Authorization in all handle_events | 15 | -10 per missing auth | | No String.to_atom with input | 10 | -10 per violation | | No raw() with untrusted content | 10 | -10 per violation | | Secrets in runtime.exs only | 15 | -15 per hardcoded secret | **Commands used:** ```bash mix sobelow --exit medium 2>&1 || true grep -r "String.to_atom" lib/ --include="*.ex" grep -r "raw(" lib/ --include="*.ex" grep -r "handle_event" lib/my_app_web/live/ -A10 | grep -v "authorize\|permit" ``` ### Test Quality (100 points) | Criterion | Points | Deductions | |-----------|--------|------------| | Coverage >70% | 30 | -5 per 10% below 70% | | No flaky test patterns | 20 | -5 per Process.sleep in test | | Async: true where possible | 15 | -2 per missing async | | verify_on_exit! in Mox tests | 15 | -5 per missing | | Reasonable test duration (<30s avg) | 10 | -5 if slow | | Error paths tested | 10 | -5 if only happy path | **Commands used:** ```bash mix test --cover 2>&1 | tail -30 grep -r "Process.sleep" test/ --include="*.exs" grep -r "async: true" test/ --include="*.exs" grep -r "verify_on_exit!" test/ --include="*.exs" ``` ### Dependencies (100 points) | Criterion | Points | Deductions | |-----------|--------|------------| | No hex.audit vulnerabilities | 40 | -20 per vulnerability | | No deps.audit issues | 20 | -10 per issue | | No major version behind (>2) | 20 | -5 per outdated | | No unused dependencies | 10 | -3 per unused | | Version pinning appropriate | 10 | -5 if all loose | **Commands used:** ```bash mix hex.audit 2>&1 mix deps.audit 2>&1 mix hex.outdated 2>&1 ``` ## Overall Score Calculation ``` overall_score = ( architecture_score * 0.20 + performance_score * 0.25 + security_score * 0.25 + test_quality_score * 0.15 + dependencies_score * 0.15 ) ``` **Weighting rationale:** - Security and Performance weighted highest (25% each) - runtime impact - Architecture weighted at 20% - long-term maintainability - Tests and Dependencies at 15% each - important but less immediate ## Grade Assignment ``` if overall_score >= 90: grade = "A" elif overall_score >= 80: grade = "B" elif overall_score >= 70: grade = "C" elif overall_score >= 60: grade = "D" else: grade = "F" ``` ## Critical Issues Override Regardless of score, flag as CRITICAL if any: - Security vulnerability detected - Hardcoded secrets found - Compile warnings present - Test suite failing
-
-
SKILL.md 4 KB
--- name: phx-audit description: Project health audit and health check — architecture, performance, tests, dependencies, code quality. Use when assessing overall project health, before releases, or after refactors. --- # Project Health Audit Comprehensive project-wide health assessment across five independent concern tracks. ## Usage ``` phx-audit # Full audit (default) phx-audit --quick # 2-3 minute pulse check phx-audit --focus=security # Deep dive single area phx-audit --focus=performance phx-audit --since abc123 # Incremental audit since commit phx-audit --since HEAD~10 # Audit last 10 commits ``` ## When to Use - **Quarterly** health checks - **Before major releases** - **After large refactors** - **New team member onboarding** (understand codebase health) ## Iron Laws 1. **Complete every selected track before synthesizing** — partial results make cross-category scores misleading 2. **Scope each track to concrete directories and checks** — vague project-wide analysis produces generic findings 3. **Never compare scores across projects** — track trends only within the same codebase 4. **Run quick mode before full mode** — catch basic failures before expensive analysis ## Portable Audit Workflow 1. Create `.claude/audit/reports/` and `.claude/audit/summaries/`. 2. Run the quick checks below. Stop and report a blocker when the project cannot compile or its test command cannot start. 3. Complete five tracks: architecture, performance, security, tests, and dependencies. Native generic workers may run independent tracks in parallel when the runtime provides them; otherwise run every track sequentially in this session. Never require named custom agents. 4. Write one evidence-focused report per track under `.claude/audit/reports/`. Report issues only, cite paths and lines, and use one summary line for a clean area. 5. After all selected reports exist, deduplicate findings, identify cross-category correlations, calculate scores using `references/scoring-methodology.md`, and write `.claude/audit/summaries/project-health-{date}.md`. If two or more optional workers fail or hit limits, finish the missing tracks sequentially. Never present an incomplete track as audited. ## Output Format Report an executive health score, per-category scores for Architecture, Performance, Security, Tests, and Dependencies, critical issues, top recommendations, and an Immediate/Short-term/Long-term action plan. ## Quick Mode (`--quick`) Only run essential checks (~2-3 minutes): Run `mix compile --warnings-as-errors`, then `mix hex.audit && mix deps.audit`, then `mix xref graph --format stats`, then `mix test --trace 2>&1 | tail -20`. Skip: Full security scan, N+1 analysis, test quality metrics, architecture deep dive. ## Focus Mode (`--focus=area`) Run only the selected concern track with its deeper checks: | Focus | Extra checks | |-------|--------------| | `security` | Full OWASP review, Sobelow, manual authorization patterns | | `performance` | Query plans, N+1 inventory, profiling evidence | | `architecture` | Full xref graph, coupling matrix, cohesion | | `tests` | Coverage by context, isolation, flaky-test indicators | | `deps` | Vulnerabilities, licenses, maintenance status | ## Incremental Mode (`--since <commit>`) Analyze only changes since a specific commit. Useful for pre-merge checks: Run `git diff --name-only <commit>...HEAD` to identify changed files, then run targeted audits on changed files only (skips full project scan). Combines with other flags: `phx-audit --since HEAD~5 --focus=security` ## Relationship to Other Commands | Command | Scope | Frequency | |---------|-------|-----------| | `phx-review` | Changed files (diff) | Every PR | | `phx-audit` | Entire project | Quarterly | | `phx-boundaries` | Context structure | On-demand | | `phx-verify` | Compile/test pass | Anytime | ## References - `references/scoring-methodology.md` - How scores are calculated - `references/architecture-checks.md` - Detailed architecture criteria
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.