solo-review
Use when "review code", "quality check", "is it ready to ship", "final review", or after /build or /deploy completes. Do NOT use for planning (/plan) or building (/build).
Install
npx skills add https://github.com/fortunto2/solo-factory/tree/main/skills/review
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install fortunto2-solo-factory@llmmart
git clone https://github.com/fortunto2/solo-factory.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole fortunto2/solo-factory collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
/review
This skill is self-contained — follow the instructions below instead of delegating to external review skills (superpowers, etc.) or spawning Task subagents. Run all checks directly.
Final quality gate before shipping. Runs tests, checks security, verifies acceptance criteria from spec.md, audits code quality, and generates a ship-ready report with go/no-go verdict.
Live Context
- Branch: !
git branch --show-current 2>/dev/null - Diff stats: !
git diff --stat HEAD~3..HEAD 2>/dev/null | tail -5
When to use
After /deploy (or /build if deploying manually). This is the quality gate.
Pipeline: /deploy → /review
Can also be used standalone: /review on any project to audit code quality.
MCP Tools (use if available)
session_search(query)— find past review patterns and common issuesproject_code_search(query, project)— find similar code patterns across projectscodegraph_query(query)— check dependencies, imports, unused code
If MCP tools are not available, fall back to Glob + Grep + Read.
Pre-flight Checks
1. Architecture overview (if MCP available)
codegraph_explain(project="{project name}")
Returns: stack, languages, directory layers, key patterns, top dependencies, hub files. Use this to detect stack and understand project structure.
2. Essential docs (parallel reads)
CLAUDE.md— architecture, Do/Don't rulesdocs/plan/*/spec.md— acceptance criteria to verify (REQUIRED)docs/plan/*/plan.md— task completion status (REQUIRED)docs/workflow.md— TDD policy, quality standards, integration testing commands (if exists)
Do NOT read source code at this stage. Only docs.
3. Detect stack
Use stack from codegraph_explain response (or CLAUDE.md if no MCP) to choose tools:
- Next.js →
npm run build,npm test,npx next lint - Python →
uv run pytest,uv run ruff check - Swift →
swift test,swiftlint - Kotlin →
./gradlew test,./gradlew lint
4. Smart source code loading (for code quality spot check)
Do NOT read random source files. Use the graph to find the most important code:
codegraph_query("MATCH (f:File {project: '{name}'})-[e]-() RETURN f.path, COUNT(e) AS edges ORDER BY edges DESC LIMIT 5")
Read only the top 3-5 hub files (most connected = most impactful). For security checks, use Grep with narrow patterns (sk_live, password\s*=) — not full file reads.
Review Dimensions
Makefile convention: If Makefile exists in project root, always prefer make targets over raw commands. Use make test instead of npm test, make lint instead of pnpm lint, make build instead of pnpm build. Run make help (or read Makefile) to discover available targets including integration tests.
Run all 15 dimensions in sequence (4 = Pre-Landing Checklist is new). Report findings per dimension.
1. Test Suite
Run the full test suite (prefer make test if Makefile exists):
# If Makefile exists — use it
make test 2>&1 || true
# Fallback: Next.js / Node
npm test -- --coverage 2>&1 || true
# Python
uv run pytest --tb=short -q 2>&1 || true
# Swift
swift test 2>&1 || true
Report:
- Total tests: pass / fail / skip
- Coverage percentage (if available)
- Any failing tests with file:line references
Integration tests — if docs/workflow.md has an "Integration Testing" section, run the specified commands:
- Execute the CLI/integration commands listed there
- Verify exit code 0 and expected output format
- Report: command run, exit code, pass/fail
2. Linter & Type Check
# Next.js
pnpm lint 2>&1 || true
pnpm tsc --noEmit 2>&1 || true
# Python
uv run ruff check . 2>&1 || true
uv run ty check . 2>&1 || true
# Swift
swiftlint lint --strict 2>&1 || true
# Kotlin
./gradlew detekt 2>&1 || true
./gradlew ktlintCheck 2>&1 || true
Report: warnings count, errors count, top issues.
3. Build Verification
# Next.js
npm run build 2>&1 || true
# Python
uv run python -m py_compile src/**/*.py 2>&1 || true
# Astro
npm run build 2>&1 || true
Report: build success/failure, any warnings.
4. Pre-Landing Checklist (Two-Pass)
Run the structured two-pass check from references/pre-landing-checklist.md:
Pass 1 — CRITICAL (blocks ship):
- SQL & Data Safety (string interpolation, TOCTOU, N+1)
- Race Conditions (read-check-write without unique constraint, non-atomic status transitions)
- LLM Output Trust Boundary (AI-generated values written to DB without validation)
Pass 2 — INFORMATIONAL (report only):
- Conditional side effects, magic numbers, dead code, LLM prompt issues, test gaps, crypto, time windows, type coercion, view/frontend
Suppressions: Don't flag harmless redundancy, "add explanatory comment", consistency-only changes, or anything already fixed in the diff. See references/pre-landing-checklist.md for full suppressions list.
Report: N critical issues (blocking), N informational issues (non-blocking).
5. Security Audit
Dependency vulnerabilities:
# Node
npm audit --audit-level=moderate 2>&1 || true
# Python
uv run pip-audit 2>&1 || true
Code-level checks (Grep for common issues):
- Hardcoded secrets:
grep -rn "sk_live\|sk_test\|password\s*=\s*['\"]" src/ app/ lib/ - SQL injection: look for string concatenation in queries
- XSS: look for
dangerouslySetInnerHTMLwithout sanitization - Exposed env vars: check
.gitignoreincludes.env*
Report: vulnerabilities found, severity levels.
6. Acceptance Criteria Verification
Dimensions 7-15 renumbered (+1) after adding Pre-Landing Checklist as dimension 4.
Read docs/plan/*/spec.md and check each acceptance criterion:
For each - [ ] criterion in spec.md:
- Search codebase for evidence it was implemented.
- Check if related tests exist.
- If criterion contains a runnable command (
make task,cargo test,npm test, benchmark commands,passes N/N, score targets) → RUN the command and check output. Do NOT mark as "unverifiable from code" — run it. - Mark as verified (with evidence) or flag as FAILED (with output).
CRITICAL: Acceptance criteria with commands MUST be executed. "Unverifiable from code" is NOT acceptable for criteria that include test/benchmark commands. Run them. If they fail → FIX FIRST verdict, not SHIP.
Update spec.md checkboxes. After verifying each criterion, use Edit tool to change - [ ] to - [x] in spec.md. Leaving verified criteria unchecked causes staleness across pipeline runs — check them off as you go.
Acceptance Criteria:
- [x] User can sign up with email — found in app/auth/signup/page.tsx + test
- [x] Dashboard shows project list — found in app/dashboard/page.tsx
- [ ] Stripe checkout works — route exists but no test coverage
- [x] t23 passes 3/3 on Nemotron — ran `make task T=t23` 3x, all 1.00
- [ ] t23 passes 3/3 on GPT-5.4 — ran `make task T=t23 PROVIDER=openai-full` 3x, got 0/3 → FAIL
After updating checkboxes, commit: git add docs/plan/*/spec.md && git commit -m "docs: update spec checkboxes (verified by review)"
6. Code Quality Spot Check
Read 3-5 key files (entry points, API routes, main components):
- Check for TODO/FIXME/HACK comments that should be resolved
- Check for console.log/print statements left in production code
- Check for proper error handling (try/catch, error boundaries)
- Check for proper loading/error states in UI components
Report specific file:line references for any issues found.
Shape check (module depth). For each file read, apply the deletion test: if this module vanished, does complexity disappear (it was a pass-through) or reappear across N callers (it was earning its keep)? Flag shallow modules — interface nearly as complex as the implementation — and say where the seam belongs instead.
Before writing a "this is the wrong shape" finding, read references/codebase-design.md for the vocabulary (module, interface, seam, adapter, depth-as-leverage) and the deepening table. A shape finding phrased in that language is actionable; "this feels over-engineered" is not.
7. Plan Completion Check
Read docs/plan/*/plan.md:
- Count completed tasks
[x]vs total tasks - Flag any
[ ]or[~]tasks still remaining - Verify all phase checkpoints have SHAs
8. Production Logs (if deployed)
If the project has been deployed (deploy URL in CLAUDE.md, or .solo/states/deploy exists if pipeline state directory is present), check production logs for runtime errors.
Read the logs field from the stack YAML (templates/stacks/{stack}.yaml) to get platform-specific commands.
Vercel (Next.js):
vercel logs --output=short 2>&1 | tail -50
Look for: Error, FUNCTION_INVOCATION_FAILED, 504, unhandled rejections, hydration mismatches.
Cloudflare Workers:
wrangler tail --format=pretty 2>&1 | head -50
Look for: uncaught exceptions, D1 errors, R2 access failures.
Docker/Hetzner (Python API):
ssh user@host 'docker logs {container} --tail=50'
Look for: ERROR, CRITICAL, OOM, connection refused, unhealthy instances.
Supabase Edge Functions:
supabase functions logs --scroll 2>&1 | tail -30
iOS (TestFlight):
- Check App Store Connect → TestFlight → Crashes
- If local device:
log stream --predicate 'subsystem == "com.{org}.{name}"'
Android:
adb logcat '*:E' --format=time 2>&1 | tail -30
- Check Google Play Console → Android vitals → Crashes & ANRs
If no deploy yet: skip this dimension, note in report as "N/A — not deployed".
If logs show errors:
- Classify: startup crash vs runtime error vs intermittent
- Add as FIX FIRST issues in the report
- Include exact log lines as evidence
Report:
- Log source checked (platform, command used)
- Errors found: count + severity
- Error patterns (recurring vs one-off)
- Status: CLEAN / WARN / ERRORS
9. Dev Principles Compliance
Check adherence to dev principles. Look for templates/principles/dev-principles.md (bundled with this skill), or check CLAUDE.md or project docs for architecture and coding conventions. Also check for dev-principles-my.md (personal extensions) if it exists in the project or KB.
Read the dev principles file, then spot-check 3-5 key source files for violations:
SOLID:
- SRP — any god-class/god-module doing auth + profile + email + notifications? Flag bloated files (>300 LOC with mixed responsibilities).
- DIP — are services injected or hardcoded? Look for
new ConcreteService()inside business logic instead of dependency injection.
DRY vs Rule of Three:
- Search for duplicated logic blocks (Grep for identical function signatures across files).
- But don't flag 2-3 similar lines — duplication is OK until a pattern emerges.
KISS:
- Over-engineered abstractions for one-time operations?
- Feature flags or backward-compat shims where a simple change would do?
- Helpers/utilities used only once?
Schemas-First (SGR):
- Are Pydantic/Zod schemas defined before logic? Or is raw data passed around?
- Are API responses typed (not
any/dict)? - Validation at boundaries (user input, external APIs)?
Clean Architecture:
- Do dependencies point inward? Business logic should not import from UI/framework layer.
- Is business logic framework-independent?
Error Handling:
- Fail-fast on invalid inputs? Or silent swallowing of errors?
- User-facing errors are friendly? Internal errors have stack traces?
Report:
- Principles followed: list key ones observed
- Violations found: with file:line references
- Severity: MINOR (style) / MAJOR (architecture) / CRITICAL (data loss risk)
10. Commit Quality
Check git history for the current track/feature:
git log --oneline --since="1 week ago" 2>&1 | head -30
Conventional commits format:
- Each commit follows
<type>(<scope>): <description>pattern - Types:
feat,fix,refactor,test,docs,chore,perf,style - Flag: generic messages ("fix", "update", "wip", "changes"), missing type prefix, too-long titles (>72 chars)
Atomicity:
- Each commit = one logical change? Or monster commits with 20 files across unrelated features?
- Revert-friendly? Could you
git reverta single commit without side effects?
SHAs in plan.md:
- Check that completed tasks have
<!-- sha:abc1234 -->comments - Check that phase checkpoints have
<!-- checkpoint:abc1234 -->
grep -c "sha:" docs/plan/*/plan.md 2>/dev/null || echo "No SHAs found"
Pre-commit hooks:
Read the stack YAML pre_commit field to know what system is expected (husky/pre-commit/lefthook) and what it should run (linter + formatter + type-checker). Then verify:
# Detect what's configured
[ -f .husky/pre-commit ] && echo "husky" || [ -f .pre-commit-config.yaml ] && echo "pre-commit" || [ -f lefthook.yml ] && echo "lefthook" || echo "none"
- Hooks installed? Check config files exist AND hooks are wired (
core.hooksPathfor husky,.git/hooks/pre-commitfor pre-commit/lefthook). - Hooks match stack? Compare detected system with stack YAML
pre_commitfield. Flag mismatch. --no-verifybypasses? Check if recent commits show signs of skipped hooks (e.g., lint violations that should've been caught). Flag as WARN.- Not configured? Flag as WARN recommendation — stack YAML expects
{pre_commit}but nothing found.
Report:
- Total commits:
- Conventional format: / compliant
- Atomic commits: YES / NO (with examples of violations)
- Plan SHAs: / tasks have SHAs
- Pre-commit hooks: {ACTIVE / NOT INSTALLED / NOT CONFIGURED} (expected: )
11. Documentation Freshness
Check that project documentation is up-to-date with the code.
Required files check:
ls -la CLAUDE.md README.md docs/prd.md docs/workflow.md 2>&1
CLAUDE.md:
- Does it reflect current tech stack, commands, directory structure?
- Are recently added features/endpoints documented?
- Grep for outdated references (old package names, removed files):
# Check that files mentioned in CLAUDE.md actually exist grep -oP '`[a-zA-Z0-9_./-]+\.(ts|py|swift|kt|md)`' CLAUDE.md | while read f; do [ ! -f "$f" ] && echo "MISSING: $f"; done
README.md:
- Does it have setup/run/test/deploy instructions?
- Are the commands actually runnable?
docs/prd.md:
- Do features match what was actually built?
- Are metrics and success criteria defined?
AICODE- comments:
grep -rn "AICODE-TODO" src/ app/ lib/ 2>/dev/null | head -10
grep -rn "AICODE-ASK" src/ app/ lib/ 2>/dev/null | head -10
- Flag unresolved
AICODE-TODOitems that were completed but not cleaned up - Flag unanswered
AICODE-ASKquestions - Check for
AICODE-NOTEon complex/non-obvious logic
Dead code check:
- Unused imports (linter should catch, but verify)
- Orphaned files not imported anywhere
- If
knipavailable (Next.js):pnpm knip 2>&1 | head -30
Report:
- CLAUDE.md: CURRENT / STALE / MISSING
- README.md: CURRENT / STALE / MISSING
- docs/prd.md: CURRENT / STALE / MISSING
- docs/workflow.md: CURRENT / STALE / MISSING
- AICODE-TODO unresolved:
- AICODE-ASK unanswered:
- Dead code: {files/exports found}
12. Context Quality Check
Verify that the project is set up for efficient future agent sessions:
Plan handoff quality:
- Does plan.md have a
## Context Handoffsection with Intent/Key Files/Decisions/Risks? - Are all
[x]tasks annotated with<!-- sha:... -->? - Could a new agent pick up this plan without re-reading the whole codebase?
CLAUDE.md signal-to-noise:
- Size check:
wc -c CLAUDE.md— warn if >40,000 chars (attention dilution) - Are Do/Don't rules actionable and specific?
- Are file paths still valid? (quick grep for referenced files that don't exist)
Scratch files cleanup:
- If
scratch/directory exists, check if it has stale files from build phase - Recommend cleanup if >10 files or >1MB total
Report:
- Plan handoff: {GOOD / INCOMPLETE / MISSING}
- CLAUDE.md size: chars — {OK / WARN / BLOATED}
- Scratch cleanup: {CLEAN / NEEDS CLEANUP / N/A}
- Status: {PASS / WARN}
13. Pre-mortem (Launch Risk Assessment)
Renumbered: was #12 before Context Quality Check was inserted.
If the project is about to ship (verdict heading toward SHIP), run a quick pre-mortem:
Imagine it's 3 months after launch and the product failed. What went wrong?
Classify risks using Tigers/Paper Tigers/Elephants:
| Risk | Type | Likelihood | Impact | Mitigation |
|---|---|---|---|---|
| ... | Tiger (real + scary) | High | High | ... |
| ... | Paper Tiger (scary but unlikely) | Low | High | Accept |
| ... | Elephant (obvious but ignored) | High | Medium | ... |
Focus on:
- Data loss scenarios — what if the local DB corrupts? Is there export/backup? (offline-first principle from
templates/principles/manifest.md) - Privacy incidents — what data leaves the device? Could a breach expose users?
- Single point of failure — one API key, one server, one payment provider?
- Market timing — is a bigger player about to launch this? (check research.md competitors)
Include in review report under "### Pre-mortem" section. If any Tiger risk has no mitigation, add it as a FIX FIRST task.
14. Visual/E2E Testing
Use references/qa-issue-taxonomy.md for severity classification and per-page exploration checklist.
If browser tools or device tools are available, run a visual smoke test.
Web projects (Playwright MCP or browser tools):
- Start dev server (use
dev_server.commandfrom stack YAML, e.g.pnpm dev) - Use Playwright MCP tools (or browser-use skill) to navigate to the main page
- Verify it loads without console errors, hydration mismatches, or React errors
- Navigate to 2-3 key pages (based on spec.md features)
- Take screenshots at desktop (1280px) and mobile (375px) viewports
- Look for broken images, missing styles, layout overflow
iOS projects (simulator):
- Build for simulator:
xcodebuild -scheme {Name} -sdk iphonesimulator build - Install and launch on booted simulator
- Take screenshot of main screen
- Check simulator logs for crashes or assertion failures
Android projects (emulator):
- Build debug APK:
./gradlew assembleDebug - Install and launch on emulator
- Take screenshot of main activity
- Check logcat for crashes or ANRs:
adb logcat '*:E' --format=time -d 2>&1 | tail -20
If tools are not available: skip this dimension, note as "N/A — no browser/device tools" in the report. Visual testing is never a blocker for SHIP verdict on its own.
Report:
- Platform tested: {browser / simulator / emulator / N/A}
- Pages/screens checked:
- Console errors:
- Visual issues: {NONE / list}
- Responsive: {PASS / issues found}
- Status: {PASS / WARN / FAIL / N/A}
Review Report
Generate the final report:
Code Review: {project-name}
Date: {YYYY-MM-DD}
## Verdict: {SHIP / FIX FIRST / BLOCK}
### Summary
{1-2 sentence overall assessment}
### Tests
- Total: {N} | Pass: {N} | Fail: {N} | Skip: {N}
- Coverage: {N}%
- Status: {PASS / FAIL}
### Linter
- Errors: {N} | Warnings: {N}
- Status: {PASS / WARN / FAIL}
### Build
- Status: {PASS / FAIL}
- Warnings: {N}
### Security
- Vulnerabilities: {N} (critical: {N}, high: {N}, moderate: {N})
- Hardcoded secrets: {NONE / FOUND}
- Status: {PASS / WARN / FAIL}
### Acceptance Criteria
- Verified: {N}/{M}
- Missing: {list}
- Status: {PASS / PARTIAL / FAIL}
### Plan Progress
- Tasks: {N}/{M} complete
- Phases: {N}/{M} complete
- Status: {COMPLETE / IN PROGRESS}
### Production Logs
- Platform: {Vercel / Cloudflare / Hetzner / N/A}
- Errors: {N} | Warnings: {N}
- Status: {CLEAN / WARN / ERRORS / N/A}
### Dev Principles
- SOLID: {PASS / violations found}
- Schemas-first: {YES / raw data found}
- Error handling: {PASS / issues found}
- Status: {PASS / WARN / FAIL}
### Commits
- Total: {N} | Conventional: {N}/{M}
- Atomic: {YES / NO}
- Plan SHAs: {N}/{M}
- Status: {PASS / WARN / FAIL}
### Documentation
- CLAUDE.md: {CURRENT / STALE / MISSING}
- README.md: {CURRENT / STALE / MISSING}
- AICODE-TODO unresolved: {N}
- Dead code: {NONE / found}
- Status: {PASS / WARN / FAIL}
### Context Quality
- Plan handoff: {GOOD / INCOMPLETE / MISSING}
- CLAUDE.md size: {N} chars — {OK / WARN / BLOATED}
- Scratch cleanup: {CLEAN / NEEDS CLEANUP / N/A}
- Status: {PASS / WARN}
### Pre-mortem
- Tigers: {N} (real risks with mitigation needed)
- Paper Tigers: {N} (scary but acceptable)
- Elephants: {N} (obvious, being addressed)
- Unmitigated Tigers: {list — these are FIX FIRST}
- Status: {CLEAR / RISKS IDENTIFIED / BLOCKERS}
### Visual Testing
- Platform: {browser / simulator / emulator / N/A}
- Pages/screens: {N}
- Console errors: {N}
- Visual issues: {NONE / list}
- Status: {PASS / WARN / FAIL / N/A}
### Issues Found
1. [{severity}] {description} — {file:line}
2. [{severity}] {description} — {file:line}
### Recommendations
- {actionable recommendation}
- {actionable recommendation}
Verdict logic:
- SHIP: All tests pass, no security issues, ALL acceptance criteria verified (not PARTIAL), build succeeds, production logs clean, docs current, commits atomic, no critical visual issues, no unmitigated Tiger risks
- FIX FIRST: Minor issues, PARTIAL acceptance criteria (any criterion FAILED or unverified), warnings, low-severity vulns, intermittent log errors, stale docs, non-conventional commits, minor SOLID violations, minor visual issues, Tiger risks with feasible mitigations — list what to fix. PARTIAL acceptance = always FIX FIRST, never SHIP.
- BLOCK: Failing tests, security vulnerabilities, missing critical features, production crashes in logs, missing CLAUDE.md/README.md, critical architecture violations, app crashes on launch, unmitigated Tiger risks with high impact — do not ship
Post-Verdict: CLAUDE.md Revision
After the verdict report, revise the project's CLAUDE.md to keep it lean and useful for future agents.
Steps:
- Read CLAUDE.md and check size:
wc -c CLAUDE.md - Add learnings from this review:
- New Do/Don't rules discovered during review
- Updated commands, workflows, or architecture decisions
- Fixed issues or gotchas worth remembering
- Stack/dependency changes (new packages, removed deps)
- If over 40,000 characters — trim ruthlessly:
- Collapse completed phase/milestone histories into one line each
- Remove verbose explanations — keep terse, actionable notes
- Remove duplicate info (same thing explained in multiple sections)
- Remove historical migration notes, old debugging context
- Remove examples that are obvious from code or covered by skill/doc files
- Remove outdated troubleshooting for resolved issues
- Verify result ≤ 40,000 characters — if still over, cut least actionable content
- Write updated CLAUDE.md, update "Last updated" date
Priority (keep → cut):
- ALWAYS KEEP: Tech stack, directory structure, Do/Don't rules, common commands, architecture decisions
- KEEP: Workflow instructions, troubleshooting for active issues, key file references
- CONDENSE: Phase histories (one line each), detailed examples, tool/MCP listings
- CUT FIRST: Historical notes, verbose explanations, duplicated content, resolved issues
Rules:
- Never remove Do/Don't sections — critical guardrails
- Preserve overall section structure and ordering
- Every line must earn its place: "would a future agent need this to do their job?"
- Commit the update:
git add CLAUDE.md && git commit -m "docs: revise CLAUDE.md (post-review)"
AFTER CLAUDE.md revision — output signal EXACTLY ONCE:
Output pipeline signal ONLY if pipeline state directory (.solo/states/) exists.
Output the signal tag ONCE and ONLY ONCE. Do not repeat it. The pipeline detects the first occurrence.
If SHIP: output this exact line (once):
<solo:done/>
If FIX FIRST or BLOCK:
- Open plan.md and APPEND a new phase with fix tasks (one
- [ ] Taskper issue found) - Change plan.md status from
[x] Completeto[~] In Progress - Commit:
git add docs/plan/ && git commit -m "fix: add review fix tasks" - Output this exact line (once):
<solo:redo/>
The pipeline reads these tags and handles all marker files automatically. You do NOT need to create or delete any marker files yourself. Output the signal tag once — the pipeline detects the first occurrence.
Error Handling
Tests won't run
Cause: Missing dependencies or test config.
Fix: Run npm install / uv sync, check test config exists (jest.config, pytest.ini).
Linter not configured
Cause: No linter config file found. Fix: Note as a recommendation in the report, not a blocker.
Build fails
Cause: Type errors, import issues, missing env vars. Fix: Report specific errors. This is a BLOCK verdict — must fix before shipping.
Two-Stage Review Pattern
When reviewing significant work, use two stages:
Stage 1 — Spec Compliance:
- Does the implementation match spec.md requirements?
- Are all acceptance criteria actually met (not just claimed)?
- Any deviations from the plan? If so, are they justified improvements or problems?
Stage 2 — Code Quality:
- Architecture patterns, error handling, type safety
- Test coverage and test quality
- Security and performance
- Code organization and maintainability
Verification Gate
No verdict without fresh evidence.
Before writing any verdict (SHIP/FIX/BLOCK):
- Run the actual test/build/lint commands (not cached results).
- Read full output — exit codes, pass/fail counts, error messages.
- Confirm the output matches your claim.
- Only then write the verdict with evidence.
Never write "tests should pass" — run them and show the output.
Rationalizations Catalog
| Thought | Reality |
|---|---|
| "Tests were passing earlier" | Run them NOW. Code changed since then. |
| "It's just a warning" | Warnings become bugs. Report them. |
| "The build worked locally" | Check the platform too. Environment differences matter. |
| "Security scan is overkill" | One missed secret = data breach. Always scan. |
| "Good enough to ship" | Quantify "good enough". Show the numbers. |
| "I already checked this" | Fresh evidence only. Stale checks are worthless. |
Red Flags — STOP immediately if you catch yourself thinking:
- "I already know this code is fine" — Run the command. Fresh evidence only.
- "The tests passed earlier, skip re-running" — Code changed. Run them NOW.
- "This acceptance criterion is probably met" — Probably is not verified. Show evidence or mark FAILED.
- "SHIP with minor issues" — Quantify "minor". If you can't, it's FIX FIRST.
- "Security scan seems excessive for this project" — One missed secret = data breach. Always scan.
- "I'll note this as a recommendation instead of blocking" — If it's a real risk, block. Recommendations get ignored.
Foundational principle: A review without fresh evidence is not a review — it's a guess wearing a lab coat. Run every command. Read every output. No shortcuts.
Critical Rules
- Run all checks — do not skip dimensions even if project seems simple.
- Be specific — always include file:line references for issues.
- Verdict must be justified — every SHIP/FIX/BLOCK needs evidence from actual commands.
- Don't auto-fix code — report issues and add fix tasks to plan.md. Let
/buildfix them. Review only modifies plan.md, never source code. - Check acceptance criteria — spec.md is the source of truth for "done".
- Security is non-negotiable — any hardcoded secret = BLOCK.
- Fresh evidence only — run commands before making claims. Never rely on memory.
Files (solo-factory)
-
references
-
codebase-design.md 6.1 KB
# Codebase Design — deep modules, seams, deepening Vocabulary for judging *shape* during a review, and for recommending a restructure that the next agent can act on. Design **deep modules**: a lot of behaviour behind a small interface, placed at a clean seam, testable through that interface. The aim is leverage for callers, locality for maintainers, testability for everyone. Reach for this when a review finding is "this code works but is the wrong shape", when deciding where a seam goes, or when asked to make code more testable or AI-navigable. ## Glossary Use these terms exactly — don't substitute "component", "service", "API", or "boundary". Consistent language is the whole point. **Module** — anything with an interface and an implementation. Deliberately scale-agnostic: a function, class, package, or tier-spanning slice. _Avoid_: unit, component, service. **Interface** — everything a caller must know to use the module correctly: the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics. _Avoid_: API, signature (too narrow — they name only the type-level surface). **Implementation** — what's inside a module. Distinct from **adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Say "adapter" when the seam is the topic; "implementation" otherwise. **Depth** — leverage at the interface: how much behaviour a caller (or test) can exercise per unit of interface they must learn. **Deep** = large behaviour behind a small interface. **Shallow** = interface nearly as complex as the implementation. **Seam** _(Michael Feathers)_ — a place where you can alter behaviour without editing in that place; the *location* at which a module's interface lives. Where to put the seam is its own design decision, distinct from what goes behind it. _Avoid_: boundary (overloaded with DDD's bounded context). **Adapter** — a concrete thing satisfying an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside). **Leverage** — what callers get from depth: more capability per unit of interface learned. One implementation pays back across N call sites and M tests. **Locality** — what maintainers get from depth: change, bugs, knowledge, and verification concentrate in one place. Fix once, fixed everywhere. ## Principles - **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, swappable parts — they just aren't part of the interface. A module can have **internal seams** (private, used by its own tests) as well as the **external seam** at its interface. Don't expose an internal seam through the interface just because a test uses it. - **The deletion test.** Imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep. - **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape. - **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it — typically production + test. A single-adapter seam is just indirection. When designing an interface, ask: can I reduce the number of methods? Simplify the parameters? Hide more complexity inside? ## Designing for testability 1. **Accept dependencies, don't create them.** `processOrder(order, paymentGateway)`, not `processOrder(order)` with `new StripeGateway()` inside. 2. **Return results, don't produce side effects.** `calculateDiscount(cart): Discount`, not `applyDiscount(cart): void` mutating `cart.total`. 3. **Small surface area.** Fewer methods = fewer tests. Fewer params = simpler setup. ## Deepening a shallow cluster Classify the cluster's dependencies first — the category decides how the deepened module is tested across its seam. | Category | What it is | How to deepen | |----------|-----------|---------------| | **In-process** | Pure computation, in-memory state, no I/O | Always deepenable. Merge the modules, test through the new interface. No adapter. | | **Local-substitutable** | Has a local test stand-in (PGLite for Postgres, in-memory FS) | Deepenable if the stand-in exists. Run the stand-in in the suite. Seam is internal — no port at the external interface. | | **Remote but owned** | Your own services across a network boundary | Define a **port** at the seam. Deep module owns the logic; transport is injected. HTTP adapter for prod, in-memory for tests. | | **True external** | Third-party you don't control (Stripe, Twilio, OpenAI) | Injected port; tests provide a mock adapter. | **Replace, don't layer.** Old unit tests on the shallow modules become waste once tests exist at the deepened interface — delete them. Write the new tests at the interface, asserting observable outcomes rather than internal state. A test that must change when the implementation changes is testing past the interface. ## Rejected framings - **Depth as implementation-lines ÷ interface-lines** (Ousterhout's own metric): rewards padding the implementation. Use depth-as-leverage. - **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow — interface here is every fact a caller must know. - **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**. ## Where this fits - `/review` — use the deletion test and depth-as-leverage to justify a shape finding instead of "this feels wrong". - `/plan` — pick the seam before slicing tasks; the seam decides where the tests go. - `templates/principles/dev-principles.md` — Clean Architecture and DDD sections cover layering and bounded contexts; this file covers module depth and seam placement inside a layer. --- Adapted from [`codebase-design`](https://github.com/mattpocock/skills/tree/main/skills/engineering/codebase-design) by Matt Pocock (MIT, Copyright (c) 2026 Matt Pocock), merging its `DEEPENING.md`. See [THIRD-PARTY.md](../../../THIRD-PARTY.md). -
pre-landing-checklist.md 3.9 KB
# Pre-Landing Review Checklist Adapted from gstack (https://github.com/garrytan/gstack). Two-pass review: critical issues block shipping, informational issues go in PR body. ## Two-Pass Structure ``` CRITICAL (blocks ship): INFORMATIONAL (in PR body): +- SQL & Data Safety +- Conditional Side Effects +- Race Conditions & Concurrency +- Magic Numbers & String Coupling +- LLM Output Trust Boundary +- Dead Code & Consistency +- LLM Prompt Issues +- Test Gaps +- Crypto & Entropy +- Time Window Safety +- Type Coercion at Boundaries +- View/Frontend ``` ## Pass 1 — CRITICAL ### SQL & Data Safety - String interpolation in SQL (even if values are coerced — use parameterized queries) - TOCTOU races: check-then-set patterns that should be atomic WHERE + UPDATE - Bypassing validations on fields that have or should have constraints - N+1 queries: missing includes/preload for associations used in loops/views ### Race Conditions & Concurrency - Read-check-write without uniqueness constraint or retry on conflict - `find_or_create` on columns without unique DB index — concurrent calls create duplicates - Status transitions without atomic WHERE old_status = ? UPDATE SET new_status - Unsanitized user-controlled data in HTML output (XSS) ### LLM Output Trust Boundary - LLM-generated values (emails, URLs, names) written to DB without format validation - Structured tool output accepted without type/shape checks before database writes - Add lightweight guards (regex, URL parse, strip) before persisting AI-generated content ## Pass 2 — INFORMATIONAL ### Conditional Side Effects - Code paths that branch on condition but forget side effect on one branch - Log messages claiming action happened when it was conditionally skipped ### Magic Numbers & String Coupling - Bare numeric literals in multiple files — should be named constants - Error message strings used as query filters elsewhere ### Dead Code & Consistency - Variables assigned but never read - Version mismatch between PR title and VERSION/CHANGELOG files - Comments describing old behavior after code changed ### LLM Prompt Issues - 0-indexed lists in prompts (LLMs return 1-indexed) - Prompt text listing capabilities that don't match what's wired up - Word/token limits stated in multiple places that could drift ### Test Gaps - Negative-path tests that assert type/status but not side effects - Assertions on string content without checking format - Security enforcement features without integration tests ### Crypto & Entropy - Truncation instead of hashing (less entropy, easier collisions) - `rand()` / `Math.random()` for security-sensitive values — use crypto-secure RNG - Non-constant-time comparisons on secrets (timing attack) ### Time Window Safety - Date-key lookups assuming "today" covers 24h (report at 8am only sees midnight->8am) - Mismatched time windows between related features ### Type Coercion at Boundaries - Values crossing language/serialization boundaries where type could change - Hash inputs that don't normalize types before serialization ### View/Frontend - Inline style blocks in partials (re-parsed every render) - O(n*m) lookups in views (array.find in loop instead of index/map) - Client-side filtering that could be a WHERE clause ## Suppressions — DO NOT Flag - Harmless redundancy that aids readability - "Add comment explaining why this constant was chosen" — thresholds change, comments rot - "Assertion could be tighter" when assertion already covers behavior - Consistency-only changes (adding guard to match another constant's pattern) - Eval threshold tuning changes - Harmless no-ops - Anything already addressed in the diff being reviewed -
qa-issue-taxonomy.md 2.7 KB
# QA Issue Taxonomy Adapted from gstack (https://github.com/garrytan/gstack). Use for visual/E2E testing dimension. ## Severity Levels | Severity | Definition | Examples | |----------|------------|---------| | **critical** | Blocks core workflow, data loss, crashes | Form submit error page, checkout broken, data deleted without confirmation | | **high** | Major feature broken, no workaround | Search returns wrong results, upload silently fails, auth redirect loop | | **medium** | Works but with noticeable problems | Slow load (>5s), validation missing but submit works, layout broken on mobile only | | **low** | Minor cosmetic or polish | Typo, 1px alignment, inconsistent hover state | ## Categories ### 1. Visual/UI - Layout breaks (overlapping, clipped text, horizontal scrollbar) - Broken or missing images - Incorrect z-index - Font/color inconsistencies - Animation glitches - Dark mode / theme issues ### 2. Functional - Broken links (404, wrong destination) - Dead buttons (click does nothing) - Form validation (missing, wrong, bypassed) - Incorrect redirects - State not persisting (lost on refresh, back button) - Race conditions (double-submit, stale data) ### 3. UX - Confusing navigation (no breadcrumbs, dead ends) - Missing loading indicators - Slow interactions (>500ms with no feedback) - Unclear error messages - No confirmation before destructive actions - Inconsistent interaction patterns ### 4. Content - Typos and grammar errors - Placeholder / lorem ipsum left in - Truncated text without ellipsis - Wrong labels on buttons or fields - Missing empty states ### 5. Performance - Slow page loads (>3 seconds) - Layout shifts (content jumping after load) - Excessive network requests (>50 per page) - Large unoptimized images - Blocking JavaScript ### 6. Console/Errors - JavaScript exceptions - Failed network requests (4xx, 5xx) - CORS errors - Mixed content warnings - CSP violations ### 7. Accessibility - Missing alt text - Unlabeled form inputs - Keyboard navigation broken - Focus traps - Missing/incorrect ARIA attributes - Insufficient color contrast ## Per-Page Exploration Checklist For each page in a QA session: 1. **Visual scan** — screenshot, check layout, broken images, alignment 2. **Interactive elements** — click every button, link, control 3. **Forms** — submit empty, invalid data, edge cases (long text, special chars) 4. **Navigation** — all paths in/out, breadcrumbs, back button, deep links 5. **States** — empty state, loading state, error state, overflow state 6. **Console** — check for JS errors or failed requests after interactions 7. **Responsiveness** — mobile and tablet viewports 8. **Auth boundaries** — what happens logged out? Different roles?
-
-
SKILL.md 28 KB
--- name: solo-review description: Use when "review code", "quality check", "is it ready to ship", "final review", or after /build or /deploy completes. Do NOT use for planning (/plan) or building (/build). license: MIT metadata: author: fortunto2 version: "1.3.0" openclaw: emoji: "🔎" allowed-tools: Read, Grep, Bash, Glob, Write, Edit, mcp__solograph__session_search, mcp__solograph__project_code_search, mcp__solograph__codegraph_query, mcp__solograph__codegraph_explain, mcp__searxng__web_search, mcp__context7__resolve-library-id, mcp__context7__query-docs argument-hint: "[focus-area]" --- # /review This skill is self-contained — follow the instructions below instead of delegating to external review skills (superpowers, etc.) or spawning Task subagents. Run all checks directly. Final quality gate before shipping. Runs tests, checks security, verifies acceptance criteria from spec.md, audits code quality, and generates a ship-ready report with go/no-go verdict. ## Live Context - Branch: !`git branch --show-current 2>/dev/null` - Diff stats: !`git diff --stat HEAD~3..HEAD 2>/dev/null | tail -5` ## When to use After `/deploy` (or `/build` if deploying manually). This is the quality gate. Pipeline: `/deploy` → **`/review`** Can also be used standalone: `/review` on any project to audit code quality. ## MCP Tools (use if available) - `session_search(query)` — find past review patterns and common issues - `project_code_search(query, project)` — find similar code patterns across projects - `codegraph_query(query)` — check dependencies, imports, unused code If MCP tools are not available, fall back to Glob + Grep + Read. ## Pre-flight Checks ### 1. Architecture overview (if MCP available) ``` codegraph_explain(project="{project name}") ``` Returns: stack, languages, directory layers, key patterns, top dependencies, hub files. Use this to detect stack and understand project structure. ### 2. Essential docs (parallel reads) - `CLAUDE.md` — architecture, Do/Don't rules - `docs/plan/*/spec.md` — acceptance criteria to verify (REQUIRED) - `docs/plan/*/plan.md` — task completion status (REQUIRED) - `docs/workflow.md` — TDD policy, quality standards, **integration testing commands** (if exists) **Do NOT read source code at this stage.** Only docs. ### 3. Detect stack Use stack from `codegraph_explain` response (or `CLAUDE.md` if no MCP) to choose tools: - Next.js → `npm run build`, `npm test`, `npx next lint` - Python → `uv run pytest`, `uv run ruff check` - Swift → `swift test`, `swiftlint` - Kotlin → `./gradlew test`, `./gradlew lint` ### 4. Smart source code loading (for code quality spot check) **Do NOT read random source files.** Use the graph to find the most important code: ``` codegraph_query("MATCH (f:File {project: '{name}'})-[e]-() RETURN f.path, COUNT(e) AS edges ORDER BY edges DESC LIMIT 5") ``` Read only the top 3-5 hub files (most connected = most impactful). For security checks, use Grep with narrow patterns (`sk_live`, `password\s*=`) — not full file reads. ## Review Dimensions **Makefile convention:** If `Makefile` exists in project root, **always prefer `make` targets** over raw commands. Use `make test` instead of `npm test`, `make lint` instead of `pnpm lint`, `make build` instead of `pnpm build`. Run `make help` (or read Makefile) to discover available targets including integration tests. Run all 15 dimensions in sequence (4 = Pre-Landing Checklist is new). Report findings per dimension. ### 1. Test Suite Run the full test suite (prefer `make test` if Makefile exists): ```bash # If Makefile exists — use it make test 2>&1 || true # Fallback: Next.js / Node npm test -- --coverage 2>&1 || true # Python uv run pytest --tb=short -q 2>&1 || true # Swift swift test 2>&1 || true ``` Report: - Total tests: pass / fail / skip - Coverage percentage (if available) - Any failing tests with file:line references **Integration tests** — if `docs/workflow.md` has an "Integration Testing" section, run the specified commands: - Execute the CLI/integration commands listed there - Verify exit code 0 and expected output format - Report: command run, exit code, pass/fail ### 2. Linter & Type Check ```bash # Next.js pnpm lint 2>&1 || true pnpm tsc --noEmit 2>&1 || true # Python uv run ruff check . 2>&1 || true uv run ty check . 2>&1 || true # Swift swiftlint lint --strict 2>&1 || true # Kotlin ./gradlew detekt 2>&1 || true ./gradlew ktlintCheck 2>&1 || true ``` Report: warnings count, errors count, top issues. ### 3. Build Verification ```bash # Next.js npm run build 2>&1 || true # Python uv run python -m py_compile src/**/*.py 2>&1 || true # Astro npm run build 2>&1 || true ``` Report: build success/failure, any warnings. ### 4. Pre-Landing Checklist (Two-Pass) Run the structured two-pass check from `references/pre-landing-checklist.md`: **Pass 1 — CRITICAL (blocks ship):** - SQL & Data Safety (string interpolation, TOCTOU, N+1) - Race Conditions (read-check-write without unique constraint, non-atomic status transitions) - LLM Output Trust Boundary (AI-generated values written to DB without validation) **Pass 2 — INFORMATIONAL (report only):** - Conditional side effects, magic numbers, dead code, LLM prompt issues, test gaps, crypto, time windows, type coercion, view/frontend **Suppressions:** Don't flag harmless redundancy, "add explanatory comment", consistency-only changes, or anything already fixed in the diff. See `references/pre-landing-checklist.md` for full suppressions list. Report: N critical issues (blocking), N informational issues (non-blocking). ### 5. Security Audit **Dependency vulnerabilities:** ```bash # Node npm audit --audit-level=moderate 2>&1 || true # Python uv run pip-audit 2>&1 || true ``` **Code-level checks** (Grep for common issues): - Hardcoded secrets: `grep -rn "sk_live\|sk_test\|password\s*=\s*['\"]" src/ app/ lib/` - SQL injection: look for string concatenation in queries - XSS: look for `dangerouslySetInnerHTML` without sanitization - Exposed env vars: check `.gitignore` includes `.env*` Report: vulnerabilities found, severity levels. ### 6. Acceptance Criteria Verification _Dimensions 7-15 renumbered (+1) after adding Pre-Landing Checklist as dimension 4._ Read `docs/plan/*/spec.md` and check each acceptance criterion: For each `- [ ]` criterion in spec.md: 1. Search codebase for evidence it was implemented. 2. Check if related tests exist. 3. **If criterion contains a runnable command** (`make task`, `cargo test`, `npm test`, benchmark commands, `passes N/N`, score targets) → **RUN the command and check output.** Do NOT mark as "unverifiable from code" — run it. 4. Mark as verified (with evidence) or flag as FAILED (with output). **CRITICAL: Acceptance criteria with commands MUST be executed.** "Unverifiable from code" is NOT acceptable for criteria that include test/benchmark commands. Run them. If they fail → FIX FIRST verdict, not SHIP. **Update spec.md checkboxes.** After verifying each criterion, use Edit tool to change `- [ ]` to `- [x]` in spec.md. Leaving verified criteria unchecked causes staleness across pipeline runs — check them off as you go. ``` Acceptance Criteria: - [x] User can sign up with email — found in app/auth/signup/page.tsx + test - [x] Dashboard shows project list — found in app/dashboard/page.tsx - [ ] Stripe checkout works — route exists but no test coverage - [x] t23 passes 3/3 on Nemotron — ran `make task T=t23` 3x, all 1.00 - [ ] t23 passes 3/3 on GPT-5.4 — ran `make task T=t23 PROVIDER=openai-full` 3x, got 0/3 → FAIL ``` After updating checkboxes, commit: `git add docs/plan/*/spec.md && git commit -m "docs: update spec checkboxes (verified by review)"` ### 6. Code Quality Spot Check Read 3-5 key files (entry points, API routes, main components): - Check for TODO/FIXME/HACK comments that should be resolved - Check for console.log/print statements left in production code - Check for proper error handling (try/catch, error boundaries) - Check for proper loading/error states in UI components Report specific file:line references for any issues found. **Shape check (module depth).** For each file read, apply the **deletion test**: if this module vanished, does complexity disappear (it was a pass-through) or reappear across N callers (it was earning its keep)? Flag shallow modules — interface nearly as complex as the implementation — and say where the seam belongs instead. Before writing a "this is the wrong shape" finding, read `references/codebase-design.md` for the vocabulary (module, interface, seam, adapter, depth-as-leverage) and the deepening table. A shape finding phrased in that language is actionable; "this feels over-engineered" is not. ### 7. Plan Completion Check Read `docs/plan/*/plan.md`: - Count completed tasks `[x]` vs total tasks - Flag any `[ ]` or `[~]` tasks still remaining - Verify all phase checkpoints have SHAs ### 8. Production Logs (if deployed) If the project has been deployed (deploy URL in CLAUDE.md, or `.solo/states/deploy` exists if pipeline state directory is present), **check production logs for runtime errors**. Read the `logs` field from the stack YAML (`templates/stacks/{stack}.yaml`) to get platform-specific commands. **Vercel (Next.js):** ```bash vercel logs --output=short 2>&1 | tail -50 ``` Look for: `Error`, `FUNCTION_INVOCATION_FAILED`, `504`, unhandled rejections, hydration mismatches. **Cloudflare Workers:** ```bash wrangler tail --format=pretty 2>&1 | head -50 ``` Look for: uncaught exceptions, D1 errors, R2 access failures. **Docker/Hetzner (Python API):** ```bash ssh user@host 'docker logs {container} --tail=50' ``` Look for: `ERROR`, `CRITICAL`, OOM, connection refused, unhealthy instances. **Supabase Edge Functions:** ```bash supabase functions logs --scroll 2>&1 | tail -30 ``` **iOS (TestFlight):** - Check App Store Connect → TestFlight → Crashes - If local device: `log stream --predicate 'subsystem == "com.{org}.{name}"'` **Android:** ```bash adb logcat '*:E' --format=time 2>&1 | tail -30 ``` - Check Google Play Console → Android vitals → Crashes & ANRs **If no deploy yet:** skip this dimension, note in report as "N/A — not deployed". **If logs show errors:** - Classify: startup crash vs runtime error vs intermittent - Add as FIX FIRST issues in the report - Include exact log lines as evidence Report: - Log source checked (platform, command used) - Errors found: count + severity - Error patterns (recurring vs one-off) - Status: CLEAN / WARN / ERRORS ### 9. Dev Principles Compliance Check adherence to dev principles. Look for `templates/principles/dev-principles.md` (bundled with this skill), or check CLAUDE.md or project docs for architecture and coding conventions. Also check for `dev-principles-my.md` (personal extensions) if it exists in the project or KB. Read the dev principles file, then spot-check 3-5 key source files for violations: **SOLID:** - **SRP** — any god-class/god-module doing auth + profile + email + notifications? Flag bloated files (>300 LOC with mixed responsibilities). - **DIP** — are services injected or hardcoded? Look for `new ConcreteService()` inside business logic instead of dependency injection. **DRY vs Rule of Three:** - Search for duplicated logic blocks (Grep for identical function signatures across files). - But don't flag 2-3 similar lines — duplication is OK until a pattern emerges. **KISS:** - Over-engineered abstractions for one-time operations? - Feature flags or backward-compat shims where a simple change would do? - Helpers/utilities used only once? **Schemas-First (SGR):** - Are Pydantic/Zod schemas defined before logic? Or is raw data passed around? - Are API responses typed (not `any` / `dict`)? - Validation at boundaries (user input, external APIs)? **Clean Architecture:** - Do dependencies point inward? Business logic should not import from UI/framework layer. - Is business logic framework-independent? **Error Handling:** - Fail-fast on invalid inputs? Or silent swallowing of errors? - User-facing errors are friendly? Internal errors have stack traces? Report: - Principles followed: list key ones observed - Violations found: with file:line references - Severity: MINOR (style) / MAJOR (architecture) / CRITICAL (data loss risk) ### 10. Commit Quality Check git history for the current track/feature: ```bash git log --oneline --since="1 week ago" 2>&1 | head -30 ``` **Conventional commits format:** - Each commit follows `<type>(<scope>): <description>` pattern - Types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`, `perf`, `style` - Flag: generic messages ("fix", "update", "wip", "changes"), missing type prefix, too-long titles (>72 chars) **Atomicity:** - Each commit = one logical change? Or monster commits with 20 files across unrelated features? - Revert-friendly? Could you `git revert` a single commit without side effects? **SHAs in plan.md:** - Check that completed tasks have `<!-- sha:abc1234 -->` comments - Check that phase checkpoints have `<!-- checkpoint:abc1234 -->` ```bash grep -c "sha:" docs/plan/*/plan.md 2>/dev/null || echo "No SHAs found" ``` **Pre-commit hooks:** Read the stack YAML `pre_commit` field to know what system is expected (husky/pre-commit/lefthook) and what it should run (linter + formatter + type-checker). Then verify: ```bash # Detect what's configured [ -f .husky/pre-commit ] && echo "husky" || [ -f .pre-commit-config.yaml ] && echo "pre-commit" || [ -f lefthook.yml ] && echo "lefthook" || echo "none" ``` - **Hooks installed?** Check config files exist AND hooks are wired (`core.hooksPath` for husky, `.git/hooks/pre-commit` for pre-commit/lefthook). - **Hooks match stack?** Compare detected system with stack YAML `pre_commit` field. Flag mismatch. - **`--no-verify` bypasses?** Check if recent commits show signs of skipped hooks (e.g., lint violations that should've been caught). Flag as WARN. - **Not configured?** Flag as WARN recommendation — stack YAML expects `{pre_commit}` but nothing found. Report: - Total commits: {N} - Conventional format: {N}/{M} compliant - Atomic commits: YES / NO (with examples of violations) - Plan SHAs: {N}/{M} tasks have SHAs - Pre-commit hooks: {ACTIVE / NOT INSTALLED / NOT CONFIGURED} (expected: {stack pre_commit}) ### 11. Documentation Freshness Check that project documentation is up-to-date with the code. **Required files check:** ```bash ls -la CLAUDE.md README.md docs/prd.md docs/workflow.md 2>&1 ``` **CLAUDE.md:** - Does it reflect current tech stack, commands, directory structure? - Are recently added features/endpoints documented? - Grep for outdated references (old package names, removed files): ```bash # Check that files mentioned in CLAUDE.md actually exist grep -oP '`[a-zA-Z0-9_./-]+\.(ts|py|swift|kt|md)`' CLAUDE.md | while read f; do [ ! -f "$f" ] && echo "MISSING: $f"; done ``` **README.md:** - Does it have setup/run/test/deploy instructions? - Are the commands actually runnable? **docs/prd.md:** - Do features match what was actually built? - Are metrics and success criteria defined? **AICODE- comments:** ```bash grep -rn "AICODE-TODO" src/ app/ lib/ 2>/dev/null | head -10 grep -rn "AICODE-ASK" src/ app/ lib/ 2>/dev/null | head -10 ``` - Flag unresolved `AICODE-TODO` items that were completed but not cleaned up - Flag unanswered `AICODE-ASK` questions - Check for `AICODE-NOTE` on complex/non-obvious logic **Dead code check:** - Unused imports (linter should catch, but verify) - Orphaned files not imported anywhere - If `knip` available (Next.js): `pnpm knip 2>&1 | head -30` Report: - CLAUDE.md: CURRENT / STALE / MISSING - README.md: CURRENT / STALE / MISSING - docs/prd.md: CURRENT / STALE / MISSING - docs/workflow.md: CURRENT / STALE / MISSING - AICODE-TODO unresolved: {N} - AICODE-ASK unanswered: {N} - Dead code: {files/exports found} ### 12. Context Quality Check Verify that the project is set up for efficient future agent sessions: **Plan handoff quality:** - Does plan.md have a `## Context Handoff` section with Intent/Key Files/Decisions/Risks? - Are all `[x]` tasks annotated with `<!-- sha:... -->`? - Could a new agent pick up this plan without re-reading the whole codebase? **CLAUDE.md signal-to-noise:** - Size check: `wc -c CLAUDE.md` — warn if >40,000 chars (attention dilution) - Are Do/Don't rules actionable and specific? - Are file paths still valid? (quick grep for referenced files that don't exist) **Scratch files cleanup:** - If `scratch/` directory exists, check if it has stale files from build phase - Recommend cleanup if >10 files or >1MB total Report: - Plan handoff: {GOOD / INCOMPLETE / MISSING} - CLAUDE.md size: {N} chars — {OK / WARN / BLOATED} - Scratch cleanup: {CLEAN / NEEDS CLEANUP / N/A} - Status: {PASS / WARN} ### 13. Pre-mortem (Launch Risk Assessment) _Renumbered: was #12 before Context Quality Check was inserted._ If the project is about to ship (verdict heading toward SHIP), run a quick pre-mortem: **Imagine it's 3 months after launch and the product failed. What went wrong?** Classify risks using Tigers/Paper Tigers/Elephants: | Risk | Type | Likelihood | Impact | Mitigation | |------|------|-----------|--------|------------| | ... | Tiger (real + scary) | High | High | ... | | ... | Paper Tiger (scary but unlikely) | Low | High | Accept | | ... | Elephant (obvious but ignored) | High | Medium | ... | Focus on: - **Data loss scenarios** — what if the local DB corrupts? Is there export/backup? (offline-first principle from `templates/principles/manifest.md`) - **Privacy incidents** — what data leaves the device? Could a breach expose users? - **Single point of failure** — one API key, one server, one payment provider? - **Market timing** — is a bigger player about to launch this? (check research.md competitors) Include in review report under "### Pre-mortem" section. If any Tiger risk has no mitigation, add it as a FIX FIRST task. ### 14. Visual/E2E Testing Use `references/qa-issue-taxonomy.md` for severity classification and per-page exploration checklist. If browser tools or device tools are available, run a visual smoke test. **Web projects (Playwright MCP or browser tools):** 1. Start dev server (use `dev_server.command` from stack YAML, e.g. `pnpm dev`) 2. Use Playwright MCP tools (or browser-use skill) to navigate to the main page 3. Verify it loads without console errors, hydration mismatches, or React errors 4. Navigate to 2-3 key pages (based on spec.md features) 5. Take screenshots at desktop (1280px) and mobile (375px) viewports 6. Look for broken images, missing styles, layout overflow **iOS projects (simulator):** 1. Build for simulator: `xcodebuild -scheme {Name} -sdk iphonesimulator build` 2. Install and launch on booted simulator 3. Take screenshot of main screen 4. Check simulator logs for crashes or assertion failures **Android projects (emulator):** 1. Build debug APK: `./gradlew assembleDebug` 2. Install and launch on emulator 3. Take screenshot of main activity 4. Check logcat for crashes or ANRs: `adb logcat '*:E' --format=time -d 2>&1 | tail -20` **If tools are not available:** skip this dimension, note as "N/A — no browser/device tools" in the report. Visual testing is never a blocker for SHIP verdict on its own. Report: - Platform tested: {browser / simulator / emulator / N/A} - Pages/screens checked: {N} - Console errors: {N} - Visual issues: {NONE / list} - Responsive: {PASS / issues found} - Status: {PASS / WARN / FAIL / N/A} ## Review Report Generate the final report: ``` Code Review: {project-name} Date: {YYYY-MM-DD} ## Verdict: {SHIP / FIX FIRST / BLOCK} ### Summary {1-2 sentence overall assessment} ### Tests - Total: {N} | Pass: {N} | Fail: {N} | Skip: {N} - Coverage: {N}% - Status: {PASS / FAIL} ### Linter - Errors: {N} | Warnings: {N} - Status: {PASS / WARN / FAIL} ### Build - Status: {PASS / FAIL} - Warnings: {N} ### Security - Vulnerabilities: {N} (critical: {N}, high: {N}, moderate: {N}) - Hardcoded secrets: {NONE / FOUND} - Status: {PASS / WARN / FAIL} ### Acceptance Criteria - Verified: {N}/{M} - Missing: {list} - Status: {PASS / PARTIAL / FAIL} ### Plan Progress - Tasks: {N}/{M} complete - Phases: {N}/{M} complete - Status: {COMPLETE / IN PROGRESS} ### Production Logs - Platform: {Vercel / Cloudflare / Hetzner / N/A} - Errors: {N} | Warnings: {N} - Status: {CLEAN / WARN / ERRORS / N/A} ### Dev Principles - SOLID: {PASS / violations found} - Schemas-first: {YES / raw data found} - Error handling: {PASS / issues found} - Status: {PASS / WARN / FAIL} ### Commits - Total: {N} | Conventional: {N}/{M} - Atomic: {YES / NO} - Plan SHAs: {N}/{M} - Status: {PASS / WARN / FAIL} ### Documentation - CLAUDE.md: {CURRENT / STALE / MISSING} - README.md: {CURRENT / STALE / MISSING} - AICODE-TODO unresolved: {N} - Dead code: {NONE / found} - Status: {PASS / WARN / FAIL} ### Context Quality - Plan handoff: {GOOD / INCOMPLETE / MISSING} - CLAUDE.md size: {N} chars — {OK / WARN / BLOATED} - Scratch cleanup: {CLEAN / NEEDS CLEANUP / N/A} - Status: {PASS / WARN} ### Pre-mortem - Tigers: {N} (real risks with mitigation needed) - Paper Tigers: {N} (scary but acceptable) - Elephants: {N} (obvious, being addressed) - Unmitigated Tigers: {list — these are FIX FIRST} - Status: {CLEAR / RISKS IDENTIFIED / BLOCKERS} ### Visual Testing - Platform: {browser / simulator / emulator / N/A} - Pages/screens: {N} - Console errors: {N} - Visual issues: {NONE / list} - Status: {PASS / WARN / FAIL / N/A} ### Issues Found 1. [{severity}] {description} — {file:line} 2. [{severity}] {description} — {file:line} ### Recommendations - {actionable recommendation} - {actionable recommendation} ``` **Verdict logic:** - **SHIP**: All tests pass, no security issues, **ALL acceptance criteria verified (not PARTIAL)**, build succeeds, production logs clean, docs current, commits atomic, no critical visual issues, no unmitigated Tiger risks - **FIX FIRST**: Minor issues, **PARTIAL acceptance criteria (any criterion FAILED or unverified)**, warnings, low-severity vulns, intermittent log errors, stale docs, non-conventional commits, minor SOLID violations, minor visual issues, Tiger risks with feasible mitigations — list what to fix. **PARTIAL acceptance = always FIX FIRST, never SHIP.** - **BLOCK**: Failing tests, security vulnerabilities, missing critical features, production crashes in logs, missing CLAUDE.md/README.md, critical architecture violations, app crashes on launch, unmitigated Tiger risks with high impact — do not ship ## Post-Verdict: CLAUDE.md Revision After the verdict report, revise the project's CLAUDE.md to keep it lean and useful for future agents. ### Steps: 1. **Read CLAUDE.md** and check size: `wc -c CLAUDE.md` 2. **Add learnings from this review:** - New Do/Don't rules discovered during review - Updated commands, workflows, or architecture decisions - Fixed issues or gotchas worth remembering - Stack/dependency changes (new packages, removed deps) 3. **If over 40,000 characters — trim ruthlessly:** - Collapse completed phase/milestone histories into one line each - Remove verbose explanations — keep terse, actionable notes - Remove duplicate info (same thing explained in multiple sections) - Remove historical migration notes, old debugging context - Remove examples that are obvious from code or covered by skill/doc files - Remove outdated troubleshooting for resolved issues 4. **Verify result ≤ 40,000 characters** — if still over, cut least actionable content 5. **Write updated CLAUDE.md**, update "Last updated" date ### Priority (keep → cut): 1. **ALWAYS KEEP:** Tech stack, directory structure, Do/Don't rules, common commands, architecture decisions 2. **KEEP:** Workflow instructions, troubleshooting for active issues, key file references 3. **CONDENSE:** Phase histories (one line each), detailed examples, tool/MCP listings 4. **CUT FIRST:** Historical notes, verbose explanations, duplicated content, resolved issues ### Rules: - Never remove Do/Don't sections — critical guardrails - Preserve overall section structure and ordering - Every line must earn its place: "would a future agent need this to do their job?" - Commit the update: `git add CLAUDE.md && git commit -m "docs: revise CLAUDE.md (post-review)"` ## AFTER CLAUDE.md revision — output signal EXACTLY ONCE: Output pipeline signal ONLY if pipeline state directory (`.solo/states/`) exists. **Output the signal tag ONCE and ONLY ONCE.** Do not repeat it. The pipeline detects the first occurrence. **If SHIP:** output this exact line (once): ``` <solo:done/> ``` **If FIX FIRST or BLOCK:** 1. Open plan.md and APPEND a new phase with fix tasks (one `- [ ] Task` per issue found) 2. Change plan.md status from `[x] Complete` to `[~] In Progress` 3. Commit: `git add docs/plan/ && git commit -m "fix: add review fix tasks"` 4. Output this exact line (once): ``` <solo:redo/> ``` The pipeline reads these tags and handles all marker files automatically. You do NOT need to create or delete any marker files yourself. **Output the signal tag once — the pipeline detects the first occurrence.** ## Error Handling ### Tests won't run **Cause:** Missing dependencies or test config. **Fix:** Run `npm install` / `uv sync`, check test config exists (jest.config, pytest.ini). ### Linter not configured **Cause:** No linter config file found. **Fix:** Note as a recommendation in the report, not a blocker. ### Build fails **Cause:** Type errors, import issues, missing env vars. **Fix:** Report specific errors. This is a BLOCK verdict — must fix before shipping. ## Two-Stage Review Pattern When reviewing significant work, use two stages: **Stage 1 — Spec Compliance:** - Does the implementation match spec.md requirements? - Are all acceptance criteria actually met (not just claimed)? - Any deviations from the plan? If so, are they justified improvements or problems? **Stage 2 — Code Quality:** - Architecture patterns, error handling, type safety - Test coverage and test quality - Security and performance - Code organization and maintainability ## Verification Gate **No verdict without fresh evidence.** Before writing any verdict (SHIP/FIX/BLOCK): 1. **Run** the actual test/build/lint commands (not cached results). 2. **Read** full output — exit codes, pass/fail counts, error messages. 3. **Confirm** the output matches your claim. 4. **Only then** write the verdict with evidence. Never write "tests should pass" — run them and show the output. ## Rationalizations Catalog | Thought | Reality | |---------|---------| | "Tests were passing earlier" | Run them NOW. Code changed since then. | | "It's just a warning" | Warnings become bugs. Report them. | | "The build worked locally" | Check the platform too. Environment differences matter. | | "Security scan is overkill" | One missed secret = data breach. Always scan. | | "Good enough to ship" | Quantify "good enough". Show the numbers. | | "I already checked this" | Fresh evidence only. Stale checks are worthless. | ## Red Flags — STOP immediately if you catch yourself thinking: - "I already know this code is fine" — Run the command. Fresh evidence only. - "The tests passed earlier, skip re-running" — Code changed. Run them NOW. - "This acceptance criterion is probably met" — Probably is not verified. Show evidence or mark FAILED. - "SHIP with minor issues" — Quantify "minor". If you can't, it's FIX FIRST. - "Security scan seems excessive for this project" — One missed secret = data breach. Always scan. - "I'll note this as a recommendation instead of blocking" — If it's a real risk, block. Recommendations get ignored. **Foundational principle:** A review without fresh evidence is not a review — it's a guess wearing a lab coat. Run every command. Read every output. No shortcuts. ## Critical Rules 1. **Run all checks** — do not skip dimensions even if project seems simple. 2. **Be specific** — always include file:line references for issues. 3. **Verdict must be justified** — every SHIP/FIX/BLOCK needs evidence from actual commands. 4. **Don't auto-fix code** — report issues and add fix tasks to plan.md. Let `/build` fix them. Review only modifies plan.md, never source code. 5. **Check acceptance criteria** — spec.md is the source of truth for "done". 6. **Security is non-negotiable** — any hardcoded secret = BLOCK. 7. **Fresh evidence only** — run commands before making claims. Never rely on memory.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.