Claude Skill

sota

Master router for the SOTA engineering skills library. Use this skill whenever the user asks to build, design, implement, refactor, harden, optimize, review, or audit an application, service, or codebase and the request spans more than one domain — or when you are unsure which sp

LLM Mart · 0 points · 13 25 0

Virus-scanned Reviewed automatically before listing.

Full trust report

Download martinholovsky-SOTA-skills-skills_sota-ec2abf6.zip · 51 KB

martinholovsky/sota-skills 23 3 forks CC-BY-4.0 Updated 6d ago

Part of martinholovsky/sota-skills — 39 skills

Install

skills CLI npx skills add https://github.com/martinholovsky/SOTA-skills/tree/main/skills/sota

Claude Code

claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install martinholovsky-sota-skills@llmmart

Git git clone https://github.com/martinholovsky/SOTA-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole martinholovsky/sota-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

SOTA Engineering Skills — Master Router

A library of 41 domain skills, each with a SKILL.md entry point and a rules/ folder of focused rule files (each under 500 lines). Each skill works in two modes:

BUILD — apply the rules while designing or writing code.
AUDIT — review existing code against the rules and emit findings in the canonical format below (it supersedes any per-skill variant): file:line | rule violated | severity (Critical/High/Medium/Low/Info) | effort (trivial/small/medium/large) | fix. Severity resolves the same way: rules/03 §1 is the floor. A skill's own Critical/High/Medium/Low table may refine it within that skill's domain and must say that it is doing so; an unscoped per-skill table does not outrank the model, and a cross-domain roll-up is always rated on it.

Read only what the task needs: first the relevant skill's SKILL.md (it has its own index of rules/ files with "read this when..." guidance), then only the rules files that match the code in front of you. Never load all skills at once.

Operating principles (always apply)

Validate every claim — mandatory. No claim ships unvalidated, in any mode. A claim is validated only by checking it against a primary source: code read in full context at the pinned commit (for findings), official docs/release notes/advisories fetched at use time (for versions, specs, CVEs, tool capabilities), or a reproduced behavior (for bugs). Training data, plausibility, and "the rules file says so" do not validate anything. A doc page states intent; only running it reports behaviour. "What does this command do when the input is missing/stale/hostile" is a behaviour question even though it reads like a tool capability, and the vendor's page can be stale or simply wrong: measured 2026-09-14, Poetry's own docs say a desynced lock produces a "Warning" while the tool exits 1, and a mechanism published from that summary had to be retracted. If a command is installable in two minutes, run it — both the failing and the passing case. A retraction is a claim and carries the same burden. Withdrawing a statement needs the primary source that falsifies it, exactly as asserting it did; "I was probably wrong" is not a finding, and it costs the reader a true statement. A wrong claim gets challenged because it asserts something; a wrong retraction sounds like humility and is waved through — nobody audits a confession. Three tells of over-correcting: it arrives within a message or two of an unrelated correction; it withdraws something you have evidence on screen for; it is a general loss of confidence rather than a named false proposition and what falsifies it. The moment after being corrected is the highest-risk moment in a session — the next claim needs more evidence, in both directions. Conceding everything is not the opposite of overconfidence; it is overconfidence about your own unreliability. What cannot be validated is either omitted or explicitly marked "needs verification" — never asserted. Before measuring, state what result would falsify the claim. If no obtainable result could, the experiment is theater — read the code path that decides the behavior instead of benchmarking its symptoms.
Freshness first. The library's version/spec/regulation facts were web-verified as of the last refresh (see README). Never trust them — or training data — for anything version- or CVE-sensitive at use time: re-verify current releases and advisories before pinning or recommending — and equally when you are merely measuring a third party's artifact, which does not feel like a version decision and is where this rule is skipped. Record the end-of-life date beside the version, every time. A version can be recalled fluently and with no felt uncertainty; an EOL date effectively cannot, so requiring it forces the lookup that "check this is current" does not. A version with no EOL date beside it has not been looked up. Detail, and what to do with a lapsed one: sota-devsecops rules/03 §3.9.
Stop-and-ask on security-relevant decisions. When a choice materially affects security posture (authn/z model, crypto primitive, trust boundary, secrets handling, network exposure), present the options with a recommendation and ask before proceeding. Do not silently pick.
Evidence over vibes. Every audit finding cites file:line, maps to a standard (CWE, OWASP, MITRE ATT&CK/ATLAS) where one applies, and proposes a concrete fix. Uncertain findings are marked "needs verification", never asserted. Borderline severities state the deciding assumption ("High if internet-facing; Medium if internal-only"). A negative claim needs more proof than a positive one: "no instances of X" and "I only looked one way" are indistinguishable from the outside, so before asserting absence, widen the search and use a second independent method — and state the search you actually ran. "Independent" means a different failure mode, not a different phrasing. Field-reported: two searches of the same tree agreed on zero and both were wrong, because both were grep -r over a directory of symlinks, which -r does not follow. The only check that works is a positive control — search for something you have already seen there, in the same invocation; if the control returns nothing, the instrument is broken and the absence is not evidence (sota-shell-scripting rules/06 §2). A control proves the instrument works; it does not prove the query asks the corpus's question. Varying the phrasing is not a second method, but it is a separate check you still owe: field-reported 2026-09-14, a gap was declared after searching "one instance | a single observation" with a control returning four files, while the corpus says "one sample" in six — a named failure mode among them. Working instrument, wrong vocabulary, false absence. Before asserting a gap, name the term the corpus itself would use — read a neighbouring section's headings — and search that. And a failed reproduction is an absence claim too — it carries this same burden and almost never gets it, because it does not feel like a search (sota-code-security rules/12 §1a.1). Where a control is unavailable because nothing is yet known to be in the target, print a denominator instead — bytes, members, total rows — beside the zero. And weigh who benefits from the source: a claim sourced to a party that sells the remedy is a hypothesis, not a measurement (sota/rules/03 §2).
Stack profile. If the repo or ~/.claude contains a profiles/*.md stack profile (preferred stores, auth provider, license policy, platform conventions), its choices are the defaults for BUILD mode and the expected baseline for AUDIT mode.
Universal build non-negotiables (apply regardless of routing). On any network-reachable endpoint or handler (HTTP, RPC, queue, webhook, upload), always include: (a) abuse control — rate limiting / quotas keyed to the caller; (b) transport enforcement — TLS, HSTS, no plaintext fallback; (c) tests for the logic; (d) structured logging without secrets/PII. These are cross-cutting, so they get silently dropped under a long, dense task even when a rules file covers them — a measured attention effect, not a coverage gap. Keep this list short and re-check it last, before you ship (BUILD step 4). If one is deliberately handled elsewhere (e.g. rate limiting at the gateway), say so — don't silently omit it.
Claim "done" only with evidence. Never report a task complete or a fix working from plausibility — "should work", "this fixes it", "Done!" are not evidence. State the check you actually ran and its result: test output with pass/fail counts and exit code, the command and its output, or the reproduced behavior. If you did not run it, say so plainly. Unverified completion is not completion — this applies to your own build output before you hand it back.
Restate from the artifact, never from your own summary. Re-reading your own write-up re-runs the reasoning that produced it — the weakest check available. Before a claim reaches anything user-visible, go back to the primary source: re-read the tool output, or re-run the command. Your earlier prose in this session is not a primary source — and neither is a verbatim copy of a file injected into context earlier in it. A quoted file reads as primary evidence in a way a summary does not, which is exactly what makes it the easier mistake: it may predate an edit made since. Re-read the path before asserting or editing against its contents. Summaries silently drop the case that contradicts them and raw output does not. Full adversarial procedure for audit findings: AUDIT step 7 and rules/03 §4.
Publishing under someone else's name raises the bar. A claim to the person who asked costs one reader's trust and is cheap to retract. A claim published as them — a PR review comment, an issue, a commit message, a mailing-list post — is public, attributed and effectively permanent. Verify every factual claim by execution rather than inference, say which parts you did not test, check the thread first for whether it is already known, and never publish on someone's behalf without approval of the final text. Full procedure: sota-docs-workflow rules/03 §8.
Match the rigour to the stakes — and name the level you chose. A spike, a one-off script, a local experiment: build it, say in one line that it is a prototype and what you left out, stop. Anything reachable by an untrusted caller or touching money, credentials or another tenant's data gets the full treatment including principle 5, whether or not the request said "quick"; if it is genuinely ambiguous, ask in one line. An unnamed shortcut is not a prototype.

Routing table

Skill	Use when the task involves...
`sota-architecture`	System design, service boundaries, monolith vs microservices, DDD, event-driven design, sagas/outbox, reconciliation against a third party that holds authoritative state, resilience (timeouts/retries/circuit breakers), scalability, multi-tenancy, 12-factor/cloud-native, architectural anti-patterns
`sota-code-security`	Writing or reviewing code that touches untrusted input, authn/authz, sessions/JWT/OAuth, crypto, XSS/CSRF/CORS, file uploads, deserialization, error/log hygiene, LLM/agent app security, silent control failure (a safeguard that looks enabled and does nothing)
`sota-threat-modeling`	Designing a new system/feature with security in mind, drawing trust boundaries and DFDs, STRIDE/LINDDUN, risk rating, reconstructing a threat model from an existing codebase
`sota-skill-security`	Installing, authoring, reviewing or auditing anything an agent loads as instructions — skills, plugins, rulesets, `AGENTS.md`/`CLAUDE.md`/`.cursorrules`: provenance and pinning, review-before-install, the instruction trust boundary (a PR that edits an agent file changes what your agent does), precedence and shadowing between overlapping skills, capability minimisation, and guidance that is confidently wrong
`sota-secrets-management`	API keys, passwords, tokens, signing/TLS/SSH keys, .env files, Vault/cloud secret managers, workload identity (OIDC), secret rotation, leak detection and remediation
`sota-sandboxing`	Isolation of untrusted code or input, least privilege, seccomp/Landlock/capabilities, container/K8s hardening, microVMs, WASM sandboxes, subprocess hygiene, sandboxing AI-agent code execution
`sota-performance`	Latency, throughput, profiling, memory usage, caching (incl. stampede protection), I/O and network efficiency, Core Web Vitals, performance regression in CI
`sota-async-concurrency`	async/await, threads, goroutines, channels, races, deadlocks, event-loop blocking, cancellation/timeouts, graceful shutdown, backpressure, bounded queues
`sota-api-design`	REST/HTTP semantics, pagination, idempotency, versioning/deprecation, GraphQL, gRPC/proto evolution, websockets/SSE/realtime, webhooks, API rate limiting and tenant isolation
`sota-devsecops`	CI/CD pipelines, GitHub Actions hardening, supply chain (SLSA, Sigstore, SBOM, dependency confusion), unused/inert dependencies and upstream-health checks, container builds, SAST/secret-scanning gates, Terraform/GitOps, admission control
`sota-databases`	Schema design, Postgres/NoSQL choice, migrations (zero-downtime), indexes/EXPLAIN, transactions/isolation, ledgers & account balances, connection pooling, replication/backups, Redis, RLS/DB security, pgvector
`sota-frontend-design`	UI/UX, visual design, typography/color/layout, design systems and tokens, components, forms, accessibility (WCAG 2.2), motion/animation design, modern CSS, responsive design
`sota-web-frameworks`	React/Next.js and Vue/Nuxt engineering — Server Components & Server Actions, the RSC/client trust boundary, Next caching (`use cache`/PPR/ISR), Nitro server routes, hydration correctness, SSR state serialization, and framework-specific security & CVEs
`sota-observability`	Logging, metrics, tracing (OpenTelemetry), SLOs/error budgets, alerting, health checks, dashboards, debugging production, "can we answer why is this slow?"
`sota-testing`	Test strategy (pyramid/trophy), unit vs integration boundaries, test design/smells, mocks/fakes/test data, contract testing, e2e, property-based/fuzzing/mutation testing, flaky tests, coverage policy
`sota-llm-engineering`	Building LLM features — evals, prompt/context engineering, structured output, RAG, agents/tool design, MCP, model selection/routing, latency/cost engineering, LLM observability — and any question about a model's tokens, context window, pricing or limits, including measuring your own files
`sota-ml-engineering`	Production ML/MLOps (classical/predictive, not LLM apps) — training→serving→monitoring lifecycle, feature stores & registries, data leakage & train/serve skew, evaluation (ML Test Score, slices), deployment (canary/shadow/rollback), drift monitoring (PSI/KS) & retraining, ML security/governance (poisoning, MITRE ATLAS, NIST AI RMF)
`sota-cloud-infrastructure`	Cloud accounts/landing zones, cloud IAM, VPC/subnet/DNS/CDN setup, compute selection (serverless vs containers vs K8s), object storage, FinOps/cost, RTO/RPO and disaster recovery
`sota-kubernetes`	Kubernetes platform security & ops — RBAC & escalation paths, admission control (PSA/Kyverno/Gatekeeper/VAP, Audit→Enforce), GitOps controllers (Argo CD/Flux, AppProject scoping), operators/CRDs/webhooks, control plane & etcd encryption, Helm supply chain, multi-tenancy, cluster lifecycle, K8s audit logging; self-hosted (Talos/k3s) and managed
`sota-identity-access`	Identity infrastructure & access management — OIDC/OAuth2.1/SAML/SCIM protocols, running an IdP (Kanidm/Keycloak/etc.), RBAC/ABAC/ReBAC authorization design, group→role mapping, joiner-mover-leaver lifecycle, deprovisioning, privileged access & break-glass, SPIFFE/workload identity, phishing-resistant MFA/passkeys, federation risk
`sota-network-security`	Network security as a discipline — zero-trust (NIST 800-207), segmentation & blast-radius, the `world`/`any` over-broad-rule trap, Kubernetes NetworkPolicy depth (Cilium L7, default-deny egress), service mesh & mTLS / internal encryption, edge/ingress/WAF, egress control & metadata-endpoint blocking, DNS/TLS/PKI & cert lifecycle, email auth (SPF/DKIM/DMARC)
`sota-confidential-computing`	Protecting workloads/data from the infrastructure operator — TEEs (AMD SEV-SNP, Intel TDX, ARM CCA, SGX enclaves, Nitro Enclaves, confidential GPUs), remote attestation (RATS, attest-then-release), confidential VMs/nodes/containers on K8s (CoCo/Kata/Trustee), and cryptographic PETs (FHE, MPC, ZKP, PSI) when hardware trust is off the table
`sota-detection-engineering`	Detective controls, SOC & IR — detection-as-code, Sigma/YARA/Suricata/Falco/Tetragon rules, ATT&CK coverage, SIEM & telemetry coverage, alert tuning/SOAR, threat hunting & intel (STIX/TAXII), deception/honeytokens, incident response (NIST 800-61), detection validation (Atomic Red Team/Caldera)
`sota-data-engineering`	Data pipelines, ELT/orchestration, dbt, Kafka/streaming, CDC, schema registry, lakehouse (Iceberg/Delta/Parquet), data quality/contracts, warehouse modeling
`sota-privacy-compliance`	PII inventory/classification, privacy by design, consent, DSAR/deletion architecture, retention, GDPR/CCPA/HIPAA/PCI/AI Act engineering obligations, SOC 2/ISO 27001 audit readiness, breach response
`sota-security-compliance`	Cybersecurity control frameworks & product-security regulations as engineering — NIST CSF 2.0, SP 800-53, 800-171/CMMC, SSDF (800-218), FedRAMP, EU Cyber Resilience Act (SBOM/CVD/signed updates), ISA/IEC 62443 (OT zones & conduits, Security Levels); control-framework-as-code crosswalks, CUI boundaries, FIPS-validated crypto
`sota-mobile`	iOS/Android/cross-platform apps — stack choice, offline-first/sync, push, mobile security (Keychain/Keystore, attestation), performance budgets, store requirements, staged rollouts
`sota-cli-ux`	CLI/developer-tool design — flags/subcommands, config precedence, stdout/stderr and --json contracts, exit codes, TTY detection, signals, completions, distribution
`sota-shell-scripting`	Bash/sh scripts, CI run blocks, entrypoints, Makefiles — safety baseline (quoting, set -euo pipefail, traps), injection, secrets in scripts, shellcheck/shfmt
`sota-docs-workflow`	Documentation (Diátaxis, READMEs, runbooks, API docs, changelogs, AGENTS.md), code review/PR workflow, commit/branch/release discipline
`sota-ux-writing`	Any user-facing interface text — microcopy, button/label wording, error messages, empty states, onboarding copy, notifications, tone of voice, terminology, alt text, i18n-ready strings
`sota-copywriting`	Outward-facing content — landing pages, headlines/CTAs, value propositions, SEO content, testimonials/social proof, claim substantiation, email marketing, app-store listings
`sota-rust`	Any Rust code — ownership/API design, error handling, unsafe discipline, tokio/async, supply chain (cargo audit/deny/vet), performance, clippy/CI
`sota-golang`	Any Go code — errors, package/interface design, goroutines/channels/leaks, net/http hardening, security (os/exec, os.Root, govulncheck), pprof/performance, golangci-lint/CI
`sota-c-cpp`	Any C/C++ code — RAII/idioms, memory safety (UAF/overflow/sanitizers), undefined behavior, security (CERT/MISRA, banned APIs, OpenSSF hardening flags), concurrency/atomics, CMake/clang-tidy/fuzzing CI, performance
`sota-jvm`	Any Java/Kotlin code — modern idioms (records/sealed/pattern-matching, Kotlin null-safety/coroutines), API/null/immutability design, concurrency (virtual threads, JMM, j.u.c, coroutines), security (deserialization/JNDI/XXE/injection, JCA crypto), GC/JFR/GraalVM performance, Maven/Gradle supply-chain & CI
`sota-python`	Any Python code — uv/ruff/typing setup, idioms/pitfalls, asyncio, security (pickle/subprocess/SQL), performance, FastAPI/Django/pytest
`sota-javascript-typescript`	Any JS/TS code — strict tsconfig/type design, idioms, promises/AbortController, Node backend hardening, XSS/supply-chain security, bundle/React performance, vitest/ESLint
`sota-dotnet`	Any C#/.NET code — modern idioms (records, nullable reference types, pattern matching, spans), API/disposal/DI design, async/await & concurrency (ConfigureAwait, cancellation, channels), security (EF/Dapper SQL, deserialization, ASP.NET Core auth, crypto), GC/Span/AOT performance, NuGet supply chain & analyzers/CI
`sota-php`	Any PHP code — strict_types/modern idioms (enums, readonly, match), security (PDO/SQLi, XSS escaping, uploads/LFI, unserialize/Phar, sessions, password_hash/sodium), framework-neutral web hardening, Composer supply chain, PHPStan/Psalm, OPcache/FPM/JIT performance
`sota-ruby`	Any Ruby code — idioms (frozen strings, pattern matching, RBS/Sorbet), security (AR/SQLi, ERB escaping, strong params, Marshal/YAML.load, command injection, ReDoS), Bundler supply chain (bundler-audit, lockfile checksums), RuboCop/Brakeman, GVL/Ractors/YJIT performance

Cross-cutting routing rules

Language skills stack on domain skills. Auditing a Go API server → sota-golang + sota- api-design + sota-code-security. The language skill covers idioms and runtime-specific traps; domain skills cover the design.
Security tasks usually need three skills. Code-level flaws → sota-code-security; design- level gaps → sota-threat-modeling; leaked or mishandled credentials → sota-secrets- management. Pipeline/supply-chain → sota-devsecops; isolation blast-radius → sota- sandboxing.
Performance complaints about queries → start in sota-databases (EXPLAIN, indexes, N+1) before sota-performance (caching, I/O).
Anything realtime (websockets, SSE, pub/sub fanout) → sota-api-design rules/05 + sota- async-concurrency (backpressure).
AI/LLM features → sota-code-security rules/08 (prompt injection, tool authorization) + sota-sandboxing rules/05 (executing model output) + sota-databases rules/07 (vectors/RAG).
Frontend work → sota-frontend-design for design/UX/a11y/motion; sota-web-frameworks for React/Next or Vue/Nuxt engineering (RSC/client boundary, Server Actions, caching, hydration, SSR security); sota-javascript-typescript for the language/TS; sota-performance rules/06 for Web Vitals. A React/Next or Vue/Nuxt security review pulls all four.
Tests accompany everything. Any BUILD task that writes logic also loads sota-testing (strategy + design rules); any AUDIT includes a suite-health pass. Language-specific runner mechanics stay in the language skills.
LLM features split three ways. Quality/architecture → sota-llm-engineering; security (prompt injection, tool authz) → sota-code-security rules/08; executing model output → sota- sandboxing rules/05; PII in prompts/logs → sota-privacy-compliance.
Infra layers split four ways. Cloud-provider setup (accounts, VPC, compute, cost, DR) → sota-cloud-infrastructure; the Kubernetes platform itself (RBAC, admission, GitOps controllers, operators, etcd) → sota-kubernetes; pod/container/workload isolation mechanics → sota-sandboxing; CI/CD and supply chain → sota-devsecops. A K8s cluster audit loads sota- kubernetes + sota-network-security + sota-sandboxing.
Identity is its own layer. App-level login/session/JWT-validation code → sota-code- security rules/02-03; identity infrastructure (IdP, OIDC/SAML config, RBAC/role-mapping design, provisioning, break-glass, SPIFFE) → sota-identity-access; the credentials themselves → sota-secrets-management.
Network: setup vs security. Cloud VPC/DNS/CDN provisioning → sota-cloud-infrastructure rules/03; segmentation, zero-trust, NetworkPolicy depth, service mesh/mTLS, egress/DNS/PKI posture → sota-network-security.
Prevention vs detection. Building the control → the relevant domain skill; verifying you'd catch the attack at runtime (logs, rules, hunting, IR) → sota-detection-engineering. Ops telemetry plumbing stays in sota-observability; design-time threat enumeration in sota- threat-modeling.
Ingesting untrusted/attacker-authored data (feeds, scraping, uploads, webhooks, RAG corpora, hostile parsers) → sota-code-security rules/09, with sota-sandboxing rules/04 for parser isolation. A tool ingesting whole repositories (scanner, SAST wrapper, review bot, agentic analyser) adds sota-sandboxing rules/05 §7 — staging, build execution, egress.
Data: OLTP vs analytics. App databases → sota-databases; pipelines, streaming, warehouse/lakehouse → sota-data-engineering; anything touching personal data → add sota- privacy-compliance.
Any handling of user/personal data (new fields, exports, logs, analytics, ML training) → check sota-privacy-compliance minimization and retention rules, even when the task isn't "about" privacy.
User-facing words split three ways. In-product UI text (labels, errors, empty states) → sota-ux-writing; marketing/site/email content → sota-copywriting; technical docs → sota- docs-workflow. The component patterns the text lives in stay sota-frontend-design.
Shell scripts hide everywhere — CI run blocks, Dockerfile RUN lines, Makefiles, entrypoints, and the one-liners you type to verify a claim: unlinted shell run against the system under test, which when wrong produces a false finding about the product (a usage error, exit 2, from the callee is the tell — but a failed glob is silent and fakes a clean result). Audit it all with sota-shell-scripting rules/01 §3 and rules/06, the file for pasted and agent-issued commands. And a script that produces or verifies evidence (attestations, ledgers, gate records, audit trails) is a security control written in shell: add sota-code-security rules/10, rules/12 and rules/15.
Cryptography fans out — there is no single crypto skill (by design). Algorithm choice, AEAD/nonce discipline, CSPRNG, in-code key handling, TLS client config, constant-time comparison, tamper-evident logs/audit ledgers (keyed hash chains, external anchoring, integrity-vs-completeness), crypto agility, and post-quantum migration → sota-code-security rules/04. The key material — storage backends (KMS/HSM, Vault, SOPS+age), lifecycle, rotation, per-credential-type handling → sota-secrets-management. Transport/PKI — TLS server config, cert lifecycle/ACME, private CA, mTLS → sota-network-security rules/06. FIPS-140-3-validated-module requirements → sota-security-compliance rules/02. Language- specific APIs (JCA, crypto/*, .NET) stay in the language skill. The stance throughout is use a vetted library, don't roll your own.
Which direction does the trust boundary point? Protecting the host from the workload (untrusted code, seccomp/microVMs/WASM sandboxes) → sota-sandboxing. Protecting the workload from the host/operator — TEEs (SEV-SNP/TDX/CCA/SGX), remote attestation, confidential VMs/containers, or computing on encrypted data (FHE/MPC/ZKP) → sota- confidential-computing. Both can apply to one system. Key custody/release stays sota- secrets-management; differential privacy and de-identification stay sota-privacy- compliance.
"It's enabled" is a claim, not a fact. Whenever a control's presence is established but its effect isn't — a banner, a config flag, a green test, "we have a scanner" — route to sota-code-security rules/10 (silent control failure). It pairs with sota-testing rules/06 (mutation-probe the control) and rules/09 (a security test must be watched to fail), sota- observability rules/05 (degradation must be visible), and sota-devsecops rules/04 (does the shipped artifact contain what the control needs at runtime?). Not for controls that are simply missing — that's the owning domain skill's audit checklist.
Model facts hide in your own tooling — the same shape as rule 17. Any question about a model's tokens, context window, pricing or limits is sota-llm-engineering (rules/02 §2 for counting, rules/05 for cost), including measuring your own files, prompts or docs, and even when the surrounding task is repo maintenance and nothing looks like an LLM feature. The tell is reaching for a chars/4 estimate or another vendor's tokenizer instead of the provider's count_tokens: measured 2026-08-26, that shortcut under-counts Claude by 54% on markdown-dense text — in the direction that makes you think you have room.

Day zero — a repo this library has not been applied to yet

Installing is ambient: the rules apply in every directory, including a repo with no gates, no agent file and no LICENSE. They then govern the code you write and nothing else — the repo keeps accepting unscanned commits. Check this once per repo, on the first BUILD task in an unfamiliar one, by looking (not inferring): is there a .pre-commit-config.yaml or any other hook manager or CI job running a secret scan; a license file (LICENSE*, COPYING* or COPYRIGHT — never match the bare name, projects name it after the licence); an AGENTS.md/CLAUDE.md; and how long is the git history? Two or more missing and a history of a few commits = day zero. A long history means a mature repo that likely decided against them; say nothing.

When it fires, say it once, in a line, with the command, then get on with the task: scripts/init-gates.sh + pre-commit install --hook-type pre-push for gates (before the first commit, while a leaked credential is still free to remove), scripts/gen-agents-md.sh for the cross-tool entry point. Full ordering and reasoning — LICENSE, .gitignore, ambient-vs-repo-resident, the core.symlinks trap — in sota-docs-workflow rules/01 §10.

Offer, never perform. These write config into the user's repo. Mention once, act only on a yes, and treat a decline as decided.

BUILD mode — workflow

Reasoning and worked examples: rules/02-build-workflow.md. Changing a step below? It is mirrored in four places — rules/02 §5 lists them, and three fail silently.

Identify the domains the feature touches (table above) and the language(s).
Load lean. Read each relevant skill's SKILL.md and, from its index, open only the rules files that match the work. Lean costs less and measures no worse; the degradation this step once claimed is not supported — rules/02 §1.
Plan first, with the checks in the plan. Before writing code, list the task's requirements as concrete, checkable items — a specific outcome you can mark done/not-done ("rate-limit login to N/min per IP", not "add rate limiting") — covering the top-10 non-negotiables of each loaded skill plus operating principle 5. Vague items don't survive to step 4. Then implement against that list.
Self-audit gate (do this LAST — do not present code until it passes). Re-read each loaded rules file's Audit checklist and operating principle 5, and verify your diff satisfies every item. For each unmet item, implement it or state why it is out of scope — silence is not allowed. For every control, ask the falsification question: if this were silently a no-op, would anything observable differ? If nothing would — no log, no metric, no failing test — it is not done (sota-code-security rules/10). If the control emits an artifact, read back the one this run just produced. Doing this last is deliberate: a long context makes mid-context rules fade, and the final re-read is what recovers the rate limiting, transport, tests and logging a model otherwise drops — measured as the bulk of the library's completeness lift (evals/run-completeness.py). For a large build, run it as a separate pass over the diff, and push the few critical invariants into deterministic gates (rules/02 §3).

AUDIT mode — workflow

Procedure lives in two rules files, read together for any full audit: rules/01-audit-methodology.md (scoping, the verified tool matrix, triage, hygiene) and rules/03-audit-findings.md (severity — chain closure, the diff baseline — evidence, the decision ledger, refutation, the report template). A new pass needs a line in this router and a section in the rules file that owns it, never here.

For a focused audit, load the matching skills and follow their AUDIT sections. For a full project audit, work in passes:

Recon. Inventory languages, frameworks, entry points (HTTP routes, queues, cron, webhooks), data stores, CI config, Dockerfiles, IaC. This determines which skills apply; skip skills with no matching surface. Full procedure: rules/01 §2.
Threat model first. sota-threat-modeling rules/06 (reconstruction): assets, trust boundaries, entry points. Its output prioritizes the rest.
Per-domain passes. For each applicable skill, follow its AUDIT mode and audit checklists. Suggested order: secrets sweep (fast, high yield) → code security (incl. rules/09 untrusted-data ingestion) → language-specific (incl. shell scripts) → API → database → async/concurrency → identity & access → sandboxing/devsecops → kubernetes platform → network security → cloud infrastructure → privacy/compliance → architecture → testing suite health → performance → observability → detection-engineering posture → frontend/a11y → LLM features, data pipelines, mobile, CLI, docs as applicable. For an infrastructure/cluster audit the heavy hitters are sota-kubernetes, sota-network-security, sota-identity-access, sota-sandboxing and sota-detection-engineering.
Silent-control pass (always run it). The per-domain passes ask "is the control there?" — this one asks "does it do anything?", then "does it cover every site it is credited with?". Apply sota-code-security rules/10 (inert controls — invisible to the other passes and to SAST), sweeping with rules/11 first to find where to look: stage duration vs work claimed, every gate's denominator (0 checked, 0 failed, exit 0), size-gated paths no fixture crosses, cache keys narrower than the behaviour, one-sample parsers, assert-as-control. Then count: sota-code-security rules/14 §6 censuses what a working mitigation guards (9 of 61 is the finding), §7 falsifies the prose.
Decision-ledger review. Code passes find defects in what was built; they cannot find the defect where the code faithfully implements a choice that stopped being right — a store picked for scale that never arrived, an expired constraint still shaping the design. Reconstruct the expensive-to-reverse decisions (ADRs, design docs, CHANGELOG, the PRs behind each major component) and classify each JUSTIFIED / STALE / UNJUSTIFIED / UNVERIFIABLE. Where a decision rests on a number, re-measure it this session. Full procedure: rules/03 §3. Then ask where else the team's knowledge lives — an agent's private memory store, an IDE's notes, a chat log. Anything in there that is a fact about the repository with no home in the repository is a finding: invisible to review, absent from a fresh clone, gone when the store is cleared. It is an absence claim and the naive search for it lies — rules/01 §4a.
Findings. Emit every finding in the canonical cross-domain format (file:line | rule | severity | effort | fix) — skill-local formats are fine within a domain pass but must carry an effort field so the roll-up can be sequenced — deduplicate across domains, and roll up into the report structure from rules/03 §5. Severity: rules/03 §1 — a Critical/High names its chain, and a diff is rated against the code it replaced.
Refute before reporting. Re-reading your own finding re-runs the reasoning that produced it — it is the weakest check available. Every Critical/High gets an independent pass prompted to kill it (a separate agent, or a fresh-context hostile read), working from the code at the pinned commit rather than your write-up, defaulting to REFUTED when the evidence is ambiguous. Survivors ship; the rest are dropped or downgraded with the refutation recorded — and swept first: the refuted pattern routinely closes somewhere else, and that sweep is where the report's strongest finding comes from. Absence claims ("no X found") get a refuter too, and carry the heavier burden of principle 3. Full procedure and failure modes: rules/03 §4.

Library map (rules files per skill)

Which rules/NN file holds what, for all 41 skills: rules/04-library-map.md. Read it when you know the domain but not the file. When you are already opening a skill's SKILL.md (BUILD step 2), use that skill's own index instead — it carries the "read this when…" guidance the map drops.

Context budget discipline

Rules files run 77–500 lines, median 237 (re-measured over all 271 on 2026-09-13; the 2026-09-11 figures — 77–497 over 270 — were correct when written and went stale in a day, and before them this said "200–310", a range half of them fall outside, which is why the count and the date are stated). So budget by the file you are actually opening, not by an average: 2–5 files is a typical focused task and can be 400 lines or 2,000. A full audit pass should load one skill at a time, finish its findings, then move on. If context is tight, prefer the skill's top-10 non-negotiables plus the single most relevant rules file.

When this library is wrong or missing something

These rules are maintained and measured, and they are still incomplete — the library has no telemetry and learns nothing from use unless someone says so. You are the only observer of the gap at the moment it appears.

If, while applying a skill, you hit one of these, say so in one line at the end of your answer and point the user at the report path — then carry on with the task:

a rule contradicted by a primary source you just checked (version, spec, API, advisory) — freshness rot;
a rule that does not fit the situation and states no exception for it;
real surface area in the task with no owning skill — a routing gap;
guidance that, followed literally, would have shipped a defect.

Two loaded rules that contradict each other are usually a scope collision: one is a general default, the other a requirement inside a narrower domain. The narrower wins in its domain — pick by which failure mode is worse here (a needless idiom vs an outage), never silently: name the rule you followed and why in a comment beside the code, or the next reader reverts it. Then report it — the fix is an explicit exception in whichever rule was too broad.

A routing gap should end as a test, not just a report. If the right skill existed and the task never reached it, the fix is the trigger — the skill's description is the only auto-loading text and is the whole classifier — and the proof is a regression case in evals/cases/desc-routing-regressions.jsonl, which pins the mis-route so it cannot return. Run that case against the PRE-change tree and watch it fail first. A case that passes in both arms pins nothing, and an absent string is not an absent capability — a model routes on meaning, so grep cannot tell you whether a task is reachable. Measured 2026-09-14: a PowerShell task claimed to be a routing gap already routed correctly with 0 of 42 descriptions naming PowerShell, and the case written to pin the "fix" scored 1.00 before and after. Same doctrine as every gate here — watch it fail first, or you have added a control that cannot fail.

Report: https://github.com/martinholovsky/SOTA-skills/issues/new/choose (bad-guidance / skill-request templates). Anything dangerous or security-sensitive goes to a private advisory instead — see SECURITY.md.

Do not fire this for personal preference, for a rule you simply did not need, or mid-task. One line, at the end, only when the library actually let the user down. An unreported gap stays in the library for everyone.

Files (sota-skills)

rules

01-audit-methodology.md 27.3 KB

# Audit Methodology — Process, Tooling, Triage & Hygiene

Scope: this file governs **how** an audit is run — scoping, inventory, tool
selection, triage, and the hygiene rules that keep it reproducible and
non-destructive. What happens to what it finds — severity, evidence, the
decision-ledger pass, adversarial verification and the report template — is
`rules/03`. It does not contain domain findings either: **what** to check comes
from each domain skill's AUDIT mode and the Audit checklist at the end of every
rules file; route into those via the table in `SKILL.md`. Read this file and
`rules/03` first in any full or multi-domain audit; the checklist at the end of
each is the quality gate on the audit deliverable itself.

---

## 1. Scoping & rules of engagement

Agree these before reading a single line of code:

- **Target**: which repos/services, which branch, **pinned to a commit hash**.
  Findings against a moving target are not reproducible.
- **Environments**: static analysis only, or dynamic testing against running
  systems too? If dynamic: which environment (never production by default),
  what traffic/load is acceptable, who is informed.
- **Stop-and-ask rule**: before touching anything live, shared, or
  destructive — running scanners against deployed endpoints, mutating CI/CD,
  rotating credentials, opening cloud consoles — stop and confirm. An audit
  that breaks the system under audit is a failed audit.
- **State the yardstick up front.** Name the standards the audit asserts
  against; this makes findings defensible and disputes resolvable:
  - OWASP ASVS (state the level: L1 baseline, L2 standard, L3 high-assurance)
  - OWASP Top 10 (2025) and OWASP API Security Top 10 (2023)
  - CWE for weakness identification
  - MITRE ATT&CK for attacker-technique mapping
  - For LLM/agent code: OWASP Top 10 for LLM Applications, OWASP Agentic AI
    guidance, MITRE ATLAS
- **Time-box and prioritize crown jewels.** When time is bounded, depth beats
  breadth. Audit first, in order: authentication/session code, secrets
  handling and history, money and sensitive-data flows, internet-facing entry
  points, any path where untrusted LLM input reaches a tool or privileged
  action. Everything else comes after.
- **Record exclusions.** Anything out of scope (vendored code, generated
  files, a service owned by another team) is written down, not silently
  skipped.

## 1a. A changeset too large to hold at once — partition it, don't skim it

The authoring side of this is covered elsewhere (`sota-docs-workflow` rules/03: keep
PRs small, because large ones converge on "LGTM"). This is the **reviewer's** half,
which is the one you need when the large PR exists anyway and is yours to review.

**The failure mode is silent and it is not laziness.** Given more changed files than
fit comfortably in one pass, a reviewer — human or model — does not announce that it
ran out of room. It reviews some files carefully, skims others, and produces a report
whose *shape* is identical to a complete one. Nothing in the output says "I covered 9
of 23 files". Reported at scale by Alibaba's Open Code Review (Apache-2.0) as the
first of three failure modes of language-driven review, and it matches this library's
own measured **completeness residual** — a cross-cutting requirement quietly dropped
as context fills (`docs/WHY-COMPLETENESS-RESIDUAL.md`).

So do not let the model decide coverage. Decide it deterministically, before review:

1. **Enumerate the units first, mechanically.** `git diff --name-only <base>...<head>`
   is the denominator. Write it down. Every file is either reviewed, or explicitly
   excluded with a reason (§1's *record exclusions*) — never neither.
2. **Bundle related files into one unit.** Files that must be read together to be
   judged go together: a handler and its test, a migration and the model it alters,
   the four locale files that must stay in sync, an interface and its implementations.
   A bundle is the unit whose *internal consistency* is the thing being checked, and
   splitting one across passes is how a mismatch survives review.
3. **Give each bundle its own pass with fresh context.** Isolated context per bundle
   is what keeps pass 9 as sharp as pass 1; it also parallelises, but that is a
   side benefit, not the reason.
4. **Report the denominator with the findings.** "23 files changed, 23 reviewed in 7
   bundles, 2 excluded (generated, vendored)" is part of the result. A findings list
   with no coverage statement is indistinguishable from a partial one — the same
   fail-closed discipline the gates in this repo apply to their own file counts.

**Partitioning is not the same as sampling.** A sample is a defensible answer to
"what is the state of this codebase"; it is never a defensible answer to "is this
change safe to merge", where the unreviewed file is exactly where the defect is.

## 2. Inventory & recon — build the map before judging

You cannot audit what you have not mapped. Enumerate:

- **Languages, frameworks, runtimes — with versions.** This selects the
  language skills, the tool matrix rows, and flags EOL runtimes immediately.
- **Entry points / attack surface**: HTTP routes, WebSocket/SSE endpoints,
  queue/stream consumers, cron and scheduled jobs, webhooks (inbound and
  outbound), CLI surfaces, MCP tools/servers and any other agent-reachable
  interfaces.
- **Trust boundaries & data flows**: where untrusted input enters, where it
  crosses a privilege boundary, where sensitive data lives and moves. Sketch
  a DFD — follow `sota-threat-modeling` rules/02 (decomposition) and rules/06
  (reconstructing a threat model from an existing codebase). The threat model
  output prioritizes every later pass.
- **Secrets surface**: how secrets are stored and injected (env, files,
  SOPS/age, Vault, cloud secret managers, workload identity), plus a history
  scan for past leaks (tools in §3).
- **Dependencies & supply chain**: lockfiles and manifests, base images,
  CI workflow definitions and third-party actions, existing SBOMs,
  signing/provenance setup. Record which declared dependencies, registered
  modules, and plugins are actually **reached from an entrypoint** — the
  declared-but-inert ones are a finding CVE scanning structurally cannot see
  (`sota-devsecops` rules/10).
- **Deploy & runtime config**: Dockerfiles/Containerfiles, K8s manifests and
  Helm charts, Terraform/IaC, network policies, GitOps definitions.

Then **map every inventory item to the routing table in `SKILL.md`** and load
the matching skills' AUDIT modes. Skip skills with no matching surface; record
that you skipped them and why. An inventory item with no owning skill is
itself a gap worth noting.

## 3. Tool matrix & triage

Tools find the mechanical 60%; manual review finds the design flaws. Run
both, never just one. The matrix below was verified current as of 2026-06;
tools rename, fork, and die — **verify the current name and version of each
tool before invoking it** (one quick search; e.g. Semgrep's OSS engine was
forked to Opengrep in 2025 after a license split). Prefer the open-source
option where capability is equivalent.

| Area | Tools (verify current before use) | Notes |
|---|---|---|
| Secrets in code & git history | gitleaks; trufflehog | gitleaks is feature-complete (security patches only); still the standard scanner. trufflehog additionally *verifies* credentials live — never run verification against creds you must not touch. detect-secrets (Yelp, actively maintained) is a solid baseline scanner; prefer the first two for breadth and live verification. |
| Python SAST + deps | bandit; Opengrep/Semgrep CE; pip-audit | pip-audit is PyPA-maintained and can suggest fixes. |
| Rust | cargo-audit; cargo-deny; clippy `-D warnings` | cargo-deny also covers licenses and banned crates; clippy ships with the toolchain. |
| Go | gosec; govulncheck; staticcheck; `go test -race` | govulncheck is the official Go team scanner — call-graph-aware, low false positives. |
| JS/TS + Node | eslint-plugin-security (eslint-community); `npm audit`/`pnpm audit`; osv-scanner | Socket.dev (commercial, free tier) adds behavioral malicious-package detection beyond CVE lookup. |
| Multi-language SAST | Opengrep or Semgrep CE + community rulesets | Opengrep (LGPL fork, multi-vendor consortium) restores cross-function taint analysis that Semgrep CE gated commercially; rule format is compatible across both. |
| SCA — any ecosystem | osv-scanner (Google); trivy; grype | Run one as primary; a second only to cross-check noisy results. |
| Containers / images | trivy; grype; dockle | Verify base-image digest pinning manually. dockle's release cadence is slow — treat as supplementary lint, not the primary gate. |
| SBOM | syft (generate) → grype (scan) | trivy can also emit SBOMs (CycloneDX/SPDX). |
| Supply-chain signing & provenance | cosign verify (with `--certificate-identity` / `--certificate-oidc-issuer` for keyless); slsa-verifier | Verify provenance/attestations actually chain to the expected builder identity, not merely that a signature exists. |
| IaC / K8s | checkov; trivy (misconfig scanning); kubescape; kube-linter | kubescape is CNCF-incubating; kube-linter is lightweight and CI-friendly. |
| CI workflow security | zizmor | Static analysis of GitHub Actions workflows: template injection, credential persistence, ref spoofing, excessive permissions. |
| Licenses | cargo-deny (Rust); trivy license scan; syft SBOM license fields | Filter against the project's allowed-license policy. |

Run each tool against the pinned commit; record the exact tool version and
command line (needed for §4 reproducibility).

### Triage discipline — tool output is raw material, not findings

- **Never paste raw scanner dumps into the report.** A scanner hit becomes a
  finding only after a human (you) confirms it.
- **Confirm each hit is real**: read the flagged code in context; filter
  false positives and unreachable code paths.
- **Deduplicate** across tools and across domain passes — one weakness
  reported by four tools is one finding.
- **Re-rate exploitability in this context.** A tool's "high" in dead code
  may be Info; a tool's "low" on an internet-facing auth path may be your
  worst finding. Tool severity is an input, never the output.
- **Suppressions are findings too**: inspect existing `#nosec`,
  `# nosemgrep`, `nolint`, audit-ignore files and the like — each one is
  either justified (note it) or a hidden finding.

### Collect deterministically, then judge

Keep the two halves apart, in this order. **Enumeration is a script's job**: it must be
exhaustive over a scope you can state, and it must print its denominator (`261 rules
files`, `47 handlers`, `12 workflows`) so the number is auditable. **Judgment is the
model's job**, and it runs over the *complete* collected set, not over whatever the
search happened to surface.

Inverting them is the standard way an audit acquires a confident blind spot. A judgment
pass over a sampled or grep-shaped set inherits the sample's gaps and reports with the
same confidence as one that saw everything — and the miss is invisible in the output,
because a finding list looks identical whether the census behind it was 12 of 12 or 12
of 61. The corollary for the write-up: a **count** is a claim about the denominator, so
"9 of 61 call sites are guarded" is a finding, while "several call sites are unguarded"
is an impression (`sota-code-security` rules/14 §6–§7). Where the enumeration cannot be
scripted, say so and bound the claim to what you did read.

### Manual review — what tools cannot see

Budget explicit manual passes for the classes SAST is structurally blind to:

- Business-logic flaws (order of operations, state machines, refund/limit
  logic).
- Authorization and object-level access (BOLA/IDOR) — tools verify *authn*
  exists, rarely that *authz* is correct per object.
- Trust-boundary crossings the DFD revealed: does validation actually happen
  at the boundary, or three layers later?
- Race conditions and TOCTOU (pair with `sota-async-concurrency` rules/07).
- Crypto misuse: right primitive, wrong protocol; key handling; nonce reuse.
- Prompt-injection, excessive-agency, and tool-poisoning paths in LLM/agent
  code (pair with `sota-code-security` rules/08).
- **Controls that exist but are inert** — a safeguard whose success and whose
  total failure look identical from outside. SAST is blind to this by
  construction: the code isn't wrong, it's a no-op. Run it as its own pass
  (`sota-code-security` rules/10) over the controls the earlier passes
  confirmed exist.

### An empty page is a fact about your fetcher, not about the source

A client-rendered site returns **HTTP 200 with no content** to a plain fetcher, and that is
indistinguishable from a page that genuinely says nothing. The failure presents as *"the
source is empty"* — a finding about the **content** — when it is a finding about the
**instrument**.

Measured 2026-09-13 on a source four separate research passes had written off as unreachable:

```console
GET evals.mitre.org/results/enterprise   → HTTP 200,   3,150 bytes,       2 words of text
GET evals.mitre.org/api/adversaries/     → HTTP 200, 1,854,829 bytes, 124,872 words
```

**The fix is usually one level down, and frequently unauthenticated.** Before concluding a
source is gated or empty, look for the API feeding the page: an `/api/` path, a
`__NEXT_DATA__` or `window.__INITIAL_STATE__` blob in the HTML, a sitemap, or an `.json`
sibling of the route. Where none exists, browser automation renders it; where even that
fails, an archive's capture of a retired **API endpoint** often survives when the archived
HTML is only a redirect stub.

- **Treat a body under a few hundred words from a documentation or data site as a broken
  instrument**, not a short page. Print the byte count and the visible-word count beside any
  claim you draw from a fetch — the denominator rule (`sota-shell-scripting` rules/06 §2)
  applied to retrieval.
- **"The page was empty" is never evidence the fact does not exist.** Say which retrieval
  method failed. An absence sourced to one fetcher carries the burden in `rules/03` §2, and
  a second method with the *same* failure mode is not a second method.

## 4. Audit hygiene

- **Reproducible**: pin the commit; record exact tool versions and full
  command lines so anyone can re-run the audit and re-verify each finding.
- **Read-only by default**: do not mutate the audited system — no fixes
  applied silently, no CI/CD edits, no secret rotation, no infra changes.
  Propose changes; apply only on explicit instruction, as a separate task.
- **No secret values in the report**: when you find a leaked secret, redact
  the value, reference its location (`file:line`, commit) and type, and flag
  rotation as the remediation. Treat the report itself as a sensitive
  artifact — it is a map of the system's weaknesses.
- **Findings stay in the report**, not scattered in code comments or TODOs
  added to the audited repo.
- **Re-audit loop**: after remediation, re-run the same tools at the new
  commit and re-execute the relevant skill checklists against the changed
  code — confirm fixes, catch regressions, and check that fixes did not
  introduce new findings. State this loop in the roadmap. **A fix is verified
  when a fresh search cannot get around it, not when the reported input stops
  working**: re-point the original hunt at the patched code with no knowledge of
  the fix. Anthropic's defending-code reference harness makes this its fourth
  patch gate — the code builds, the proof of concept no longer fires, the test
  suite passes, *and* *"a fresh find agent can't find a way around the fix."* A
  patch that closes one input and leaves the class open passes all three of the
  narrower checks.
- **Read the yield curve across waves.** Repeated audits of one codebase should
  show the finding **count fall while the difficulty rises** — earlier findings
  are fixed, so later passes have to reach deeper; the harness reports the same
  shape (*"the number of findings will likely go down, but the complexity will
  likely also go up"*). A count that stays flat wave after wave is a statement
  about the audit, not about the code: the waves were not independent — same
  prompt, same salient files, nothing carried over. Carry the already-reported
  findings into the next wave as an explicit exclusion so it is steered past them
  instead of re-deriving them, and treat a wave that returns the previous wave's
  list as a failed wave.

---

## 4a. Knowledge that lives only in an agent's memory

An assistant working a repository over months accumulates a private store — Claude Code's
`~/.claude/projects/<repo>/memory/`, an IDE's workspace notes, a chat history someone greps.
It is genuinely useful and some of it *must* stay there: facts about the machine (which
binary `grep` really is, which commands the harness refuses), account state, and anything a
public repo's own denylist would reject. **The audit question is narrower: is anything in
there a fact about the repository that the repository does not have?**

That fact is invisible to review, absent from a fresh clone, missing on a second machine, and
gone when the store is cleared — and nobody discovers this until the one person or session
that held it is not there. Known gaps, open items, why a decision was made, a measured
number, a deferral: if the only copy is in an agent's memory, the repo has a hole shaped
exactly like the thing everyone assumes is written down.

**How to run it**

1. **Enumerate the store** and classify each entry: *environment* (stays), *behavioural
   correction to the agent* (stays), or *repository fact* (must exist in the repo).
2. For every repository fact, **find its home**: an item ledger, a decision record, a results
   file, a changelog entry. Not "is it mentioned somewhere" — *which file owns it*.
3. **Report the ones with no home as findings**, and fix them by moving the fact into the
   repo, leaving the memory entry as a pointer.

**This is an absence claim, so it carries §3's burden — and the naive form of it lies.**
Measured on a 50-file store against a 16-million-character corpus: a literal scan of 148
distinct measured claims flagged **11** as memory-only, and **every one that was checked
turned out to be in the repo under different wording** — "10 of 32" was written there as *"10
of the old 32 freshness cases"*. A numeral is the worst possible search key for this, because
prose paraphrases numerals freely. Use a **distinctive neighbouring phrase** as the second
method (`signed-char`, `plugin-scanner`, `per-case progress`), run a positive control in the
same invocation, and read the hits rather than the count
(`sota-shell-scripting` rules/06 §2).

**The durable fix is a convention, not a sweep.** Memory should *cite* the repo, never
restate it — "landed v1.36.2", "ROADMAP 44" — so a stale note reads as a pointer to re-check
rather than a competing claim. Restated status drifts; that is the same failure the summary
table has (`rules/03` §2), one layer further out, and it is the reason this pass exists at
all rather than being a one-off cleanup.

## 4b. Resolve every citation before you ship it — position drift

`rules/03` §2 requires each finding's location to be "exact, clickable, reproducible".
That states the requirement and checks nothing, which is this library's most common
gap shape: the rule is written, the probe is missing. Here is the probe.

**Position drift is a finding that is right about the defect and wrong about where it
is.** The mechanism is ordinary: line numbers read off a diff hunk rather than the
file, a quote paraphrased from memory after the file scrolled out of context, a
`file:line` carried forward while the surrounding analysis moved on, or a path that is
correct in one module and repeated for its near-identical sibling. Alibaba's Open Code
Review (Apache-2.0) reports it from two years of production review as one of three
dominant failure modes; this repo has its own version, which is why **invariant 18**
exists — roughly 1,300 prose `§` references that broke silently on a renumber, and 20
more caught during a single rules-file split.

**It is dangerous because it degrades trust rather than triggering an error.** A
report whose citations do not resolve looks exactly like one whose citations do. The
reader who cannot find the code usually assumes they are looking in the wrong place;
if they do apply the fix, they apply it to whatever is at that line now.

Before any finding ships — and **before the adversarial pass of `rules/03` §4, not
after** — resolve every citation mechanically:

- [ ] **Re-read the file at the pinned commit.** Not your notes, not the diff, not
      the snippet you already quoted. Restating from your own earlier output re-runs
      the reasoning that produced the error (`SKILL.md` principle 7).
- [ ] **The quoted evidence appears at the cited line, byte-identical.** A paraphrase
      that "means the same thing" is a failed check: it means the quote was
      reconstructed, and a reconstructed quote is not evidence.
- [ ] **Diff line numbers were converted.** Hunk-relative and file-absolute numbers
      differ by the hunk offset, and the mistake is invisible because the result is
      still a plausible line in a real file.
- [ ] **The path is the one you read**, not its sibling — check the full path, not the
      basename. `src/auth/session.go` and `src/authz/session.go` both exist.

A citation that does not resolve **fails the finding, it does not soften it**. Fix the
location or drop the finding; never ship it with an approximate one. The cost
asymmetry is the whole argument for putting this first: this check is mechanical and
takes seconds, the refutation pass in `rules/03` §4 is expensive, and a finding that
cannot even be located does not deserve a refuter's attention.

**Run it as a real check, not an intention.** Verify it can fail before you trust a
clean result — point it at a citation you have deliberately broken and watch it
complain (`sota-code-security` rules/11 §7). A verification pass that has never
produced a failure is not evidence that the citations are right.

## 5. Changing the AUDIT workflow? Change all three places

The audit workflow lives in **three** surfaces and they drift independently:

| Surface | What it holds |
|---|---|
| `skills/sota/SKILL.md` §AUDIT | the seven passes, one imperative each — read on every audit |
| this file | scoping, recon, the tool matrix, triage, hygiene |
| `rules/03` | severity, evidence, the decision ledger, refutation, report template |

The router's §AUDIT is deliberately terse because it is read every time; detail belongs in
the two rules files. So a new pass needs **a line in the router and a section in whichever
rules file owns it**, and a change to an existing pass needs both updated together — a step
whose procedure contradicts the file it points at is worse than no step, because the reader
follows whichever they loaded. The split itself is a drift risk: a pass about *rating or
reporting* a finding belongs in `rules/03`, one about *running* the audit belongs here, and
a section added to the wrong file is found by nobody looking for it.

§AUDIT **is** hash-pinned, as of 2026-09-01 — invariant 20 in `scripts/check-invariants.sh`
holds `ROUTER_AUDIT_SHA`, and the build fails when the section moves. The pin does not know
whether the two rules files still agree; it only guarantees that **someone had to come and
look**, because bumping it is a deliberate edit in the same commit. So the sequence is:
change §AUDIT, re-read this section and `rules/03`, fix whichever half is now wrong, then set
the new hash. (Before that date this paragraph read *"nothing catches this automatically"*,
and it was true — the gate was parked on a trigger that had already been met without anyone
noticing: `run-repo-audit.py` pastes the whole router, §AUDIT included.)

## Audit checklist — quality gate on running the audit

- [ ] **Any source written off as empty or unreachable?** (§3) A client-rendered page returns
      **200 with almost no text** — measured, 2 words of visible text where the API behind it
      returned 124,872. Look for the `/api/` path, a `__NEXT_DATA__`/`__INITIAL_STATE__` blob
      or a sitemap before concluding a source says nothing, and print bytes plus visible-word
      count beside any claim drawn from a fetch.

Finding quality and report structure are checked by `rules/03`'s checklist; this
one covers coverage, tooling and hygiene. Both run.

**Coverage**
- [ ] Scope agreed: repos, branch, pinned commit, environments,
      static-vs-dynamic — and exclusions documented?
- [ ] Standards set named up front (ASVS level, OWASP Top 10 2025,
      API Top 10 2023, CWE, ATT&CK; LLM/ATLAS where applicable)?
- [ ] Full inventory done: languages+versions, entry points, trust
      boundaries/DFD, secrets surface, dependencies, deploy configs?
- [ ] Every inventory item mapped to a skill via the routing table, and each
      applicable skill's AUDIT mode executed (skips recorded with reasons)?
- [ ] Crown-jewel paths (auth, secrets, money/data flows, internet-facing,
      untrusted-LLM-input) audited in depth, first?

**Tooling & triage**
- [ ] Tool names/versions verified current before running (renames/forks
      checked), versions and commands recorded?
- [ ] Matrix coverage run per detected language plus secrets-history, SCA,
      containers, IaC/K8s, CI workflows, signing as applicable?
- [ ] Every reported finding human-confirmed — no raw scanner dumps,
      false positives filtered, duplicates merged?
- [ ] Exploitability re-rated in context (tool severity treated as input)?
- [ ] Existing suppression comments reviewed?
- [ ] Manual passes done for logic, authz/BOLA, boundary crossings, races,
      crypto misuse, prompt-injection paths?
- [ ] **Silent-control pass run** over the controls confirmed to exist — inert
      safeguards, fail-open catches, degradation nothing logs, tests that pass
      against a no-op'd body (`sota-code-security` rules/10)?
- [ ] **Census, not spot-check**, for every mitigation the audit confirms exists:
      the protected operation enumerated and each call site marked guarded or
      unguarded, with the ratio reported (`sota-code-security` rules/14 §6)?
- [ ] **Universal claims in the security prose falsified by counting** — threat
      model, `security_model.md`, module docstrings, ADRs (`sota-code-security`
      rules/14 §7)?

**Coverage and citations**
- [ ] For a changeset too large for one pass: units **enumerated mechanically**
      first, related files **bundled** so each bundle is judged whole, every file
      either reviewed or explicitly excluded, and the **denominator reported**
      alongside the findings (§1a)?
- [ ] Every finding's `file:line` **resolved against the file at the pinned
      commit**, with the quoted evidence byte-identical, diff-relative numbers
      converted, and the full path checked against its siblings — run *before*
      the adversarial pass, and watched to fail at least once (§4b)?

**Hygiene**
- [ ] Audit was read-only; nothing in the target mutated without explicit
      instruction?
- [ ] Re-audit loop defined for verifying remediation — and does it verify each fix
      by **re-pointing the original hunt at the patched code**, rather than only
      confirming the reported input stopped working (§4)?
- [ ] On a repeat audit, was the **yield curve** read — count falling while
      difficulty rises — and were the previous wave's findings carried in as an
      explicit exclusion so the waves are independent (§4)?

02-build-workflow.md 8.9 KB

# BUILD mode — the reasoning behind the four steps

**The router holds the imperatives; this file holds the *why*.** `skills/sota/SKILL.md`
§BUILD is deliberately terse — it is read on every build, and length there is paid by
every task. Load this file when a build is large or unusual, when a step is being
skipped, or when you are changing the workflow itself.

**If you change anything here, see §5 — three other places may need the same change.**

## 1. Why "load lean" — and what it is *not* worth claiming

Load lean: read each skill's index and open only the rules files that match the work.

**Corrected 2026-09-01, against our own measurement.** This section used to say that a long
context of similar-looking guidance *"measurably reduces how many rules the model applies"*.
That was never measured, and when it finally was, it did not hold: adding **400 lines of
genuine, unrelated rules prose** to the with-library arm of the completeness eval moved the
mean from 1.00 to **0.99 — a delta of −0.01**, with six of seven cases unchanged
([COMPLETENESS-PADDING](../../../evals/results/2026-09-01/COMPLETENESS-PADDING.md)). The
claim had been generalised from a different experiment
([WHY-COMPLETENESS-RESIDUAL](../../../docs/WHY-COMPLETENESS-RESIDUAL.md)), where adding a
**relevant** rule to a checklist made application *worse* and a short reminder fixed it. That
result stands; the generalisation from it to "any extra context costs applied rules" does not.

So the honest case for lean is narrower, and still sufficient:

- **It costs nothing to follow.** Fewer tokens, faster, cheaper, and measured no worse.
- **The self-audit gate absorbs the effect — measured 2026-09-10, no longer a conjecture.**
  The padded arm ran *with* step 4 active, which is precisely the countermeasure. Turning it
  off under the same padding costs **−0.05**, and the gate recovers **+0.062** (SE 0.019,
  95% CI **[+0.024, +0.099]**, 7 tasks × 3 samples, temp 0.7 —
  [GATE-ABSORPTION-N3](../../../evals/results/2026-09-09/GATE-ABSORPTION-N3.md)). So the
  correct reading of the null is **"lean plus a terminal re-read is robust to competing
  context"**, and *not* "context length is free": the padding is free only while step 4 is
  there to absorb it. At that higher power the gated padded arm reads **+0.01**, so even
  −0.01/−0.03 overstated the cost of padding *with* the gate on.
- **The salience mechanism is still real** — it is what step 4 exists for. What is not
  established is that *irrelevant* context triggers it.

**Do not restore the old sentence** without a run that shows a drop. A number this project
asserts is a number it must be able to produce — and note which sentence the measurement
licensed: *"drop step 4 and competing context costs you"* is supported; *"any extra context
costs applied rules"* still is not.

## 2. Why the plan comes before the code, and must be concrete

Named up front and verified at the end, constraints are followed far better than when
left implicit. The failure mode is a plan of vague intentions — "handle errors", "make it
secure" — which cannot be marked done or not-done at step 4 and therefore never is.

A checkable item states an outcome with a number or a subject:

- "rate-limit login to N/min per IP", not "add rate limiting"
- "reject uploads over N MB", not "validate uploads"
- "structured log on auth failure, no credentials in the line", not "add logging"

## 3. Why the self-audit gate runs LAST

A long build context makes mid-context rules fade. The final re-read is what recovers the
rate limiting, transport, tests and logging that a model otherwise drops silently.

**What the ablation actually shows** (`evals/run-completeness.py`, `sonnet-4.6`,
2026-07-13): base **0.60** → +rules **0.89** → +this self-audit **0.93** → +principle 5
**0.99**. So the rules carry the largest single step (+0.29), and the last two — the
terminal re-read and the short cross-cutting reminder — close **0.10 of the 0.11 that
remained**. Both readings matter: loading the rules is what puts the knowledge in reach,
and re-reading last is what gets the peripheral items actually written.

*Corrected 2026-09-01.* This paragraph previously said the re-read was "the bulk of the
library's completeness lift" and cited the 0.62 → 1.00 `sonnet-5` run. That run has **two
arms**, so it measures the whole library and cannot apportion credit to any component —
and the ablation that can, above, does not support the claim. A number cited for something
it cannot show is the same defect as a number that is wrong.

Doing it first, or continuously, does not work: at that point there is no diff to audit.

**For a large build, run it as a separate pass over the diff.** A fresh, minimal context
beats a long polluted one — the same reason the gate exists at all.

**Push the few truly critical invariants into deterministic gates.** A lint or CI check
that fails when an endpoint has no rate limiting or no TLS does not depend on attention.
Attention is not an enforcement mechanism; a test is.

## 4. The questions that catch inert work

- **The falsification question** — *if this control were silently a no-op, would anything
  observable differ?* No log, no metric, no failing test means the control is not done
  (`sota-code-security` rules/10).
- **Read back the artifact this run produced.** Where a control emits something — a
  record, a ledger line, a signature — it is wrong in the *output* long before it looks
  wrong in the source. Re-reading the code that writes it re-runs the reasoning that
  produced it, which is the weakest check available.
- **Ask for a fact to produce, not a judgment to make.** "Did you handle the error
  paths?" is answered from the same context that wrote them, and the answer is yes.
  "Paste every `except` in the diff and name what each one re-raises" cannot be answered
  without going and looking, and it is the *going and looking* that changes the output —
  the checked item comes back with the file:line attached rather than a verdict. So write
  each gate item as an artifact you must return: the grep and its hit count, the command
  and its exit code, the specific line that implements the requirement. A checklist of
  yes/no questions grades itself and passes; the same checklist phrased as evidence to
  produce cannot be satisfied without doing the work. This is why step 4 says *re-read*
  the checklist rather than *confirm* it, and why an item's denominator matters
  (`sota-code-security` rules/11 §2.2): "0 checked, 0 failed" is a pass shaped exactly
  like a real one.

## 5. Changing BUILD? Change these too

The BUILD workflow is **mirrored in four places**. They drift independently, and three of
the four fail silently.

| Surface | What it holds | What happens if you forget |
|---|---|---|
| `skills/sota/SKILL.md` §BUILD | the imperatives an agent reads every build | the change never ships |
| this file | the reasoning | the *why* rots away from the *what* |
| `evals/run-completeness.py` → `BUILD_WORKFLOW` | a hand-compressed mirror used as the eval's treatment arm | **the eval measures a workflow that is not shipped** |
| `evals/run-completeness.py` → `ROUTER_BUILD_SHA` | a hash pin over the router's BUILD section | the eval **aborts** — this one is loud, and is the guard that catches the row above |

The pin is the only reason this is survivable: edit §BUILD and the next eval run stops
and prints the new hash. **Never bump the hash without first re-reading the mirror
against the new §BUILD, clause by clause.** Then one of two things is true, and you must
say which in the commit:

- **an imperative changed** → re-sync `BUILD_WORKFLOW`, *then* set the new hash;
- **only prose moved** (rationale relocated here, wording tightened) → the mirror is
  already accurate; record the clause-by-clause check and set the hash alone.

Bumping the hash *without that check* is exactly the drift the guard exists to prevent,
and it happened once (2026-07-20, PR #119: a falsification clause added to the
router was missing from the mirror for four days while the project's most-cited number
was measured against a workflow that no longer shipped).

## Audit checklist (meta — for changes to this workflow)

- [ ] A change to §BUILD in the router is reflected in this file's reasoning, in
      `BUILD_WORKFLOW`, and in `ROUTER_BUILD_SHA` — all four, same commit.
- [ ] `ROUTER_BUILD_SHA` was updated only **after** the mirror was re-read against the
      new §BUILD, and the commit says whether an imperative changed or only prose moved.
      Run the guard once and confirm it passes rather than assuming it.
- [ ] The router's §BUILD still states every imperative; only reasoning lives here. An
      agent that never opens this file must still build correctly.
- [ ] Any new step is checkable at step 4 — it states an outcome, not an intention.
- [ ] Every gate item is phrased as **evidence to produce** (a grep, a command, a
      file:line), not a yes/no question about your own work (§4).

03-audit-findings.md 31.3 KB

# Audit Findings — Severity, Evidence, Decisions, Refutation & Reporting

Scope: what happens to what an audit finds — how a finding is **rated**,
**evidenced**, **verified against itself**, and **reported**, plus the
decision-ledger pass that produces findings the code passes cannot.
`rules/01` owns how the audit is *run* (scoping, recon, tooling, triage,
hygiene); this file owns the deliverable. Read both for any full audit.

Split out of `01-audit-methodology.md` on 2026-08-29, when that file reached
455 of its 500 lines. **The section numbers changed**: severity, evidence, the
decision ledger, refutation and reporting were sections 4 to 8 of `rules/01` and
are §1–§5 here, in the same order. Prose written before the split that points at
`rules/01` for any of those five means this file, shifted down by three.

---

## 1. Severity model

Rate **impact × likelihood/exploitability, in context**. CVSS may inform the
rating; it is never the rating. The deployment context (internet-facing vs
internal, data sensitivity, existing mitigations) decides the final level.

- **Critical** — exploitable now with severe impact: RCE, auth bypass,
  secrets/keys exposed in repo or logs, unauthenticated access to sensitive
  data, prompt injection reaching a privileged tool. Fix immediately; ask
  whether it is already an incident (was the secret ever live? rotate first,
  then fix).
- **High** — serious impact or likely exploitation: injection (SQL/command/
  NoSQL), broken access control (BOLA/IDOR), missing authn on a sensitive
  route, weak or hand-rolled crypto, SSRF. Fix this sprint.
- **Medium** — real weakness requiring conditions or chaining: missing rate
  limits, verbose error leakage, absent security headers/CSP, weak
  validation behind an authenticated boundary.
- **Low** — defense-in-depth and hygiene: minor info disclosure, hardening
  gaps with low standalone impact, lint-level issues with a security flavor.
- **Info** — no direct risk: observations, tech-debt notes, future-proofing,
  positive-pattern caveats.

Five hard rules (this list said *"two"* while carrying four until 2026-09-11):

1. **Borderline ratings state the deciding assumption explicitly** — "High
   if this endpoint is internet-facing; Medium if internal-only" — and ask
   when the answer is knowable. Do not silently pick the scarier level.
2. **Uncertain findings are marked "needs verification", never asserted.**
   A speculative Critical that turns out false costs the whole report its
   credibility.
3. **Name the primitive before you rate.** A severity is a claim about a
   *chain*, and a chain has legs: attacker-controlled input **reaches** the
   code; something **acts** on it (a write, a read, an exec, a spawn); the
   effect **crosses a trust boundary** (another tenant, the host, a third
   party, the operator's own credentials); and where exfiltration is the
   impact, a **channel** carries it out. Write the legs down and point each at
   code. A missing leg does not make the finding smaller, it makes it a
   *different* finding — hostile bytes staged into a directory that nothing
   ever executes is a hardening gap, not RCE, and rating it as RCE spends
   credibility you need for the real one. The inverse is the higher-yield
   half: when every leg coexists **in one path**, that is what the report
   leads with, however unremarkable the file looks. "And then presumably it
   runs" is not a leg; the toolchain that is not in the image, the build step
   that compiles without executing, and the temp directory that never leaves
   the operator's machine are all legs that turn out to be missing on
   inspection.
4. **On a diff, the baseline is the code the change replaced — not perfect.**
   A hardening change that closes three exposures and leaves a fourth is not a
   High: the High was the state *before* it. Read the pre-change path (`git
   show <base>:<file>`) before rating anything on a branch that moves in the
   right direction, and say which baseline you used. Filing a High against a
   fix is not a conservative error — it teaches authors that hardening attracts
   findings, which is how the next mitigation does not get written. The residual
   gap is still reported: as a hardening item against the *codebase*, at its own
   severity, with the improvement stated. Where the change is a pure regression,
   this rule costs nothing — the baseline comparison makes that case stronger,
   not weaker.

5. **This model outranks a per-skill severity table unless that table says it is
   narrower.** At least four skills ship their own Critical/High/Medium/Low rubric
   (`sota-sandboxing`, `sota-threat-modeling`, `sota-kubernetes`,
   `sota-network-security`). They exist to be domain-specific and are useful; what
   was missing until 2026-09-11 is which one wins. The router resolved the finding
   **format** ("supersedes any per-skill variant") from the start and said nothing
   equivalent for **severity** — an asymmetry visible in the text with no judgement
   needed, and confirmed as one of three real conflicts by the 2026-09-09 conflict-rate
   measurement (ROADMAP 46). The resolution: §1 is the floor for every finding and for
   any cross-domain roll-up; a skill's table may **refine** it inside that skill's
   domain and must **say so**, the way `sota-kubernetes` marks a row
   *"(requirement-level; depth → network-security)"*. An unscoped table that simply
   rates the same class differently is the conflict, not a refinement — and where two
   scoped tables still disagree, rate on §1 and name both in the finding.

## 2. Evidence standard — no finding without it

Every finding carries all of the following. A finding missing any item is
not ready to ship:

1. **Title** — concise statement of what is wrong.
2. **Severity** + one-line justification (impact × likelihood, per §1).
3. **Location** — `file:line` at the pinned commit (or manifest key, route,
   workflow step). Exact, clickable, reproducible — and *verified* to resolve
   there before shipping, which is a mechanical pass of its own (`rules/01` §4b).
4. **Evidence** — the minimal code/config snippet or triaged tool output
   that proves the issue. Minimal: enough to verify, no page-long dumps.
5. **Standard mapping** — CWE id; OWASP Top 10 / API Top 10 / ASVS item;
   MITRE ATT&CK or ATLAS technique where it applies.
6. **Impact** — what the attacker (or affected user) *actually gets*:
   "reads any tenant's invoices", not "improper access control".
7. **Remediation** — concrete and diff-level where possible ("parameterize
   this query", with the changed line), referencing the relevant skill's
   rules file for the full pattern. Never "sanitize input".
8. **Effort estimate** — trivial / small / medium / large. Severity says
   what hurts; effort enables the roadmap in §5.

Two asymmetries the evidence standard has to carry:

- **Negative claims need more proof than positive ones.** "No hardcoded secrets
  remain", "authorization is enforced everywhere", "nothing in this class was
  found" — a narrow search and a true absence produce identical output. Before
  any absence claim, widen the search (synonyms, other languages, generated and
  vendored trees, config as well as code) and confirm with a **second
  independent method** (grep *and* AST/call-graph *and* a dynamic or mutation
  probe). Then state the search performed, so the reader can judge its reach.
  An unqualified absence claim is the one finding-type nobody can falsify.
- **A status can be TRUE and still be about the wrong subject.** The evidence standard
  above catches *wrong* answers. This one is not wrong: the value is correct, unambiguous,
  and describes **a different subject than the reader assumes** — so no amount of care
  applied to the number detects it. Field-reported, six in one session, every one literally
  true: *"exit code 0"* was the **wrapper's**, not the job's; *"no advisories for this
  package"* was true of **the database that was opened**, not of the package; *"the harness
  returned no result"* was true of **a container that never started**, not of the target;
  *"38,861 passed"* was true of **the tree fifty minutes ago**, not as committed. **When a
  check reports OK, name what the OK is about. If the subject is implied rather than
  stated, the check has not reported anything.** In practice that is one clause: *the
  series arrived* rather than *the exporter is up*; *0 rows in `<path>`* rather than *no
  advisories*; *green at `<sha>`* rather than *green*.
- **"The control is present" is not "the control works."** Evidence for a
  positive observation must show *effect*, not existence — the log line it
  emitted, the request it rejected, the test that fails when it's disabled.
  See `sota-code-security` rules/10; this applies to §5's positive-observations
  section too, where an inert control praised as a strength is the worst
  possible reporting error.

**A correlation across the whole population is still not a mechanism** — and a *complete*
one is the most persuasive version of the mistake, because it looks like the N-of-N evidence
above rather than like a guess. Repeating an **observation** N of N times bounds noise;
observing that N of N **members share a property** bounds nothing, because you have not
touched the thing that would have to implement it. Field-reported: 8 of 8 working scrape
objects lived in one namespace and the only one elsewhere collected nothing — a perfect
correlation across the entire population, and the inference drawn from it
("cross-namespace scraping does not work here") was **wrong**. One query against the
collector's own target list refuted it: that target was discovered normally and the real
cause was the URL scheme. The correlation was an artifact of where a previous author had put
files. **Name the component that would have to implement the mechanism and ask it directly**
— the selector, the config, the target list, the resolver. That check is usually one command,
and the correlation is precisely what stops you looking for it. Reported as a near-miss,
caught before acting: judge it accordingly, and note that the sibling failure —
generalising from a *single* sample — is `sota-code-security` rules/15 §2.1.

**A reproduction you ran once is a coincidence you have not ruled out.** Where the
evidence is a *behaviour* rather than a line of code — a crash, a race, a
timing-dependent bypass, a fuzzer hit, or anything an agent or a sampled model
produced — reproduce it **N of N** and report both numbers. Anthropic's
defending-code reference harness sets the reference point: its find agents run
the instrumented binary *"until a given input produces a crash 3 out of 3
times"*, and until then it is not a crash. The threshold matters less than having
one, because `1/1` and `3/3` are typeset identically in a report and mean
entirely different things. This binds your own instruments too: a criterion that
flips run to run at temperature 0-ish (`sota-llm-engineering` rules/01) cannot be
settled by a single run, so a one-shot measurement quoted as a result is an
unstated `1/1`.

The library's short finding format (`file:line | rule | severity | effort | fix`)
is the working format during passes; expand each surviving finding into the
full evidence block for the report. Skill-local block formats are fine during
a single-domain pass, but they must carry the effort field — §5's roadmap is
sequenced by risk-reduction-per-effort and can't be built without it.

**Keep the output that produced the finding.** A command whose result you will cite is
evidence: redirect it to a file rather than piping it through `tail`/`head`, because a
consumed pipe cannot be re-read and the truncation keeps the summary while discarding the
context that would qualify it (`sota-shell-scripting` rules/01 §3). An audit that cannot
reproduce its own quoted output without re-running the job has a weaker evidence chain
than it appears to.

**Weigh who benefits from the source.** A claim from a party that sells the remedy is not
disqualified by that — it is *unblinded* by it, and the tell is structural, not tonal:
independent sources converging is evidence, **sources converging on the conclusion that
sells their product is one hypothesis held by several interested parties**. Field-reported:
both sources establishing that LSM hooks resist a bypass syscall hooks do not were vendors
of LSM-based sensors. **Name the interest in the finding itself** (the row outlives the
reasoning); prefer a test you run yourself; and where the test is pending, label it a
**design argument, not a measurement**, with a severity reflecting the unverified premise.

## 3. Decision-ledger review — audit the decisions, not just the code

Code review finds defects in what was built. It cannot find the defect where the
code is a faithful implementation of a choice that **stopped being right** — a
datastore picked for a scale that never arrived, a benchmark-justified rewrite
whose benchmark no longer reproduces, a constraint that expired two years ago and
is still shaping the design. That class is invisible to every pass above and is
often the most expensive thing in the repo.

`sota-architecture` rules/01 §4 owns **writing** ADRs. This is the audit side:
reading them back and asking whether they still hold.

**Reconstruct the ledger.** Sources, in order of reliability: ADRs
(`docs/adr/`), design docs and RFCs, the CHANGELOG, PR descriptions on the
commits that introduced each major component, and — last — comments. Where no
record exists, the decision is still there, just undocumented: infer it from the
code and label it *reconstructed, unconfirmed*. A major component with no
discoverable rationale is itself a finding.

**For each significant decision, classify it:**

- **JUSTIFIED** — the reasoning holds and the evidence still reproduces.
- **STALE** — sound when made, no longer: the constraint expired, the alternative
  got better, the load never materialized, the dependency went EOL. Not a mistake;
  a decision that has outlived its inputs. Say what changed.
- **UNJUSTIFIED** — the stated reasoning does not support the decision, or the
  evidence cited was never checked. Distinguish this from STALE plainly; it is a
  judgment about the decision as made.
- **UNVERIFIABLE** — no rationale survives and none can be reconstructed. Record
  it rather than guessing.

**The reason must be self-contained and decision-enabling.** A verdict is read
later, by someone who does not have your session — most often to decide whether to
reopen the question. A reason that only makes sense next to the thing it judged is
a verdict with no audit trail, and it quietly re-opens the decision anyway, because
the next reader cannot tell what was checked. Ban the four reasons that carry no
information: *"unchanged"*, *"superseded"*, *"overlaps with X"*, *"looks fine"*.
Restate the evidence every time, even when the verdict is unchanged from last pass.

| Verdict | Not this | This |
|---|---|---|
| JUSTIFIED | "Still fine." | "Postgres over the queue: the 8k msg/s that justified a broker still has not arrived — peak measured 240/s this session (`bench/throughput.py`, 2026-08-31). Holds." |
| STALE | "Outdated." | "Sharding by `tenant_id`: justified by a 40-tenant forecast; 3 tenants after two years, one holds 96% of rows, so the shard key now concentrates rather than spreads. Reversal is large — see roadmap item 4." |
| UNJUSTIFIED | "Bad call." | "ADR-007 cites a 3× benchmark for the rewrite; the benchmark script compares release-vs-debug builds (`bench/run.sh:12`), so it measures build flags, not the rewrite. Re-run like-for-like: 1.04×." |
| UNVERIFIABLE | "No docs." | "No ADR, no PR body, original author gone. Would be settled by the load figures behind the 2024 capacity plan, if anyone still has them." |

The same bar applies to any verdict this library asks you to record — a refuted
finding (§4), an intake decision in `docs/ADOPTION-LOG.md`, a triage dismissal
(`rules/01` §3). "Already covered" without a `file:line` is the same empty verdict
wearing different clothes.


**Re-measure anything a decision rests on.** When the justification is a number —
a benchmark, a latency or throughput target, a recall/false-positive rate, "X is
faster than Y", "this doesn't scale" — **re-run it this session** and report the
result, including in heavyweight environments when that is the only honest way to
check. A number in a two-year-old ADR is a historical claim, not a current fact.
If you cannot re-run it, mark the decision UNVERIFIABLE and say precisely what
would confirm it (principles 0 and 6 apply here with full force). Respect the
repo's documented environment constraints and teardown rules when you do.

**Check the ledger against reality, both directions.** A decision recorded but
never implemented is as much a finding as one implemented but never recorded —
the ADR says "we use the outbox pattern", the code dual-writes. And a superseded
ADR still describing current behavior misleads every future reader.

**Report as findings.** STALE and UNJUSTIFIED entries carry a severity like any
other finding (impact of continuing on the current path × likelihood it bites),
and feed §5's roadmap — reversing an expensive decision is usually *large* effort
and belongs sequenced, not buried in prose. Quote both sides: the recorded
rationale and what you measured.

Scope it: the decisions worth this treatment are the expensive-to-reverse ones —
datastore, broker, service boundaries, auth model, tenancy model, language or
framework, build/deploy topology. Do not ledger-review every merged PR.

## 4. Adversarial verification — try to kill your own findings

Re-reading your own finding is the weakest possible check: you re-run the
reasoning that produced it and reach the same conclusion. Confirmation bias is
not defeated by attention. Before a finding ships, someone — a separate agent, a
colleague, or you in a deliberately hostile pass with fresh context — must try
to **refute** it.

The pass:

1. **State the finding as a falsifiable claim.** "An unauthenticated caller can
   read any tenant's invoices via `GET /invoices/{id}`" — not "weak access
   control in the invoices module". A claim you cannot refute is a claim you
   cannot verify.
2. **Assign refuters, prompted to kill it.** The instruction is *find the reason
   this is wrong*, not *check this*. Default the verdict to REFUTED when the
   evidence is ambiguous — an unrefutable finding must earn its survival.
3. **Use distinct lenses when a finding can fail in more than one way.** Three
   identical reviewers are worth less than three different questions:
   - **Correctness** — is the mechanism real? Read the full path, not the
     snippet. Is there an upstream guard the finding missed?
   - **Reachability** — can attacker-controlled input actually get here? Dead
     code, an unregistered route, or a caller that always sanitizes downgrades
     it to hardening debt.
   - **Severity inflation** — does the stated impact follow, or is a Medium
     dressed as a Critical? Rate the *demonstrated* impact.
   - **Chain closure** — walk the legs from §1 rule 3 and demand each one in
     code. This lens's whole job is to find the leg that is *missing*, and it
     is the one that most often kills a plausible finding: the language that
     routes here has no toolchain in the image; the pinned build command runs
     no generator, no test and no `main`; the staged bytes land in a
     mode-`0700` directory the operator already owns. A finding that cannot
     name its execution primitive is reachability plus speculation.
4. **Majority-refute kills it.** Survivors ship; the rest are dropped or
   downgraded with the refutation recorded — a refuted finding is a result, not
   waste, and stops the next auditor re-raising it.
5. **A refuted finding is a template — sweep before you drop it.** The
   refutation hands you two reusable things: the *pattern*, and the *leg* that
   was missing. Grep the pattern across the whole repository — not just the
   audited scope — and check each hit for that leg. The strongest instance of a
   class is routinely **outside** the scope where you first noticed it: a
   diff-scoped review that refuted a symlink-dereferencing staging finding on
   the branch (nothing on that path executes the staged code) found the same
   staging in an *unchanged* module where the analysis step compiles the target
   and runs its build scripts, with no network isolation — three legs at once,
   and the only High the review shipped. Record the sweep next to the
   refutation, with its denominator: "refuted here; swept 6 call sites; one
   survives at `file:line`". A refutation with no sweep closes the class on a
   single instance and leaves the report asserting more than it checked.
6. **Verify the negatives too.** "Authorization is enforced everywhere" is a
   finding-shaped claim with the heavier burden of `SKILL.md` principle 3. Give
   absence claims a refuter whose job is to find one counter-example.

### 4a. Bound what the refuter gets, and make its verdict a number

Step 2 says *prompt it to kill the finding*. Three conditions decide whether that
prompt does anything, and all three are cheap.

**Give the refuter less than the finder had.** A refuter holding the finder's
tools re-runs the finder's investigation and arrives where it arrived. Take the
tools away: it reads code and reasons, it does not execute, write files, or
re-derive the reproduction. Anthropic's `/security-review` says this to its own
filter sub-tasks — *"you do not need to run commands to reproduce the
vulnerability, just read the code to determine if it is a real vulnerability. Do
not use the bash tool or write to any files."* It is the cheaper half of the
discipline, because a refuter that cannot re-run anything has to engage with the
claim exactly as written.

**Bound what crosses to a single artifact.** Fresh context is not enough on its
own: hand over your write-up and the refuter inherits its framing; hand over your
session and it inherits your dead ends. The strong form ships **only the
reproducible artifact** — the proof of concept, the failing command, the
`file:line` — and nothing else. Anthropic's defending-code reference harness
states it precisely: its grader reproduces each crash *"in a fresh container that
the find agent hasn't touched"*, and *"the only thing that crosses over from the
find agent to the grader is the proof of concept it produced."* Where no artifact
can be produced, that is itself the result: a finding with nothing to hand over
is a finding you have not reproduced (§2).

**Score the verdict; do not narrate it.** A refuter that returns prose returns a
judgement you then re-judge, which is the anchoring the pass exists to escape.
Require a number — `/security-review` scores confidence 1–10 and **drops
everything below 8** — and fix the bar *before* you see the findings. The
threshold is a policy about what the report is for, so state it in the
methodology (`rules/01` §1): a pre-merge gate that must not cry wolf sits high; a
one-off deep audit, where a missed Critical is the expensive error, sits lower
and ships the near-misses marked "needs verification" rather than dropping them
silently. Either way the cut is reviewable, which a paragraph of hedging is not.

A number is not a fact. It is a forcing function that makes disagreement visible,
and like every other number in this file it is a claim about a process you must
have watched work (`sota-code-security` rules/15 §2.2).

Scale it to stakes: every Critical/High gets refuted, always. Mediums get a pass
when the audit is high-stakes or the finding drives an expensive fix. Skip it for
Low/Info hygiene items — the overhead outruns the value.

Two failure modes to avoid:

- **The rubber-stamp refuter.** An agent told to "verify" agrees. Prompt it to
  *refute*, give it the code rather than your summary, and do not show it your
  confidence level — a refuter that reads "I'm certain this is exploitable"
  inherits the certainty.
- **Refuting the description instead of the code.** The refuter must open the
  file at the pinned commit. A refutation built on the finding's prose only
  tests your writing.

## 5. Report structure

Deliver in exactly this order:

1. **Executive summary** — overall posture in plain language, counts by
   severity, the top 3–5 risks and what they mean for the business. A
   non-engineer must be able to read only this section and make decisions.
   **The posture is capped by the worst thing standing**, not averaged over the
   findings: no summary may read better than *"not ready"* while an unfixed
   Critical, a missing authorization check on sensitive data, a non-idempotent
   payment or fulfilment path, or an unrecoverable migration is in the list — and
   none may read better than *"ready with caveats"* while CI is red or the
   crown-jewel path was never exercised end to end. A single blocker outranks
   twenty clean domains; the executive summary is the one place where averaging
   is a lie, because it is the only section most readers will finish.
2. **Scope & methodology** — repos and commit hash, what was and was not
   covered (with the recorded exclusions from `rules/01` §1), standards asserted
   against, tools run with exact versions and commands, audit date. This
   makes the audit reproducible and bounds its claims. Close it with **evidence
   not obtained** — the specific artifact that was unavailable and *what it would
   change*: "no production logs, so the rate limiter's effect at real traffic is
   unverified — a day of 429 counts would settle finding H-3 either way." An
   exclusion says what you skipped; this says what the reader can go get to move
   a verdict. It is also the honest home for every claim that had to be softened:
   an unobtainable check named here is bounded, the same check left unmentioned
   reads as one that passed.
3. **Decision ledger** — the §3 table: each significant decision →
   JUSTIFIED / STALE / UNJUSTIFIED / UNVERIFIABLE, with the recorded rationale
   and the evidence you re-checked. Omit the section only if the repo has no
   discoverable decisions; say so if you do.
4. **Findings** — grouped Critical → High → Medium → Low → Info; within a
   severity, ordered by exploitability. Each in the full §2 evidence block, and
   each Critical/High having survived the §4 refutation pass.
5. **Prioritized remediation roadmap** — *not a finding dump in severity
   order*. Sequence by **risk-reduction-per-effort**: quick critical wins
   first (trivial/small fixes to Critical/High), then high-impact larger
   work, then hardening. Group related fixes that share a root cause or a
   code area into one work item. The reader should be able to start work
   from the roadmap alone.
6. **Positive observations** — what is already done well (good patterns,
   solid boundaries, tooling in place), so it is preserved through
   remediation rather than accidentally regressed.
7. **Appendix** — full triaged tool output, the inventory from `rules/01` §2, DFDs and
   trust-boundary sketches, suppression-comment review.

---

## Audit checklist — quality gate on the findings and the report

**Finding quality**
- [ ] No inference rests on a **correlation alone**, however complete — the component that
      would have to implement the mechanism was queried directly (§2).
- [ ] **Does every reported OK name its subject?** (§2) "exit 0", "no results", "green",
      "up" — say what each is *about*: the job not the launcher, the store's path not just a
      row count, the revision not just the suite.
- [ ] **Does any finding rest on a source that sells the remedy?** (§2) If so, is the
      interest stated in the row, and is the claim labelled a design argument rather than a
      measurement until someone runs the test? Several interested sources agreeing is one
      hypothesis, not corroboration.
- [ ] Every finding has title, severity+justification, file:line@commit,
      minimal evidence, standard mapping, concrete impact, diff-level
      remediation, and effort estimate?
- [ ] Borderline severities state the deciding assumption explicitly?
- [ ] Every recorded verdict — ledger entry, refutation, triage dismissal — is
      **self-contained and decision-enabling**, restating the evidence rather than
      saying "unchanged" / "superseded" / "already covered" (§3)?
- [ ] Executive-summary posture **capped by the worst blocker standing**, not
      averaged, and no better than "ready with caveats" on a red CI or an
      unexercised crown-jewel path (§5)?
- [ ] Scope & methodology ends with **evidence not obtained** — what was
      unavailable and which verdict it would move (§5)?
- [ ] Uncertain findings marked "needs verification", not asserted?
- [ ] **Every Critical/High names its chain** — reach, primitive, boundary
      crossing, and (where exfiltration is the impact) channel — each leg
      pointed at code rather than assumed (§1)?
- [ ] On a **diff-scoped** review, findings rated against the code the change
      replaced, with a residual gap in a new mitigation reported as a hardening
      item against the codebase rather than as a regression in the fix (§1)?
- [ ] **Decision ledger reviewed** — expensive-to-reverse decisions reconstructed
      and classified JUSTIFIED / STALE / UNJUSTIFIED / UNVERIFIABLE, every number a
      decision rests on **re-measured this session** (or the decision marked
      unverifiable), and ledger-vs-code checked both directions (§3)?
- [ ] **Every Critical/High refuted by an independent pass** — a separate agent
      or a fresh-context hostile read prompted to *kill* the finding, working
      from the code and not from your write-up, with survivors kept and
      refutations recorded (§4)?
- [ ] Did each refuter get **less than the finder** — no execution, no writes —
      and did **only the artifact** cross over (the PoC, the failing command, the
      `file:line`), rather than your write-up or your session (§4a)?
- [ ] Does each refutation carry a **number** against a threshold fixed *before* the
      findings were seen, with the threshold stated in the methodology (§4a)?
- [ ] Is every finding whose evidence is a **behaviour** (crash, race, timing,
      agent- or model-produced) reproduced **N of N**, with both numbers reported
      rather than an unstated `1/1` (§2)?
- [ ] Every **refuted or downgraded** finding **swept** across the repository for
      the same pattern before it was dropped, with the sweep and its denominator
      recorded beside the refutation (§4)?
- [ ] Every **absence claim** ("no X found", "enforced everywhere") backed by a
      widened search plus a second independent method, with the search stated?
- [ ] Positive observations evidenced by **effect** (a rejection, a log, a test
      that fails when disabled), not by the control's mere presence?

**Report**
- [ ] Executive summary in plain language with severity counts and top
      3–5 risks?
- [ ] Scope/methodology section sufficient to reproduce the audit?
- [ ] Findings grouped by severity, ordered by exploitability within?
- [ ] Remediation roadmap sequenced by risk-reduction-per-effort, related
      fixes grouped — actionable without re-reading every finding?
- [ ] Positive observations included?
- [ ] No secret values anywhere in the report; leaks redacted and referenced
      by location only?

A report that ships unverified findings, raw tool dumps, or no prioritized
roadmap is itself a failed deliverable — treat missing evidence or missing
remediation as a blocker on the audit, not a polish item.

04-library-map.md 11.2 KB

# 04 — Library map: which `rules/` file holds what

The per-skill index of every `rules/NN` file in the library, so a routing decision can
name the file to open without loading a `SKILL.md` first.

**This lived in the router until it was offloaded.** It is pure lookup — every entry is a
condensed second copy of the index inside that skill's own `SKILL.md`, which the BUILD
workflow (`rules/02` step 2) tells you to read anyway. Keeping it in the router charged
its tokens to *every* session that loaded the router, including the ones that never
consult it. The router's length was measured as **not** costing routing accuracy (flat at
2.6x the length, `evals/results/2026-08-26/ROUTER-LENGTH.md`), so this is a
tokens-per-load change, not an accuracy one.

Read it when you know the domain but not the file. If you are already opening a skill's
`SKILL.md`, prefer that skill's own index — it carries "read this when…" guidance this
map deliberately drops.

**Both directions are gated.** Invariant 7 requires every skill to appear here; invariant
15 requires every `rules/NN` file that exists to be listed, and every listed file to
exist. `sota-code-security/rules/11` sat unlisted for two releases before 15 existed.

## Library map (rules files per skill)

- **sota/rules**: 01 audit methodology (scoping, tool matrix, triage, hygiene), 02 build workflow
  (the four BUILD steps, and the four surfaces a BUILD change must be mirrored into), 03 audit
  findings (severity & chain closure, evidence, decision ledger, refutation, report template),
  04 library map (this file — which `rules/NN` holds what, across every skill)
- **sota-architecture/rules**: 01 styles & decisions, 02 domain modeling, 03 distributed systems &
  events, 04 resilience, 05 scalability & state, 06 cloud-native config & delivery, 07 anti-
  patterns catalog, 08 NATS JetStream messaging
- **sota-code-security/rules**: 01 input & injection, 02 authentication, 03 authorization, 04
  cryptography, 05 web security, 06 memory & resource safety, 07 data exposure, 08 LLM/AI
  security, 09 untrusted-data ingestion, 10 silent control failure, 11 dead-path diagnostics, 12
  verifying the verifier, 13 context-dependent silence, 14 the control that is not in force,
  15 instruments & guards, 16 where silent no-ops hide (the catalogue split out of 10)
- **sota-skill-security/rules**: 01 provenance & installation, 02 trust boundary & capability,
  03 authoring & auditing skills
- **sota-threat-modeling/rules**: 01 methodologies, 02 decomposition, 03 threat catalogs, 04 risk
  rating & treatment, 05 outputs & operationalization, 06 audit reconstruction
- **sota-secrets-management/rules**: 01 lifecycle & workload identity, 02 storage backends, 03
  application patterns, 04 detection & remediation, 05 credential types
- **sota-sandboxing/rules**: 01 isolation boundaries, 02 Linux/OS hardening, 03 containers &
  microVMs, 04 process/app sandboxing, 05 AI-agent sandboxing
- **sota-performance/rules**: 01 methodology, 02 algorithms & data structures, 03 memory, 04 I/O &
  network, 05 caching, 06 frontend/web
- **sota-async-concurrency/rules**: 01 models & structure, 02 correctness, 03 primitives, 04
  event-loop hygiene, 05 cancellation/timeouts/shutdown, 06 backpressure & flow control, 07 audit
  bug catalog
- **sota-api-design/rules**: 01 REST/HTTP, 02 versioning & evolution, 03 GraphQL, 04 gRPC &
  protocols, 05 realtime/websockets/SSE, 06 webhooks, 07 security & operations
- **sota-devsecops/rules**: 01 pipeline security, 02 provenance & signing, 03 dependencies,
  04 build & containers, 05 analysis gates, 06 IaC & deployment, 07 runtime & ops,
  08 registry security, 09 gates that hold, 10 declared-but-not-reached (the inert-dependency
  sweep, split out of 03 in v1.36.0), 11 after the gate fails (durable verdicts, reproducing the
  gate's invocation, preserving a red run's log — split out of 09, · v1.43.0)
- **sota-databases/rules**: 01 choosing & modeling, 02 schema & migrations, 03 queries & indexes,
  04 transactions & concurrency, 05 reliability & scale, 06 security & compliance, 07 vector & AI,
  08 SurrealDB & multi-model
- **sota-frontend-design/rules**: 01 typography & color, 02 layout/spacing/ responsive, 03 design
  systems & components, 04 UX patterns, 05 accessibility, 06 motion design, 07 visual craft &
  distinctiveness
- **sota-web-frameworks/rules**: 01 baseline (versions/support, render modes), 02 React 19 (hooks,
  Actions, React Compiler), 03 Next.js (App Router, Server Actions, caching, CVEs), 04 Vue 3
  (Composition API, reactivity, XSS), 05 Nuxt 4 (data fetching, server routes, CVEs), 06 SSR &
  hydration (mismatches, serialization, caching, CSP), 07 framework security & CVEs
- **sota-observability/rules**: 01 structured logging, 02 metrics, 03 tracing, 04 SLOs & alerting,
  05 operational readiness, 06 audit playbook
- **sota-testing/rules**: 01 strategy & shape, 02 test design & quality, 03 doubles & test data,
  04 integration/contract/system, 05 e2e & UI, 06 property/fuzzing/mutation, 07 suite health & CI,
  08 BDD/spec-by-example, 09 security testing
- **sota-llm-engineering/rules**: 01 evals, 02 prompt & context engineering, 03 RAG & retrieval,
  04 agents & tools, 05 production engineering, 06 data & lifecycle
- **sota-ml-engineering/rules**: 01 ML systems architecture, 02 data & features (leakage/skew), 03
  training & experimentation, 04 evaluation & validation, 05 deployment & serving, 06 monitoring &
  drift, 07 security & governance
- **sota-cloud-infrastructure/rules**: 01 org/accounts/governance, 02 IAM design, 03 networking,
  04 compute selection, 05 data & storage, 06 cost/FinOps, 07 resilience & DR
- **sota-kubernetes/rules**: 01 control plane & etcd, 02 RBAC & serviceaccounts, 03 admission &
  policy, 04 GitOps controllers, 05 operators/CRDs/webhooks, 06 workloads & tenancy, 07 supply
  chain & audit
- **sota-identity-access/rules**: 01 federation protocols, 02 IdP operations, 03 authorization
  models, 04 lifecycle & provisioning, 05 privileged & workload identity, 06
  MFA/federation/assurance, 07 Active Directory & Kerberos/ADCS hardening
- **sota-network-security/rules**: 01 zero-trust architecture, 02 segmentation & blast radius, 03
  K8s network policy, 04 service mesh & mTLS, 05 edge/ingress/egress, 06 DNS/TLS/PKI
- **sota-confidential-computing/rules**: 01 threat model & selection, 02 TEE technologies, 03
  remote attestation, 04 confidential Kubernetes, 05 PETs & computing on encrypted data
- **sota-detection-engineering/rules**: 01 detection-engineering discipline, 02 telemetry & SIEM
  data layer, 03 rule languages & engines, 04 alerting/ triage/SOC/SOAR, 05
  hunting/intel/deception, 06 incident response & validation, 07 AD attack detection
  (Kerberoasting/DCSync/ADCS)
- **sota-data-engineering/rules**: 01 architecture & modeling, 02 pipelines & orchestration, 03
  streaming & CDC, 04 data quality & contracts, 05 storage & performance, 06 operations &
  governance
- **sota-privacy-compliance/rules**: 01 data inventory & classification, 02 privacy by design, 03
  consent & user rights, 04 regulatory landscape, 05 audit-ready engineering, 06 incident & breach
  readiness
- **sota-security-compliance/rules**: 01 control-frameworks-as-code (CSF 2.0 spine), 02 NIST
  800-53/800-171/CMMC/FedRAMP, 03 SSDF secure SDLC, 04 EU Cyber Resilience Act, 05 ISA/IEC 62443
  (OT/ICS)
- **sota-mobile/rules**: 01 platform & stack, 02 architecture & state, 03 offline/background/push,
  04 security, 05 performance, 06 release & operations, 07 Swift language (Swift 6 concurrency,
  ARC, SPM)
- **sota-cli-ux/rules**: 01 commands/flags/config, 02 output & interaction, 03 behavior &
  lifecycle, 04 distribution & docs
- **sota-shell-scripting/rules**: 01 safety baseline (shebang, `set -e`, quoting, zsh deviations),
  02 robustness & correctness, 03 security, 04 CI & operational scripts, 05 constructs & cleanup
  (arrays, IFS, traps/mktemp, tests, globbing), 06 ad-hoc commands (zsh deviations,
  searcher exclusions, patterns parsed as flags, stderr-suppressed absence checks, a hardcoded
  verdict label, non-portable word-boundary escapes — split out of 01 in v1.38.0), 07 PowerShell (`pwsh` CI steps, deploy scripts, entrypoints: the missing
  `set -euo pipefail`, disagreeing status variables, GitHub Actions shell defaults, execution
  policy, PSScriptAnalyzer), 08 ad-hoc side effects (blast radius, process-table exhaustion —
  split out of 06 in v1.42.2),
  09 listing & selection (a lister's default page read as a population, a selector answering a
  neighbouring question — split out of 06 · v1.43.1)
- **sota-docs-workflow/rules**: 01 documentation architecture, 02 API reference & changelogs, 03
  code review & PR workflow, 04 commits/branches/ releases, 05 spec-driven development
- **sota-ux-writing/rules**: 01 voice/tone & plain language, 02 microcopy & components, 03 errors
  & feedback, 04 accessibility & localization
- **sota-copywriting/rules**: 01 positioning & value proposition, 02 headlines/landing pages/CTAs,
  03 SEO content, 04 claims/legal/trust
- **sota-rust/rules**: 01 ownership & API design, 02 errors & panics, 03 unsafe discipline, 04
  async/tokio, 05 security & supply chain, 06 performance, 07 tooling & CI
- **sota-golang/rules**: 01 errors, 02 design, 03 concurrency, 04 HTTP services, 05 security, 06
  performance, 07 tooling & CI
- **sota-c-cpp/rules**: 01 idioms (RAII/ownership), 02 memory safety, 03 undefined behavior, 04
  security (CERT/MISRA/hardening), 05 concurrency, 06 build/tooling & CI, 07 performance
- **sota-jvm/rules**: 01 idioms (Java/Kotlin), 02 API/null/immutability design, 03 concurrency
  (virtual threads, JMM, coroutines), 04 security (deserialization/injection/XXE/JNDI/crypto), 05
  performance (GC/JFR/GraalVM), 06 build/tooling & CI
- **sota-python/rules**: 01 tooling & project setup, 02 typing & correctness, 03 idioms &
  pitfalls, 04 async, 05 security, 06 performance, 07 frameworks & testing
- **sota-javascript-typescript/rules**: 01 tsconfig & types, 02 language idioms, 03 async
  patterns, 04 Node backend, 05 security, 06 performance, 07 testing & tooling
- **sota-dotnet/rules**: 01 idioms (records/NRT/patterns), 02 API/disposal/DI design, 03 async &
  concurrency, 04 security (SQL/deserialization/ASP.NET Core/crypto), 05 performance
  (GC/Span/AOT), 06 build/tooling & CI
- **sota-php/rules**: 01 language baseline & idioms, 02 injection (SQL/XSS), 03
  files/deserialization/SSRF, 04 sessions/auth/web hardening, 05 Composer & tooling, 06
  performance & runtime
- **sota-ruby/rules**: 01 language & idioms, 02 security, 03 web hardening, 04 supply chain &
  tooling, 05 concurrency & performance

## Audit checklist

- [ ] Does every `skills/sota-*/rules/NN-*.md` in the tree appear in the map above, and
      does every number in the map name a file that exists (invariant 15 checks both
      directions — a missing entry is invisible to a router-driven load, and a stale one
      sends the model after nothing)?
- [ ] Does every domain skill have a `**<skill>/rules**` entry (invariant 7)?
- [ ] When a `rules/NN` file is added, split or renumbered, was this map updated in the
      **same commit** — and the owning `SKILL.md` index too (invariant 10)?
- [ ] Is this file still only a lookup table? Guidance that belongs at the point of use
      has drifted here if an entry has grown a rule rather than a title.

SKILL.md 40.6 KB

---
name: sota
description: >-
  Master router for the SOTA engineering skills library. Use this skill whenever the user asks to
  build, design, implement, refactor, harden, optimize, review, or audit an application, service, or
  codebase and the request spans more than one domain — or when you are unsure which specific sota-*
  skill applies. Also use when working on a codebase you do not own: reviewing a pull request or
  diff, responding to code review, evaluating someone else's patch, or preparing an upstream
  contribution — including mid-session, once you are already reading source, a diff, or CI config.
  It maps the task (build or audit mode) to the right domain and language skills (Rust, Go, C/C++,
  JVM, Python, JS/TS, .NET/C#, PHP, Ruby). Trigger keywords: SOTA, best practices, audit my code,
  security review, compliance, hardening, prod readiness, code quality, pull request, PR review,
  review comment, maintainer feedback, patch, diff, upstream, contribute, merge request.
---

# SOTA Engineering Skills — Master Router

A library of 41 domain skills, each with a `SKILL.md` entry point and a `rules/`
folder of focused rule files (each under 500 lines). Each skill works in two
modes:

- **BUILD** — apply the rules while designing or writing code.
- **AUDIT** — review existing code against the rules and emit findings in the
  canonical format below (it supersedes any per-skill variant):
  `file:line | rule violated | severity (Critical/High/Medium/Low/Info) |
  effort (trivial/small/medium/large) | fix`.
  **Severity resolves the same way:** `rules/03` §1 is the floor. A skill's own
  Critical/High/Medium/Low table may refine it *within that skill's domain* and must
  say that it is doing so; an unscoped per-skill table does not outrank the model, and
  a cross-domain roll-up is always rated on it.

Read only what the task needs: first the relevant skill's `SKILL.md` (it has its
own index of `rules/` files with "read this when..." guidance), then only the
rules files that match the code in front of you. Never load all skills at once.

## Operating principles (always apply)

0. **Validate every claim — mandatory.** No claim ships unvalidated, in any
   mode. A claim is validated only by checking it against a primary source:
   code read in full context at the pinned commit (for findings), official
   docs/release notes/advisories fetched at use time (for versions, specs,
   CVEs, tool capabilities), or a reproduced behavior (for bugs). Training
   data, plausibility, and "the rules file says so" do not validate anything.
   **A doc page states intent; only running it reports behaviour.** "What does
   this command do when the input is missing/stale/hostile" is a *behaviour*
   question even though it reads like a tool capability, and the vendor's page
   can be stale or simply wrong: measured 2026-09-14, Poetry's own docs say a
   desynced lock produces a "Warning" while the tool exits **1**, and a mechanism
   published from that summary had to be retracted. If a command is installable
   in two minutes, run it — both the failing and the passing case.
   **A retraction is a claim and carries the same burden.** Withdrawing a
   statement needs the primary source that falsifies it, exactly as asserting it
   did; *"I was probably wrong"* is not a finding, and it costs the reader a true
   statement. A wrong claim gets challenged because it asserts something; a wrong
   retraction sounds like humility and is waved through — **nobody audits a
   confession**. Three tells of over-correcting: it arrives within a message or
   two of an *unrelated* correction; it withdraws something you have evidence on
   screen for; it is a general loss of confidence rather than a named false
   proposition and what falsifies it. **The moment after being corrected is the
   highest-risk moment in a session** — the next claim needs *more* evidence, in
   **both** directions. Conceding everything is not the opposite of
   overconfidence; it is overconfidence about your own unreliability.
   What cannot be validated is either omitted or explicitly marked
   "needs verification" — never asserted. **Before measuring, state what result
   would falsify the claim.** If no obtainable result could, the experiment is
   theater — read the code path that decides the behavior instead of
   benchmarking its symptoms.
1. **Freshness first.** The library's version/spec/regulation facts were
   web-verified as of the last refresh (see README). Never trust them — or
   training data — for anything version- or CVE-sensitive at use time:
   re-verify current releases and advisories before pinning or recommending —
   **and equally when you are merely *measuring* a third party's artifact**,
   which does not feel like a version decision and is where this rule is
   skipped. **Record the end-of-life date beside the version**, every time. A
   version can be recalled fluently and with no felt uncertainty; an EOL date
   effectively cannot, so requiring it forces the lookup that "check this is
   current" does not. A version with no EOL date beside it has not been looked
   up. Detail, and what to do with a lapsed one: `sota-devsecops` rules/03 §3.9.
2. **Stop-and-ask on security-relevant decisions.** When a choice materially
   affects security posture (authn/z model, crypto primitive, trust boundary,
   secrets handling, network exposure), present the options with a
   recommendation and ask before proceeding. Do not silently pick.
3. **Evidence over vibes.** Every audit finding cites file:line, maps to a
   standard (CWE, OWASP, MITRE ATT&CK/ATLAS) where one applies, and proposes a
   concrete fix. Uncertain findings are marked "needs verification", never
   asserted. Borderline severities state the deciding assumption ("High if
   internet-facing; Medium if internal-only"). **A negative claim needs more
   proof than a positive one**: "no instances of X" and "I only looked one way"
   are indistinguishable from the outside, so before asserting absence, widen
   the search and use a second independent method — and state the search you
   actually ran. **"Independent" means a different failure mode, not a different
   phrasing.** Field-reported: two searches of the same tree agreed on zero and
   both were wrong, because both were `grep -r` over a directory of symlinks,
   which `-r` does not follow. The only check that works is a **positive
   control** — search for something you have already seen there, in the same
   invocation; if the control returns nothing, the instrument is broken and the
   absence is not evidence (`sota-shell-scripting` rules/06 §2).
   **A control proves the instrument works; it does not prove the query asks the
   corpus's question.** Varying the phrasing is not a second *method*, but it is
   a separate check you still owe: field-reported 2026-09-14, a gap was declared
   after searching *"one instance | a single observation"* with a control
   returning four files, while the corpus says **"one sample"** in six — a named
   failure mode among them. Working instrument, wrong vocabulary, false absence.
   Before asserting a gap, name the term the corpus itself would use — read a
   neighbouring section's headings — and search that. **And a failed reproduction is
   an absence claim too** — it carries this same burden and almost never gets it,
   because it does not feel like a search (`sota-code-security` rules/12 §1a.1). Where a
   control is unavailable because nothing is yet known to be in the target,
   print a **denominator** instead — bytes, members, total rows — beside the
   zero. **And weigh who benefits from the source**: a claim sourced to a party
   that sells the remedy is a hypothesis, not a measurement
   (`sota/rules/03` §2).
4. **Stack profile.** If the repo or `~/.claude` contains a `profiles/*.md`
   stack profile (preferred stores, auth provider, license policy, platform
   conventions), its choices are the defaults for BUILD mode and the expected
   baseline for AUDIT mode.
5. **Universal build non-negotiables (apply regardless of routing).** On **any
   network-reachable endpoint or handler** (HTTP, RPC, queue, webhook, upload),
   always include: **(a)** abuse control — rate limiting / quotas keyed to the
   caller; **(b)** transport enforcement — TLS, HSTS, no plaintext fallback;
   **(c)** tests for the logic; **(d)** structured logging without secrets/PII.
   These are cross-cutting, so they get silently dropped under a long, dense
   task even when a rules file covers them — a measured attention effect, not a
   coverage gap. Keep this list short and **re-check it last, before you ship**
   (BUILD step 4). If one is deliberately handled elsewhere (e.g. rate limiting
   at the gateway), say so — don't silently omit it.
6. **Claim "done" only with evidence.** Never report a task complete or a fix
   working from plausibility — "should work", "this fixes it", "Done!" are not
   evidence. State the check you actually ran and its result: test output with
   pass/fail counts and exit code, the command and its output, or the reproduced
   behavior. If you did not run it, say so plainly. Unverified completion is not
   completion — this applies to your own build output before you hand it back.
7. **Restate from the artifact, never from your own summary.** Re-reading your
   own write-up re-runs the reasoning that produced it — the weakest check
   available. Before a claim reaches anything user-visible, go back to the
   primary source: re-read the tool output, or re-run the command. Your earlier
   prose in this session is not a primary source — **and neither is a verbatim
   copy of a file injected into context earlier in it.** A quoted file reads as
   primary evidence in a way a summary does not, which is exactly what makes it
   the easier mistake: it may predate an edit made since. Re-read the path before
   asserting or editing against its contents. Summaries silently drop the case
   that contradicts them and raw output does not. Full adversarial procedure for
   audit findings: AUDIT step 7 and `rules/03` §4.
8. **Publishing under someone else's name raises the bar.** A claim to the
   person who asked costs one reader's trust and is cheap to retract. A claim
   published as them — a PR review comment, an issue, a commit message, a
   mailing-list post — is public, attributed and effectively permanent. Verify
   every factual claim by execution rather than inference, say which parts you
   did not test, check the thread first for whether it is already known, and
   never publish on someone's behalf without approval of the final text. Full
   procedure: `sota-docs-workflow` rules/03 §8.
9. **Match the rigour to the stakes — and name the level you chose.** A spike, a
   one-off script, a local experiment: build it, say in one line that it is a
   prototype and what you left out, stop. Anything reachable by an untrusted caller
   or touching money, credentials or another tenant's data gets the full treatment
   including principle 5, whether or not the request said "quick"; if it is
   genuinely ambiguous, ask in one line. An *unnamed* shortcut is not a prototype.

## Routing table

| Skill | Use when the task involves... |
|---|---|
| `sota-architecture` | System design, service boundaries, monolith vs microservices, DDD, event-driven design, sagas/outbox, reconciliation against a third party that holds authoritative state, resilience (timeouts/retries/circuit breakers), scalability, multi-tenancy, 12-factor/cloud-native, architectural anti-patterns |
| `sota-code-security` | Writing or reviewing code that touches untrusted input, authn/authz, sessions/JWT/OAuth, crypto, XSS/CSRF/CORS, file uploads, deserialization, error/log hygiene, LLM/agent app security, silent control failure (a safeguard that looks enabled and does nothing) |
| `sota-threat-modeling` | Designing a new system/feature with security in mind, drawing trust boundaries and DFDs, STRIDE/LINDDUN, risk rating, reconstructing a threat model from an existing codebase |
| `sota-skill-security` | Installing, authoring, reviewing or auditing anything an **agent loads as instructions** — skills, plugins, rulesets, `AGENTS.md`/`CLAUDE.md`/`.cursorrules`: provenance and pinning, review-before-install, the instruction trust boundary (a PR that edits an agent file changes what your agent does), precedence and shadowing between overlapping skills, capability minimisation, and guidance that is confidently wrong |
| `sota-secrets-management` | API keys, passwords, tokens, signing/TLS/SSH keys, .env files, Vault/cloud secret managers, workload identity (OIDC), secret rotation, leak detection and remediation |
| `sota-sandboxing` | Isolation of untrusted code or input, least privilege, seccomp/Landlock/capabilities, container/K8s hardening, microVMs, WASM sandboxes, subprocess hygiene, sandboxing AI-agent code execution |
| `sota-performance` | Latency, throughput, profiling, memory usage, caching (incl. stampede protection), I/O and network efficiency, Core Web Vitals, performance regression in CI |
| `sota-async-concurrency` | async/await, threads, goroutines, channels, races, deadlocks, event-loop blocking, cancellation/timeouts, graceful shutdown, backpressure, bounded queues |
| `sota-api-design` | REST/HTTP semantics, pagination, idempotency, versioning/deprecation, GraphQL, gRPC/proto evolution, websockets/SSE/realtime, webhooks, API rate limiting and tenant isolation |
| `sota-devsecops` | CI/CD pipelines, GitHub Actions hardening, supply chain (SLSA, Sigstore, SBOM, dependency confusion), unused/inert dependencies and upstream-health checks, container builds, SAST/secret-scanning gates, Terraform/GitOps, admission control |
| `sota-databases` | Schema design, Postgres/NoSQL choice, migrations (zero-downtime), indexes/EXPLAIN, transactions/isolation, ledgers & account balances, connection pooling, replication/backups, Redis, RLS/DB security, pgvector |
| `sota-frontend-design` | UI/UX, visual design, typography/color/layout, design systems and tokens, components, forms, accessibility (WCAG 2.2), motion/animation design, modern CSS, responsive design |
| `sota-web-frameworks` | React/Next.js and Vue/Nuxt engineering — Server Components & Server Actions, the RSC/client trust boundary, Next caching (`use cache`/PPR/ISR), Nitro server routes, hydration correctness, SSR state serialization, and framework-specific security & CVEs |
| `sota-observability` | Logging, metrics, tracing (OpenTelemetry), SLOs/error budgets, alerting, health checks, dashboards, debugging production, "can we answer why is this slow?" |
| `sota-testing` | Test strategy (pyramid/trophy), unit vs integration boundaries, test design/smells, mocks/fakes/test data, contract testing, e2e, property-based/fuzzing/mutation testing, flaky tests, coverage policy |
| `sota-llm-engineering` | Building LLM features — evals, prompt/context engineering, structured output, RAG, agents/tool design, MCP, model selection/routing, latency/cost engineering, LLM observability — **and any question about a model's tokens, context window, pricing or limits**, including measuring your own files |
| `sota-ml-engineering` | Production ML/MLOps (classical/predictive, *not* LLM apps) — training→serving→monitoring lifecycle, feature stores & registries, data leakage & train/serve skew, evaluation (ML Test Score, slices), deployment (canary/shadow/rollback), drift monitoring (PSI/KS) & retraining, ML security/governance (poisoning, MITRE ATLAS, NIST AI RMF) |
| `sota-cloud-infrastructure` | Cloud accounts/landing zones, cloud IAM, VPC/subnet/DNS/CDN setup, compute selection (serverless vs containers vs K8s), object storage, FinOps/cost, RTO/RPO and disaster recovery |
| `sota-kubernetes` | Kubernetes platform security & ops — RBAC & escalation paths, admission control (PSA/Kyverno/Gatekeeper/VAP, Audit→Enforce), GitOps controllers (Argo CD/Flux, AppProject scoping), operators/CRDs/webhooks, control plane & etcd encryption, Helm supply chain, multi-tenancy, cluster lifecycle, K8s audit logging; self-hosted (Talos/k3s) and managed |
| `sota-identity-access` | Identity infrastructure & access management — OIDC/OAuth2.1/SAML/SCIM protocols, running an IdP (Kanidm/Keycloak/etc.), RBAC/ABAC/ReBAC authorization design, group→role mapping, joiner-mover-leaver lifecycle, deprovisioning, privileged access & break-glass, SPIFFE/workload identity, phishing-resistant MFA/passkeys, federation risk |
| `sota-network-security` | Network security as a discipline — zero-trust (NIST 800-207), segmentation & blast-radius, the `world`/`any` over-broad-rule trap, Kubernetes NetworkPolicy depth (Cilium L7, default-deny egress), service mesh & mTLS / internal encryption, edge/ingress/WAF, egress control & metadata-endpoint blocking, DNS/TLS/PKI & cert lifecycle, email auth (SPF/DKIM/DMARC) |
| `sota-confidential-computing` | Protecting workloads/data from the infrastructure operator — TEEs (AMD SEV-SNP, Intel TDX, ARM CCA, SGX enclaves, Nitro Enclaves, confidential GPUs), remote attestation (RATS, attest-then-release), confidential VMs/nodes/containers on K8s (CoCo/Kata/Trustee), and cryptographic PETs (FHE, MPC, ZKP, PSI) when hardware trust is off the table |
| `sota-detection-engineering` | Detective controls, SOC & IR — detection-as-code, Sigma/YARA/Suricata/Falco/Tetragon rules, ATT&CK coverage, SIEM & telemetry coverage, alert tuning/SOAR, threat hunting & intel (STIX/TAXII), deception/honeytokens, incident response (NIST 800-61), detection validation (Atomic Red Team/Caldera) |
| `sota-data-engineering` | Data pipelines, ELT/orchestration, dbt, Kafka/streaming, CDC, schema registry, lakehouse (Iceberg/Delta/Parquet), data quality/contracts, warehouse modeling |
| `sota-privacy-compliance` | PII inventory/classification, privacy by design, consent, DSAR/deletion architecture, retention, GDPR/CCPA/HIPAA/PCI/AI Act engineering obligations, SOC 2/ISO 27001 audit readiness, breach response |
| `sota-security-compliance` | Cybersecurity control frameworks & product-security regulations as engineering — NIST CSF 2.0, SP 800-53, 800-171/CMMC, SSDF (800-218), FedRAMP, EU Cyber Resilience Act (SBOM/CVD/signed updates), ISA/IEC 62443 (OT zones & conduits, Security Levels); control-framework-as-code crosswalks, CUI boundaries, FIPS-validated crypto |
| `sota-mobile` | iOS/Android/cross-platform apps — stack choice, offline-first/sync, push, mobile security (Keychain/Keystore, attestation), performance budgets, store requirements, staged rollouts |
| `sota-cli-ux` | CLI/developer-tool design — flags/subcommands, config precedence, stdout/stderr and --json contracts, exit codes, TTY detection, signals, completions, distribution |
| `sota-shell-scripting` | Bash/sh scripts, CI run blocks, entrypoints, Makefiles — safety baseline (quoting, set -euo pipefail, traps), injection, secrets in scripts, shellcheck/shfmt |
| `sota-docs-workflow` | Documentation (Diátaxis, READMEs, runbooks, API docs, changelogs, AGENTS.md), code review/PR workflow, commit/branch/release discipline |
| `sota-ux-writing` | Any user-facing interface text — microcopy, button/label wording, error messages, empty states, onboarding copy, notifications, tone of voice, terminology, alt text, i18n-ready strings |
| `sota-copywriting` | Outward-facing content — landing pages, headlines/CTAs, value propositions, SEO content, testimonials/social proof, claim substantiation, email marketing, app-store listings |
| `sota-rust` | Any Rust code — ownership/API design, error handling, unsafe discipline, tokio/async, supply chain (cargo audit/deny/vet), performance, clippy/CI |
| `sota-golang` | Any Go code — errors, package/interface design, goroutines/channels/leaks, net/http hardening, security (os/exec, os.Root, govulncheck), pprof/performance, golangci-lint/CI |
| `sota-c-cpp` | Any C/C++ code — RAII/idioms, memory safety (UAF/overflow/sanitizers), undefined behavior, security (CERT/MISRA, banned APIs, OpenSSF hardening flags), concurrency/atomics, CMake/clang-tidy/fuzzing CI, performance |
| `sota-jvm` | Any Java/Kotlin code — modern idioms (records/sealed/pattern-matching, Kotlin null-safety/coroutines), API/null/immutability design, concurrency (virtual threads, JMM, j.u.c, coroutines), security (deserialization/JNDI/XXE/injection, JCA crypto), GC/JFR/GraalVM performance, Maven/Gradle supply-chain & CI |
| `sota-python` | Any Python code — uv/ruff/typing setup, idioms/pitfalls, asyncio, security (pickle/subprocess/SQL), performance, FastAPI/Django/pytest |
| `sota-javascript-typescript` | Any JS/TS code — strict tsconfig/type design, idioms, promises/AbortController, Node backend hardening, XSS/supply-chain security, bundle/React performance, vitest/ESLint |
| `sota-dotnet` | Any C#/.NET code — modern idioms (records, nullable reference types, pattern matching, spans), API/disposal/DI design, async/await & concurrency (ConfigureAwait, cancellation, channels), security (EF/Dapper SQL, deserialization, ASP.NET Core auth, crypto), GC/Span/AOT performance, NuGet supply chain & analyzers/CI |
| `sota-php` | Any PHP code — strict_types/modern idioms (enums, readonly, match), security (PDO/SQLi, XSS escaping, uploads/LFI, unserialize/Phar, sessions, password_hash/sodium), framework-neutral web hardening, Composer supply chain, PHPStan/Psalm, OPcache/FPM/JIT performance |
| `sota-ruby` | Any Ruby code — idioms (frozen strings, pattern matching, RBS/Sorbet), security (AR/SQLi, ERB escaping, strong params, Marshal/YAML.load, command injection, ReDoS), Bundler supply chain (bundler-audit, lockfile checksums), RuboCop/Brakeman, GVL/Ractors/YJIT performance |

## Cross-cutting routing rules

1. **Language skills stack on domain skills.** Auditing a Go API server → `sota-golang` + `sota-
   api-design` + `sota-code-security`. The language skill covers idioms and runtime-specific
   traps; domain skills cover the design.
2. **Security tasks usually need three skills.** Code-level flaws → `sota-code-security`; design-
   level gaps → `sota-threat-modeling`; leaked or mishandled credentials → `sota-secrets-
   management`. Pipeline/supply-chain → `sota-devsecops`; isolation blast-radius → `sota-
   sandboxing`.
3. **Performance complaints about queries** → start in `sota-databases` (EXPLAIN, indexes, N+1)
   before `sota-performance` (caching, I/O).
4. **Anything realtime** (websockets, SSE, pub/sub fanout) → `sota-api-design` rules/05 + `sota-
   async-concurrency` (backpressure).
5. **AI/LLM features** → `sota-code-security` rules/08 (prompt injection, tool authorization) +
   `sota-sandboxing` rules/05 (executing model output) + `sota-databases` rules/07 (vectors/RAG).
6. **Frontend work** → `sota-frontend-design` for design/UX/a11y/motion; `sota-web-frameworks` for
   React/Next or Vue/Nuxt engineering (RSC/client boundary, Server Actions, caching, hydration,
   SSR security); `sota-javascript-typescript` for the language/TS; `sota-performance` rules/06
   for Web Vitals. A React/Next or Vue/Nuxt security review pulls all four.
7. **Tests accompany everything.** Any BUILD task that writes logic also loads `sota-testing`
   (strategy + design rules); any AUDIT includes a suite-health pass. Language-specific runner
   mechanics stay in the language skills.
8. **LLM features split three ways.** Quality/architecture → `sota-llm-engineering`; security
   (prompt injection, tool authz) → `sota-code-security` rules/08; executing model output → `sota-
   sandboxing` rules/05; PII in prompts/logs → `sota-privacy-compliance`.
9. **Infra layers split four ways.** Cloud-provider setup (accounts, VPC, compute, cost, DR) →
   `sota-cloud-infrastructure`; the Kubernetes platform itself (RBAC, admission, GitOps
   controllers, operators, etcd) → `sota-kubernetes`; pod/container/workload isolation mechanics →
   `sota-sandboxing`; CI/CD and supply chain → `sota-devsecops`. A K8s cluster audit loads `sota-
   kubernetes` + `sota-network-security` + `sota-sandboxing`.
10. **Identity is its own layer.** App-level login/session/JWT-validation code → `sota-code-
    security` rules/02-03; identity *infrastructure* (IdP, OIDC/SAML config, RBAC/role-mapping
    design, provisioning, break-glass, SPIFFE) → `sota-identity-access`; the credentials
    themselves → `sota-secrets-management`.
11. **Network: setup vs security.** Cloud VPC/DNS/CDN provisioning → `sota-cloud-infrastructure`
    rules/03; segmentation, zero-trust, NetworkPolicy depth, service mesh/mTLS, egress/DNS/PKI
    posture → `sota-network-security`.
12. **Prevention vs detection.** Building the control → the relevant domain skill; verifying you'd
    *catch* the attack at runtime (logs, rules, hunting, IR) → `sota-detection-engineering`. Ops
    telemetry plumbing stays in `sota-observability`; design-time threat enumeration in `sota-
    threat-modeling`.
13. **Ingesting untrusted/attacker-authored data** (feeds, scraping, uploads, webhooks, RAG
    corpora, hostile parsers) → `sota-code-security` rules/09, with `sota-sandboxing` rules/04 for
    parser isolation. A tool ingesting whole **repositories** (scanner, SAST wrapper, review bot,
    agentic analyser) adds `sota-sandboxing` rules/05 §7 — staging, build execution, egress.
14. **Data: OLTP vs analytics.** App databases → `sota-databases`; pipelines, streaming,
    warehouse/lakehouse → `sota-data-engineering`; anything touching personal data → add `sota-
    privacy-compliance`.
15. **Any handling of user/personal data** (new fields, exports, logs, analytics, ML training) →
    check `sota-privacy-compliance` minimization and retention rules, even when the task isn't
    "about" privacy.
16. **User-facing words split three ways.** In-product UI text (labels, errors, empty states) →
    `sota-ux-writing`; marketing/site/email content → `sota-copywriting`; technical docs → `sota-
    docs-workflow`. The component patterns the text lives in stay `sota-frontend-design`.
17. **Shell scripts hide everywhere** — CI run blocks, Dockerfile RUN lines, Makefiles,
    entrypoints, **and the one-liners you type to verify a claim**: unlinted shell run
    against the system under test, which when wrong produces a false finding *about the
    product* (a usage error, exit 2, from the callee is the tell — but a failed glob is
    **silent** and fakes a clean result). Audit it all with `sota-shell-scripting`
    rules/01 §3 and **rules/06**, the file for pasted and agent-issued commands. And a script that **produces or
    verifies evidence** (attestations, ledgers, gate records, audit trails) is a security
    control written in shell: add `sota-code-security` rules/10, rules/12 and rules/15.
18. **Cryptography fans out — there is no single crypto skill (by design).** Algorithm choice,
    AEAD/nonce discipline, CSPRNG, in-code key handling, TLS client config, constant-time
    comparison, tamper-evident logs/audit ledgers (keyed hash chains, external anchoring,
    integrity-vs-completeness), crypto agility, and post-quantum migration → `sota-code-security`
    rules/04. The key *material* — storage backends (KMS/HSM, Vault, SOPS+age), lifecycle,
    rotation, per-credential-type handling → `sota-secrets-management`. Transport/PKI — TLS server
    config, cert lifecycle/ACME, private CA, mTLS → `sota-network-security` rules/06.
    FIPS-140-3-validated-module requirements → `sota-security-compliance` rules/02. Language-
    specific APIs (JCA, `crypto/*`, .NET) stay in the language skill. The stance throughout is
    **use a vetted library, don't roll your own**.
19. **Which direction does the trust boundary point?** Protecting the *host from the workload*
    (untrusted code, seccomp/microVMs/WASM sandboxes) → `sota-sandboxing`. Protecting the
    *workload from the host/operator* — TEEs (SEV-SNP/TDX/CCA/SGX), remote attestation,
    confidential VMs/containers, or computing on encrypted data (FHE/MPC/ZKP) → `sota-
    confidential-computing`. Both can apply to one system. Key custody/release stays `sota-
    secrets-management`; differential privacy and de-identification stay `sota-privacy-
    compliance`.
20. **"It's enabled" is a claim, not a fact.** Whenever a control's *presence* is established but
    its *effect* isn't — a banner, a config flag, a green test, "we have a scanner" — route to
    `sota-code-security` rules/10 (silent control failure). It pairs with `sota-testing` rules/06
    (mutation-probe the control) and rules/09 (a security test must be watched to fail), `sota-
    observability` rules/05 (degradation must be visible), and `sota-devsecops` rules/04 (does the
    shipped artifact contain what the control needs at runtime?). Not for controls that are simply
    *missing* — that's the owning domain skill's audit checklist.

21. **Model facts hide in your own tooling** — the same shape as rule 17. Any question
    about a model's **tokens, context window, pricing or limits** is `sota-llm-engineering`
    (rules/02 §2 for counting, rules/05 for cost), *including* measuring your own files,
    prompts or docs, and even when the surrounding task is repo maintenance and nothing
    looks like an LLM feature. The tell is reaching for a chars/4 estimate or another
    vendor's tokenizer instead of the provider's `count_tokens`: measured 2026-08-26, that
    shortcut under-counts Claude by **54%** on markdown-dense text — in the direction that
    makes you think you have room.

## Day zero — a repo this library has not been applied to yet

Installing is *ambient*: the rules apply in every directory, including a repo
with no gates, no agent file and no LICENSE. They then govern the code you write
and nothing else — the repo keeps accepting unscanned commits. Check this
**once per repo**, on the first BUILD task in an unfamiliar one, by looking (not
inferring): is there a `.pre-commit-config.yaml` **or any other hook manager or
CI job** running a secret scan; a license file (`LICENSE*`, `COPYING*` or
`COPYRIGHT` — never match the bare name, projects name it after the licence);
an `AGENTS.md`/`CLAUDE.md`; and how long is the git history? Two or more missing
**and** a history of a few commits = day zero. A long history means a mature
repo that likely decided against them; say nothing.

When it fires, say it **once, in a line, with the command**, then get on with the
task: `scripts/init-gates.sh` + `pre-commit install --hook-type pre-push` for
gates (before the first commit, while a leaked credential is still free to
remove), `scripts/gen-agents-md.sh` for the cross-tool entry point. Full ordering
and reasoning — LICENSE, `.gitignore`, ambient-vs-repo-resident, the
`core.symlinks` trap — in `sota-docs-workflow` rules/01 §10.

**Offer, never perform.** These write config into the user's repo. Mention once,
act only on a yes, and treat a decline as decided.

## BUILD mode — workflow

Reasoning and worked examples: `rules/02-build-workflow.md`. **Changing a step below?
It is mirrored in four places — `rules/02` §5 lists them, and three fail silently.**

1. Identify the domains the feature touches (table above) and the language(s).
2. **Load lean.** Read each relevant skill's `SKILL.md` and, from its index, open **only**
   the rules files that match the work. Lean costs less and measures no worse; the
   degradation this step once claimed is **not supported** — `rules/02` §1.
3. **Plan first, with the checks in the plan.** Before writing code, list the task's
   requirements as **concrete, checkable items** — a specific outcome you can mark
   done/not-done ("rate-limit login to N/min per IP", not "add rate limiting") — covering
   the top-10 non-negotiables of each loaded skill plus operating principle 5. Vague items
   don't survive to step 4. Then implement against that list.
4. **Self-audit gate (do this LAST — do not present code until it passes).** Re-read each
   loaded rules file's **Audit checklist** *and* operating principle 5, and verify your
   diff satisfies every item. For each unmet item, **implement it** or state why it is out
   of scope — silence is not allowed. For every control, ask the **falsification
   question**: *if this were silently a no-op, would anything observable differ?* If
   nothing would — no log, no metric, no failing test — it is not done
   (`sota-code-security` rules/10). **If the control emits an artifact, read back the one
   this run just produced.** Doing this *last* is deliberate: a long context makes
   mid-context rules fade, and the final re-read is what recovers the rate limiting,
   transport, tests and logging a model otherwise drops — measured as the bulk of the
   library's completeness lift (`evals/run-completeness.py`). For a large build, run it as
   a separate pass over the diff, and push the few critical invariants into deterministic
   gates (`rules/02` §3).

## AUDIT mode — workflow

Procedure lives in two rules files, read together for any full audit: `rules/01-audit-methodology.md`
(scoping, the verified tool matrix, triage, hygiene) and `rules/03-audit-findings.md` (severity —
chain closure, the diff baseline — evidence, the decision ledger, refutation, the report template).
**A new pass needs a line in this router and a section in the rules file that owns it, never here.**

For a focused audit, load the matching skills and follow their AUDIT sections. For a
**full project audit**, work in passes:

1. **Recon.** Inventory languages, frameworks, entry points (HTTP routes, queues, cron,
   webhooks), data stores, CI config, Dockerfiles, IaC. This determines which skills
   apply; skip skills with no matching surface. Full procedure: `rules/01` §2.
2. **Threat model first.** `sota-threat-modeling` rules/06 (reconstruction): assets, trust
   boundaries, entry points. Its output prioritizes the rest.
3. **Per-domain passes.** For each applicable skill, follow its AUDIT mode and audit
   checklists. Suggested order: secrets sweep (fast, high yield) → code security (incl.
   rules/09 untrusted-data ingestion) → language-specific (incl. shell scripts) → API →
   database → async/concurrency → identity & access → sandboxing/devsecops → kubernetes
   platform → network security → cloud infrastructure → privacy/compliance → architecture
   → testing suite health → performance → observability → detection-engineering posture →
   frontend/a11y → LLM features, data pipelines, mobile, CLI, docs as applicable. For an
   infrastructure/cluster audit the heavy hitters are `sota-kubernetes`,
   `sota-network-security`, `sota-identity-access`, `sota-sandboxing` and
   `sota-detection-engineering`.
4. **Silent-control pass (always run it).** The per-domain passes ask "is the control
   there?" — this one asks "does it *do* anything?", then "does it cover **every** site it
   is credited with?". Apply `sota-code-security` rules/10 (inert controls — invisible to
   the other passes and to SAST), sweeping with `rules/11` first to find *where* to look:
   stage duration vs work claimed, every gate's denominator (`0 checked, 0 failed, exit 0`),
   size-gated paths no fixture crosses, cache keys narrower than the behaviour, one-sample
   parsers, `assert`-as-control. Then **count**: `sota-code-security` rules/14 §6 censuses
   what a *working* mitigation guards (9 of 61 is the finding), §7 falsifies the prose.
5. **Decision-ledger review.** Code passes find defects in what was built; they cannot
   find the defect where the code faithfully implements a choice that **stopped being
   right** — a store picked for scale that never arrived, an expired constraint still
   shaping the design. Reconstruct the expensive-to-reverse decisions (ADRs, design docs,
   CHANGELOG, the PRs behind each major component) and classify each **JUSTIFIED / STALE /
   UNJUSTIFIED / UNVERIFIABLE**. Where a decision rests on a number, **re-measure it this
   session**. Full procedure: `rules/03` §3. **Then ask where else the team's knowledge
   lives** — an agent's private memory store, an IDE's notes, a chat log. Anything in
   there that is a *fact about the repository* with no home in the repository is a
   finding: invisible to review, absent from a fresh clone, gone when the store is
   cleared. It is an absence claim and the naive search for it lies — `rules/01` §4a.
6. **Findings.** Emit every finding in the canonical cross-domain format (`file:line |
   rule | severity | effort | fix`) — skill-local formats are fine within a domain pass but
   must carry an effort field so the roll-up can be sequenced — deduplicate across domains,
   and roll up into the report structure from `rules/03` §5. **Severity: `rules/03` §1** —
   a Critical/High names its chain, and a diff is rated against the code it replaced.
7. **Refute before reporting.** Re-reading your own finding re-runs the reasoning that
   produced it — it is the weakest check available. Every Critical/High gets an
   **independent pass prompted to kill it** (a separate agent, or a fresh-context hostile
   read), working from the code at the pinned commit rather than your write-up, defaulting
   to REFUTED when the evidence is ambiguous. Survivors ship; the rest are dropped or
   downgraded **with the refutation recorded** — and **swept first**: the refuted pattern
   routinely closes somewhere else, and that sweep is where the report's strongest finding
   comes from. Absence claims ("no X found") get a refuter too, and carry the heavier
   burden of principle 3. Full procedure and failure modes: `rules/03` §4.

## Library map (rules files per skill)

Which `rules/NN` file holds what, for all 41 skills: **[rules/04-library-map.md](rules/04-library-map.md)**.
Read it when you know the domain but not the file. When you are already opening a skill's
`SKILL.md` (BUILD step 2), use that skill's own index instead — it carries the "read this
when…" guidance the map drops.

## Context budget discipline

Rules files run **77–500 lines, median 237** (re-measured over all 271 on
2026-09-13; the 2026-09-11 figures — 77–497 over 270 — were correct when written and
went stale in a day, and before them this said "200–310", a range half of them fall
outside, which is why the count and the date are stated). So budget by
the file you are actually opening, not by an average: 2–5 files is a typical
focused task and can be 400 lines or 2,000. A full audit pass should load one
skill at a time, finish its findings, then move on. If context is tight, prefer
the skill's top-10 non-negotiables plus the single most relevant rules file.

## When this library is wrong or missing something

These rules are maintained and measured, and they are still incomplete — the
library has no telemetry and learns nothing from use unless someone says so.
You are the only observer of the gap at the moment it appears.

If, while applying a skill, you hit one of these, **say so in one line at the end
of your answer** and point the user at the report path — then carry on with the
task:

- a rule contradicted by a primary source you just checked (version, spec, API,
  advisory) — freshness rot;
- a rule that does not fit the situation and states no exception for it;
- real surface area in the task with **no owning skill** — a routing gap;
- guidance that, followed literally, would have shipped a defect.

**Two loaded rules that contradict each other** are usually a scope collision:
one is a general default, the other a requirement inside a narrower domain. The
narrower wins *in its domain* — pick by which failure mode is worse here (a
needless idiom vs an outage), never silently: name the rule you followed and why
in a comment beside the code, or the next reader reverts it. Then report it —
the fix is an explicit exception in whichever rule was too broad.

**A routing gap should end as a test, not just a report.** If the right skill existed
and the task never reached it, the fix is the *trigger* — the skill's `description` is the
only auto-loading text and is the whole classifier — and the proof is a regression case in
`evals/cases/desc-routing-regressions.jsonl`, which pins the mis-route so it cannot return.
**Run that case against the PRE-change tree and watch it fail first.** A case that passes in
both arms pins nothing, and an absent *string* is not an absent *capability* — a model routes
on meaning, so grep cannot tell you whether a task is reachable. Measured 2026-09-14: a
PowerShell task claimed to be a routing gap already routed correctly with **0 of 42
descriptions naming PowerShell**, and the case written to pin the "fix" scored 1.00 before
*and* after. Same doctrine as every gate here — watch it fail first, or you have added a
control that cannot fail.

Report: `https://github.com/martinholovsky/SOTA-skills/issues/new/choose`
(bad-guidance / skill-request templates). Anything dangerous or
security-sensitive goes to a **private advisory** instead — see `SECURITY.md`.

Do not fire this for personal preference, for a rule you simply did not need, or
mid-task. One line, at the end, only when the library actually let the user down.
An unreported gap stays in the library for everyone.