Claude Skill

sota-kubernetes

State-of-the-art Kubernetes platform security and operations (2026) for cloud-managed (EKS/GKE/AKS) and self-hosted clusters (kubeadm, k3s/k0s, Talos). Use when building, operating, hardening, or auditing the cluster PLATFORM layer: control plane and etcd, API server/kubelet, RBA

LLM Mart · 0 points · 9 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download martinholovsky-SOTA-skills-skills_sota-kubernetes-ec2abf6.zip · 39 KB
Part of martinholovsky/sota-skills — 39 skills

Install

skills CLI npx skills add https://github.com/martinholovsky/SOTA-skills/tree/main/skills/sota-kubernetes
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install martinholovsky-sota-skills@llmmart
Git git clone https://github.com/martinholovsky/SOTA-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole martinholovsky/sota-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

SOTA Kubernetes Platform Security & Operations

Purpose

Engineer and audit the Kubernetes platform so that a compromised workload, a hostile chart, a leaked token, or a malicious controller cannot pivot to cluster-admin, read every Secret, or take the cluster down. This skill owns the layer above the pod: control plane and etcd, API server and kubelet hardening, RBAC and ServiceAccount identity, admission control and policy-as-code, GitOps controllers, operators/CRDs and admission webhooks, Helm/Kustomize at admission, multi-tenancy boundaries, cluster lifecycle, and K8s audit logging.

It does not re-teach pod isolation mechanics. Boundaries it defers:

  • Pod securityContext, seccomp, AppArmor, capabilities, PSA pod-level fields → sota-sandboxing (rules/03 containers & microVMs; rules/01 boundaries). This skill owns the admission-time enforcement of those fields, not their internals.
  • NetworkPolicy semantics, CNI choice, service mesh, mTLS → sota-network-security. This skill states only the requirement (default-deny per namespace, enforced at admission).
  • OIDC/SSO and RBAC-role design methodology → sota-identity-access. This skill owns the K8s RBAC mechanics and escalation traps.
  • Runtime/audit-log detection content (Falco/Tetragon rules, detections) → sota-detection-engineering. This skill owns producing and shipping the audit stream.
  • Workload identity, Secret storage backends (ESO, sealed-secrets, CSI) → sota-secrets-management (rules/01, rules/02). This skill owns etcd encryption-at-rest.
  • Cloud IAM, managed-K8s selection, DR → sota-cloud-infrastructure (rules/02, rules/04, rules/07).
  • CI/CD provenance, image signing, IaC/GitOps pipeline, runtime ops → sota-devsecops (rules/02, rules/04, rules/06, rules/07).

Two modes. Pick one explicitly at the start of the task.


BUILD mode

Use when provisioning a cluster, writing RBAC/policies/manifests, configuring a GitOps controller, or installing an operator/chart.

  1. Name the cluster topology first: managed (EKS/GKE/AKS) vs self-hosted (kubeadm, k3s/k0s, Talos), single- vs multi-tenant, who the threat actors are (workload → control plane? tenant → tenant? compromised CI → cluster?). Read the matching rules.
  2. Least privilege by default: no wildcard RBAC verbs/resources, automountService AccountToken: false unless the pod calls the API, scoped AppProjects, operators get the narrowest RBAC that works. Loosen only with a comment stating why.
  3. Encrypt etcd at rest with a KMS provider (rules/01) — Secrets are base64, not encrypted, by default. This is the single most common "we thought we were covered" gap.
  4. Admission is fail-closed and ENFORCING, not auditing. A policy left in audit/ warn forever is documentation, not a control (rules/03). Ship the AUDIT→ENFORCE rollout plan with the policy.
  5. GitOps is the only write path to the cluster. Humans propose via PR; the controller reconciles. Scope the controller's own privileges and AppProjects tightly (rules/04).
  6. Plan the upgrade before you build: version skew, EOL date, CVE-response runbook, tested etcd restore (rules/01, rules/07).

Deliverables: topology + threat statement, the concrete manifests/configs, the AUDIT→ENFORCE rollout, and the residual-risk/assumptions list (cloud-managed control plane internals you cannot see, org IAM, CNI behavior).

AUDIT mode

Use when reviewing an existing cluster, its RBAC, policies, GitOps config, or operators.

Procedure: inventory the cluster surface (control plane flags or managed equivalent, RBAC graph, admission policies, GitOps controllers, operators/CRDs, namespaces/tenancy); for each, walk the relevant rules-file Audit checklist; verify empirically with kubectl, kubectl auth can-i, rbac-tool/krane, kube-bench, and helm template where possible; report findings in the format below. Do not report style nits as security findings.

Severity conventions

Severity Meaning Examples
Critical Cluster-admin, all-Secrets read, or cluster takedown reachable now anonymous-auth enabled on API server/kubelet; etcd unencrypted AND reachable; ClusterRoleBinding granting cluster-admin to a workload SA or system:authenticated; wildcard */* ClusterRole bound broadly; Argo CD AppProject clusterResourceWhitelist: [{group: '*', kind: '*'}] with broad SSO; unpatched control plane on a known-RCE CVE
High Escalation/secret-read by an in-cluster or contributor principal, or a single event from it escalate/bind/impersonate verbs granted; secret-reader → token-mint → privilege chain; admission policy in audit mode for a control that should enforce; image-verification policy only Audits signatures; operator with cluster-wide secrets:*; kubelet read-only-port open; no etcd backup or untested restore
Medium Weakens defense in depth or detection PSA not enforced (only warn/audit); automountServiceAccountToken defaulted on for non-API pods; no default-deny NetworkPolicy (requirement-level; depth → network-security); audit policy missing or None/Metadata-only for Secret access; aggregated ClusterRole accreting verbs; no PDB on critical workloads
Low Hygiene, hardening headroom RBAC subjects for departed users; unused ClusterRoles; namespaces without resource quotas; :latest image tags admitted; missing PolicyException expiry

Severity is judged by reachability (anonymous > workload/tenant > contributor > admin) × yield (cluster-admin/all-Secrets > namespace compromise > info leak > availability).

Finding format

file:line | rule | severity | effort | fix

Where effort is one of trivial / small / medium / large, rule is the rules-file section (e.g. 02 §RBAC-wildcards), and fix is the concrete change. Expand below the line with: what is wrong, who exploits it and how (concrete attack path), and a snippet when short. Example:

clusterrole-ci.yaml:14 | 02 §escalate-verbs | Critical | small | remove the bind/escalate grant; scope to the 3 named ClusterRoles the controller actually applies
  Issue: ClusterRole bound to the CI ServiceAccount grants rbac.authorization.k8s.io/{bind,escalate} on clusterroles.
  Attack path: anyone who can run a job as this SA (any merged PR) can bind themselves cluster-admin.

End every audit with: counts per severity, top 3 fixes by risk-reduction-per-effort, and an explicit OUT-OF-SCOPE list (managed control-plane internals, org IAM, CNI/mesh, runtime detection content).

Rules index

File Read this when...
rules/01-control-plane-etcd.md HA control plane, API server flags (anonymous-auth, authz modes, audit), etcd encryption-at-rest with KMS v2 + backup/defrag/restore, kubelet hardening, node + immutable-distro hardening (Talos no-SSH/machine-config/SecureBoot+TPM, k3s/k0s), CIS benchmark + kube-bench
rules/02-rbac-serviceaccounts.md Roles/ClusterRoles least-privilege, the escalation traps (wildcards, bind/escalate/impersonate, cluster-admin bindings, aggregated roles, secret-reader chains), ServiceAccount hygiene (automount off, bound/projected/audience-scoped tokens, no long-lived token Secrets), RBAC auditing (auth can-i, rbac-tool/krane/who-can)
rules/03-admission-policy.md Pod Security Admission (restricted/baseline/privileged, enforce/audit/warn) and its limits, Kyverno vs Gatekeeper/OPA vs ValidatingAdmissionPolicy/MutatingAdmissionPolicy, the AUDIT→ENFORCE rollout discipline, image verification at admission (cosign/Kyverno verifyImages), PolicyException discipline
rules/04-gitops-controllers.md Argo CD / Flux security: AppProject scoping (the clusterResourceWhitelist:[{*,*}] trap), project/RBAC/SSO, the controller's own privileges and self-management, repo/SSH creds, ApplicationSet injection, auto-sync vs approval, drift, recent Argo CD CVEs, promotion/rollback as git ops
rules/05-operators-crds-webhooks.md The operator privilege problem (broad RBAC → CRD-mediated escalation), vetting operator RBAC, CRD validation/security, trusted controllers (ESO, cert-manager), admission webhooks as attack surface (failurePolicy, timeout, TLS, namespaceSelector)
rules/06-workloads-tenancy.md Resource requests/limits as availability+QoS, PodDisruptionBudget, topology spread, anti-affinity, priorityClass/preemption, namespace-as-SOFT-boundary reality, hard multi-tenancy (vCluster/separate clusters/node isolation), Secrets in K8s (etcd encryption + ESO/sealed-secrets pointer)
rules/07-supply-chain-audit.md Helm/Kustomize review before apply (helm template, RBAC-in-charts trap, privilege-granting values), OCI chart signing/provenance, only-signed-images enforcement, digest pinning at the manifest layer, K8s audit policy (what/levels), shipping audit logs, detection + runtime-security pointers

When a task spans layers (most do), read every matching file. For a full cluster audit, read all seven plus the referenced sibling skills.

Top 10 non-negotiables

Violations are at minimum High in AUDIT mode and must never be introduced in BUILD mode:

  1. anonymous-auth=false on the API server AND kubelet; kubelet read-only-port=0, kubelet authz mode Webhook (not AlwaysAllow). Anonymous or unauthenticated access to either is cluster-game-over (01).
  2. etcd encrypted at rest with a KMS v2 provider (envelope DEK/KEK), not the default base64. Secrets are NOT encrypted out of the box (01, 06).
  3. No wildcard RBAC. No * verbs/resources in Roles/ClusterRoles bound to workloads or users; no cluster-admin bound to a ServiceAccount, system:authenticated, or system:unauthenticated (02).
  4. bind, escalate, impersonate are privilege-escalation verbs — grant only to a named, audited admin path, never to a workload or CI SA (02).
  5. automountServiceAccountToken: false is the default; opt in only for pods that call the API. No long-lived Secret-based SA tokens — use bound/projected tokens (02).
  6. Pod Security Admission restricted ENFORCED on workload namespaces (label pod-security.kubernetes.io/enforce: restricted), not merely warn/audit. PSA is stable; PSP is gone (03). Pod-level field internals → sota-sandboxing.
  7. Admission policies enforce, with a documented AUDIT→ENFORCE rollout. A policy parked in Audit/warn indefinitely is a real, recurring finding (03).
  8. GitOps is the only cluster write path; the controller and its AppProjects are tightly scoped. Never clusterResourceWhitelist: [{group:'*',kind:'*'}] with broad project access; pin Argo CD/Flux versions and patch known CVEs (04).
  9. Operators get least-privilege RBAC; admission webhooks have correct failurePolicy/timeoutSeconds/namespaceSelector and TLS — a webhook is both a control and a single point of failure/attack (05).
  10. API server audit logging enabled (RequestResponse for sensitive verbs/Secrets), shipped to tamper-resistant storage; etcd backups taken AND restore-tested (07,01). Detection content on the stream → sota-detection-engineering.

If the user asks for something that violates a non-negotiable, implement the secure alternative and explain the delta; comply only after they acknowledge the risk explicitly.

Files (sota-skills)
  • rules
    • 01-control-plane-etcd.md 14.8 KB
      # 01 — Control Plane, etcd & Node/Distro Hardening
      
      Scope: the cluster's brain — HA control plane, API server and kubelet hardening, etcd
      encryption/backup/restore, node and immutable-distro hardening (Talos, k3s/k0s), version
      skew and CVE response. Frame against the **CIS Kubernetes Benchmark** (v1.12.0 at writing,
      matched to your minor version) and the **NSA/CISA Kubernetes Hardening Guidance** (v1.2,
      Aug 2022 — still the current edition; verify before citing). On managed clusters
      (EKS/GKE/AKS) the provider owns the control plane and etcd — you cannot set these flags;
      audit the *managed equivalents* (provider security posture, control-plane logging,
      secrets-encryption setting) and focus your effort on RBAC, admission, and nodes.
      
      ---
      
      ## 1. HA control plane
      
      - **Three (or five) control-plane nodes** across failure domains; etcd quorum needs an
        odd count. Two nodes is *worse* than one (split-brain risk, no quorum gain).
      - **API server behind a load balancer**; control-plane components talk to it via a stable
        endpoint, not a single node IP.
      - **etcd: stacked vs external.** Stacked (etcd co-located with control plane) is simpler
        and fine for most; external etcd isolates blast radius and is preferred for large or
        high-security clusters. Either way etcd is the crown jewel — everything below applies.
      - Control-plane nodes run **no workloads** (taint `node-role.kubernetes.io/control-plane:
        NoSchedule`). A tenant pod on a control-plane node is a node-escape away from etcd.
      
      ## 2. API server hardening
      
      These are kube-apiserver flags (or managed-cluster equivalents). Each is a CIS control.
      
      | Setting | Required value | Why |
      |---|---|---|
      | `--anonymous-auth` | `false` | Anonymous requests hit RBAC as `system:anonymous`/`system:unauthenticated`; combined with any loose binding = unauthenticated access. **Critical if true.** |
      | `--authorization-mode` | `Node,RBAC` (never includes `AlwaysAllow`) | `AlwaysAllow` disables authz entirely. `Node` authorizer + `RBAC` is the baseline. |
      | `--enable-admission-plugins` | includes `NodeRestriction` | Stops a compromised kubelet from editing other nodes/pods or escalating via node labels. |
      | `--audit-policy-file` / `--audit-log-path` | set (see `rules/07`) | No audit log = no forensics, no detection. |
      | `--encryption-provider-config` | set with KMS v2 (see §4) | Secrets at rest. |
      | `--service-account-lookup` | `true` | Revoked SA tokens stop working. |
      | `--tls-min-version` | `VersionTLS12`+ ; strong cipher suites | Defense in depth on the API. |
      | `--profiling` | `false` in prod | `/debug/pprof` leaks data and is a DoS vector. |
      | `--request-timeout`, `--max-requests-inflight` | tuned | API server DoS resistance. |
      | `--enable-priority-and-fairness` | `true` (the default) | API Priority & Fairness — never set `false`. |
      
      **API Priority & Fairness (APF)** is stable since v1.29 and **on by default**; it
      supersedes the raw `--max-requests-inflight` limits by splitting that total
      concurrency across priority levels via `FlowSchema` + `PriorityLevelConfiguration`,
      with fair queuing so one runaway controller can't starve `leader-election` or
      node heartbeats. Audit: confirm it isn't disabled (`--enable-priority-and-fairness=false`
      is a finding), give a dedicated `PriorityLevelConfiguration` to high-value
      controllers, and alert on `apiserver_flowcontrol_rejected_requests_total`. A
      catch-all FlowSchema that lumps everything into one level re-creates the global-limit
      DoS APF exists to prevent.
      
      **Authentication**: prefer **OIDC** for human users (RBAC-design and SSO are
      `sota-identity-access` territory — wire it there). Never distribute the cluster-admin
      kubeconfig / client cert as the day-to-day human credential; client certs cannot be
      revoked short of CA rotation. Service-to-API auth uses ServiceAccount tokens (`rules/02`).
      
      ```yaml
      # BAD — kube-apiserver manifest fragments that are Critical/High findings
      - --anonymous-auth=true
      - --authorization-mode=AlwaysAllow
      - --insecure-port=8080            # removed in modern K8s; if present, you're ancient
      - --token-auth-file=/etc/tokens   # static token files are unrevocable plaintext creds
      ```
      
      ## 3. Kubelet hardening
      
      The kubelet is a root-capable agent on every node with its own API. Harden it explicitly
      — defaults have historically been loose.
      
      | Setting | Required value | Why |
      |---|---|---|
      | `anonymous.enabled` (`--anonymous-auth`) | `false` | An open kubelet API = run any pod, read any Secret mounted on the node, exec into containers. **Critical if anonymous.** |
      | `authorization.mode` (`--authorization-mode`) | `Webhook` | `AlwaysAllow` lets any authenticated client drive the kubelet. |
      | `readOnlyPort` (`--read-only-port`) | `0` | The read-only port (10255) exposes pod/node metadata unauthenticated. |
      | `--rotate-certificates`, `serverTLSBootstrap` | `true` | Short-lived, rotated kubelet certs. |
      | `protectKernelDefaults` | `true` | Kubelet refuses unsafe sysctls. |
      | `streamingConnectionIdleTimeout` | non-zero | Reaps idle exec/attach streams. |
      | `makeIPTablesUtilChains` | `true` | Expected networking baseline. |
      
      Hunt: `curl -sk https://NODE:10250/pods` returning data without a token is a Critical
      finding (anonymous kubelet). `curl http://NODE:10255/pods` returning data means the
      read-only port is open.
      
      ## 4. etcd security — "Secrets are base64, not encrypted"
      
      **The reality:** by default Kubernetes stores Secret objects in etcd as **base64-encoded
      plaintext**. Anyone who reads etcd (etcd client access, an etcd backup file, a disk image,
      a node with the data dir) reads every Secret in the cluster. Base64 is encoding, not
      encryption.
      
      **Encrypt at rest with a KMS v2 provider** (GA since Kubernetes v1.29; KMS v1 deprecated
      v1.28, disabled-by-default v1.29 — do not build on v1). KMS v2 does envelope encryption: a
      local DEK encrypts data, a remote KMS KEK wraps the DEK, and v2 derives single-use DEKs
      from a rotated seed for performance.
      
      ```yaml
      # EncryptionConfiguration — KMS v2 first, so Secrets get the strong provider
      apiVersion: apiserver.config.k8s.io/v1
      kind: EncryptionConfiguration
      resources:
        - resources: ["secrets"]
          providers:
            - kms:
                apiVersion: v2
                name: cloud-kms
                endpoint: unix:///run/kmsplugin/socket.sock
            - identity: {}        # fallback for reads of not-yet-rewritten data; never first
      ```
      
      ```yaml
      # BAD — identity first means everything is written in plaintext
      providers:
        - identity: {}
        - aescbc: { keys: [...] }   # also: aescbc is weaker than aesgcm/KMS; local key on disk
      ```
      
      - The **provider listed first encrypts** new writes. `identity` must never be first.
      - After enabling/rotating, **rewrite all Secrets** so existing data is re-encrypted:
        `kubectl get secrets -A -o json | kubectl replace -f -`.
      - On managed clusters, enable the provider's secrets-encryption feature (e.g. EKS KMS
        envelope encryption, GKE Application-layer Secrets Encryption, AKS KMS etcd encryption).
      - **etcd transport**: peer and client TLS with cert auth (`--client-cert-auth=true`,
        `--peer-client-cert-auth=true`); etcd reachable only from control-plane nodes, never
        exposed to the pod network or internet.
      
      ### etcd backup, defrag, restore — TESTED
      
      - **Scheduled `etcdctl snapshot save`** (or managed backup), stored **off-cluster** in a
        separate trust domain, encrypted, immutable, not deletable by cluster credentials.
      - A backup of an *encrypted* etcd needs the KMS KEK to restore — back up your key access /
        document the KEK recovery, or the snapshot is unrestorable.
      - **Periodic defrag** (`etcdctl defrag`) — fragmented etcd causes `mvcc: database space
        exceeded` and a hard API-server outage. Alert on etcd DB size approaching `--quota-
        backend-bytes`.
      - **Restore drills**: an untested backup is a hope. Drill restore-into-a-throwaway-cluster
        on a schedule; record RTO. Cross-link DR posture with `sota-cloud-infrastructure`
        rules/07.
      
      ## 5. Node hardening
      
      - **Minimal, hardened OS**; CIS-bench the nodes; auto-patch or immutable-rebuild.
      - **No SSH to nodes as a routine workflow** — node access is break-glass, audited.
      - Container runtime hardened (containerd with sane defaults); no Docker socket on nodes.
      - Pod-level isolation (seccomp/AppArmor/securityContext, runtime class gVisor/Kata) is
        **`sota-sandboxing`** (rules/03) — enforce it via admission (`rules/03`), don't re-spec
        it here.
      - Protect node metadata endpoints: block pod access to the cloud metadata service
        (169.254.169.254) unless via workload identity — see `sota-cloud-infrastructure`
        rules/02 and `sota-network-security`.
      
      ## 6. Immutable distros (e.g. Talos, including on ARM)
      
      **Talos Linux** — API-driven, immutable, minimal Linux purpose-built for K8s:
      - **No SSH, no shell, no package manager, no interactive login.** The entire node is
        managed via the gRPC `talosctl` API (mTLS-authenticated). This removes the single
        biggest node-attack surface: there is no interactive foothold to gain.
      - **Machine config is the security boundary.** Treat `machineconfig` like a Secret: it
        holds the cluster CA and join material. Store it encrypted, deliver via a trusted
        channel, and scope `talosconfig` credentials (the client cert) tightly — it is
        root-equivalent on the node.
      - **SecureBoot + TPM disk encryption.** Modern Talos (systemd-boot + Unified Kernel Image
        is the default for new UEFI installs since v1.10) supports SecureBoot; combine with
        **LUKS2 disk encryption keyed to the TPM** (`machine.systemDiskEncryption`) for measured
        boot and at-rest disk protection. On ARM, confirm board/firmware SecureBoot + TPM 2.0
        support before relying on it; where TPM is unavailable, use a `nodeID` or KMS key source
        and document the weaker guarantee.
      - **KubePrism / API access**: restrict the Talos API and Kubernetes API endpoints to
        management networks; the Talos API at :50000 is as sensitive as the kube-apiserver.
      - Apply config changes via versioned, reviewed `talosctl apply-config` from git — Talos
        config is GitOps-able; treat it like the rest of `rules/04`.
      
      ```yaml
      # Talos machine config fragment — TPM-bound disk encryption (verify slot/keys per version)
      machine:
        systemDiskEncryption:
          state:     { provider: luks2, keys: [{ tpm: {}, slot: 0 }] }
          ephemeral: { provider: luks2, keys: [{ tpm: {}, slot: 0 }] }
      ```
      
      **k3s / k0s** (lightweight self-hosted):
      - k3s ships SQLite by default for single-server; use **embedded etcd (HA)** or an external
        datastore for multi-server, and apply the same etcd encryption discipline (§4) — k3s
        supports `--secrets-encryption` to enable at-rest encryption.
      - k3s bundles components; pin the version, track its CVE feed, and disable bundled add-ons
        you don't use (`--disable traefik,servicelb` etc.) to shrink surface.
      - k0s separates controller/worker cleanly; harden the same API-server/kubelet flags (§2,
        §3) via its config. Both still need RBAC/admission/audit from the other rules files.
      
      ## 7. Version skew, EOL & CVE response
      
      - **Supported window**: the project maintains the **latest three minor releases**, each
        with ~1 year of patch support. Run a supported minor; an EOL control plane gets no CVE
        fixes. (Verify current numbers at kubernetes.io/releases — at writing latest is the 1.36
        line; 1.34/1.35/1.36 supported.)
      - **Version skew policy** (since 1.28): the **control plane may be up to 3 minor versions
        ahead of kubelets**; kube-apiserver instances within ≤1 minor of each other; kubectl
        within ±1 of the API server. Upgrade control plane first, then nodes — never the reverse.
      - **Upgrade cadence**: minor releases ~3×/year. Plan a rolling upgrade every 1–2 minors;
        don't fall to EOL. On managed clusters, stay on a supported channel and don't defer past
        the provider's forced-upgrade date.
      - **CVE response runbook**: subscribe to the `kubernetes-announce` list / CVE feed; for a
        control-plane RCE or auth-bypass, patch on the emergency track, not the quarterly one.
        Track CVEs for *every* control too: Argo CD (`rules/04`), Kyverno/Gatekeeper (`rules/03`),
        operators (`rules/05`), **ingress controllers and CSI drivers** (the official feed's 2026
        wave hit both — ingress-nginx config-injection/auth-bypass/DoS CVE-2026-24512/-24513/
        -24514/-1580/-3288/-4342; CSI SMB/NFS subDir path traversal CVE-2026-3864/-3865), and
        the distro (Talos/k3s/k0s).
      - **ingress-nginx is EOL** (retired by SIG Network/SRC; maintenance ended March 2026, repo
        read-only). The 2026 CVE wave *was* patched in the final releases (>=1.13.9/1.14.5/1.15.1);
        the standing risk is that any CVE found after EOL gets **no** fix. A deployed ingress-nginx is a
        standing High finding: migrate to Gateway API or an actively maintained ingress
        controller (ingress/edge config depth is `sota-network-security`).
      
      ## Audit checklist
      
      - [ ] API server: `--anonymous-auth=false`, `--authorization-mode` includes RBAC and not `AlwaysAllow`, `NodeRestriction` enabled, profiling off, audit configured? (`grep -E 'anonymous-auth|authorization-mode|NodeRestriction|profiling' /etc/kubernetes/manifests/kube-apiserver.yaml`; managed → check provider posture)
      - [ ] API Priority & Fairness left enabled (no `--enable-priority-and-fairness=false`), high-value controllers on a dedicated `PriorityLevelConfiguration`, `apiserver_flowcontrol_rejected_requests_total` alerted?
      - [ ] Kubernetes Dashboard: not deployed unless required; if deployed, NOT exposed publicly (no LoadBalancer/Ingress to it), reached only via `kubectl proxy`/authenticating proxy, and its ServiceAccount is least-privilege (never `cluster-admin`) — a privileged, exposed Dashboard is a one-click takeover (historic Tesla cryptojacking). Talos does not ship it; keep it that way.
      - [ ] Kubelet: anonymous-auth off, authz `Webhook`, `read-only-port=0`? (`curl -sk https://NODE:10250/pods` should 401; `curl http://NODE:10255/pods` should refuse)
      - [ ] etcd encrypted at rest with KMS v2, `identity` not first, all existing Secrets rewritten? (`kubectl get secret -A -o json | head` against an etcd dump; check `EncryptionConfiguration`)
      - [ ] etcd reachable only from control plane, client/peer TLS cert-auth on? (`etcdctl` from a worker should fail)
      - [ ] etcd backups scheduled, off-cluster, immutable, restore-DRILLED, KEK recoverable? (when was the last restore drill?)
      - [ ] etcd defrag scheduled and DB-size alerting wired?
      - [ ] Control-plane nodes tainted to run no workloads?
      - [ ] Nodes: minimal/hardened OS, kube-bench passing, no routine SSH? Talos: SecureBoot + TPM/LUKS2 disk encryption, machineconfig/talosconfig stored as secrets and scoped? k3s/k0s: `--secrets-encryption` on, unused add-ons disabled?
      - [ ] Running a SUPPORTED minor (not EOL)? Skew within policy (control plane ≥ nodes, ≤3 minors)? (`kubectl version`, `kubectl get nodes -o wide`)
      - [ ] CVE-response runbook exists and covers control plane + every add-on/controller + ingress/CSI + distro?
      - [ ] No EOL ingress-nginx still deployed (retired March 2026 — final releases patched the 2026 CVE wave, but any post-EOL CVE is unpatched — High; migration to Gateway API or a maintained controller done or dated)?
      
    • 02-rbac-serviceaccounts.md 11.5 KB
      # 02 — RBAC & ServiceAccounts
      
      Scope: the K8s authorization graph and workload identity — least-privilege Roles/
      ClusterRoles, the privilege-escalation traps, ServiceAccount hygiene and token model, and
      how to audit who-can-do-what. RBAC *role-design methodology and SSO* is
      `sota-identity-access`; this file owns the **K8s RBAC mechanics and the escalation traps**.
      Workload identity to cloud and secret backends is `sota-secrets-management` (rules/01).
      
      RBAC is the cluster's authorization fabric: `Role`/`ClusterRole` (a set of allowed
      verbs×resources) bound to subjects (users, groups, ServiceAccounts) by `RoleBinding`
      (namespaced) / `ClusterRoleBinding` (cluster-wide). Authorization is additive and
      allow-only — there is no deny rule. So the only lever is **granting less**.
      
      ---
      
      ## 1. Least privilege — the shape of a good Role
      
      - **Enumerate verbs and resources explicitly.** Name the API groups, resources, and verbs
        the workload actually uses. Default to `get`/`list`/`watch`; add `create`/`update`/
        `patch`/`delete` only where proven.
      - **Namespaced `Role` over `ClusterRole`** unless the resource is cluster-scoped or the
        subject genuinely needs all namespaces. A `ClusterRoleBinding` is cluster-wide blast radius.
      - **`resourceNames`** to scope to specific objects where the API supports it (e.g. read
        one named ConfigMap, not all ConfigMaps).
      - **Separate read from write**; separate by namespace/team. One mega-role bound everywhere
        is the distributed `cluster-admin`.
      
      ```yaml
      # GOOD — narrow, namespaced, explicit
      kind: Role
      metadata: { name: orders-reader, namespace: orders }
      rules:
        - apiGroups: [""]
          resources: ["configmaps"]
          resourceNames: ["orders-config"]
          verbs: ["get", "watch"]
      ```
      
      ## 2. The escalation traps (each a High/Critical finding)
      
      ### 2.1 Wildcards
      `verbs: ["*"]`, `resources: ["*"]`, or `apiGroups: ["*"]` grant **everything that exists
      now and everything added in future API versions**. A `*/*` ClusterRole is effectively
      cluster-admin. Never on a workload, CI, or tenant subject.
      
      ```yaml
      # BAD — this is cluster-admin with extra steps
      rules:
        - apiGroups: ["*"]
          resources: ["*"]
          verbs: ["*"]
      ```
      
      ### 2.2 The escalation verbs: `bind`, `escalate`, `impersonate`
      These are not ordinary write verbs — they let a subject **grant itself more than it has**:
      - **`escalate`** (on `roles`/`clusterroles`): create/edit a role with *more* permissions
        than you hold. Normally RBAC stops you from authoring a role above your own privileges;
        `escalate` removes that guard. → self-grant cluster-admin.
      - **`bind`** (on `roles`/`clusterroles`): bind an existing powerful role to yourself.
        `bind` on `cluster-admin` = `bind` yourself to cluster-admin.
      - **`impersonate`** (on `users`/`groups`/`serviceaccounts`): act as another principal —
        impersonate a cluster-admin user/group, or `system:masters`. Total bypass.
      
      Grant these only to a **named, audited human-admin path**, never to a workload or
      automation SA. `impersonate` on `groups` for `system:masters` is an instant Critical.
      
      ### 2.3 ClusterRoleBinding to `cluster-admin` (or to broad subjects)
      - A ClusterRoleBinding of the built-in `cluster-admin` ClusterRole to a **ServiceAccount**
        means owning that pod owns the cluster. Critical.
      - Binding a role to the group **`system:authenticated`** grants it to every authenticated
        principal; to **`system:unauthenticated`** grants it to unauthenticated callers. **There
        is no bridge between them**: an anonymous request gets the username `system:anonymous` and
        the group `system:unauthenticated`, and `system:authenticated` is assigned only when
        authentication *succeeds*
        ([K8s authentication](https://kubernetes.io/docs/reference/access-authn-authz/authentication/),
        verified 2026-09-16).
      - **Rate the verbs and resources, not the group name.** Kubernetes itself binds
        `system:public-info-viewer` to *"system:authenticated and system:unauthenticated groups"*
        by default, so "bound to a broad group" cannot be Critical on its own — that would flag the
        shipped defaults. Critical is a *broad grant* to a broad group: write/exec/secrets-read, or
        wildcard verbs on wildcard resources. Compare against the default bindings
        (`kubectl get clusterrolebinding -o wide`) before filing.
      - **Know which defaults changed at v1.14**, because an old cluster inverts this check.
        `system:public-info-viewer` was *introduced* in v1.14; before it existed,
        **`system:basic-user` and `system:discovery` were themselves bound to
        `system:unauthenticated`** and were unbound from it at that release. So on a pre-1.14
        cluster those two bindings are the shipped default, and on a current one they are a finding.
        Read the table rather than recalling it — a summary of this page got `system:discovery`
        wrong in exactly this way (default bindings table, verified 2026-09-16:
        [K8s RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)).
      
      ### 2.4 Aggregated ClusterRoles
      ClusterRoles with `aggregationRule` automatically absorb the rules of any ClusterRole
      matching the label selector. A new operator/chart that ships a ClusterRole with the
      aggregation label (e.g. `rbac.authorization.k8s.io/aggregate-to-admin: "true"`) silently
      **widens the built-in `admin`/`edit`/`view` roles cluster-wide**. Audit what aggregates
      into the powerful roles; a hostile or careless chart uses this to escalate quietly.
      
      ### 2.5 The secret-reader → privilege chain
      `get`/`list` on `secrets` is rarely "just read a config value." Secrets hold
      ServiceAccount tokens, TLS keys, kubeconfigs, cloud creds. **Read on secrets in a
      namespace ≈ the union of every identity whose token/cred lives there.** Specifically:
      - Read SA token Secrets → authenticate as those SAs → inherit their RBAC.
      - `create` on `serviceaccounts/token` (TokenRequest) or `create pods` with a privileged SA
        → mint/borrow a stronger identity.
      - `create`/`update` on `pods` lets you schedule a pod that *mounts a more-privileged SA*
        or mounts host paths/Secrets — pod-create is a classic lateral/escalation primitive.
      
      Treat broad `secrets` read, `pods` create, and `serviceaccounts/token` create as
      near-Secret-equivalent and near-impersonate-equivalent; scope them hard. Policy-engine
      CRDs belong in the same class: create/update on Kyverno (namespaced) policies runs
      attacker logic inside the admission-controller pod (`rules/03` §2, CVE-2026-4789).
      
      ### 2.6 The Helm-chart-grants-cluster-admin trap
      Charts and operators bundle their own RBAC. A values toggle like
      `rbac.clusterAdministrator: true`, `rbac.create: true` with a `*/*` ClusterRole, or a
      default ServiceAccount bound to `cluster-admin` hands the workload the cluster. **Always
      `helm template | grep -A20 -iE 'ClusterRole|ClusterRoleBinding'` before install** and read
      the rules (`rules/07` covers chart review). A logging agent does not need cluster-admin.
      
      ## 3. ServiceAccount hygiene & the token model
      
      - **`automountServiceAccountToken: false` by default** — on the ServiceAccount and/or pod
        spec. A pod that never calls the Kubernetes API should not carry a credential to it.
        Opt in only for pods that talk to the API. A mounted token + an SSRF/RCE in the pod =
        the attacker holds that SA's RBAC.
      
      ```yaml
      # GOOD — SA carries no token unless a pod explicitly needs it
      apiVersion: v1
      kind: ServiceAccount
      metadata: { name: web, namespace: shop }
      automountServiceAccountToken: false
      ---
      apiVersion: v1
      kind: Pod
      spec:
        serviceAccountName: web
        automountServiceAccountToken: true   # explicit, only because this pod calls the API
      ```
      
      - **Bound, projected, short-lived tokens are the model** (bound-SA-token volumes GA since
        1.22). Pods get an auto-rotating, time-bound token (default ~1h) via a projected volume,
        audience-scoped, tied to the pod's lifetime. This is the default mount mechanism — good.
      - **No long-lived Secret-based SA tokens.** Since 1.24 the API server **no longer
        auto-creates** a forever-token Secret per ServiceAccount (GA 1.26). Do not manually
        create `kubernetes.io/service-account-token` Secrets for routine use — they are
        non-expiring, non-rotating bearer credentials that leak into logs/backups/etcd. If an
        external system needs an SA token, mint a **short-lived audience-scoped token** via the
        TokenRequest API (`kubectl create token sa --audience=... --duration=...`) and refresh it.
      - **Audience-scoped tokens**: a token minted for audience `vault` is rejected by the API
        server and by any verifier expecting a different audience — limits replay if leaked.
      - **One ServiceAccount per workload**, never the namespace `default` SA for real
        workloads, never shared across apps. The `default` SA should have an empty token mount
        and no bindings.
      
      ## 4. Auditing RBAC
      
      RBAC is allow-only and additive, so the real question is always "what is the transitive
      closure of what subject X can do, and can it escalate?" Tools:
      
      - **`kubectl auth can-i`** — point checks, including as another subject:
        ```bash
        kubectl auth can-i '*' '*' --as=system:serviceaccount:ci:deployer    # cluster-admin?
        kubectl auth can-i create clusterrolebindings --as=...
        kubectl auth can-i list secrets -A --as=...
        ```
      - **`kubectl auth whoami`** — confirm your own identity/groups.
      - **rbac-tool** (insights/aquasecurity), **krane**, **kubectl-who-can** (aquasecurity) —
        build the reverse index: *who* can `escalate`, `bind`, `impersonate`, read secrets,
        create pods, mint tokens. Run who-can on every escalation primitive:
        ```bash
        kubectl who-can create pods -A
        kubectl who-can '*' '*'
        kubectl who-can impersonate users
        ```
      - **Hunt patterns** across the RBAC manifests in git / `kubectl get clusterroles -o yaml`:
        ```bash
        # wildcards in cluster roles
        kubectl get clusterroles -o json | jq -r '.items[]|select(.rules[]?|(.verbs[]?=="*") or (.resources[]?=="*"))|.metadata.name'
        # escalation verbs anywhere
        grep -rE 'escalate|impersonate|"bind"' rbac/
        # cluster-admin bound to a ServiceAccount or broad group
        kubectl get clusterrolebindings -o json | jq -r '.items[]|select(.roleRef.name=="cluster-admin")|{name:.metadata.name,subjects:.subjects}'
        ```
      - **Find dead subjects**: bindings to departed users / deleted SAs accrue silently — Low,
        but they're attack surface and noise. Reconcile against the identity source.
      
      ## Audit checklist
      
      - [ ] No wildcard `*` verbs/resources/apiGroups in any Role/ClusterRole bound to a workload, CI, or tenant? (`kubectl get clusterroles,roles -A -o json | jq` for `"*"`)
      - [ ] `escalate`/`bind`/`impersonate` granted only to a named, audited admin path — never automation? (`kubectl who-can escalate clusterroles`, etc.)
      - [ ] No `cluster-admin` (or any broad role) bound to a ServiceAccount, `system:authenticated`, or `system:unauthenticated`? (`kubectl get clusterrolebindings -o json | jq '...roleRef.name=="cluster-admin"'`)
      - [ ] Aggregated ClusterRoles reviewed — nothing unexpected aggregates into `admin`/`edit`/`view`?
      - [ ] Broad `secrets` read, `pods` create, `serviceaccounts/token` create, and policy-engine CRD writes (`rules/03`) scoped tightly (treated as escalation primitives)?
      - [ ] `automountServiceAccountToken: false` is the default; only API-calling pods opt in? (`grep -rL automountServiceAccountToken` deployments; check SA spec)
      - [ ] No manually-created long-lived `service-account-token` Secrets; external consumers use short-lived audience-scoped TokenRequest tokens?
      - [ ] One SA per workload, `default` SA unused/unbound, no SA shared across apps?
      - [ ] who-can run on every escalation primitive and the transitive closure for high-value SAs reviewed?
      - [ ] Bindings reconciled against the identity source — no dead users/SAs?
      
    • 03-admission-policy.md 11.1 KB
      # 03 — Admission Control & Policy-as-Code
      
      Scope: the gate between "a manifest was submitted" and "the object exists in etcd." Pod
      Security Admission, the policy engines (Kyverno, Gatekeeper/OPA, ValidatingAdmission
      Policy/MutatingAdmissionPolicy), the AUDIT→ENFORCE rollout discipline, image verification
      at admission, and PolicyException hygiene. **Pod-level securityContext/seccomp field
      internals are `sota-sandboxing` (rules/03);** this file owns *enforcing them at admission*.
      Image *signing/provenance production* is `sota-devsecops` (rules/02); this file owns
      *verifying signatures at admission*.
      
      Admission runs after authn/authz, on the object: **validating** webhooks/policies accept
      or reject; **mutating** ones modify (inject sidecars, set defaults). Admission is where
      policy becomes enforcement — RBAC says *who*, admission says *what's allowed to exist*.
      
      ---
      
      ## 1. Pod Security Admission (PSA) — the built-in floor
      
      PSA is the in-tree replacement for the removed PodSecurityPolicy (PSA stable since K8s
      v1.25; PSP gone since 1.25). It enforces the three **Pod Security Standards** by namespace
      **label**, in three **modes**:
      
      - **Standards**: `privileged` (no restrictions), `baseline` (blocks known escalations:
        hostNetwork/PID/IPC, privileged, hostPath, most added caps), `restricted` (hardened:
        non-root, no privilege escalation, seccomp RuntimeDefault, drop ALL caps, etc.).
      - **Modes**: `enforce` (reject), `audit` (allow + audit-log annotation), `warn` (allow +
        client warning). You can set all three independently and pin a `-version`.
      
      ```yaml
      # GOOD — enforce restricted; audit/warn at the same level catch drift in subresources
      apiVersion: v1
      kind: Namespace
      metadata:
        name: payments
        labels:
          pod-security.kubernetes.io/enforce: restricted
          pod-security.kubernetes.io/enforce-version: latest   # or pin to your minor
          pod-security.kubernetes.io/audit: restricted
          pod-security.kubernetes.io/warn: restricted
      ```
      
      **Enforce `restricted` on workload namespaces.** `baseline` is a transitional floor, not a
      destination. **PSA's limits** (this is why you also need an engine, §2):
      - PSA is **per-namespace and standard-only** — it cannot express "images must come from
        our registry," "no `:latest` tag," "every pod has resource limits," or "the SA isn't
        `default`." It only checks the built-in pod-security fields.
      - It does **not mutate** — it won't add a missing `securityContext`, only reject.
      - Privileged-namespace exemptions (kube-system, some operators) are broad. Don't let a
        workload land in an exempt/privileged namespace to dodge PSA.
      
      The *meaning* of `runAsNonRoot`, `seccompProfile`, capability drops → `sota-sandboxing`.
      
      ## 2. Policy engines — choosing one
      
      You need a general engine for everything PSA can't express. The 2026 options:
      
      | Option | What it is | Use when |
      |---|---|---|
      | **ValidatingAdmissionPolicy (VAP)** | In-tree, CEL-based validating policies. GA since K8s **v1.30**. No external webhook/pod — runs in the API server. | The check is *validation only* and expressible in CEL. Lowest operational risk (no webhook to fail, no extra pod). Prefer for simple invariants. |
      | **MutatingAdmissionPolicy (MAP)** | In-tree, CEL-based *mutating* policies. GA in K8s **v1.36** (beta 1.34, feature-gated). | Mutation you'd otherwise run a mutating webhook for, once you're on a GA-supporting version. Verify your cluster's version before relying on it. |
      | **Kyverno** | YAML-native policy engine (validate/mutate/generate/verifyImages/cleanup). Current line ~v1.18 (verify). Runs as an admission webhook + controllers. | You want readable, K8s-native policies, image verification, resource generation, and don't want to write Rego. Most teams' default. |
      | **Gatekeeper (OPA)** | OPA/Rego policies via ConstraintTemplates + Constraints. Current line ~v3.22 (verify). Webhook + audit controller; can also generate VAPs. | You already use Rego/OPA org-wide, or need very expressive logic. Heavier; Rego learning curve. |
      
      Guidance: **use VAP for what it cleanly covers** (it's the lowest-risk, in-tree option),
      and one of **Kyverno / Gatekeeper** for the rest (image verification, mutation, generation,
      cross-resource logic). Don't run all three doing overlapping things. Whatever you pick:
      policies live in git, are GitOps-deployed (`rules/04`), and have **CI-tested deny cases**
      (kyverno CLI `kyverno test`, gatekeeper-library tests, VAP unit tests) so an "improvement"
      can't silently stop blocking.
      
      **Policy objects are an escalation primitive — never delegate their creation to tenants.**
      Policies execute inside the engine's privileged pod: CVE-2026-4789 (CVSS 9.8, disclosed
      2026-03) let any author of a Kyverno `NamespacedValidatingPolicy` use the CEL
      `http.Get()`/`http.Post()` library to make arbitrary HTTP requests *from the Kyverno
      admission-controller pod* — reaching cloud metadata (169.254.169.254) and internal
      services, bypassing RBAC; it was one of a 2026 wave of Kyverno SSRF/apiCall advisories.
      So: treat create/update on (namespaced) policy CRDs like `bind`/`escalate` (`rules/02`
      §2.5); put an egress NetworkPolicy on policy-engine pods (block the metadata endpoint and
      all unneeded egress); and confirm the running line includes the CVE-2026-4789 fix (merged
      2026-04; patched releases per the project's security advisories).
      
      ```yaml
      # Kyverno — require resource limits AND block :latest (PSA cannot do either)
      apiVersion: kyverno.io/v1
      kind: ClusterPolicy
      metadata: { name: workload-baseline }
      spec:
        validationFailureAction: Enforce          # NOT Audit (see §3)
        background: true
        rules:
          - name: require-limits
            match: { any: [{ resources: { kinds: ["Pod"] } }] }
            validate:
              message: "containers must set cpu/memory limits"
              pattern:
                spec: { containers: [{ resources: { limits: { memory: "?*", cpu: "?*" } } }] }
          - name: disallow-latest
            match: { any: [{ resources: { kinds: ["Pod"] } }] }
            validate:
              message: ":latest tag is not allowed; pin a digest"
              pattern:
                spec: { containers: [{ image: "!*:latest" }] }
      ```
      
      ## 3. The AUDIT→ENFORCE rollout discipline (and the trap)
      
      New policies break workloads if you enforce blind. The discipline:
      1. **Deploy in audit/warn** (`validationFailureAction: Audit` / PSA `audit`+`warn`,
         Gatekeeper `enforcementAction: dryrun`).
      2. **Watch the audit signal** — collect every would-be violation, fix the workloads (or
         add a *scoped* exception, §5).
      3. **Flip to enforce** once the violation rate is zero/known. Then enforce stays on.
      
      **The trap (a real, recurring audit finding):** policies left in **Audit/dryrun forever**.
      Audit-only is not a control — it produces a dashboard nobody reads while violations sail
      through. Treat "policy exists but never enforces" as **High**: the org believes it's
      protected and isn't. For each Audit-mode policy, demand a flip-date and an owner, or
      downgrade it to honest "we don't enforce this."
      
      Hunt: `grep -rE 'validationFailureAction:\s*Audit|enforcementAction:\s*dryrun' policies/`
      and `kubectl get ns -o json | jq '.items[].metadata.labels | select(.["pod-security.
      kubernetes.io/enforce"]==null)'` (namespaces with warn/audit but no enforce).
      
      ## 4. Image verification at admission
      
      Signing an image (cosign, `sota-devsecops` rules/02) does nothing unless **admission
      verifies the signature and rejects unsigned/unattested images**. Use Kyverno `verifyImages`
      or the sigstore **policy-controller**:
      
      ```yaml
      # Kyverno verifyImages — Enforce, keyless, exact identity, mutate tag→digest
      apiVersion: kyverno.io/v1
      kind: ClusterPolicy
      metadata: { name: verify-signed-images }
      spec:
        validationFailureAction: Enforce
        rules:
          - name: verify
            match: { any: [{ resources: { kinds: ["Pod"] } }] }
            verifyImages:
              - imageReferences: ["registry.example.com/*"]
                mutateDigest: true            # pin the verified digest into the spec
                required: true
                attestors:
                  - entries:
                      - keyless:
                          issuer: "https://token.actions.githubusercontent.com"
                          subject: "https://github.com/org/repo/.github/workflows/release.yml@refs/heads/main"
      ```
      
      Traps:
      - **"Signing exists but admission only Audits it."** Same as §3 — unsigned images still
        run. High. Enforce, or it's theater.
      - **Prereq ordering**: enforce *can block scheduling* if your build/sign path isn't
        complete for every in-use image (base images, third-party charts, kube-system). Inventory
        what's deployed, get everything signed/allowlisted, *then* enforce — or you'll wedge the
        cluster. Scope `imageReferences` to registries you control and allowlist the rest
        explicitly (with expiry) rather than disabling enforcement.
      - Verify the exact **issuer + subject identity**, not just "a signature exists" — an
        attacker's valid signature from their own identity passes a check that doesn't pin who.
      - Require **provenance attestations** (SLSA) too, not only signatures, for high-value
        workloads (`sota-devsecops` rules/02).
      
      ## 5. PolicyException / exemption discipline
      
      Every engine has an escape hatch (Kyverno `PolicyException`, Gatekeeper Constraint
      `excludedNamespaces`/`match`, VAP `matchConditions`, PSA namespace exemptions). Without
      discipline these become permanent holes:
      - **Scoped**: exact namespace + resource + rule, never cluster-wide "skip this policy."
      - **Owned and time-bound**: an owner annotation and an expiry; CI/cron fails or alerts on
        expired exceptions.
      - **PR-reviewed and inventoried**: exceptions live in git, are reviewed like code, and a
        query lists all live ones. An untracked, unexpiring exception is how "we enforce
        restricted" quietly becomes "except in these 30 namespaces."
      
      ## Audit checklist
      
      - [ ] PSA `enforce: restricted` (or justified `baseline`) on every workload namespace, not just `warn`/`audit`? (`kubectl get ns -L pod-security.kubernetes.io/enforce`)
      - [ ] No workloads parked in privileged/exempt namespaces (kube-system, operator ns) to dodge PSA?
      - [ ] A policy engine covers what PSA can't (registry allowlist, no `:latest`, required limits, non-default SA, host-path/host-namespace bans)?
      - [ ] Engine choice sane (VAP for simple validation; Kyverno/Gatekeeper for the rest; not three overlapping)? Version current/supported?
      - [ ] Policy CRD writes (incl. Kyverno namespaced policies) never delegated to tenants; policy-engine pods behind an egress NetworkPolicy (metadata endpoint blocked); running line includes the CVE-2026-4789 fix?
      - [ ] All security policies in `Enforce`, not parked in `Audit`/`dryrun` indefinitely? (`grep -rE 'Audit|dryrun' policies/`) Each Audit-mode policy has a flip-date + owner?
      - [ ] Image verification ENFORCED for controlled registries: exact signer issuer+subject, provenance required, tag→digest mutation, full image inventory covered before enforce? (not Audit-only)
      - [ ] Policies in git, GitOps-deployed, with CI-tested deny cases (`kyverno test` / gatekeeper tests / VAP units)?
      - [ ] Exceptions scoped, owned, time-bound, PR-reviewed, inventoried, expiry-alerted? (`kubectl get polex -A`; list Gatekeeper excludedNamespaces)
      - [ ] Mutating policies/webhooks reviewed for what they inject (a hostile mutation adds a sidecar/hostPath) — see `rules/05`?
      
    • 04-gitops-controllers.md 14.3 KB
      # 04 — GitOps Controller Security (Argo CD, Flux)
      
      Scope: the controller that reconciles git into the cluster — Argo CD and Flux. Its own
      privileges and self-management risk, project/tenant scoping (the `clusterResourceWhitelist`
      escalation trap), SSO/RBAC, repo/credential scoping, ApplicationSet/templating injection,
      sync strategy, drift, recent CVEs, and promotion/rollback as git operations. CI/CD
      *pipeline* and deployment *strategy* are `sota-devsecops` (rules/06); this file owns the
      in-cluster controller's security model. SSO/OIDC *design* is `sota-identity-access`.
      
      **Core principle:** GitOps is the *only* write path to the cluster. Humans don't `kubectl
      apply` to prod — they open a PR; the controller reconciles the merged state. This makes
      git the audit log and review gate, and makes the **controller a cluster-admin-class
      component** whose compromise = cluster compromise. Scope it accordingly.
      
      ---
      
      ## 1. The controller is privileged — treat it as crown jewels
      
      Argo CD's application-controller / Flux's controllers apply arbitrary manifests, so they
      hold broad (often cluster-admin-equivalent) RBAC by design. Therefore:
      - **The git repo + the controller's RBAC + who can change either = your real attack
        surface.** Anyone who can merge to the watched branch, or who can edit the controller's
        config, can deploy anything the controller can apply.
      - **Branch protection + required review + signed commits** on GitOps repos are not
        optional (`sota-devsecops` rules/01). A self-approved merge to `main` is a deploy.
      - **Self-management risk**: if the controller manages *its own* manifests (RBAC, config,
        the App that defines other Apps — "app-of-apps"), a malicious PR can widen its own
        privileges or repoint it at a hostile repo. Put the controller's own bootstrap under
        extra-tight review, or manage it out-of-band.
      - Run the controller in its own namespace; restrict who has RBAC *to* Argo CD/Flux
        resources (Applications, AppProjects, GitRepositories, Kustomizations) — editing an
        Application is editing a deployment.
      
      ## 2. Argo CD AppProject scoping — the `clusterResourceWhitelist` trap
      
      `AppProject` is Argo CD's tenancy/blast-radius boundary. It constrains which repos, which
      destination clusters/namespaces, and which **resource kinds** an Application may deploy.
      The traps:
      
      ```yaml
      # BAD — a project that can deploy anything, anywhere, from any repo
      apiVersion: argoproj.io/v1alpha1
      kind: AppProject
      metadata: { name: team-a, namespace: argocd }
      spec:
        sourceRepos: ["*"]
        destinations: [{ server: "*", namespace: "*" }]
        clusterResourceWhitelist: [{ group: "*", kind: "*" }]   # can create ClusterRoles, etc.
      ```
      
      - **`clusterResourceWhitelist: [{group:'*',kind:'*'}]`** lets the project create *cluster-
        scoped* resources of any kind — including `ClusterRole`/`ClusterRoleBinding`. A tenant
        who can commit to that project's repo can bind themselves cluster-admin via a manifest.
        **Combined with broad SSO/RBAC into the project, this is Critical.** Allow only the
        specific cluster-scoped kinds the tenant legitimately needs (often *none* — leave it
        empty/unset so cluster resources are denied).
      - **`sourceRepos: ["*"]`** lets the project deploy from any repo, defeating provenance.
        Pin to the exact repos.
      - **`destinations` with `*`** lets one project write to every cluster/namespace. Pin to the
        tenant's clusters and namespaces.
      - The **`default` AppProject is wide open** — don't run production Apps in it; lock it down
        (empty sourceRepos/destinations) and use named projects.
      - Use `namespaceResourceBlacklist` / `clusterResourceBlacklist` to forbid dangerous kinds
        (e.g. `ResourceQuota`, `LimitRange`, RBAC) even within an otherwise-scoped project.
      
      ```yaml
      # GOOD — scoped project; no cluster-scoped kinds, pinned repo + destination
      spec:
        sourceRepos: ["https://github.com/org/team-a-config.git"]
        destinations: [{ server: "https://kubernetes.default.svc", namespace: "team-a-*" }]
        clusterResourceWhitelist: []        # cluster-scoped resources denied
        namespaceResourceBlacklist: [{ group: "rbac.authorization.k8s.io", kind: "*" }]
      ```
      
      ## 3. SSO, RBAC, and credentials
      
      - **SSO over local accounts.** Disable the built-in `admin` account in prod (`admin.
        enabled: false`) after bootstrap, or scope it to break-glass; rotate its secret. Wire
        OIDC/SSO (design → `sota-identity-access`).
      - **Argo CD RBAC** (`policy.csv`): map SSO groups to roles; default policy `role:''`
        (no access). Scope roles to specific projects/Applications — not `role:admin` for
        everyone. The `g, <group>, role:admin` line is the equivalent of a cluster-admin binding.
      - **Repository credentials are secrets**: scope read-only deploy keys / fine-grained PATs
        per-repo, store them as Argo CD repo Secrets (or via ESO — `sota-secrets-management`),
        never embed write creds. A controller with org-write git creds can poison source.
      - **Cluster credentials**: when one Argo CD manages multiple clusters, each cluster
        credential is a foothold; scope and rotate them.
      
      ## 4. ApplicationSet & templating injection
      
      ApplicationSet generates Applications from generators (git directories, PR/SCM, lists,
      clusters). Generated fields are **templated** — untrusted input in a generator can inject
      into the Application spec:
      - **PR/SCM generators reading fork PRs** can let an external contributor influence
        generated Applications (repo URL, path, namespace) — treat like the `pull_request_target`
        problem (`sota-devsecops` rules/01). Restrict generators to trusted repos/branches; don't
        template attacker-controlled fields into `project`, `destination`, or `source`.
      - Keep ApplicationSet `templatePatch`/Go-template values constrained; an injected
        `project:` field can move an App into a more-privileged AppProject.
      
      ## 5. Sync strategy & drift
      
      - **Auto-sync vs sync-with-approval**: auto-sync makes a merge an immediate deploy — good
        for low-risk envs, paired with strong PR review. For prod, gate with manual sync /
        **sync windows** / approval so a merge doesn't instantly hit prod without a release step.
      - **`selfHeal: true`** reverts out-of-band `kubectl` changes back to git — this is the
        point of GitOps (it kills drift and unaudited changes). Pair with **`prune: true`**
        carefully (prune deletes resources removed from git; a bad git change can delete prod).
      - **Drift detection is a security signal**: an OutOfSync resource nobody changed in git
        means someone wrote to the cluster directly (or a controller is fighting). Alert on
        unexpected drift; investigate it as a possible intrusion, not just noise.
      - **Promotion and rollback are git operations**: promote by moving a verified digest
        through environment overlays/branches (build-once-promote-many, `sota-devsecops`
        rules/06); roll back by reverting the git commit, not by hand-editing the cluster.
      
      ## 6. Patch the controller — recent Argo CD CVEs
      
      The GitOps controller is high-value and has a live CVE stream; track and patch it like the
      control plane (`rules/01` §7). Recent examples (verify against the Argo CD security
      advisories before citing exact IDs/versions):
      - **GHSA-3v3m-wc6v-x4x3 (CVE-2026-42880), ~May 2026, Critical** — Kubernetes Secret
        extraction via the ServerSideDiff feature: a low-privilege/read-only user could obtain
        plaintext Secret data. Patched in the then-current 3.2.x/3.3.x patch releases; upgrade.
      - **GHSA-786q-9hcg-v9ff (CVE-2025-55190), Sep 2025, Critical (CVSS ~10)** — project API
        tokens with even `get` permission could retrieve repository credentials. Patched in
        2.13.9 / 2.14.16 / 3.0.14 / 3.1.2.
      - Plus assorted webhook-parser DoS and stored-XSS (annotation) advisories.
      
      Current Argo CD line is ~3.4.x (verify). Stay on a supported minor, watch the advisories
      feed, and patch Critical auth/secret-exposure issues on the emergency track. Flux likewise
      publishes advisories — track its controllers' CVEs.
      
      ## 7. Write-back controllers — the success log is not the commit
      
      §1–§6 cover what the controller may do *inward*: git → cluster. A controller that
      also writes **outward** — an image-update controller, a PR bot, config sync
      committing rendered manifests back — is an instrument reporting on itself, so
      `sota-code-security` rules/15 §2 applies to it in full. Its log describes the
      update it *decided* to make, not the write landing.
      
      **R7.1 — Verify at the remote, never from the controller's own counters.**
      `git ls-remote` the branch, read the file on `origin`, or diff the rendered
      manifest. A line like `images_considered=2 images_skipped=0 images_updated=1
      errors=0` reports decisions taken in memory, and is emitted independently of any
      commit.
      
      Measured (2026-08-18, a GitOps image-update controller on a private cluster).
      A second image alias was added to one custom resource so that two images built
      from the same commit would move in lockstep. The controller then logged
      `Committing 1 parameter update(s)` and `Successfully updated the live application
      spec` **every reconcile cycle for ~15 minutes across ~7 cycles**, while making no
      `git commit` or `git push` at all — nothing in its own log, and the remote branch
      never moved. Net effect: the primary application silently stopped auto-deploying
      and the two images skewed apart, which is the exact invariant the second alias
      existed to guarantee. Removing the alias, with the primary image deliberately left
      on the old tag so the next reconcile was a clean experiment, produced a commit
      within **84 seconds** — that is what established cause rather than correlation.
      The controller-side root cause was never found and did not need to be: the
      operator has to check the remote, not explain the bug.
      
      **R7.2 — A per-object counter counts decisions; an aggregate hides the failure.**
      Both traps are in that incident. `updated=1` counts what the reconciler resolved
      to do. And where a controller manages N objects, one object's success masks
      another's silent failure in any rolled-up total — the write kept working for one
      image and stopped for the other while the aggregate stayed healthy. **Assert on
      all N, not on the sum.** Alert on the **age of the last observed remote change**,
      which goes stale by itself when writes stop; a counter of successes never will.
      
      **R7.3 — This is a liveness gap, not an integrity one.** Nothing was corrupted and
      no alarm was wrong: a write that should have happened did not, and every signal
      reporting on it was a signal about intent. Freshness is the only observable that
      degrades when an outbound write stops — the same distinction `sota-code-security`
      rules/04 §8 draws between integrity and completeness for audit ledgers.
      
      ## 7a. What you wrote is not what lands — verify a rendered object by rendering
      
      An overlay, a transformer or a templating layer sits between the file you edited and the
      object the controller applies, and **the field you wrote can be overridden without a
      warning**. The common one: a manifest declares `namespace: monitoring`, the kustomization
      sets `namespace: argo-workflows`, and the object lands in the second. The tell is a
      diagnostic that reads like a *different* failure — `kubectl get <kind> -n monitoring`
      returns nothing, which reads as **"it was never applied"** rather than **"you are looking in
      the wrong place"**, and the next hour goes into re-applying something that already exists.
      
      - **Render, then grep the render** — `kustomize build .`, `helm template`, the controller's
        own diff/preview — and confirm the field's *final* value before you use it to look
        something up. Same discipline as reproducing a gate's exact invocation
        (`sota-devsecops` rules/09 §5): what you believe you ran and what ran are two questions.
      - **Ask the cluster where the object actually is** (`kubectl get <kind> -A`) before
        concluding it is absent. An empty namespaced query is an absence claim, and it carries the
        burden of one (`sota/rules/03` §2).
      - **Deleting a GitOps-managed object means removing it from the rendered set**, not deleting
        its file. The controller prunes what it no longer renders, so dropping the reference
        converges the cluster immediately and the file tidy-up can follow under normal review.
        Useful when file deletion is blocked; it also means an object can be *functionally* gone
        while its manifest is still in the repository.
      
      ## Audit checklist
      
      - [ ] **Is any conclusion drawn from a field an overlay can override?** (§7a) Namespace,
            name prefixes, labels and images are transformer-owned — render (`kustomize build`,
            `helm template`) and grep the render before treating an empty namespaced lookup as
            "not applied"; `kubectl get <kind> -A` before any absence claim.
      - [ ] GitOps is the only write path to prod (no routine human `kubectl apply`); GitOps repos have branch protection + required review + signed commits?
      - [ ] Controller runs least-privilege where possible; RBAC *to* Argo CD/Flux resources (Applications/AppProjects/Kustomizations) is restricted; self-management/bootstrap under extra review?
      - [ ] No AppProject with `clusterResourceWhitelist: [{group:'*',kind:'*'}]` (or it's justified + tightly access-controlled)? (`kubectl get appprojects -A -o yaml | grep -A3 clusterResourceWhitelist`)
      - [ ] AppProjects pin `sourceRepos` and `destinations` (no `*`); `default` project locked down; RBAC-creating kinds blacklisted where not needed?
      - [ ] Argo CD SSO wired, built-in admin disabled/break-glass, RBAC `policy.csv` scoped per project (no blanket `role:admin`)?
      - [ ] Repo/cluster credentials scoped (read-only deploy keys, per-repo), stored as secrets/ESO, rotated — no org-write creds in the controller?
      - [ ] ApplicationSet generators restricted to trusted repos/branches; no untrusted input templated into `project`/`destination`/`source`?
      - [ ] Prod sync gated (approval/sync windows), not blind auto-sync; `prune`/`selfHeal` reviewed; drift alerted and triaged as a security signal?
      - [ ] Promotion = move verified digest through git; rollback = git revert (not hand-edits)?
      - [ ] Argo CD/Flux on a supported version, CVE feed tracked, recent Critical advisories (Secret-extraction, cred-exposure) patched? (`argocd version`)
      - [ ] Any controller that writes **outward** (image updater, PR bot, config sync committing back) is verified **at the remote** — `git ls-remote` / the file on `origin` / a rendered-manifest diff — not from its own `updated=N errors=0`; freshness of the last remote change is alerted, and a controller managing N objects asserts on all N rather than an aggregate? (§7)
      
    • 05-operators-crds-webhooks.md 8.4 KB
      # 05 — Operators, CRDs & Admission Webhooks
      
      Scope: extension components that run with cluster privilege — operators (controllers
      reconciling CRDs), the CRDs themselves, and admission webhooks (validating/mutating). These
      are the third-party code you grant standing power inside the cluster; each is both a
      control and an attack surface. Trusted-controller examples: ExternalSecrets Operator (ESO),
      cert-manager. Secret *backends/ESO usage* is `sota-secrets-management` (rules/02); this
      file owns *vetting the operator's cluster privilege and webhook surface*.
      
      ---
      
      ## 1. The operator privilege problem
      
      An operator is a long-running controller with a ServiceAccount that, by design, holds broad
      RBAC: it watches and mutates the resources it manages, often cluster-wide, often including
      **Secrets, RBAC objects, or pods**. That makes every installed operator a standing,
      privileged principal — and **CRD-mediated escalation** the recurring risk:
      
      - The operator reads a CRD (a custom resource you or a tenant created) and *acts* with its
        own (broad) privileges. So **anyone who can create/edit that CRD can often induce the
        operator to do something privileged on their behalf** — a confused-deputy. Example: a
        CRD field that becomes a pod's `serviceAccountName`, `hostPath` mount, image, or RBAC
        binding lets a CRD-author escalate via the operator's hands.
      - An operator with `clusterroles`/`clusterrolebindings` write, or `secrets` cluster-wide,
        or `pods` create with arbitrary SA, is effectively cluster-admin-adjacent. Owning the
        operator pod (its image, its dependencies, an RCE) = owning that power.
      
      **Mitigations:**
      - **Vet the operator's RBAC before install.** `helm template` / read the bundled
        ClusterRole (`rules/07`, `rules/02`). Reject or scope wildcard verbs, cluster-wide
        `secrets:*`, and RBAC-write unless the operator genuinely needs them. Prefer
        **namespaced-scoped operators** (watch one/few namespaces via OLM `OperatorGroup` or the
        operator's `--namespace` flag) over cluster-wide where the operator supports it.
      - **Limit who can create the operator's CRs.** RBAC on the CRD's API group is an
        escalation primitive — treat `create`/`update` on a CRD whose fields drive pods/RBAC/
        secrets like `pods` create (`rules/02` §2.5). Don't hand tenants CR-create on a CRD the
        operator turns into privileged actions.
      - **Pin and patch the operator** like any privileged software (image digest, CVE feed).
      - **Watch what the operator mutates**: an operator that injects sidecars or modifies pod
        specs is doing admission-time mutation — audit it like a mutating webhook (§3).
      
      Trusted, widely-used operators still need scoping: **ESO** holds creds to your secret
      backend and writes Secrets — scope its `SecretStore`/`ClusterSecretStore` and the
      namespaces it serves; **cert-manager** can issue certs and holds issuer credentials —
      scope `ClusterIssuer` usage and ACME/CA access. "Popular" is not "harmless when
      over-privileged."
      
      ## 2. CRD validation & security
      
      CRDs extend the API; a sloppy CRD is an injection and DoS vector into the operator.
      - **Structural schema with strict validation.** Define `openAPIV3Schema` with types,
        enums, patterns, and `x-kubernetes-validations` (CEL) for cross-field rules. A CRD that
        accepts arbitrary fields (`x-kubernetes-preserve-unknown-fields: true`) lets attackers
        smuggle data the operator may mishandle — avoid except where genuinely needed.
      - **Validate fields that become privileged.** If a CR field becomes a container image, an
        SA name, a host path, a command, or an RBAC subject, the CRD schema (and/or an admission
        policy) must constrain it — don't trust the operator to sanitize.
      - **CRD scope**: cluster-scoped CRDs are visible/creatable per cluster-wide RBAC;
        namespaced CRDs are easier to delegate safely. Prefer namespaced unless the resource is
        genuinely global.
      - **Conversion webhooks** (for multi-version CRDs) are admission-webhook-class surface —
        same hardening as §3.
      - Removing/replacing a CRD with `deletionPolicy` implications can cascade-delete CRs and
        the operator-managed resources — review before applying CRD changes via GitOps.
      
      ## 3. Admission webhooks as attack surface AND single point of failure
      
      Validating/mutating webhooks are external HTTPS endpoints the API server calls on every
      matching request. They are powerful (they can reject or rewrite any object) and fragile (if
      they're down or misconfigured, they can wedge the cluster). Configure deliberately:
      
      ```yaml
      # Knobs that matter on a (Validating|Mutating)WebhookConfiguration
      webhooks:
        - name: policy.example.com
          failurePolicy: Fail            # Fail = secure (reject if webhook down) for SECURITY gates
                                         # Ignore = available (allow if down) — only for non-security mutators
          timeoutSeconds: 5              # short; a slow/hung webhook stalls every API write
          namespaceSelector:             # EXCLUDE kube-system / the webhook's own ns to avoid deadlock
            matchExpressions:
              - { key: kubernetes.io/metadata.name, operator: NotIn, values: ["kube-system","webhook-system"] }
          matchPolicy: Equivalent
          sideEffects: None
          admissionReviewVersions: ["v1"]
          clientConfig:
            caBundle: <pinned CA>        # TLS verified; rotate before expiry
      ```
      
      The hard tradeoffs and traps:
      - **`failurePolicy`**: `Fail` is correct for **security** webhooks (an unavailable policy
        engine must not mean "allow everything") — but a `Fail` webhook that covers its own
        namespace or kube-system can **deadlock the cluster** (you can't restart the webhook
        because admission needs the webhook). Always set a `namespaceSelector`/`objectSelector`
        that **excludes the webhook's own namespace and kube-system**. Run the webhook HA
        (multiple replicas, PDB — `rules/06`) so `Fail` doesn't take you down on a single pod
        restart. Non-security convenience mutators may use `Ignore`, but know that means
        fail-open.
      - **`timeoutSeconds`**: keep it small (≤5–10s). Every matching API write blocks on the
        webhook; a hung webhook is a cluster-wide write outage.
      - **TLS**: the `caBundle` must be valid and **rotated before expiry** — an expired webhook
        cert with `failurePolicy: Fail` is a self-inflicted outage; with `Ignore` it silently
        disables your control. Use cert-manager (and its CA-injector) to manage and rotate
        webhook certs; alert on cert expiry.
      - **Scope (`rules`/`namespaceSelector`/`objectSelector`)**: match only what you must. An
        over-broad webhook intercepting every resource is latency + blast radius. A *malicious or
        compromised* webhook scoped to `pods`/`*` can read every spec (data exfil) and a mutating
        one can inject sidecars, hostPath mounts, or credentials into every pod — so **who can
        create/edit WebhookConfigurations is an escalation primitive** (it's cluster-wide RBAC;
        guard `admissionregistration.k8s.io` writes like RBAC writes, `rules/02`).
      - Prefer **in-tree CEL policies (ValidatingAdmissionPolicy)** over a webhook where the
        logic fits (`rules/03` §2) — no external endpoint means no webhook outage/attack surface.
      
      ## Audit checklist
      
      - [ ] Every installed operator's RBAC reviewed; no unjustified wildcard verbs, cluster-wide `secrets:*`, RBAC-write, or `pods` create with arbitrary SA? (`kubectl get clusterrole -o yaml` for each operator SA; trace its ClusterRoleBinding)
      - [ ] Operators run namespaced-scoped where supported, not cluster-wide by default?
      - [ ] RBAC to create/edit operator CRDs is restricted (treated as an escalation primitive when the CRD drives pods/RBAC/secrets)?
      - [ ] CRDs have structural schemas with validation (`x-kubernetes-validations`), no needless `preserve-unknown-fields`; privileged-becoming fields constrained?
      - [ ] Trusted controllers (ESO, cert-manager) scoped — SecretStore/Issuer usage and served namespaces limited, backend creds tight?
      - [ ] Operators pinned by digest and on a tracked CVE/patch cadence?
      - [ ] Security webhooks `failurePolicy: Fail` BUT with `namespaceSelector` excluding kube-system and the webhook's own ns; webhook runs HA + PDB?
      - [ ] Webhook `timeoutSeconds` small; webhook scope (`rules`/selectors) minimal, not `*`?
      - [ ] Webhook TLS `caBundle` valid, managed by cert-manager, rotated before expiry, expiry-alerted?
      - [ ] Writes to `admissionregistration.k8s.io` (WebhookConfigurations) guarded like RBAC writes? (`kubectl who-can create validatingwebhookconfigurations`)
      - [ ] Logic that fits CEL moved to ValidatingAdmissionPolicy instead of a webhook where practical?
      
    • 06-workloads-tenancy.md 8 KB
      # 06 — Workloads, Scheduling Availability & Multi-Tenancy
      
      Scope: the platform-level properties of workloads that affect **availability and tenant
      isolation** — resource requests/limits as QoS (not isolation), PodDisruptionBudgets,
      topology spread / anti-affinity, priority/preemption, the namespace-as-soft-boundary
      reality and when you need hard multi-tenancy, and where Secrets live. **Pod isolation
      mechanics (securityContext, seccomp, runtime classes) are `sota-sandboxing` (rules/03);**
      this file references the *isolation decision* but does not re-spec the pod fields. Secret
      *backends* are `sota-secrets-management`; this file states the etcd-encryption requirement
      and points there.
      
      ---
      
      ## 1. Resource requests/limits — availability + QoS, NOT isolation
      
      Requests/limits govern scheduling and the cgroup budget; they are an **availability and
      fairness** control, not a security boundary (a shared kernel means a noisy/hostile neighbor
      can still affect others — real isolation is `sota-sandboxing`'s node/runtime story).
      
      - **Set both `requests` and `limits`** for CPU and memory on every container.
        - `requests` drive scheduling (the scheduler packs to requested capacity).
        - **Memory `limit` is a hard kill**: exceed it → OOMKill. Set it from real usage +
          headroom; too low = crashloop, too high = node memory overcommit and node-level OOM
          that evicts *other* pods.
        - **CPU `limit` throttles** (CFS), it doesn't kill — and aggressive CPU limits cause
          latency cliffs. Many shops set CPU *requests* and omit CPU *limits* (or set generous
          ones) to avoid throttling while keeping scheduling fairness; decide deliberately.
      - **QoS classes** follow from this: **Guaranteed** (requests == limits for all
        containers) is evicted last; **Burstable** (requests < limits) next; **BestEffort** (none
        set) is evicted first under node pressure. Critical workloads should be Guaranteed.
      - **A pod with no requests is a scheduling and eviction hazard** — it can be packed onto a
        full node and is first to be evicted. Enforce limits/requests at admission (`rules/03`)
        and set namespace **`LimitRange`** (defaults) + **`ResourceQuota`** (caps) so a tenant
        can't starve the cluster or schedule unbounded pods.
      
      ## 2. Disruption, spread & priority — staying up
      
      - **PodDisruptionBudget (PDB)** on every multi-replica critical workload: caps how many
        pods *voluntary* disruptions (node drain/upgrade) may take down at once
        (`minAvailable`/`maxUnavailable`). Without a PDB, a node drain during an upgrade can take
        all replicas at once → outage. (PDBs don't protect against involuntary disruptions like
        node crashes — that's replicas + spread.)
      
      ```yaml
      apiVersion: policy/v1
      kind: PodDisruptionBudget
      spec:
        minAvailable: 2                      # or maxUnavailable: 1
        selector: { matchLabels: { app: api } }
      ```
      
      - **Topology spread constraints** spread replicas across zones/nodes so one failure domain
        loss doesn't kill the service:
      
      ```yaml
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector: { matchLabels: { app: api } }
      ```
      
      - **Pod anti-affinity** (`podAntiAffinity`) keeps replicas off the same node — use
        `requiredDuringScheduling` for hard, `preferred` for soft. Topology spread is usually the
        cleaner modern tool; don't double-specify conflicting rules.
      - **PriorityClass & preemption**: assign higher `priorityClassName` to platform-critical
        pods so they preempt lower-priority workloads under contention. Beware: a too-high
        default priority on tenant workloads lets a tenant preempt platform pods — reserve top
        priorities for the platform, and consider `preemptionPolicy: Never` for non-urgent batch.
      
      ## 3. Multi-tenancy — namespace is a SOFT boundary
      
      The most common dangerous assumption: **a Namespace is not a security boundary.** It scopes
      names and RBAC, but tenants in different namespaces still share **the same nodes, kernel,
      control plane, CNI, and CRDs**. A container escape, a kernel CVE, a node-level secret read,
      or a cluster-scoped resource crosses namespaces. So:
      
      - **Soft multi-tenancy (namespace-per-tenant)** is appropriate only for **mutually trusted
        or low-sensitivity** tenants (teams in one org). Even then, enforce per-namespace: RBAC
        isolation (`rules/02`), `ResourceQuota`/`LimitRange`, PSA `restricted` (`rules/03`), and
        **default-deny NetworkPolicy** — note the requirement here, but NetworkPolicy/CNI depth
        is `sota-network-security`.
      
      - **When you need HARD multi-tenancy** (untrusted tenants, strong compliance/regulatory
        separation, hostile-neighbor model), namespaces are insufficient. Escalate the boundary:
        - **Separate clusters per tenant** — the strongest, simplest-to-reason-about boundary;
          no shared control plane/etcd. Default for genuinely untrusted tenants.
        - **vCluster** (virtual clusters) — each tenant gets its own API server/control plane
          syncing into a host namespace; far stronger control-plane isolation than a namespace,
          cheaper than separate clusters. Good middle ground; the *data plane* still shares host
          nodes unless combined with node isolation.
        - **Node isolation** — dedicate node pools per tenant (taints/tolerations + node
          affinity) so workloads don't co-reside; combine with a **stronger runtime boundary
          (gVisor/Kata)** for untrusted code — this is `sota-sandboxing` (rules/01 boundary
          choice, rules/03 runtime classes). Co-residency of untrusted tenants on a shared kernel
          is the thing node isolation + microVM/gVisor exists to fix.
      
        State the tenancy model and its boundary explicitly in design; "we use namespaces for
        isolation" against an untrusted tenant is a **High** finding.
      
      ## 4. Secrets in Kubernetes (platform view)
      
      - **etcd encryption-at-rest with KMS v2 is mandatory** — K8s Secrets are base64, not
        encrypted, by default (full treatment in `rules/01` §4).
      - **Don't inject secrets as plain env vars** where avoidable — env is readable via
        `/proc`, crash dumps, `kubectl describe`, and child processes; prefer **mounted files**
        (projected/volume) and tools that support file-based secrets. (App-level secret handling
        is `sota-secrets-management`.)
      - **Use a real secrets workflow**: **ExternalSecrets Operator** (sync from Vault/cloud
        secret managers — `sota-secrets-management` rules/02, and scope it per `rules/05`),
        **sealed-secrets** (encrypt secrets safe to commit to git for GitOps), or the **Secrets
        Store CSI driver** (mount from an external store, optionally never landing in etcd). Pick
        per your GitOps model; the goal is that the source of truth is an encrypted store, not a
        plaintext manifest in git.
      - **Restrict who can read Secrets** (`rules/02` §2.5) — `get`/`list secrets` is
        near-credential-equivalent. Audit it.
      
      ## Audit checklist
      
      - [ ] Every container sets memory requests+limits and CPU requests (CPU limits a deliberate decision)? Critical workloads Guaranteed QoS? (`kubectl get pods -A -o json | jq '..|.resources? // empty'`)
      - [ ] Namespaces have `ResourceQuota` + `LimitRange` so a tenant can't starve the cluster or schedule unbounded pods? (`kubectl get resourcequota,limitrange -A`)
      - [ ] Multi-replica critical workloads have a PDB; PDB values won't block legitimate drains or allow full takedown? (`kubectl get pdb -A`)
      - [ ] Topology spread / anti-affinity spreads replicas across zones/nodes (no single-node single-zone critical service)?
      - [ ] PriorityClasses reserve top priorities for the platform; tenants can't preempt platform pods?
      - [ ] Tenancy model stated explicitly: is "namespace" being relied on as a security boundary against untrusted tenants? (that's a High finding — escalate to vCluster / separate clusters / node isolation + gVisor/Kata per sota-sandboxing)
      - [ ] Soft-tenant namespaces enforce RBAC isolation, quota, PSA restricted, and default-deny NetworkPolicy (depth → sota-network-security)?
      - [ ] etcd Secrets encrypted with KMS v2 (rules/01); secrets mounted as files not env where possible; ESO/sealed-secrets/CSI used over plaintext-in-git (sota-secrets-management)?
      - [ ] Secret read RBAC scoped and audited (rules/02)?
      
    • 07-supply-chain-audit.md 8.7 KB
      # 07 — Helm/Kustomize Supply Chain & K8s Audit Logging
      
      Scope: two platform concerns the other files lean on — (a) reviewing what a chart/overlay
      actually applies before it hits the cluster, plus chart/image provenance enforced **at
      admission**; and (b) producing and shipping the Kubernetes API audit log. Image *signing
      production* and *CI provenance* are `sota-devsecops` (rules/02, rules/04); admission
      *verification mechanics* are `rules/03`; **detection content on the audit stream and
      runtime detection (Falco/Tetragon) are `sota-detection-engineering`** — this file owns
      *generating and shipping* the stream, not writing the detections.
      
      ---
      
      ## 1. `helm template` review before apply
      
      A Helm chart is arbitrary templated YAML that becomes live cluster objects. Treat
      installing a third-party chart like running third-party code with the controller's
      privileges. **Always render and review before apply** — never `helm install` an unread
      chart into a privileged cluster:
      
      ```bash
      helm template rel chart/ -f values.yaml > rendered.yaml
      # what RBAC does this chart grant?
      grep -nE 'kind:\s*(ClusterRole|ClusterRoleBinding|Role|RoleBinding)' rendered.yaml
      # privilege-granting values / specs
      grep -nE 'privileged:\s*true|hostPID|hostNetwork|hostPath|cluster-admin|automountServiceAccountToken:\s*true|"\*"' rendered.yaml
      # images and tags (are they pinned? from your registry?)
      grep -nE 'image:' rendered.yaml
      ```
      
      Then diff against the running state (`helm diff upgrade`, or render-and-`kubectl diff`) so
      an upgrade can't silently widen privileges. In GitOps (`rules/04`), the rendered output is
      what the controller applies — review the chart version bump like code.
      
      ## 2. The RBAC-in-charts / privilege-values trap
      
      (Cross-ref `rules/02` §2.6.) Charts ship their own RBAC and expose values that grant
      privilege. The recurring traps:
      - **`rbac.create: true` paired with a `*/*` ClusterRole**, or a values toggle like
        `rbac.clusterAdministrator: true` / `clusterRole.rules: [{apiGroups:["*"],...}]` — the
        chart hands its ServiceAccount the cluster. A logging/monitoring agent that "needs to see
        everything" rarely needs *write* on everything; scope it.
      - **`securityContext`/`podSecurityContext` values defaulting to root/privileged**, or
        `hostNetwork`/`hostPID`/`hostPath` toggles. Override to non-root, no host namespaces;
        admission (`rules/03`) should reject these anyway — if the chart can't deploy under PSA
        `restricted`, that's a signal.
      - **`serviceAccount.create` + a powerful binding** — confirm the SA it creates isn't bound
        to a built-in `admin`/`cluster-admin` role.
      - **Operators bundled in charts** — apply `rules/05` (vet the operator RBAC and CRDs).
      
      Don't disable your own admission policies to make a chart install. If a vendor chart
      demands cluster-admin or privileged pods, push back, scope it, or sandbox it — vendor
      convenience is not a risk acceptance.
      
      ## 3. Chart & image provenance — enforced at admission
      
      - **Sign and verify Helm charts.** OCI-registry charts can be **cosign-signed**; verify the
        signature/provenance before install (`cosign verify <oci-chart-ref>`), and pin the chart
        by **digest**, not a mutable version tag, in your GitOps source. An unsigned chart pulled
        by floating version is mutable supply-chain surface.
      - **Only-signed-images enforcement** is the admission control from `rules/03` §4 — the
        manifests a chart produces must reference images that pass signature/provenance
        verification (cosign/Kyverno verifyImages / policy-controller). Signing without admission
        enforcement is theater (`sota-devsecops` rules/02 produces the signatures; this is where
        they're checked).
      - **Digest-pin images at the manifest layer.** Deploy manifests reference
        `image@sha256:<digest>` (or a signed tag your admission policy mutates to a verified
        digest), never `:latest` or a floating tag — build-once-promote-many
        (`sota-devsecops` rules/06). Mutable tags admit a different artifact than you reviewed.
      - **Kustomize**: pin `images:` with `digest:`; review `patches`/`patchesStrategicMerge`
        for privilege escalation the same way (a patch can add `privileged: true` or a hostPath);
        beware remote bases (`resources:` pointing at a URL) — pin and review them like remote
        charts.
      
      ## 4. Kubernetes API audit logging — produce the stream
      
      The API server audit log is the authoritative record of *who did what to the cluster* —
      the basis for forensics and the detections in `sota-detection-engineering`. Without it, a
      cluster compromise is uninvestigable. (On managed clusters, enable the provider's
      control-plane audit logging — you configure the policy via the provider, e.g. EKS audit
      logs to CloudWatch, GKE audit logs to Cloud Logging, AKS to the diagnostic settings.)
      
      **Audit levels** (per rule, least→most): `None` → `Metadata` (who/what/when, no bodies) →
      `Request` (+ request body) → `RequestResponse` (+ response body). Tune per sensitivity —
      log everything at `Metadata`, escalate sensitive verbs/resources to `RequestResponse`, and
      drop noise (read-only system loops) to `None`.
      
      ```yaml
      # audit-policy.yaml — log sensitive things richly, drop noise, default to Metadata
      apiVersion: audit.k8s.io/v1
      kind: Policy
      omitStages: ["RequestReceived"]
      rules:
        # Secrets/configmaps: log requests but DON'T capture bodies (would log secret values)
        - level: Metadata
          resources: [{ group: "", resources: ["secrets","configmaps"] }]
        # RBAC and admission changes: full bodies — these are escalation actions
        - level: RequestResponse
          resources:
            - { group: "rbac.authorization.k8s.io", resources: ["*"] }
            - { group: "admissionregistration.k8s.io", resources: ["*"] }
        # exec/attach/portforward into pods: high-signal, log them
        - level: Request
          resources: [{ group: "", resources: ["pods/exec","pods/attach","pods/portforward"] }]
        # drop noisy authenticated read loops from system components
        - level: None
          users: ["system:kube-scheduler","system:kube-controller-manager"]
          verbs: ["get","list","watch"]
        - level: Metadata           # default for everything else
      ```
      
      Note: **don't log Secret/ConfigMap bodies** (`Request`/`RequestResponse` on `secrets`)
      or you write secret values into the audit log — keep them at `Metadata`.
      
      ## 5. Ship the stream (tamper-resistant, retained)
      
      - **Forward audit logs off the control-plane node** to tamper-resistant, append-only
        storage in a **separate trust domain** — credentials that can compromise the cluster must
        not be able to delete its audit trail. Retention ≥1 year (align with compliance —
        `sota-privacy-compliance`).
      - Pipe to your SIEM / log platform; the **plumbing** is `sota-observability` territory, the
        **detections** (privilege escalation, anonymous access, exec into prod, token abuse,
        policy disable) are `sota-detection-engineering`.
      - **Pipeline/controller identities are alertable principals**: an action by the Argo CD SA
        outside a sync, or by a CI SA at an odd time, is a signal.
      
      ## 6. Runtime security pointer
      
      Audit logs capture the *API* surface; they don't see in-container behavior (a process that
      never calls the API). For runtime detection — anomalous syscalls, unexpected egress,
      process execution, container drift — deploy **Falco** or **Tetragon (eBPF)**. Their
      deployment-as-privileged-DaemonSet hardening overlaps `sota-sandboxing`, and the
      **detection rules/content are `sota-detection-engineering`**. This skill's job is to ensure
      the audit-log source exists and is shipped; pair it with runtime detection there.
      
      ## Audit checklist
      
      - [ ] Third-party charts rendered (`helm template`) and reviewed for RBAC, privileged/host-* specs, and images BEFORE install; upgrades diffed so privilege can't silently widen?
      - [ ] No chart values granting cluster-admin / `*/*` ClusterRole / root-privileged pods accepted unscoped (RBAC-in-charts trap)? Admission policies not disabled to force an install?
      - [ ] Charts signed (cosign) and pinned by digest; remote Kustomize bases pinned + reviewed?
      - [ ] Images digest-pinned at the manifest layer (no `:latest`/floating tags); only-signed-images enforced at admission (rules/03)?
      - [ ] API server audit logging enabled with a tuned policy: Metadata default, RequestResponse on RBAC/admission changes, Request on exec/attach, secrets/configmaps NOT capturing bodies, system read-loops dropped? (`grep -E 'audit-policy-file|audit-log' kube-apiserver.yaml`; managed → provider audit logging on)
      - [ ] Audit logs shipped off-node to tamper-resistant, append-only storage in a separate trust domain, retained ≥1y, not deletable by cluster-compromising creds?
      - [ ] Audit stream wired to SIEM; controller/CI/automation SAs treated as alertable principals (detections → sota-detection-engineering)?
      - [ ] Runtime detection (Falco/Tetragon) deployed to cover in-container behavior the audit log can't see (content → sota-detection-engineering)?
      
  • SKILL.md 12.5 KB
    ---
    name: sota-kubernetes
    description: >-
      State-of-the-art Kubernetes platform security and operations (2026) for
      cloud-managed (EKS/GKE/AKS) and self-hosted clusters (kubeadm, k3s/k0s,
      Talos). Use when building, operating, hardening, or auditing the cluster
      PLATFORM layer: control plane and etcd, API server/kubelet, RBAC, admission
      control and policy-as-code, GitOps controllers, operators/CRDs/admission
      webhooks, Helm/Kustomize supply chain, multi-tenancy, cluster
      lifecycle/upgrades/CVE response, and audit logging. Trigger keywords:
      Kubernetes, k8s, cluster, control plane, etcd, kube-apiserver, kubelet,
      RBAC, ClusterRole, ServiceAccount, admission controller, Pod Security
      Admission, PSA, Kyverno, Gatekeeper, OPA, ValidatingAdmissionPolicy, Argo
      CD, Flux, GitOps, operator, CRD, admission webhook, Helm, Kustomize,
      multi-tenancy, vCluster, Talos, k3s, k0s, kubeadm, EKS, GKE, AKS, CIS
      benchmark, kube-bench, audit log, version skew. NOT pod-level
      securityContext/seccomp (sota-sandboxing) or NetworkPolicy/CNI depth
      (sota-network-security).
    ---
    
    # SOTA Kubernetes Platform Security & Operations
    
    ## Purpose
    
    Engineer and audit the Kubernetes **platform** so that a compromised workload, a
    hostile chart, a leaked token, or a malicious controller cannot pivot to cluster-admin,
    read every Secret, or take the cluster down. This skill owns the layer above the pod:
    control plane and etcd, API server and kubelet hardening, RBAC and ServiceAccount
    identity, admission control and policy-as-code, GitOps controllers, operators/CRDs and
    admission webhooks, Helm/Kustomize at admission, multi-tenancy boundaries, cluster
    lifecycle, and K8s audit logging.
    
    It does **not** re-teach pod isolation mechanics. Boundaries it defers:
    - **Pod `securityContext`, seccomp, AppArmor, capabilities, PSA pod-level fields** → `sota-sandboxing` (rules/03 containers & microVMs; rules/01 boundaries). This skill owns the *admission-time enforcement* of those fields, not their internals.
    - **NetworkPolicy semantics, CNI choice, service mesh, mTLS** → `sota-network-security`. This skill states only the *requirement* (default-deny per namespace, enforced at admission).
    - **OIDC/SSO and RBAC-role *design* methodology** → `sota-identity-access`. This skill owns the K8s RBAC *mechanics and escalation traps*.
    - **Runtime/audit-log detection content (Falco/Tetragon rules, detections)** → `sota-detection-engineering`. This skill owns *producing and shipping* the audit stream.
    - **Workload identity, Secret storage backends (ESO, sealed-secrets, CSI)** → `sota-secrets-management` (rules/01, rules/02). This skill owns etcd encryption-at-rest.
    - **Cloud IAM, managed-K8s selection, DR** → `sota-cloud-infrastructure` (rules/02, rules/04, rules/07).
    - **CI/CD provenance, image signing, IaC/GitOps pipeline, runtime ops** → `sota-devsecops` (rules/02, rules/04, rules/06, rules/07).
    
    Two modes. Pick one explicitly at the start of the task.
    
    ---
    
    ## BUILD mode
    
    Use when provisioning a cluster, writing RBAC/policies/manifests, configuring a GitOps
    controller, or installing an operator/chart.
    
    1. **Name the cluster topology first**: managed (EKS/GKE/AKS) vs self-hosted (kubeadm,
       k3s/k0s, Talos), single- vs multi-tenant, who the threat actors are (workload →
       control plane? tenant → tenant? compromised CI → cluster?). Read the matching rules.
    2. **Least privilege by default**: no wildcard RBAC verbs/resources, `automountService
       AccountToken: false` unless the pod calls the API, scoped AppProjects, operators get
       the narrowest RBAC that works. Loosen only with a comment stating why.
    3. **Encrypt etcd at rest with a KMS provider** (`rules/01`) — Secrets are base64, not
       encrypted, by default. This is the single most common "we thought we were covered" gap.
    4. **Admission is fail-closed and ENFORCING, not auditing.** A policy left in `audit`/
       `warn` forever is documentation, not a control (`rules/03`). Ship the AUDIT→ENFORCE
       rollout plan with the policy.
    5. **GitOps is the only write path to the cluster.** Humans propose via PR; the
       controller reconciles. Scope the controller's own privileges and AppProjects tightly
       (`rules/04`).
    6. **Plan the upgrade before you build**: version skew, EOL date, CVE-response runbook,
       tested etcd restore (`rules/01`, `rules/07`).
    
    Deliverables: topology + threat statement, the concrete manifests/configs, the
    AUDIT→ENFORCE rollout, and the residual-risk/assumptions list (cloud-managed control
    plane internals you cannot see, org IAM, CNI behavior).
    
    ## AUDIT mode
    
    Use when reviewing an existing cluster, its RBAC, policies, GitOps config, or operators.
    
    Procedure: inventory the cluster surface (control plane flags or managed equivalent,
    RBAC graph, admission policies, GitOps controllers, operators/CRDs, namespaces/tenancy);
    for each, walk the relevant rules-file Audit checklist; verify empirically with
    `kubectl`, `kubectl auth can-i`, rbac-tool/krane, kube-bench, and `helm template` where
    possible; report findings in the format below. Do not report style nits as security
    findings.
    
    ### Severity conventions
    
    | Severity | Meaning | Examples |
    |---|---|---|
    | **Critical** | Cluster-admin, all-Secrets read, or cluster takedown reachable now | `anonymous-auth` enabled on API server/kubelet; etcd unencrypted AND reachable; ClusterRoleBinding granting `cluster-admin` to a workload SA or `system:authenticated`; wildcard `*/*` ClusterRole bound broadly; Argo CD AppProject `clusterResourceWhitelist: [{group: '*', kind: '*'}]` with broad SSO; unpatched control plane on a known-RCE CVE |
    | **High** | Escalation/secret-read by an in-cluster or contributor principal, or a single event from it | `escalate`/`bind`/`impersonate` verbs granted; secret-reader → token-mint → privilege chain; admission policy in `audit` mode for a control that should enforce; image-verification policy only `Audit`s signatures; operator with cluster-wide `secrets:*`; kubelet `read-only-port` open; no etcd backup or untested restore |
    | **Medium** | Weakens defense in depth or detection | PSA not enforced (only `warn`/`audit`); `automountServiceAccountToken` defaulted on for non-API pods; no default-deny NetworkPolicy (requirement-level; depth → network-security); audit policy missing or `None`/Metadata-only for Secret access; aggregated ClusterRole accreting verbs; no PDB on critical workloads |
    | **Low** | Hygiene, hardening headroom | RBAC subjects for departed users; unused ClusterRoles; namespaces without resource quotas; `:latest` image tags admitted; missing PolicyException expiry |
    
    Severity is judged by **reachability** (anonymous > workload/tenant > contributor > admin)
    × **yield** (cluster-admin/all-Secrets > namespace compromise > info leak > availability).
    
    ### Finding format
    
    ```
    file:line | rule | severity | effort | fix
    ```
    Where `effort` is one of trivial / small / medium / large, `rule` is the rules-file
    section (e.g. `02 §RBAC-wildcards`), and `fix` is the concrete change. Expand below the
    line with: what is wrong, who exploits it and how (concrete attack path), and a snippet
    when short. Example:
    
    ```
    clusterrole-ci.yaml:14 | 02 §escalate-verbs | Critical | small | remove the bind/escalate grant; scope to the 3 named ClusterRoles the controller actually applies
      Issue: ClusterRole bound to the CI ServiceAccount grants rbac.authorization.k8s.io/{bind,escalate} on clusterroles.
      Attack path: anyone who can run a job as this SA (any merged PR) can bind themselves cluster-admin.
    ```
    
    End every audit with: counts per severity, top 3 fixes by risk-reduction-per-effort, and
    an explicit OUT-OF-SCOPE list (managed control-plane internals, org IAM, CNI/mesh,
    runtime detection content).
    
    ## Rules index
    
    | File | Read this when... |
    |---|---|
    | [rules/01-control-plane-etcd.md](rules/01-control-plane-etcd.md) | HA control plane, API server flags (anonymous-auth, authz modes, audit), etcd encryption-at-rest with KMS v2 + backup/defrag/restore, kubelet hardening, node + immutable-distro hardening (Talos no-SSH/machine-config/SecureBoot+TPM, k3s/k0s), CIS benchmark + kube-bench |
    | [rules/02-rbac-serviceaccounts.md](rules/02-rbac-serviceaccounts.md) | Roles/ClusterRoles least-privilege, the escalation traps (wildcards, `bind`/`escalate`/`impersonate`, cluster-admin bindings, aggregated roles, secret-reader chains), ServiceAccount hygiene (automount off, bound/projected/audience-scoped tokens, no long-lived token Secrets), RBAC auditing (`auth can-i`, rbac-tool/krane/who-can) |
    | [rules/03-admission-policy.md](rules/03-admission-policy.md) | Pod Security Admission (restricted/baseline/privileged, enforce/audit/warn) and its limits, Kyverno vs Gatekeeper/OPA vs ValidatingAdmissionPolicy/MutatingAdmissionPolicy, the AUDIT→ENFORCE rollout discipline, image verification at admission (cosign/Kyverno verifyImages), PolicyException discipline |
    | [rules/04-gitops-controllers.md](rules/04-gitops-controllers.md) | Argo CD / Flux security: AppProject scoping (the `clusterResourceWhitelist:[{*,*}]` trap), project/RBAC/SSO, the controller's own privileges and self-management, repo/SSH creds, ApplicationSet injection, auto-sync vs approval, drift, recent Argo CD CVEs, promotion/rollback as git ops |
    | [rules/05-operators-crds-webhooks.md](rules/05-operators-crds-webhooks.md) | The operator privilege problem (broad RBAC → CRD-mediated escalation), vetting operator RBAC, CRD validation/security, trusted controllers (ESO, cert-manager), admission webhooks as attack surface (failurePolicy, timeout, TLS, namespaceSelector) |
    | [rules/06-workloads-tenancy.md](rules/06-workloads-tenancy.md) | Resource requests/limits as availability+QoS, PodDisruptionBudget, topology spread, anti-affinity, priorityClass/preemption, namespace-as-SOFT-boundary reality, hard multi-tenancy (vCluster/separate clusters/node isolation), Secrets in K8s (etcd encryption + ESO/sealed-secrets pointer) |
    | [rules/07-supply-chain-audit.md](rules/07-supply-chain-audit.md) | Helm/Kustomize review before apply (`helm template`, RBAC-in-charts trap, privilege-granting values), OCI chart signing/provenance, only-signed-images enforcement, digest pinning at the manifest layer, K8s audit policy (what/levels), shipping audit logs, detection + runtime-security pointers |
    
    When a task spans layers (most do), read every matching file. For a full cluster audit,
    read all seven plus the referenced sibling skills.
    
    ## Top 10 non-negotiables
    
    Violations are at minimum **High** in AUDIT mode and must never be introduced in BUILD mode:
    
    1. **`anonymous-auth=false` on the API server AND kubelet; kubelet `read-only-port=0`,
       kubelet authz mode `Webhook` (not `AlwaysAllow`).** Anonymous or unauthenticated
       access to either is cluster-game-over (`01`).
    2. **etcd encrypted at rest with a KMS v2 provider** (envelope DEK/KEK), not the default
       base64. Secrets are NOT encrypted out of the box (`01`, `06`).
    3. **No wildcard RBAC.** No `*` verbs/resources in Roles/ClusterRoles bound to workloads
       or users; no `cluster-admin` bound to a ServiceAccount, `system:authenticated`, or
       `system:unauthenticated` (`02`).
    4. **`bind`, `escalate`, `impersonate` are privilege-escalation verbs** — grant only to
       a named, audited admin path, never to a workload or CI SA (`02`).
    5. **`automountServiceAccountToken: false`** is the default; opt in only for pods that
       call the API. No long-lived Secret-based SA tokens — use bound/projected tokens (`02`).
    6. **Pod Security Admission `restricted` ENFORCED** on workload namespaces (label
       `pod-security.kubernetes.io/enforce: restricted`), not merely `warn`/`audit`. PSA is
       stable; PSP is gone (`03`). Pod-level field internals → sota-sandboxing.
    7. **Admission policies enforce, with a documented AUDIT→ENFORCE rollout.** A policy
       parked in `Audit`/`warn` indefinitely is a real, recurring finding (`03`).
    8. **GitOps is the only cluster write path; the controller and its AppProjects are
       tightly scoped.** Never `clusterResourceWhitelist: [{group:'*',kind:'*'}]` with broad
       project access; pin Argo CD/Flux versions and patch known CVEs (`04`).
    9. **Operators get least-privilege RBAC; admission webhooks have correct
       `failurePolicy`/`timeoutSeconds`/`namespaceSelector` and TLS** — a webhook is both a
       control and a single point of failure/attack (`05`).
    10. **API server audit logging enabled** (RequestResponse for sensitive verbs/Secrets),
        shipped to tamper-resistant storage; etcd backups taken AND restore-tested (`07`,`01`).
        Detection content on the stream → sota-detection-engineering.
    
    If the user asks for something that violates a non-negotiable, implement the secure
    alternative and explain the delta; comply only after they acknowledge the risk explicitly.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related