Claude Skill

sg-deceptive-reachability-auditor

Audit a fleet of AWS security groups for the multi-hop lateral-movement path that no single ingress rule reveals. Builds a directed reachability graph from the SG-to-SG references (an ingress rule on SG B naming SG A means a host in A can reach B), adds an internet edge for every

LLM Mart · 0 points · 4 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download anyshift-io-sre-skills-skills_sg-deceptive-reachability-auditor-a7af922.zip · 210 KB
anyshift-io/sre-skills 17 0 forks Apache-2.0 Updated 28d ago
Part of anyshift-io/sre-skills — 5 skills

Install

skills CLI npx skills add https://github.com/anyshift-io/sre-skills/tree/main/skills/sg-deceptive-reachability-auditor
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install anyshift-io-sre-skills@llmmart
Git git clone https://github.com/anyshift-io/sre-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole anyshift-io/sre-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

sg-deceptive-reachability-auditor

Reachability-audit skill for a fleet of AWS security groups. Takes the describe-security-groups output for the fleet (plus describe-instances for the instance-to-SG membership), composes the SG-to-SG references into a directed graph, and answers one question a per-rule read cannot: from a named entry point, what can actually be reached, and does any path reach the crown-jewel tier. It returns the shortest path, the blast radius, and any pivot hub, ranked by severity, then names exactly where the SG graph stops being able to answer the question.

The job is a graph problem, and the whole point of the skill is the composition the graph makes visible. A per-rule read sees "app accepts from web" and "db accepts from app" as two individually fine rules and never assembles that internet -> web -> app -> db is one reachable path. In a fleet of 10-13 security groups, that chain is buried among scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue, logs), and the loud 0.0.0.0/0 rule on the front door draws the eye away from it. This skill composes the edges instead of clearing each rule in isolation.

When to invoke

  • An agent is asked to review a security-group fleet for lateral movement, blast radius, or "can the internet reach the database."
  • A fleet is being shipped or changed and the question is whether a low-trust tier can reach a sensitive one transitively, not just directly.
  • An incident assumes a host is compromised and the question is what that foothold can pivot to.
  • A fleet looks segmented and the claim "the database is isolated" needs to be confirmed against the actual edges rather than taken on trust.

What this skill reads, and what it does not

It reads the static configuration of a fleet of security groups plus the instance-to-SG membership. Both are EC2 control-plane reads (describe-security-groups, describe-instances). That is the entire input. The audit is correct and complete for what the SG graph can tell you, and it is explicit about the rest. Reachability-on-paper is not exploitability, and every audit ends by naming the joins it cannot make:

  • It does not confirm a live host is listening. An edge means an SG accepts the referenced SG; it does not mean an instance in that SG is running and serving. An empty SG is a path to nothing. Join: SG graph to the live ENIs/instances in each SG.
  • It does not read route tables. Two SGs on unrouted subnets are not on a routable path no matter what the ingress rules allow. Join: SG graph to the subnet route tables.
  • It does not read network ACLs. A NACL is a stateless layer below security groups and can deny traffic the SG graph would allow. Join: SG graph to the subnet NACLs.
  • It does not read app-layer auth. A database password, an mTLS handshake, or an app token can stop a network-reachable hop from becoming access. Join: network reachability to the app-layer auth on each tier.

Every audit ends by naming these. A clean (segmented) fleet still gets a boundary section, because a network-segmented fleet is not a proven-safe system.

The model

Build a directed graph over security groups. An edge A -> B exists when SG B has an ingress rule whose UserIdGroupPairs includes SG A (B accepts traffic from A), meaning a host in A can reach a host in B. A synthetic node internet has an edge internet -> X for every SG X with a 0.0.0.0/0 (or ::/0) ingress rule.

From the entry point named for the audit (internet, or a compromised instance id that resolves to that instance's SGs), compute the transitive closure with BFS. A visited set makes cycles terminate. The findings fall out of the closure.

The methodology, in order

1. Parse the fleet into edges

Before any judgment, turn the JSON into the graph:

  • For each SG, read its IpPermissions (ingress). Every UserIdGroupPairs entry naming another SG in the fleet is an incoming edge: referenced-SG -> this-SG. This is the step a naive read skips. The SG-reference arrays are where the chain lives.
  • Every 0.0.0.0/0 / ::/0 IpRanges / Ipv6Ranges entry makes the SG internet-facing: internet -> this-SG.
  • Read describe-instances for the instance-to-SG membership, so the entry point (a compromised host) resolves to a set of start SGs, and so a tier with no running host can be flagged as a path to nothing at the boundary.
  • Label each SG by its tier / Name tag, then GroupName, then GroupId, so the path reads as internet -> web -> app -> db, not as a list of sg- ids.

2. Compute the closure and the path (P1)

Run BFS from the entry's start set. The reachable set is the closure minus the start. If the crown-jewel tier is in the closure, compute the shortest path (fewest hops) to it and report it as the headline:

  • P1 (critical) — a reachable path from the entry to the crown jewel. Report the shortest path as an explicit ordered hop list (internet -> cdn -> waf -> gw -> app -> svc -> db). No single ingress rule is alarming; each tier accepting the tier in front of it is routine. The edges compose into one path a per-rule read never assembles. This is the lateral-movement chain the audit exists to surface, and on a needle fleet it is the primary finding, named end to end, not a footnote under the loud public rule.

3. Report the blast radius (B1)

  • B1 (high) — the blast radius. When the closure composes at least one lateral hop (distance >= 2 from the entry, i.e. beyond the directly-exposed front-door tier), report the full reachable set: the tiers a foothold at the entry can pivot to with no further misconfiguration. State explicitly whether the crown jewel is inside the radius. A fleet whose chain breaks after the first hop has no lateral reach and does not fire B1 — that distinction is load-bearing for the clean fleets.

4. Find the pivot hub (H1)

  • H1 (high) — a pivot/hub SG bridging otherwise-isolated regions. For each intermediate reachable SG, recompute the closure with that node removed. If its removal disconnects two or more mutually-isolated regions of the blast radius, it is a true pivot (an articulation point), not just an ordinary hop on a linear chain. A shared-services SG (monitoring, CI, a jump tier) that every tier references is exactly this shape: it quietly joins tiers that were never meant to reach each other. Do not report the entry node or the directly-internet-facing front door as a hub; those are a different auditor's finding.

5. Stay quiet on the deceptive-clean fleet

This is the half of the skill that the naive read gets wrong in the other direction. A segmented, orphaned, or broken fleet where no path reaches the crown jewel is CLEAN, and the audit must say so instead of manufacturing a path. The same composition discipline is what proves it. Specifically:

  • An orphaned deep chain (the deep tiers reference each other, but the front tier accepts only an internal service-mesh CIDR, not the public SG) is not a reachable path. Do not report it as one.
  • An intended public ALB taking 0.0.0.0/0 is the expected ingress, not the lateral path and not the headline.
  • A disjoint data island (a public region and an unconnected private region) must not be spliced into a manufactured internet -> db route.
  • A broken mid-chain segment (the chain is cut at one hop) is not reachable across the cut.
  • Do not drown the real finding, or the clean verdict, in a wall of low-value nitpicks about correctly-scoped tiers (bastion, monitoring, ci, ssm).

On a clean fleet the audit reports: no reachable path to the crown jewel, the bounded blast radius (and the boundary it cannot cross), and the join checks that would confirm the segmentation holds. It does not invent a critical.

6. Rank and report, then name the boundary

Order findings by severity (critical, high). For each: the path/SG it is grounded in, what it means, and the fix. Then list the boundary from step "What this skill reads." A clean fleet still gets a boundary section.

Severity model

Severity Meaning
critical A composed path from the entry reaches the crown-jewel tier. P1.
high A foothold at the entry can pivot laterally, or a single SG bridges isolated regions. B1, H1.

There is no low band here: a reachability finding is grounded in the composed graph, not in a heuristic. The uncertainty lives entirely in the boundary (is a host live, is the subnet routed, does a NACL deny, is there app auth), which is why the boundary section is mandatory rather than a footnote.

Rule reference

Code Rule Severity Grounded in
P1 Reachable path from the entry to the crown-jewel tier critical shortest path in the SG closure
B1 Blast radius spans a lateral hop (distance >= 2) beyond the front door high transitive closure from the entry
H1 Pivot/hub SG bridges two or more otherwise-isolated reachable regions high articulation point in the reachable subgraph

The matching half of every rule is the clean verdict: P1 absent (no path), B1 absent (no lateral hop), H1 absent (no bridge) on a segmented fleet is the correct, complete output, not a failure to find something.

Output format

The agent's final message in any invocation must include:

  1. Fleet: SG count, the entry point, the crown-jewel tier.
  2. Findings: ranked by severity, each with the code, the path/SG it is grounded in, what it means, and the fix. The P1 path named hop by hop. Or "no reachable path to the crown jewel" for a segmented fleet, with the bounded blast radius stated.
  3. Boundary: the joins this audit could not make (live membership, route tables, NACLs, app-layer auth), stated explicitly so the gap is visible instead of silent.

Worked examples

Seven end-to-end fixtures are committed under fixtures/, each a fleet of 10-13 security groups with a runnable replay test. They are split between buried needles and deceptive-clean fleets, with no short obvious 2-3 hop path in either set:

Replay tests

Every fixture has a replay test in tests/ that runs the methodology (via the deterministic reference engine tests/_reach_engine.py, wrapped by tests/_deep.py) against the committed JSON, with no external credentials. Run from the skill directory:

for t in tests/replay_*.py; do python "$t" || exit 1; done

The seven tests cover the three needle paths (P1/B1/H1 present, hops correct) and the four deceptive-clean fleets (no path fabricated). Tests exit non-zero if the audit composes the wrong path or invents one on a clean fleet. See tests/README.md for the fixture schema.

Failure modes

This skill is wrong in predictable ways. Read FAILURE_MODES.md before relying on it. Highlights:

  • It audits reachability, not exploitability. A path that passes every edge can reach a tier with no live host, an unrouted subnet, a denying NACL, or app-layer auth that stops the hop. Reachability-on-paper is a hypothesis to confirm, not a breach.
  • The crown-jewel tier and the entry point are supplied by the caller. A wrong entry or a mislabelled crown jewel changes the path.
  • It reasons over the SG references and the internet edge only. A reachable route via a peering connection, a transit gateway, or a VPC endpoint that does not appear as an SG reference is outside the graph this skill builds.

Anyshift integration (opt-in)

The audit above runs end-to-end against the describe-security-groups + describe-instances output the user already has. No Anyshift dependency.

Every boundary note in this skill is a join: SG graph to live instance membership, SG graph to the route tables, SG graph to the subnet NACLs, network reachability to the app-layer auth on each tier. The Anyshift MCP can act as a context primer by resolving those joins from a versioned resource graph, so a P1 path can be confirmed (the host is live, the subnet is routed, no NACL denies) instead of left as a hypothesis at the boundary. A measured "with vs without" delta will be published here once the integration has been exercised against the replay fixtures.

Files (sre-skills)
  • fixtures
    • 01-orphaned-front-internal-cidr
      • instances.json 1.2 KB 
        {
  "Instances": [
    {"InstanceId": "i-01lb00a01", "PublicIpAddress": "203.0.113.11", "SecurityGroups": [{"GroupId": "sg-01lb00000000000001"}], "Tags": [{"Key": "Name", "Value": "alb-a"}, {"Key": "tier", "Value": "lb"}]},
    {"InstanceId": "i-01web0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-01web00000000000002"}], "Tags": [{"Key": "Name", "Value": "web-1"}, {"Key": "tier", "Value": "web"}]},
    {"InstanceId": "i-01app0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-01app00000000000003"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-01cache01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-01cache0000000000004"}], "Tags": [{"Key": "Name", "Value": "cache-1"}, {"Key": "tier", "Value": "cache"}]},
    {"InstanceId": "i-01db00001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-01db00000000000005"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-01bast001", "PublicIpAddress": "203.0.113.19", "SecurityGroups": [{"GroupId": "sg-01bastion000000006"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 378 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 11 security groups. A deep service chain (web to app to session cache to db) is wired with SG references, but the web frontend accepts only the internal service-mesh CIDR, not the public ALB security group, so the chain is orphaned from the internet entry. The crown jewel is the database tier."
}
      • security-groups.json 6.8 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-01lb00000000000001",
      "GroupName": "public-alb",
      "VpcId": "vpc-0orph0001",
      "Description": "Public application load balancer, HTTPS from internet",
      "Tags": [{"Key": "tier", "Value": "lb"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01web00000000000002",
      "GroupName": "web-frontend",
      "VpcId": "vpc-0orph0001",
      "Description": "Web frontend, accepts the internal service mesh CIDR on 8443 (ALB target registration is by IP, not SG)",
      "Tags": [{"Key": "tier", "Value": "web"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8443, "ToPort": 8443, "IpRanges": [{"CidrIp": "10.50.0.0/16", "Description": "internal service mesh"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01app00000000000003",
      "GroupName": "app-tier",
      "VpcId": "vpc-0orph0001",
      "Description": "Application tier, accepts the web frontend on 8080",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-01web00000000000002", "Description": "from web"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01cache0000000000004",
      "GroupName": "session-cache",
      "VpcId": "vpc-0orph0001",
      "Description": "Redis session cache, accepts the app tier on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-01app00000000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01db00000000000005",
      "GroupName": "db-primary",
      "VpcId": "vpc-0orph0001",
      "Description": "Postgres primary, accepts the session cache host on 5432 (cache warmer writes through)",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-01cache0000000000004", "Description": "from session cache warmer"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01bastion000000006",
      "GroupName": "bastion",
      "VpcId": "vpc-0orph0001",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01mon0000000000007",
      "GroupName": "monitoring",
      "VpcId": "vpc-0orph0001",
      "Description": "Prometheus, scoped to the monitoring subnet",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01ci00000000000008",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0orph0001",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01queue00000000009",
      "GroupName": "rabbitmq",
      "VpcId": "vpc-0orph0001",
      "Description": "RabbitMQ, accepts the app tier on 5672",
      "Tags": [{"Key": "tier", "Value": "queue"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5672, "ToPort": 5672, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-01app00000000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01ssm0000000000010",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0orph0001",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-01logs000000000011",
      "GroupName": "log-aggregator",
      "VpcId": "vpc-0orph0001",
      "Description": "Log aggregator, accepts app on 5044",
      "Tags": [{"Key": "tier", "Value": "logs"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5044, "ToPort": 5044, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-01app00000000000003"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 02-public-alb-no-sg-ref
      • instances.json 1.2 KB 
        {
  "Instances": [
    {"InstanceId": "i-02lb00a01", "PublicIpAddress": "198.51.100.21", "SecurityGroups": [{"GroupId": "sg-02lb00000000000001"}], "Tags": [{"Key": "Name", "Value": "alb-a"}, {"Key": "tier", "Value": "lb"}]},
    {"InstanceId": "i-02web0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-02web00000000000002"}], "Tags": [{"Key": "Name", "Value": "web-1"}, {"Key": "tier", "Value": "web"}]},
    {"InstanceId": "i-02app0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-02app00000000000003"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-02svc0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-02svc00000000000004"}], "Tags": [{"Key": "Name", "Value": "svc-1"}, {"Key": "tier", "Value": "svc"}]},
    {"InstanceId": "i-02db00001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-02db00000000000005"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-02bast001", "PublicIpAddress": "198.51.100.29", "SecurityGroups": [{"GroupId": "sg-02bastion000000006"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 477 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 11 security groups. A public ALB carries the only 0.0.0.0/0 and ::/0 ingress on 443/80. A deep service chain (web to app to internal services to db) is wired downstream with SG references, but nothing references the public ALB security group: it forwards to its target group by registered IP, so no SG-to-SG edge composes from the internet into the fleet. The crown jewel is the database tier."
}
      • security-groups.json 7.1 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-02lb00000000000001",
      "GroupName": "public-alb",
      "VpcId": "vpc-0noref001",
      "Description": "Public application load balancer, 443 and 80 from internet, forwards to target groups by registered IP",
      "Tags": [{"Key": "tier", "Value": "lb"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []},
        {"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTP redirect"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02web00000000000002",
      "GroupName": "web-frontend",
      "VpcId": "vpc-0noref001",
      "Description": "Web frontend, accepts the internal ALB target subnet on 8080",
      "Tags": [{"Key": "tier", "Value": "web"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [{"CidrIp": "10.40.10.0/24", "Description": "alb target subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02app00000000000003",
      "GroupName": "app-tier",
      "VpcId": "vpc-0noref001",
      "Description": "Application tier, accepts the web frontend on 9000",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9000, "ToPort": 9000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-02web00000000000002", "Description": "from web"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02svc00000000000004",
      "GroupName": "internal-services",
      "VpcId": "vpc-0noref001",
      "Description": "Internal microservices, accepts the app tier on 7000",
      "Tags": [{"Key": "tier", "Value": "svc"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 7000, "ToPort": 7000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-02app00000000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02db00000000000005",
      "GroupName": "db-primary",
      "VpcId": "vpc-0noref001",
      "Description": "MySQL primary, accepts the internal services tier on 3306",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 3306, "ToPort": 3306, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-02svc00000000000004", "Description": "from internal services"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02bastion000000006",
      "GroupName": "bastion",
      "VpcId": "vpc-0noref001",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02mon0000000000007",
      "GroupName": "monitoring",
      "VpcId": "vpc-0noref001",
      "Description": "Grafana + Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 3000, "ToPort": 3000, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02ci00000000000008",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0noref001",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02cache00000000009",
      "GroupName": "redis-cache",
      "VpcId": "vpc-0noref001",
      "Description": "Redis cache, accepts the internal services tier on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-02svc00000000000004", "Description": "from internal services"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02ssm0000000000010",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0noref001",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-02logs000000000011",
      "GroupName": "log-aggregator",
      "VpcId": "vpc-0noref001",
      "Description": "Log aggregator, accepts app + svc on 5044",
      "Tags": [{"Key": "tier", "Value": "logs"}, {"Key": "az", "Value": "eu-west-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5044, "ToPort": 5044, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-02app00000000000003"}, {"GroupId": "sg-02svc00000000000004"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 03-disjoint-public-vpn-islands
      • instances.json 1.4 KB 
        {
  "Instances": [
    {"InstanceId": "i-03lb00a01", "PublicIpAddress": "192.0.2.31", "SecurityGroups": [{"GroupId": "sg-03lb00000000000001"}], "Tags": [{"Key": "Name", "Value": "alb-a"}, {"Key": "tier", "Value": "lb"}]},
    {"InstanceId": "i-03web0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-03web00000000000002"}], "Tags": [{"Key": "Name", "Value": "web-1"}, {"Key": "tier", "Value": "web"}]},
    {"InstanceId": "i-03wapp001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-03webapp0000000003"}], "Tags": [{"Key": "Name", "Value": "web-app-1"}, {"Key": "tier", "Value": "web-app"}]},
    {"InstanceId": "i-03admin01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-03admin00000000004"}], "Tags": [{"Key": "Name", "Value": "admin-1"}, {"Key": "tier", "Value": "admin"}]},
    {"InstanceId": "i-03dproc01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-03dataproc00000005"}], "Tags": [{"Key": "Name", "Value": "dataproc-1"}, {"Key": "tier", "Value": "dataproc"}]},
    {"InstanceId": "i-03db00001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-03db00000000000006"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-03bast001", "PublicIpAddress": "192.0.2.39", "SecurityGroups": [{"GroupId": "sg-03bastion000000008"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 569 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 11 security groups in two AZs forming two disconnected islands. The internet-facing island (public ALB to web to web-app) is wired by CIDR, not SG references, and dead-ends. The data island carries a deep SG-reference chain (admin plane to data processor to db, plus an analytics cache), but the admin plane accepts only the corp VPN CIDR and has no inbound SG reference, so the data island is reachable only from the VPN, never from the internet. The crown jewel is the database tier."
}
      • security-groups.json 6.8 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-03lb00000000000001",
      "GroupName": "public-alb",
      "VpcId": "vpc-0islands1",
      "Description": "Public application load balancer, HTTPS from internet",
      "Tags": [{"Key": "tier", "Value": "lb"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03web00000000000002",
      "GroupName": "web-frontend",
      "VpcId": "vpc-0islands1",
      "Description": "Public web frontend, accepts the ALB target subnet by CIDR on 8443",
      "Tags": [{"Key": "tier", "Value": "web"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8443, "ToPort": 8443, "IpRanges": [{"CidrIp": "10.60.10.0/24", "Description": "alb target subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03webapp0000000003",
      "GroupName": "web-app",
      "VpcId": "vpc-0islands1",
      "Description": "Public-facing app tier, accepts the web subnet by CIDR on 8080",
      "Tags": [{"Key": "tier", "Value": "web-app"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [{"CidrIp": "10.60.20.0/24", "Description": "web subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03admin00000000004",
      "GroupName": "admin-plane",
      "VpcId": "vpc-0islands1",
      "Description": "Internal admin plane, corp VPN CIDR only on 443",
      "Tags": [{"Key": "tier", "Value": "admin"}, {"Key": "az", "Value": "us-west-2b"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03dataproc00000005",
      "GroupName": "data-processor",
      "VpcId": "vpc-0islands1",
      "Description": "Data processing tier, accepts the admin plane on 9443",
      "Tags": [{"Key": "tier", "Value": "dataproc"}, {"Key": "az", "Value": "us-west-2b"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9443, "ToPort": 9443, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-03admin00000000004", "Description": "from admin plane"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03db00000000000006",
      "GroupName": "db-primary",
      "VpcId": "vpc-0islands1",
      "Description": "Postgres primary, accepts the data processor on 5432",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "us-west-2b"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-03dataproc00000005", "Description": "from data processor"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03cache00000000007",
      "GroupName": "analytics-cache",
      "VpcId": "vpc-0islands1",
      "Description": "Analytics cache, accepts the data processor on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "us-west-2b"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-03dataproc00000005", "Description": "from data processor"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03bastion000000008",
      "GroupName": "bastion",
      "VpcId": "vpc-0islands1",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03mon0000000000009",
      "GroupName": "monitoring",
      "VpcId": "vpc-0islands1",
      "Description": "Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03ci00000000000010",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0islands1",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "us-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-03ssm0000000000011",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0islands1",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "us-west-2b"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 04-broken-segment-midchain
      • instances.json 1.2 KB 
        {
  "Instances": [
    {"InstanceId": "i-04edge001", "PublicIpAddress": "203.0.113.41", "SecurityGroups": [{"GroupId": "sg-04edge00000000000002"}], "Tags": [{"Key": "Name", "Value": "edge-1"}, {"Key": "tier", "Value": "edge"}]},
    {"InstanceId": "i-04web0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-04web00000000000003"}], "Tags": [{"Key": "Name", "Value": "web-1"}, {"Key": "tier", "Value": "web"}]},
    {"InstanceId": "i-04app0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-04app00000000000004"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-04cache01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-04cache0000000000005"}], "Tags": [{"Key": "Name", "Value": "cache-1"}, {"Key": "tier", "Value": "cache"}]},
    {"InstanceId": "i-04db00001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-04db00000000000006"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-04bast001", "PublicIpAddress": "203.0.113.49", "SecurityGroups": [{"GroupId": "sg-04bastion000000007"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 541 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 10 security groups. A long service chain is visible (public edge proxy to web to app to session cache to db). The public edge proxy carries the only 0.0.0.0/0 ingress, but the chain is cut at the first internal hop: the web frontend accepts only the internal mesh CIDR, not the edge proxy SG, so the edge-to-web link is not an SG edge and the rest of the chain (web to app to cache to db) is orphaned from the internet. The crown jewel is the database tier."
}
      • security-groups.json 6.3 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-04edge00000000000002",
      "GroupName": "edge-proxy",
      "VpcId": "vpc-0broken01",
      "Description": "Public edge reverse proxy, HTTPS from internet",
      "Tags": [{"Key": "tier", "Value": "edge"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04web00000000000003",
      "GroupName": "web-frontend",
      "VpcId": "vpc-0broken01",
      "Description": "Web frontend, accepts the internal mesh CIDR on 8080 (NOT the edge proxy SG: traffic from the public edge proxy is NAT'd onto the mesh subnet so registration is by CIDR)",
      "Tags": [{"Key": "tier", "Value": "web"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [{"CidrIp": "10.70.0.0/16", "Description": "internal mesh"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04app00000000000004",
      "GroupName": "app-tier",
      "VpcId": "vpc-0broken01",
      "Description": "Application tier, accepts the web frontend on 9000",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9000, "ToPort": 9000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-04web00000000000003", "Description": "from web"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04cache0000000000005",
      "GroupName": "session-cache",
      "VpcId": "vpc-0broken01",
      "Description": "Redis session cache, accepts the app tier on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-04app00000000000004", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04db00000000000006",
      "GroupName": "db-primary",
      "VpcId": "vpc-0broken01",
      "Description": "Postgres primary, accepts the session cache on 5432",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-04cache0000000000005", "Description": "from session cache"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04bastion000000007",
      "GroupName": "bastion",
      "VpcId": "vpc-0broken01",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04mon0000000000008",
      "GroupName": "monitoring",
      "VpcId": "vpc-0broken01",
      "Description": "Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04ci00000000000009",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0broken01",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04queue00000000010",
      "GroupName": "rabbitmq",
      "VpcId": "vpc-0broken01",
      "Description": "RabbitMQ, accepts the app tier on 5672",
      "Tags": [{"Key": "tier", "Value": "queue"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5672, "ToPort": 5672, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-04app00000000000004", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-04logs000000000011",
      "GroupName": "log-aggregator",
      "VpcId": "vpc-0broken01",
      "Description": "Log aggregator, accepts app + web on 5044",
      "Tags": [{"Key": "tier", "Value": "logs"}, {"Key": "az", "Value": "eu-central-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5044, "ToPort": 5044, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-04app00000000000004"}, {"GroupId": "sg-04web00000000000003"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 05-six-hop-cdn-waf-gw-app-svc-db
      • instances.json 1.4 KB 
        {
  "Instances": [
    {"InstanceId": "i-05cdn001", "PublicIpAddress": "203.0.113.51", "SecurityGroups": [{"GroupId": "sg-05cdn00000000000001"}], "Tags": [{"Key": "Name", "Value": "cdn-1"}, {"Key": "tier", "Value": "cdn"}]},
    {"InstanceId": "i-05waf001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-05waf00000000000002"}], "Tags": [{"Key": "Name", "Value": "waf-1"}, {"Key": "tier", "Value": "waf"}]},
    {"InstanceId": "i-05gw0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-05gw00000000000003"}], "Tags": [{"Key": "Name", "Value": "gw-1"}, {"Key": "tier", "Value": "gw"}]},
    {"InstanceId": "i-05app001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-05app00000000000004"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-05svc001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-05svc00000000000005"}], "Tags": [{"Key": "Name", "Value": "billing-1"}, {"Key": "tier", "Value": "svc"}]},
    {"InstanceId": "i-05db0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-05db00000000000006"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-05bast01", "PublicIpAddress": "203.0.113.59", "SecurityGroups": [{"GroupId": "sg-05bastion000000007"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 394 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 13 security groups. A six-hop service chain (CDN origin to WAF to API gateway to app to billing service to db) is wired entirely with ordinary single-upstream SG references, surrounded by scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue, logs). The crown jewel is the database tier."
}
      • security-groups.json 8 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-05cdn00000000000001",
      "GroupName": "cdn-origin",
      "VpcId": "vpc-0sixhop01",
      "Description": "CDN origin edge, HTTPS from internet",
      "Tags": [{"Key": "tier", "Value": "cdn"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05waf00000000000002",
      "GroupName": "waf-tier",
      "VpcId": "vpc-0sixhop01",
      "Description": "WAF inspection tier, accepts the CDN origin on 8443",
      "Tags": [{"Key": "tier", "Value": "waf"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8443, "ToPort": 8443, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05cdn00000000000001", "Description": "from cdn origin"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05gw00000000000003",
      "GroupName": "api-gateway",
      "VpcId": "vpc-0sixhop01",
      "Description": "API gateway, accepts the WAF tier on 8080",
      "Tags": [{"Key": "tier", "Value": "gw"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05waf00000000000002", "Description": "from waf"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05app00000000000004",
      "GroupName": "app-tier",
      "VpcId": "vpc-0sixhop01",
      "Description": "Application tier, accepts the API gateway on 9000",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9000, "ToPort": 9000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05gw00000000000003", "Description": "from api gateway"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05svc00000000000005",
      "GroupName": "billing-service",
      "VpcId": "vpc-0sixhop01",
      "Description": "Billing microservice, accepts the app tier on 7000",
      "Tags": [{"Key": "tier", "Value": "svc"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 7000, "ToPort": 7000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05app00000000000004", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05db00000000000006",
      "GroupName": "db-primary",
      "VpcId": "vpc-0sixhop01",
      "Description": "Postgres primary, accepts the billing service on 5432 (settlement writer)",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05svc00000000000005", "Description": "from billing service settlement writer"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05bastion000000007",
      "GroupName": "bastion",
      "VpcId": "vpc-0sixhop01",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05mon0000000000008",
      "GroupName": "monitoring",
      "VpcId": "vpc-0sixhop01",
      "Description": "Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05ci00000000000009",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0sixhop01",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05cache00000000010",
      "GroupName": "redis-cache",
      "VpcId": "vpc-0sixhop01",
      "Description": "Redis cache, accepts the app tier on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05app00000000000004", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05queue00000000011",
      "GroupName": "kafka",
      "VpcId": "vpc-0sixhop01",
      "Description": "Kafka brokers, accepts the app tier on 9092",
      "Tags": [{"Key": "tier", "Value": "queue"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9092, "ToPort": 9092, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05app00000000000004", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05ssm0000000000012",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0sixhop01",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-05logs000000000013",
      "GroupName": "log-aggregator",
      "VpcId": "vpc-0sixhop01",
      "Description": "Log aggregator, accepts app + svc on 5044",
      "Tags": [{"Key": "tier", "Value": "logs"}, {"Key": "az", "Value": "us-east-1a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5044, "ToPort": 5044, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-05app00000000000004"}, {"GroupId": "sg-05svc00000000000005"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 06-compromised-ci-runner-deep
      • instances.json 1.6 KB 
        {
  "Instances": [
    {"InstanceId": "i-06ci0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06ci00000000000001"}], "Tags": [{"Key": "Name", "Value": "ci-runner-1"}, {"Key": "tier", "Value": "ci"}]},
    {"InstanceId": "i-06build01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06build0000000002"}], "Tags": [{"Key": "Name", "Value": "build-1"}, {"Key": "tier", "Value": "build"}]},
    {"InstanceId": "i-06artif01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06artifact000003"}], "Tags": [{"Key": "Name", "Value": "artifact-1"}, {"Key": "tier", "Value": "artifact"}]},
    {"InstanceId": "i-06deploy1", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06deploy00000004"}], "Tags": [{"Key": "Name", "Value": "deploy-1"}, {"Key": "tier", "Value": "deploy"}]},
    {"InstanceId": "i-06app001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06app00000000005"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-06db0001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06db00000000000006"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-06lb0001", "PublicIpAddress": "203.0.113.61", "SecurityGroups": [{"GroupId": "sg-06lb00000000000007"}], "Tags": [{"Key": "Name", "Value": "alb-1"}, {"Key": "tier", "Value": "lb"}]},
    {"InstanceId": "i-06web001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-06web00000000000008"}], "Tags": [{"Key": "Name", "Value": "web-1"}, {"Key": "tier", "Value": "web"}]}
  ]
}
      • meta.json 466 B 
        {
  "entry": "i-06ci0001",
  "crown_jewel": "db",
  "context": "Production fleet, 12 security groups. The entry point is a COMPROMISED CI runner host (instance i-06ci0001), not the internet. From that foothold a deep build-pipeline chain composes (CI runner to build coordinator to artifact store to deploy agent to app to db). A separate public ALB to web to web-cache front door exists but never references the data plane. The crown jewel is the database tier."
}
      • security-groups.json 7.5 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-06ci00000000000001",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0cirun01",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06build0000000002",
      "GroupName": "build-coordinator",
      "VpcId": "vpc-0cirun01",
      "Description": "Build coordinator, accepts the CI runners on 8500",
      "Tags": [{"Key": "tier", "Value": "build"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8500, "ToPort": 8500, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06ci00000000000001", "Description": "from ci runners"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06artifact000003",
      "GroupName": "artifact-store",
      "VpcId": "vpc-0cirun01",
      "Description": "Artifact store, accepts the build coordinator on 8081",
      "Tags": [{"Key": "tier", "Value": "artifact"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8081, "ToPort": 8081, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06build0000000002", "Description": "from build coordinator"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06deploy00000004",
      "GroupName": "deploy-agent",
      "VpcId": "vpc-0cirun01",
      "Description": "Deploy agent, accepts the artifact store on 8090 (pulls signed artifacts then pushes to app)",
      "Tags": [{"Key": "tier", "Value": "deploy"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8090, "ToPort": 8090, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06artifact000003", "Description": "from artifact store"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06app00000000005",
      "GroupName": "app-tier",
      "VpcId": "vpc-0cirun01",
      "Description": "Application tier, accepts the deploy agent on 9000 (rolling deploy push)",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9000, "ToPort": 9000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06deploy00000004", "Description": "from deploy agent"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06db00000000000006",
      "GroupName": "db-primary",
      "VpcId": "vpc-0cirun01",
      "Description": "Postgres primary, accepts the app tier on 5432",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06app00000000005", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06lb00000000000007",
      "GroupName": "public-alb",
      "VpcId": "vpc-0cirun01",
      "Description": "Public application load balancer, HTTPS from internet (separate public front door)",
      "Tags": [{"Key": "tier", "Value": "lb"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06web00000000000008",
      "GroupName": "web-frontend",
      "VpcId": "vpc-0cirun01",
      "Description": "Public web frontend, accepts the public ALB on 8443",
      "Tags": [{"Key": "tier", "Value": "web"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8443, "ToPort": 8443, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06lb00000000000007", "Description": "from public alb"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06cache00000000009",
      "GroupName": "web-cache",
      "VpcId": "vpc-0cirun01",
      "Description": "Web edge cache, accepts the web frontend on 6379 (public front door only, no data-plane reference)",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-06web00000000000008", "Description": "from web"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06bastion0000010",
      "GroupName": "bastion",
      "VpcId": "vpc-0cirun01",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06mon00000000011",
      "GroupName": "monitoring",
      "VpcId": "vpc-0cirun01",
      "Description": "Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-06ssm00000000012",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0cirun01",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "us-east-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
    • 07-five-hop-ingress-mesh-broker-db
      • instances.json 1.2 KB 
        {
  "Instances": [
    {"InstanceId": "i-07ingr01", "PublicIpAddress": "198.51.100.71", "SecurityGroups": [{"GroupId": "sg-07ingress00000001"}], "Tags": [{"Key": "Name", "Value": "ingress-1"}, {"Key": "tier", "Value": "ingress"}]},
    {"InstanceId": "i-07mesh01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-07mesh00000000002"}], "Tags": [{"Key": "Name", "Value": "mesh-1"}, {"Key": "tier", "Value": "mesh"}]},
    {"InstanceId": "i-07app01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-07app00000000003"}], "Tags": [{"Key": "Name", "Value": "app-1"}, {"Key": "tier", "Value": "app"}]},
    {"InstanceId": "i-07brkr01", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-07broker0000000004"}], "Tags": [{"Key": "Name", "Value": "broker-1"}, {"Key": "tier", "Value": "broker"}]},
    {"InstanceId": "i-07db001", "PublicIpAddress": null, "SecurityGroups": [{"GroupId": "sg-07db00000000000005"}], "Tags": [{"Key": "Name", "Value": "db-primary"}, {"Key": "tier", "Value": "db"}]},
    {"InstanceId": "i-07bast01", "PublicIpAddress": "198.51.100.79", "SecurityGroups": [{"GroupId": "sg-07bastion0000006"}], "Tags": [{"Key": "Name", "Value": "bastion-1"}, {"Key": "tier", "Value": "bastion"}]}
  ]
}
      • meta.json 394 B 
        {
  "entry": "internet",
  "crown_jewel": "db",
  "context": "Production fleet, 12 security groups. A five-hop service chain (ingress controller to mesh sidecar to app to event broker to db) is wired entirely with ordinary single-upstream SG references, surrounded by scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue, logs). The crown jewel is the database tier."
}
      • security-groups.json 7.4 KB 
        {
  "SecurityGroups": [
    {
      "GroupId": "sg-07ingress00000001",
      "GroupName": "ingress-controller",
      "VpcId": "vpc-0mesh0001",
      "Description": "Kubernetes ingress controller, HTTPS from internet",
      "Tags": [{"Key": "tier", "Value": "ingress"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "public HTTPS"}], "Ipv6Ranges": [{"CidrIpv6": "::/0"}], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07mesh00000000002",
      "GroupName": "mesh-sidecar",
      "VpcId": "vpc-0mesh0001",
      "Description": "Service mesh sidecar gateway, accepts the ingress controller on 15443",
      "Tags": [{"Key": "tier", "Value": "mesh"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 15443, "ToPort": 15443, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07ingress00000001", "Description": "from ingress controller"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07app00000000003",
      "GroupName": "app-tier",
      "VpcId": "vpc-0mesh0001",
      "Description": "Application tier, accepts the mesh sidecar on 9000",
      "Tags": [{"Key": "tier", "Value": "app"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9000, "ToPort": 9000, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07mesh00000000002", "Description": "from mesh sidecar"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07broker0000000004",
      "GroupName": "event-broker",
      "VpcId": "vpc-0mesh0001",
      "Description": "Event broker, accepts the app tier on 5671",
      "Tags": [{"Key": "tier", "Value": "broker"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5671, "ToPort": 5671, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07app00000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07db00000000000005",
      "GroupName": "db-primary",
      "VpcId": "vpc-0mesh0001",
      "Description": "Postgres primary, accepts the event broker on 5432 (broker persists events through to the journal table)",
      "Tags": [{"Key": "tier", "Value": "db"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5432, "ToPort": 5432, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07broker0000000004", "Description": "from event broker journal writer"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07bastion0000006",
      "GroupName": "bastion",
      "VpcId": "vpc-0mesh0001",
      "Description": "SSH bastion, corp VPN only",
      "Tags": [{"Key": "tier", "Value": "bastion"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "10.20.0.0/16", "Description": "corp VPN"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07mon00000000007",
      "GroupName": "monitoring",
      "VpcId": "vpc-0mesh0001",
      "Description": "Prometheus, monitoring subnet only",
      "Tags": [{"Key": "tier", "Value": "monitoring"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 9090, "ToPort": 9090, "IpRanges": [{"CidrIp": "10.80.1.0/24", "Description": "monitoring subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07ci00000000008",
      "GroupName": "ci-runner",
      "VpcId": "vpc-0mesh0001",
      "Description": "CI runners, runner subnet only",
      "Tags": [{"Key": "tier", "Value": "ci"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.80.7.0/24", "Description": "runner subnet"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07cache00000009",
      "GroupName": "redis-cache",
      "VpcId": "vpc-0mesh0001",
      "Description": "Redis cache, accepts the app tier on 6379",
      "Tags": [{"Key": "tier", "Value": "cache"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 6379, "ToPort": 6379, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07app00000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07queue0000010",
      "GroupName": "sqs-poller",
      "VpcId": "vpc-0mesh0001",
      "Description": "SQS poller fleet, accepts the app tier on 8444",
      "Tags": [{"Key": "tier", "Value": "queue"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 8444, "ToPort": 8444, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07app00000000003", "Description": "from app"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07ssm0000011",
      "GroupName": "ssm-endpoints",
      "VpcId": "vpc-0mesh0001",
      "Description": "SSM interface endpoints, internal CIDR only",
      "Tags": [{"Key": "tier", "Value": "ssm"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "10.0.0.0/8"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    },
    {
      "GroupId": "sg-07logs000012",
      "GroupName": "log-aggregator",
      "VpcId": "vpc-0mesh0001",
      "Description": "Log aggregator, accepts app + mesh on 5044",
      "Tags": [{"Key": "tier", "Value": "logs"}, {"Key": "az", "Value": "eu-west-2a"}],
      "IpPermissions": [
        {"IpProtocol": "tcp", "FromPort": 5044, "ToPort": 5044, "IpRanges": [], "Ipv6Ranges": [], "UserIdGroupPairs": [{"GroupId": "sg-07app00000000003"}, {"GroupId": "sg-07mesh00000000002"}]}
      ],
      "IpPermissionsEgress": [{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}], "Ipv6Ranges": [], "UserIdGroupPairs": []}]
    }
  ]
}
  • tests
    • eval
      • eval_results.json 561.5 KB 
        [
  {
    "fixture": "01-orphaned-front-internal-cidr",
    "condition": "control",
    "trial": 0,
    "score": 2,
    "scores_by_item": [
      1,
      0,
      0,
      0,
      0,
      1,
      0
    ],
    "notes": [
      "Agent correctly parses SG-to-SG edges (UserIdGroupPairs), reads 0.0.0.0/0 as internet entry, and reads instance-to-SG membership throughout.",
      "This is a CLEAN fixture with no reachable path to the crown jewel; the agent failed by fabricating a critical 'cache-to-db lateral path' instead of correctly reporting no real path exists.",
      "Agent fatally over-flags the orphaned deep chain (cache->db) as a critical reachable lateral path, which is exactly the bait the ground truth says to reject; the internet reaches only the LB and the deep chain is disconnected.",
      "Agent assembles a multi-hop chain (internet->ALB->web->app->cache->db) but this chain is orphaned from the internet due to the CIDR-only web-frontend rule, making the composition incorrect on the facts.",
      "Agent headlines a fabricated cache-to-db lateral path as CRITICAL when ground truth is CLEAN; this is the opposite of correct criticality ranking.",
      "Agent correctly names route tables, NACLs, instance membership, and live traffic as things outside SG-graph visibility.",
      "Agent recommends removing the cache->db rule as the top fix, but ground truth says no fix is needed; the correct recommendation is no change plus confirming the boundary."
    ],
    "agent_output": "# Security Review: VPC Network Configuration\n\n## Methodology Note\n\nI'm working from security-group rules and instance-to-SG mappings only. I'll flag what I can prove from this data, what I can infer with reasonable confidence, and what requires additional evidence.\n\n---\n\n## Findings, Ranked by Severity\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 1: Cache tier has unnecessary write-through path to the database\n\n**The specific misconfiguration:**\n\n`sg-01db00000000000005` (db-primary) grants inbound TCP/5432 to `sg-01cache0000000000004` (session-cache).\n\nThe description says *\"cache warmer writes through\"* \u2014 meaning the Redis cache tier has a direct, permitted network path to the Postgres primary on the database port.\n\n**Why this is critical:**\n\nThe intended trust boundary is:\n\n```\ninternet \u2192 ALB \u2192 web \u2192 app \u2192 db\n```\n\nThis rule creates a **second, lateral path to the crown jewel**:\n\n```\ninternet \u2192 ALB \u2192 web \u2192 app \u2192 cache \u2192 db\n```\n\nThe cache tier now serves as a pivot point. Any RCE or severe vuln on a cache node \u2014 a tier that exists to serve the app tier, which is itself reachable from the internet through two more hops \u2014 gives an attacker a direct connection to the database without ever touching the app tier directly. Redis nodes are also frequently under-hardened relative to application servers (older auth configs, less EDR coverage, less frequent patching scrutiny).\n\n**The deeper design problem:** A cache warming pattern that requires the cache to *write to* the primary database inverts the normal data flow. Typically the application tier reads from the DB and populates the cache, or a dedicated cache-warming service with its own tightly scoped SG does so. The session cache should never need to initiate connections to Postgres. If this architecture is intentional, it deserves a dedicated `sg-cachewarmer` SG with a named human owner and a separate instance, not a blanket rule on the general cache SG.\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 2: Public ALB is an EC2 instance with a public IP, not an ELB resource\n\n**The specific misconfiguration:**\n\n`i-01lb00a01` has `PublicIpAddress: 203.0.113.11` and carries only `sg-01lb00000000000001`.\n\nThis is a publicly routable IP assigned directly to an EC2 instance, not an Elastic Load Balancing resource. Genuine ALBs do not appear in `describe-instances` output. They have DNS names (e.g., `*.<region>.elb.amazonaws.com`), not instance IDs.\n\n**Why this is critical:**\n\n- A real ALB is a managed AWS service: AWS handles patching, DDoS absorption (Shield Standard), and the data plane. This is a self-managed instance running load balancer software, meaning the OS and LB software are your responsibility to patch.\n- `sg-01lb00000000000001` allows TCP/443 inbound from `0.0.0.0/0` **and** `::/0` with a fully open (`-1`, `0.0.0.0/0`) egress rule. This instance is directly internet-facing with unrestricted outbound.\n- If this instance is compromised, the attacker has an internet-routable foothold inside the VPC with full outbound capability, and can reach the web tier (and through the chain) everything downstream.\n- There is no WAF integration, no managed TLS termination audit trail, and no AWS-native access logging that would come for free with a real ALB.\n\n**Caveat:** It is *possible* this is a legitimate pattern (e.g., HAProxy/Nginx on EC2 used intentionally, or an NLB with an EIP). But the SG is described as \"Public application load balancer\" and the instance is tagged `tier: lb`, which strongly implies this is meant to be an ALB-equivalent. This warrants immediate confirmation.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 3: web-frontend uses CIDR-based source instead of SG reference for ALB traffic\n\n**The specific misconfiguration:**\n\n`sg-01web00000000000002` allows TCP/8443 from CIDR `10.50.0.0/16` described as *\"internal service mesh\"* with the comment that *\"ALB target registration is by IP, not SG.\"*\n\n**Why this is high severity:**\n\nThe SG reference model (`UserIdGroupPairs`) is AWS's mechanism for proving *identity* of the source, not just address. By falling back to a /16 CIDR:\n\n- Any host anywhere in the `10.50.0.0/16` range can reach the web tier on 8443 \u2014 that is up to 65,534 possible source addresses, not one ALB.\n- If any other workload is or ever becomes present in that /16 (misconfigured instance, compromised host in a peered VPC, a future service deployed carelessly), it has a permitted path to the web tier.\n- The /16 is a very wide mask for what should be a single load balancer's source addresses.\n\n**The likely root cause:** The team correctly identified that when an ALB registers targets by IP (IP target type), the source IP seen by the target is the ALB node's private IP, not a security group identity. This is a real constraint. But the correct mitigation is to use the **ALB's specific subnet CIDRs** (e.g., `/28` or `/27` per AZ), not the entire /16. If this is a genuine ALB (finding 2 notwithstanding), you'd use the ALB's subnet ranges.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 4: Bastion host has unrestricted outbound egress\n\n**The specific misconfiguration:**\n\n`sg-01bastion000000006` ingress is correctly locked to `10.20.0.0/16` (corp VPN) on TCP/22. However, its egress is `protocol: -1, 0.0.0.0/0` \u2014 all protocols, all destinations, including the internet.\n\n**Why this is high:**\n\nA bastion host's egress should be tightly scoped to the specific internal hosts and ports it needs to reach (typically TCP/22 to the internal instance subnet CIDRs). The current rule means:\n\n- A compromised bastion can exfiltrate data to any internet destination.\n- A compromised bastion can initiate outbound connections to C2 infrastructure.\n- The bastion can reach every other SG in the VPC \u2014 there is nothing in any downstream SG that explicitly blocks the bastion's IP/SG from connecting to, e.g., the database port.\n\nThe bastion has no inbound rule to the db-primary SG \u2014 the DB SG only allows the cache SG. But there is nothing preventing someone from modifying the bastion's egress or the DB's ingress if they have IAM access, and the current egress rule makes the blast radius of a bastion compromise much larger.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 5: All security groups have unrestricted outbound egress (`-1, 0.0.0.0/0`)\n\n**The specific misconfiguration:**\n\nEvery single one of the 11 security groups has an identical egress rule:\n```json\n{ \"IpProtocol\": \"-1\", \"IpRanges\": [{ \"CidrIp\": \"0.0.0.0/0\" }] }\n```\n\n**Why this is high:**\n\nThis is the AWS default and it is almost always wrong for a production tiered fleet. Defense-in-depth requires egress controls at each tier:\n\n| Instance | Should be able to reach | Currently can reach |\n|---|---|---|\n| db-primary | Nothing outbound (or specific: OS patch repos via NAT, nothing else) | Everything |\n| session-cache | Nothing outbound | Everything |\n| app-tier | DB port, cache port, queue port, log port, SSM endpoints | Everything |\n| web-frontend | App port only | Everything |\n\nUnrestricted egress means that if any tier is compromised, the attacker can exfiltrate data, download tooling, beacon to C2, and pivot to any reachable destination. The database instance, specifically, should have no legitimate reason to initiate any outbound connection to the public internet.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 6: `ssm-endpoints` SG source is `10.0.0.0/8` \u2014 a /8 supernet\n\n**The specific misconfiguration:**\n\n`sg-01ssm0000000000010` allows TCP/443 from `10.0.0.0/8`.\n\n**Why this matters:**\n\n`10.0.0.0/8` encompasses ~16.7 million addresses. If this VPC is peered, connected via Transit Gateway, or has VPN connections to other networks in RFC-1918 space, every host on all of those networks can reach the SSM VPC endpoints. SSM endpoints are sensitive because they're the control plane for Systems Manager \u2014 access to them (combined with appropriate IAM) enables session manager tunnels, parameter store reads, and more. The source should be scoped to the VPC's actual CIDR(s), not the entire class A.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 7: Single-AZ deployment (all resources tagged `us-east-1a`)\n\n**The specific misconfiguration:**\n\nEvery instance and security group is tagged `az: us-east-1a`. There is no evidence of resources in `us-east-1b`, `us-east-1c`, etc.\n\n**Why this matters (security angle, not just reliability):**\n\nFrom a security posture perspective:\n- A single-AZ deployment means failover or DR likely involves manual intervention, which increases the blast radius of an availability incident (including a security incident requiring instance isolation/termination).\n- Real ALBs require at least two AZs. If this is a real ALB (see Finding 2), the single-AZ tag is inconsistent. If it's not a real ALB, see Finding 2.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 8: `log-aggregator` SG rule is missing a description on the UserIdGroupPair\n\n**The specific misconfiguration:**\n\n```json\n// sg-01logs000000000011\n\"UserIdGroupPairs\": [\n  {\n    \"GroupId\": \"sg-01app00000000000003\"\n    // no \"Description\" field\n  }\n]\n```\n\nAll other SG-reference rules in this config include a `Description`. This one does not.\n\n**Why this matters:**\n\nMinor in isolation, but security group rule descriptions are the primary human-readable audit trail for *why* a rule exists. An undescribed rule is harder to audit, harder to review in automated compliance tooling, and easier to leave in place after it's no longer needed (\"no one knows what it does, don't touch it\"). In a production environment this violates the tagging/documentation standard apparently in use everywhere else.\n\n---\n\n## Most Important Single Fix\n\n**Fix the cache-to-database path (Finding 1).**\n\nConcretely:\n\n```\nRemove from sg-01db00000000000005 IpPermissions:\n  TCP/5432 source: sg-01cache0000000000004\n```\n\nThen re-evaluate whether the cache-warming architecture actually requires the Redis tier to connect to Postgres at all. If it does, create a dedicated `sg-cachewarmer` SG attached only to the specific instance(s) that perform warming, with a narrower rule, a named owner tag, and a documented change ticket. Do not grant the general session-cache SG database access.\n\nThis is the most important fix because it closes a lateral movement path to the crown jewel that bypasses the intended app-tier control point, with a relatively small blast radius required to exploit (compromise app \u2192 compromise cache \u2192 connect to DB, or more directly, compromise cache directly if it's reachable from app tier with RCE).\n\n---\n\n## What Cannot Be Determined From This Configuration Alone\n\nThe following are material to a complete risk assessment but are not visible in SG + instance data:\n\n1. **VPC flow logs status** \u2014 Whether there is any network-level visibility into actual traffic. If flow logs are disabled, none of the lateral paths described above would produce forensic evidence.\n\n2. **NACL rules** \u2014 Network ACLs are a separate, stateless control layer applied at the subnet level. SGs can be perfectly configured while NACLs are wide open, or vice versa. The actual effective network policy is the intersection of both.\n\n3. **Route tables and subnet topology** \u2014 Which subnets are public (IGW route) vs. private (NAT route) vs. isolated (no internet route). The DB and cache instances have no public IPs, which is consistent with private subnets, but is not proven by SG data alone. A misconfigured route table could make a \"private\" subnet publicly routable.\n\n4. **VPC peering / Transit Gateway attachments** \u2014 Whether `10.0.0.0/8` reachability to the SSM endpoints (Finding 6) or `10.50.0.0/16` reachability to the web tier (Finding 3) includes any external networks. If the VPC is peered with a less-trusted environment, the effective attack surface is significantly larger.\n\n5. **IAM roles on instances** \u2014 Particularly the EC2 instance profile on the ALB/bastion/app instances. A misconfigured IAM role could allow an attacker who compromises any instance to escalate to broad AWS API access regardless of SG controls.\n\n6. **Whether `i-01lb00a01` is actually running ALB software or is an AWS ALB** \u2014 As noted in Finding 2, the instance appearing in `describe-instances` is anomalous for a managed ALB. Confirmation of what is actually running on this instance is required.\n\n7. **Security group membership of the monitoring instance** \u2014 The `monitoring` SG (`sg-01mon0000000000007`) has no corresponding instance in the `describe-instances` output. It is unknown what host (if any) is attached to it, whether it has a scrape config that reaches all other tiers, and what its own outbound access pattern looks like.\n\n8. **TLS certificate and termination configuration** \u2014 The ALB terminates HTTPS (443), but it is unknown whether backend connections (ALB\u2192web on 8443, web\u2192app on 8080) use TLS. Port 8080 is plaintext HTTP by convention; if app-tier traffic is unencrypted inside the VPC, a compromised hop anywhere in the chain can read session data.\n\n9. **RabbitMQ management port (15672)** \u2014 The `rabbitmq` SG only shows TCP/5672 (AMQP). The management UI on 15672 may be open under a rule not captured here, or may be controlled by a separate SG not in scope. If exposed internally, the RabbitMQ management interface is a high-value target.\n\n10. **Patch levels, OS configuration, and host-based controls** \u2014 SGs control network access; they say nothing about whether instances are running vulnerable software versions, whether SSHd is hardened, or whether any host-based firewall (iptables/nftables) adds an additional layer or contradicts the SG rules.",
    "agent_chars": 13431,
    "agent_s": 81.1,
    "elapsed_s": 89.8
  },
  {
    "fixture": "01-orphaned-front-internal-cidr",
    "condition": "control",
    "trial": 1,
    "score": 2,
    "scores_by_item": [
      1,
      0,
      0,
      0,
      0,
      1,
      0
    ],
    "notes": [
      "Agent correctly parses UserIdGroupPairs as SG-to-SG edges, reads 0.0.0.0/0 as internet entry, and traces instance-to-SG membership throughout.",
      "This is a CLEAN fixture with no reachable path to the crown jewel; the agent incorrectly asserts a 4-hop internet-to-database path as its primary critical finding, which is false \u2014 the web-frontend accepts only 10.50.0.0/16 (internal mesh CIDR), not the public ALB SG, orphaning the deep chain.",
      "Agent badly over-flags: it asserts the orphaned deep chain (lb\u2192web\u2192app\u2192cache\u2192db) is a live reachable internet-to-database path, which is exactly the bait the fixture is designed to expose; the chain is disconnected because the web-frontend ingress is CIDR-based on 10.50.0.0/16, not the ALB SG, so the ALB cannot reach the web tier from the internet.",
      "Agent traces a 4-hop path but the path it traces is incorrect because the first hop (ALB\u2192web-frontend) does not exist \u2014 the web-frontend accepts only the internal service-mesh CIDR, not the ALB SG, so multi-hop assembly is built on a false premise.",
      "Agent marks the fabricated internet\u2192db path as CRITICAL, which is wrong for a clean fixture; no critical finding should exist here.",
      "Agent explicitly names route tables, NACLs, live SG membership, and instance membership as things it cannot determine from the SG graph alone, matching the ground-truth boundary statement.",
      "The recommended top fix (remove cache\u2192db rule, add app\u2192db direct reference) is incorrect for a clean fixture where no fix is warranted; the correct answer is no change beyond confirming the boundary."
    ],
    "agent_output": "# Security Review: VPC Network Configuration\n\n## Methodology Note\n\nI'm analyzing security group rules, instance-to-SG membership, and the implicit trust chain from internet to database. I'm reading the actual JSON, not summaries.\n\n---\n\n## Findings, Ranked by Severity\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 #1: The Cache-to-Database Trust Chain Creates an Unintended Lateral Movement Path\n\n**The core architectural problem.**\n\nThe access chain as configured:\n\n```\nInternet (0.0.0.0/0)\n  \u2192 alb (sg-01lb00000000000001)        [HTTPS/443, open internet]\n  \u2192 web-frontend (sg-01web00000000000002) [port 8443, CIDR-based]\n  \u2192 app-tier (sg-01app00000000000003)   [port 8080, SG reference \u2713]\n  \u2192 session-cache (sg-01cache0000000000004) [port 6379, SG reference \u2713]\n  \u2192 db-primary (sg-01db00000000000005)  [port 5432, SG reference \u2713]\n```\n\n`db-primary` accepts connections on **5432 from `sg-01cache0000000000004`** (the Redis session cache), justified in the description as a \"cache warmer writes through.\"\n\n**The problem is not the description \u2014 it's what the rule actually permits.** The inbound rule on `sg-01db00000000000005` grants access to **any instance carrying `sg-01cache0000000000004`**, not just a specific cache-warmer process or host. This means:\n\n- Any compromise of the cache tier (i-01cache01) gives direct Postgres access\n- Any future instance accidentally assigned `sg-01cache0000000000004` gets Postgres access\n- The app tier already has a direct, legitimate path to the DB (implicitly expected). If it doesn't \u2014 and the cache is the *only* DB client \u2014 then the app tier has *indirect* DB access through a Redis intermediary, which is a worse design than a direct controlled SG reference\n\n**The deeper issue:** If the application tier needs database access (which is almost universally true), it should have a *direct* SG reference from `sg-01app00000000000003` \u2192 `sg-01db00000000000005`. Routing DB writes through a session cache is an unconventional architecture that widens the blast radius and obscures the trust model. There is no rule currently granting app \u2192 db directly.\n\n**The explicit lateral movement path to the crown jewel:**\n```\nAttacker compromises web-1 (i-01web0001)\n  \u2192 web SG can reach app SG on 8080 \u2713\n  \u2192 app SG can reach cache SG on 6379 \u2713\n  \u2192 cache SG can reach db on 5432 \u2713\n```\nThat is a **4-hop internet-to-database path** where each hop is permitted by the configured rules.\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 #2: All Egress Rules Are Unrestricted (`0.0.0.0/0`, Protocol `-1`)\n\n**Every single security group** \u2014 all 11 \u2014 has an egress rule of:\n```json\n{\n  \"IpProtocol\": \"-1\",\n  \"IpRanges\": [{ \"CidrIp\": \"0.0.0.0/0\" }]\n}\n```\n\nProtocol `-1` means **all protocols, all ports**. This is the AWS default that most teams never remove.\n\n**Concrete consequences:**\n\n| Instance | What unrestricted egress enables |\n|---|---|\n| `db-primary` (i-01db00001) | Can initiate outbound connections anywhere on the internet \u2014 ideal for exfiltration after compromise |\n| `session-cache` (i-01cache01) | Can connect out to attacker C2 infrastructure |\n| `app-tier` (i-01app0001) | Can reach any internal or external service, not just declared dependencies |\n| `bastion` (i-01bast001) | Can pivot to any host, any port |\n\nThe database server has no legitimate reason to initiate outbound internet connections. At minimum, the db, cache, and app tiers should have egress locked to their declared downstream dependencies only.\n\n**What least-privilege egress should look like for `db-primary`:**\n- Outbound to backup/snapshot destination (specific S3 VPC endpoint or subnet)\n- Outbound to monitoring on port 9090 (if push-based)\n- Nothing else\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 #3: ALB Instance Has a Public IP and Unrestricted Egress\n\n`i-01lb00a01` has `PublicIpAddress: 203.0.113.11` and carries `sg-01lb00000000000001`.\n\nThe inbound rule is correctly restricted to TCP/443. However:\n\n1. **The egress rule is `0.0.0.0/0 / -1`**. The ALB can initiate connections to anywhere on the internet on any protocol/port. If this is a software load balancer (HAProxy, nginx) rather than an AWS-managed ALB, that egress posture is a significant exfiltration and C2 risk.\n\n2. **If this is an AWS-managed Application Load Balancer**, it should not appear in `describe-instances` at all \u2014 managed ALBs are not EC2 instances. Its presence here as an EC2 instance with a public IP means this is almost certainly a **self-managed load balancer running on an EC2 instance**, which carries its own patch/management burden and attack surface that a managed ALB would not.\n\n3. The instance should only need egress to the web-frontend tier on port 8443, plus potentially port 443 to ACM/OCSP for certificate validation. All other egress should be denied.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 #4: Bastion Has a Public IP and Egress to `0.0.0.0/0`\n\n`i-01bast001` has `PublicIpAddress: 203.0.113.19`.\n\n- Inbound SSH is restricted to `10.20.0.0/16` (corp VPN) \u2014 **this is correct and good**\n- However, egress is `0.0.0.0/0 / -1`\n\n**The risk:** A compromised bastion (via a vulnerability in sshd, a stolen VPN credential, or a misconfigured authorized_keys) can reach any host inside the VPC on any port, plus any external host. There is no declared list of what the bastion is permitted to reach internally.\n\n**Additional concern:** The bastion is not in `describe-instances` \u2014 wait, it is: `i-01bast001`. But there is no SG rule on any other security group that explicitly permits inbound connections *from* `sg-01bastion000000006`. This means:\n\n- **The bastion cannot currently SSH into the web, app, cache, or db tiers** based on the security group rules shown \u2014 none of those groups have an inbound rule referencing `sg-01bastion000000006` on port 22.\n- Either SSH access from the bastion is broken (undiscoverable from SGs alone), or those instances' SGs have additional rules not shown, or the bastion uses a different mechanism.\n\nThis is either a gap in access (bastion is present but can't reach anything) or a configuration that isn't fully captured in this data.\n\n**Recommendation:** Replace the bastion entirely with AWS Systems Manager Session Manager. The `ssm-endpoints` SG (`sg-01ssm0000000000010`) already exists, suggesting SSM is partially deployed \u2014 the bastion may be redundant and should be decommissioned. If retained, egress should be locked to specific internal CIDRs on port 22 only.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 #5: `web-frontend` Ingress Is CIDR-Based (`10.50.0.0/16`), Not Security-Group-Based\n\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 8443,\n  \"ToPort\": 8443,\n  \"IpRanges\": [{ \"CidrIp\": \"10.50.0.0/16\", \"Description\": \"internal service mesh\" }]\n}\n```\n\nThe description acknowledges this: *\"ALB target registration is by IP, not SG.\"*\n\n**The problem:** `10.50.0.0/16` is a /16 \u2014 **65,536 addresses**. Any host in that entire CIDR range can reach the web frontend on 8443, not just the ALB instance (`203.0.113.11` is public-facing, so the ALB's internal/private IP must fall within 10.50.0.0/16 for this rule to function).\n\nThis rule cannot distinguish the ALB from:\n- Any other instance in the 10.50.0.0/16 range\n- A compromised host in that subnet range\n- Future instances provisioned in overlapping space\n\nThe stated reason (IP-based ALB target registration) is a real constraint for AWS-managed ALBs. But since the ALB here appears to be an EC2 instance (finding #3), there is **no technical reason this cannot be converted to a security group reference** (`sg-01lb00000000000001` \u2192 `sg-01web00000000000002`), which would be strictly more precise.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 #6: `ssm-endpoints` Ingress Allows `10.0.0.0/8` \u2014 An Extremely Broad CIDR\n\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 443,\n  \"ToPort\": 443,\n  \"IpRanges\": [{ \"CidrIp\": \"10.0.0.0/8\" }]\n}\n```\n\n`10.0.0.0/8` covers **16.7 million addresses** \u2014 the entire RFC 1918 Class A space. If this VPC is peered with other VPCs, connected via Transit Gateway, or has Direct Connect/VPN, any host in any connected network can reach these SSM endpoints.\n\nSSM VPC interface endpoints should be restricted to the specific subnets or security groups of the instances that need SSM access, not the entire private IP space. A compromised workload in any connected environment could use these endpoints to issue SSM commands.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 #7: No Inbound Rules on Any SG Reference the Monitoring SG (`sg-01mon0000000000007`)\n\n`monitoring` accepts Prometheus scrapes on 9090 from `10.80.1.0/24`. But no other security group has an **egress or inbound rule that explicitly references `sg-01mon0000000000007`** to allow the Prometheus server *to scrape* the other instances.\n\nFor Prometheus pull-based scraping to work:\n- The Prometheus server (in `sg-01mon0000000000007`) needs to initiate connections **to** the exporters on the other instances (typically port 9100 for node_exporter, or application-specific ports)\n- None of the other SGs (web, app, cache, db, etc.) have inbound rules permitting `sg-01mon0000000000007`\n\nEither monitoring is non-functional as configured, or node exporters are listening on ports covered by some implicit CIDR rule not shown. Either way, monitoring coverage of the db and app tiers cannot be confirmed, which is itself a risk.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 #8: `ci-runner` Accepts Inbound 443 from Runner Subnet \u2014 Inverted Direction\n\n```json\n{\n  \"GroupId\": \"sg-01ci00000000000008\",\n  \"GroupName\": \"ci-runner\",\n  \"IpPermissions\": [\n    {\n      \"IpProtocol\": \"tcp\",\n      \"FromPort\": 443,\n      \"ToPort\": 443,\n      \"IpRanges\": [{ \"CidrIp\": \"10.80.7.0/24\", \"Description\": \"runner subnet\" }]\n    }\n  ]\n}\n```\n\nThis SG is attached to instances described as \"CI runners,\" but the inbound rule allows the runner subnet to reach *these* instances on 443. That means something *else* in `10.80.7.0/24` is initiating connections to CI runner instances on HTTPS.\n\nThis is likely inverted or misattributed \u2014 typically CI runners initiate outbound connections (to pull jobs from a CI controller, to reach SCM, to push artifacts). If runners need to pull from a controller, the controller's SG needs inbound rules, not the runners'.\n\nAdditionally, CI runners that have access to the internal network (per unrestricted egress) and can receive inbound connections from a /24 runner subnet could be used to pivot if compromised during a build pipeline execution (e.g., supply chain attack via a malicious dependency).\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 #9: No Instance Is Assigned Both a Production SG and the Monitoring or SSM SG\n\nFrom `describe-instances`, each instance has exactly one security group. For SSM Session Manager to function, instances need to reach the SSM endpoints \u2014 but the SSM endpoint SG (`sg-01ssm0000000000010`) is not attached to any instance shown, and none of the production instances are shown with multiple SGs.\n\nThis may be fine (instances contact VPC endpoints; endpoint SGs control who can reach the endpoint, not what the instance carries). But it means SSM reachability for each production instance depends entirely on their single SG's egress rules allowing 443 to the endpoint's private IP \u2014 which the current unrestricted egress rules do permit. If egress is ever tightened (which it should be per Finding #2), SSM access must be explicitly preserved or these instances become unmanageable without the bastion.\n\n---\n\n### \ud83d\udfe2 LOW \u2014 #10: IPv6 Not Configured Consistently\n\n`sg-01lb00000000000001` (public ALB) has:\n```json\n\"Ipv6Ranges\": [{ \"CidrIpv6\": \"::/0\" }]\n```\n\nNo other security group has any IPv6 rules. If the VPC has an IPv6 CIDR and instances have IPv6 addresses, traffic flows could bypass security group rules that only specify IPv4 CIDRs. This needs to be consistent with whether IPv6 is actually enabled on this VPC and its subnets.\n\n---\n\n### \ud83d\udfe2 LOW \u2014 #11: All Resources Tagged with Single AZ (`us-east-1a`)\n\nEvery resource has `\"az\": \"us-east-1a\"`. Security groups themselves are VPC-scoped (not AZ-scoped), so this tag is informational. However, it suggests the entire fleet may be single-AZ, which is a **availability risk** rather than a security risk per se \u2014 but single-AZ deployments often also indicate that HA/DR planning (including security incident recovery) may be underdeveloped.\n\n---\n\n## The Single Most Important Fix\n\n**Remove the `sg-01cache0000000000004` \u2192 `sg-01db00000000000005` inbound rule on port 5432, and replace it with a direct `sg-01app00000000000003` reference if app-to-db access is needed.**\n\nThe cache-warmer-to-database trust chain is the highest-risk architectural decision in this configuration. It:\n\n1. Creates a 4-hop path from the internet to the database through individually-compromisable tiers\n2. Grants the Redis cache (an in-memory store with a historically exploitable attack surface \u2014 unauthenticated by default in older versions) direct write access to the Postgres database\n3. Obscures the actual data flow so that future engineers may not recognize the risk\n4. Means a Redis SSRF, eval abuse, or unauthorized replication attack could pivot directly to Postgres\n\n**Concrete change:**\n\nOn `sg-01db00000000000005`, replace:\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 5432,\n  \"ToPort\": 5432,\n  \"UserIdGroupPairs\": [\n    { \"GroupId\": \"sg-01cache0000000000004\", \"Description\": \"from session cache warmer\" }\n  ]\n}\n```\nWith (if app tier needs DB access, which it almost certainly does):\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 5432,\n  \"ToPort\": 5432,\n  \"UserIdGroupPairs\": [\n    { \"GroupId\": \"sg-01app00000000000003\", \"Description\": \"from app tier\" }\n  ]\n}\n```\nAnd separately, re-architect the cache warmer to not require direct DB write access \u2014 it should receive data *from* the application tier, not bypass it.\n\n---\n\n## What Cannot Be Determined From This Configuration Alone\n\nThese are genuine unknowns that require additional data sources:\n\n| Unknown | Why It Matters |\n|---|---|\n| **VPC subnet topology and route tables** | Whether subnets are truly private (no IGW route) cannot be determined from SGs alone. A \"private\"",
    "agent_chars": 13929,
    "agent_s": 83.6,
    "elapsed_s": 95.2
  },
  {
    "fixture": "01-orphaned-front-internal-cidr",
    "condition": "control",
    "trial": 2,
    "score": 2,
    "scores_by_item": [
      1,
      0,
      0,
      0,
      0,
      1,
      0
    ],
    "notes": [
      "Agent correctly parses UserIdGroupPairs as SG-to-SG edges, reads 0.0.0.0/0 as internet entry, and reads instance-to-SG membership.",
      "This is a CLEAN fixture with no reachable path to the crown jewel; the agent incorrectly asserts a fully-connected 5-hop kill chain from internet to DB, which is the opposite of the correct finding.",
      "Agent fatally over-flags: it declares the orphaned deep chain (web->app->cache->db) as a CRITICAL reachable path, when ground truth shows the chain is disconnected from the internet because web-frontend accepts only the internal service-mesh CIDR 10.50.0.0/16, not the public ALB SG.",
      "Agent traces a multi-hop path but the path it traces is non-existent on this fixture; the web tier is not reachable from the ALB via SG reference, making the entire chain orphaned.",
      "Agent headlines a fabricated critical path (cache->db as lateral movement) on a clean fixture; correct answer is no critical finding and no reachable path to the crown jewel.",
      "Agent explicitly names route tables, NACLs, live instance membership, and subnet ownership as things it cannot determine from the SG graph alone, matching the ground-truth boundary.",
      "Agent recommends removing cache->db rule as the top fix, but the correct answer is no fix needed (clean fixture); the recommendation is based on a fabricated finding."
    ],
    "agent_output": "# Security Group Configuration Review\n\n## Methodology Note\n\nI'm working from SG rules and instance-to-SG mappings only. I'll flag what I can prove from the data, separate from what requires additional context, and be explicit about assumptions.\n\n---\n\n## Findings, Ranked by Severity\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 1: The DB Tier Accepts Connections from the Cache Tier, Creating an Unintended Lateral Movement Path\n\n**What the config says:**\n\n`sg-01db00000000000005` (`db-primary`) allows inbound TCP 5432 from `sg-01cache0000000000004` (`session-cache`).\n\n**Why this is the most serious finding:**\n\nThe described intent is *\"cache warmer writes through\"* \u2014 meaning the Redis instance has a direct authenticated path to Postgres. This means the kill chain from the internet to the crown jewel is:\n\n```\nInternet \u2192 ALB (443) \u2192 Web Frontend (8443) \u2192 App Tier (8080) \u2192 Session Cache (6379) \u2192 DB Primary (5432)\n```\n\nThat is **five hops, but four of them are permitted by explicit SG rules with no gaps.** The chain is fully connected. An attacker who compromises any node in that chain has a permitted network path to the database \u2014 no firewall rule stops them, only application-layer auth.\n\nMore specifically: if the Redis instance is compromised (e.g., via an unauth RCE in Redis, which has a documented history), the attacker has a direct, SG-permitted TCP connection to Postgres. Redis instances are frequently misconfigured or unauthenticated internally.\n\n**The deeper design problem:** A session cache should not need to write to the primary database. Cache warmers that require DB write access are typically an architectural smell. If this pattern is genuinely required, the write path should belong to the app tier SG, not the cache tier SG. The app tier already has a DB-adjacent trust position and is a more defensible source.\n\n**The single most important fix (see dedicated section below).**\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 2: Every Security Group Has Unrestricted Egress (`0.0.0.0/0`)\n\n**What the config says:**\n\nAll 11 security groups have identical egress rules:\n```json\n{\n  \"IpProtocol\": \"-1\",\n  \"IpRanges\": [{ \"CidrIp\": \"0.0.0.0/0\" }]\n}\n```\n\n**Why this is critical:**\n\nUnrestricted egress on every tier means:\n- A compromised instance can exfiltrate data to any external IP on any port\n- A compromised DB or cache instance can initiate outbound connections to attacker-controlled infrastructure (reverse shells, C2 beaconing, DNS exfiltration)\n- The DB instance (`i-01db00001`) \u2014 the crown jewel \u2014 can freely open outbound connections to the internet despite having no documented reason to do so\n\nThis is the **widest blast radius** finding in the config. Defense-in-depth requires that egress be scoped as tightly as ingress. For a database tier, legitimate egress is typically: Postgres replication to a replica SG, possibly SSM endpoint (443), and nothing else.\n\n**Concrete example of risk:** If an attacker exploits a SQL injection to achieve RCE on the DB host, the unrestricted egress rule permits them to immediately beacon out, download tooling, or exfiltrate the entire database over port 443 to an S3-compatible endpoint \u2014 none of which would be blocked at the SG layer.\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 3: The ALB Instance Has a Public IP and Unrestricted Egress\n\n**What the config says:**\n\n`i-01lb00a01` has `PublicIpAddress: 203.0.113.11` and is in `sg-01lb00000000000001`, which has `0.0.0.0/0` egress on all protocols.\n\n**Why this matters beyond Finding 2:**\n\nAn internet-facing EC2 instance with unrestricted egress is particularly dangerous. The ALB security group is the first trust boundary from the internet. If this is a self-managed load balancer (not an AWS-managed ALB resource \u2014 see \"Cannot Determine\" section), it has the largest attack surface and any compromise immediately grants outbound internet access.\n\nNote: If this is actually an AWS-managed ALB *resource* (not an EC2 instance), the risk profile differs \u2014 but the instance record shows it as an EC2 instance with a public IP, so it must be treated as such.\n\n---\n\n### \ud83d\udd34 CRITICAL \u2014 Finding 4: The Bastion Has a Public IP with No Egress Restriction\n\n**What the config says:**\n\n`i-01bast001` has `PublicIpAddress: 203.0.113.19`, accepts SSH from `10.20.0.0/16` (corp VPN), and has `0.0.0.0/0` egress.\n\n**Issue 1 \u2014 Ingress source is a /16 CIDR, not a SG:**\n\n`10.20.0.0/16` contains 65,534 hosts. If \"corp VPN\" means a VPN gateway, the appropriate source is the specific IP(s) of the VPN concentrator's internal interface, or a much tighter CIDR. Any host on that /16 \u2014 including potentially compromised corp laptops, shared build machines, etc. \u2014 can initiate SSH to the bastion.\n\n**Issue 2 \u2014 The bastion has a public IP:**\n\nThe description says \"corp VPN only,\" which implies access should come through the VPN, not the public internet. If the VPN terminates inside the VPC, the bastion doesn't need a public IP at all. If the VPN terminates externally, the SSH source CidrIp should be the VPN concentrator's *public* IP, and it should appear in `IpRanges` as a /32. The current rule allows `10.20.0.0/16` (RFC-1918), which is only reachable internally \u2014 so the public IP serves no apparent function, but enlarges the attack surface.\n\n**Issue 3 \u2014 No SG-reference chain from bastion to anything:**\n\nThe bastion SG has no egress restrictions and no ingress rules from other SGs referencing it. This means we cannot determine from SG rules alone which tiers the bastion can SSH into. The bastion could be connecting to any instance on any port. This should be explicit and scoped.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 5: The `web-frontend` SG Uses a CIDR-Based Rule for Inter-Tier Traffic Instead of a Security Group Reference\n\n**What the config says:**\n\n`sg-01web00000000000002` allows inbound TCP 8443 from `10.50.0.0/16` with description \"internal service mesh.\" All other inter-tier rules in this config use SG references (`UserIdGroupPairs`).\n\n**Why this is weaker than the pattern used elsewhere:**\n\nA CIDR-based rule grants access to any resource in the `10.50.0.0/16` range, regardless of whether it's the ALB. The SG reference model (used correctly in app\u2192web, app\u2192cache, app\u2192queue, app\u2192logs) means only instances *actually attached to that SG* can connect \u2014 it's identity-based, not address-based.\n\nThe description acknowledges why this was done: *\"ALB target registration is by IP, not SG.\"* This is a real constraint with EC2-based load balancers in some configurations, but it means the web tier trusts an entire /16 (65,534 addresses) rather than a specific SG. Any host in that subnet range \u2014 including potentially unrelated workloads \u2014 can reach the web tier on 8443.\n\nIf the ALB is an **AWS-managed ALB resource**, it *can* be referenced by SG ID in target group rules and in downstream SG ingress rules. The CIDR workaround suggests this may be an EC2-based proxy/load balancer, which has its own implications.\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 6: The Bastion Instance Does Not Appear to Have SSM Agent Coverage, and the SSM Endpoint SG Has No Instance Attached\n\n**What the config says:**\n\n`sg-01ssm0000000000010` (`ssm-endpoints`) exists and accepts 443 from `10.0.0.0/8`, but **no instance in the `describe-instances` output is a member of this SG.** The bastion instance (`i-01bast001`) uses only `sg-01bastion000000006`.\n\n**Risk:**\n\n- The SSM endpoint SG appears orphaned (no attached instances visible)\n- The bastion relies on SSH from a VPN CIDR rather than SSM Session Manager, which is the recommended modern pattern for bastion-less access\n- If SSM is not functional (because no instances are enrolled in SSM or the endpoint SG isn't attached to the endpoint ENI), the bastion represents the *only* administrative access path \u2014 a single point of failure and a high-value target\n\n---\n\n### \ud83d\udfe0 HIGH \u2014 Finding 7: Monitoring SG and CI Runner SG Are CIDR-Based with No Instance Attachment Visible\n\n**What the config says:**\n\n- `sg-01mon0000000000007` accepts 9090 from `10.80.1.0/24` \u2014 no instance in `describe-instances` is attached to this SG\n- `sg-01ci00000000000008` accepts 443 from `10.80.7.0/24` \u2014 no instance is attached to this SG\n\n**Risks:**\n\n1. **Prometheus (monitoring SG):** The monitoring SG has no ingress reference from any application tier SG. This means Prometheus cannot scrape metrics from the application tiers unless those tiers have additional SG rules not shown, or Prometheus uses a push model. If scraping is happening via CIDR rules defined on the *application* SGs (not visible here because they'd be ingress rules on the app SGs from a monitoring CIDR), this is a hidden rule not in the provided data.\n\n2. **Orphaned SGs:** SGs with no attached resources are noise that complicates auditing and may represent forgotten infrastructure. They should be removed or the attachment documented.\n\n3. **CI Runner with DB-adjacent access:** The CI runner SG accepts 443 from a runner subnet. CI systems are high-value compromise targets (supply chain). The egress rule is `0.0.0.0/0`, and we cannot determine from this data whether CI runners are able to reach the DB tier \u2014 that depends on the CI runner instances' SG membership and route tables, neither of which is shown.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 8: All Security Groups Share the Same AZ Tag (`us-east-1a`)\n\n**What the config says:**\n\nEvery SG has `\"az\": \"us-east-1a\"`.\n\n**Why this matters:**\n\nSecurity groups are VPC-wide resources, not AZ-scoped. The tag suggests instances may only be deployed in a single AZ, which means:\n- No high availability \u2014 an `us-east-1a` outage takes down the entire fleet\n- More critically for security: no blast-radius isolation between AZs\n\nThis is an availability finding with security implications \u2014 if instances are single-AZ, there's no architectural separation that would limit the scope of an AZ-level event.\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 9: `log-aggregator` Ingress Rule Is Missing a Description\n\n**What the config says:**\n\n```json\n{\n  \"GroupId\": \"sg-01app00000000000003\"\n}\n```\n\nThe `UserIdGroupPairs` entry for `sg-01logs000000000011` has no `Description` field, unlike every other SG-reference rule in the config.\n\nThis is a low-severity hygiene issue on its own, but descriptions on SG rules are the primary inline documentation mechanism in AWS. In a production fleet, undescribed rules create audit friction and can mask unauthorized rule additions (an attacker who adds a rule won't add a description; missing descriptions normalize the pattern).\n\n---\n\n### \ud83d\udfe1 MEDIUM \u2014 Finding 10: RabbitMQ Egress Is Unrestricted\n\nThis is a specific instance of Finding 2 worth calling out separately. `sg-01queue00000000009` (RabbitMQ) has `0.0.0.0/0` egress. Message queues are common targets for data exfiltration \u2014 large volumes of structured data flow through them, and an unrestricted egress rule means a compromised queue broker can forward messages to an external endpoint. RabbitMQ's legitimate egress needs are minimal: responses to connected consumers (handled by stateful SG rules automatically), management API if applicable, and possibly clustering ports.\n\n---\n\n## The Single Most Important Fix\n\n**Remove the direct `session-cache` \u2192 `db-primary` connection and replace it with `app-tier` \u2192 `db-primary`.**\n\nConcretely:\n\n**Remove from `sg-01db00000000000005` IpPermissions:**\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 5432,\n  \"ToPort\": 5432,\n  \"UserIdGroupPairs\": [\n    {\n      \"GroupId\": \"sg-01cache0000000000004\",\n      \"Description\": \"from session cache warmer\"  \u2190 REMOVE THIS ENTRY\n    }\n  ]\n}\n```\n\n**Add to `sg-01db00000000000005` IpPermissions:**\n```json\n{\n  \"IpProtocol\": \"tcp\",\n  \"FromPort\": 5432,\n  \"ToPort\": 5432,\n  \"UserIdGroupPairs\": [\n    {\n      \"GroupId\": \"sg-01app00000000000003\",\n      \"Description\": \"from app tier\"\n    }\n  ]\n}\n```\n\n**Rationale for this being the #1 fix:**\n\nIt eliminates the only fully-connected kill chain from the internet to the database. After this change, the attack path becomes:\n\n```\nInternet \u2192 ALB \u2192 Web \u2192 App \u2192 DB  \u2713 (expected, with app-layer auth as the control)\nInternet \u2192 ALB \u2192 Web \u2192 App \u2192 Cache  \u2713 (expected)\nCache \u2192 DB  \u2717 BLOCKED\n```\n\nThe cache tier no longer has a permitted network path to the database. Any \"cache warmer\" functionality must be refactored to run in the app tier (where it already has SG access to both cache and, after this fix, the DB), eliminating the Redis-as-a-DB-proxy architecture that creates the lateral movement path.\n\nThis fix also reduces the blast radius of a Redis compromise from \"full DB access\" to \"cache data only.\"\n\n---\n\n## What Cannot Be Determined From This Configuration Alone\n\nThe following are material to a complete risk assessment but are not answerable from `describe-security-groups` and `describe-instances` alone:\n\n| Gap | Why It Matters |\n|---|---|\n| **VPC route tables and subnet associations** | SG rules are necessary but not sufficient. A permissive route table could allow traffic that bypasses SG intent, or a missing route could mean a permitted SG rule is functionally dead. |\n| **Whether the ALB instance is an EC2 instance or an AWS-managed ALB resource** | AWS-managed ALBs have different security properties; the EC2 API returning it as an instance with a public IP is ambiguous. |\n| **NACLs (Network ACLs)** | NACLs are stateless and operate at the subnet level. They could be adding restrictions not visible in SG rules, or they could be `ALLOW ALL` (the default). |\n| **Whether the VPN CIDR `10.20.0.0/16` is reachable from the internet** | If VPN traffic is routed through a public-facing concentrator into the VPC, the effective attack surface for the bastion is different from a fully private VPN. |\n| **IAM instance profiles and roles attached to each instance** | Network-level controls are one layer. An EC2 instance with an overly-permissive IAM role can exfiltrate data, modify infrastructure, or pivot to other AWS services entirely bypassing SG rules. This is frequently the *actual* kill chain in AWS breaches. |\n| **Whether SSM Session Manager is functional** | We can see the SSM endpoint SG exists but no instance is attached to it in the describe-instances output. We cannot determine if SSM is working as an alternative access path. |\n| **The `10.50.0.0/16` subnet ownership** | We cannot determine what other resources exist in this range that can reach the web frontend on 8443. |\n| **VPC Flow Logs status** | Whether any of these permitted paths are actively being used, or whether anomalous traffic is being logged, is not determinable from configuration alone. |\n| **Whether the DB instance has a security group allowing replication to a replica** | There is one DB instance visible. If there's a replica, its replication path may depend on rules not shown. |\n| **Encryption in transit between tiers** | SG rules say nothing about whether connections are TLS-wrapped. Redis on 6379 and RabbitMQ on 5672 are plaintext by default. |",
    "agent_chars": 14876,
    "agent_s": 88.6,
    "elapsed_s": 101.2
  },
  {
    "fixture": "02-public-alb-no-sg-ref",
    "condition": "control",
    "trial": 0,
    "score": 2,
    "scores_by_item": [
      1,
      0,
      0,
      0,
      0,
      1,
      0
    ],
    "notes": [
      "Agent correctly parses UserIdGroupPairs as SG-to-SG edges, reads 0.0.0.0/0 as internet entry, and reads instance-to-SG membership throughout the review.",
      "This is a clean fixture with no path to the crown jewel; the agent fails to correctly report this \u2014 instead it invents multiple reachability concerns and does not recognize the configuration as clean.",
      "Agent over-flags extensively: it headlines the intended public ALB 0.0.0.0/0 as a significant concern, invents lateral paths via the bastion, and drowns the review in nitpicks about correctly-scoped tiers (bastion egress, monitoring gaps, CI runner inbound) rather than recognizing the clean segmentation.",
      "Not applicable \u2014 there is no long path on this clean fixture, and the agent does not correctly identify the absence of a path; instead it fabricates multi-hop concerns that do not exist in the ground truth.",
      "Agent invents Critical findings on a clean fixture (bastion blast radius, ALB EC2 instance concern), directly violating the requirement not to manufacture critical findings on a clean fleet.",
      "Agent correctly names multiple things it cannot determine from SG data alone \u2014 route tables, NACLs, VPC peering, IAM roles \u2014 matching the ground-truth boundary acknowledgment.",
      "Top fix (lock bastion egress / decommission bastion) does not match the ground-truth fix, which is 'no change needed \u2014 the fleet is clean and correctly segmented'; the agent should have reported no fix required."
    ],
    "agent_output": "# Security Group Configuration Review\n\n## Methodology Note\n\nI'm working from SG rules and instance-to-SG mappings only. I'll flag what I can prove from this data and explicitly separate what requires additional context.\n\n---\n\n## Findings, Ranked by Severity\n\n---\n\n### CRITICAL\n\n#### 1. Bastion host has unrestricted egress and can reach every tier \u2014 including the database\n\n**What the data shows:**\n\n`sg-02bastion000000006` has a single egress rule: `protocol -1, 0.0.0.0/0`. There are no egress rules scoping where the bastion *can talk to*. The bastion instance `i-02bast001` has a public IP (`198.51.100.29`).\n\nThe database SG `sg-02db00000000000005` permits inbound on 3306 only from `sg-02svc00000000000004`. The bastion is **not** in that group. So the bastion cannot reach the DB *on port 3306 via the DB's own inbound rule* \u2014 but that is not the full picture.\n\nThe real risk is: if `i-02bast001` is compromised, the attacker is on a host inside the VPC with unrestricted outbound. They can:\n\n- Probe every private IP on any port\n- Reach the MySQL port on `i-02db00001` if the DB instance's inbound rule is the only control (which it is \u2014 there is no evidence of NACLs, host firewall, or subnet isolation here)\n- Reach Redis on 6379, the log aggregator on 5044, the SSM endpoints, the app tier on 9000\n\nThe bastion's egress should be locked to the specific ports and SG targets it legitimately needs (typically SSH/22 to internal instance SGs only). `0.0.0.0/0` egress on a publicly-addressed host is a significant blast-radius amplifier.\n\n---\n\n#### 2. ALB instance is an EC2 instance with a public IP \u2014 this is almost certainly wrong\n\n**What the data shows:**\n\n`i-02lb00a01` is an EC2 instance tagged `tier: lb`, carrying `sg-02lb00000000000001`, and has a public IP `198.51.100.21`.\n\nAn AWS ALB is a managed service. It does **not** appear as an EC2 instance in `describe-instances`. If this is a genuine AWS ALB, it would appear in `describe-load-balancers`, not here. The SG description says \"forwards to target groups by registered IP\", which is ALB language \u2014 but the resource is an EC2 instance.\n\nThis means one of the following is true, and each has different implications:\n\n- **This is a self-managed reverse proxy (nginx, HAProxy, etc.) on an EC2 instance**, not an AWS ALB. If so, it is a single instance with a public IP doing TLS termination. There is no managed scaling, no AWS-native WAF attachment point, and the SG description is misleading. A compromise of this instance gives an attacker a foothold inside the VPC with broad egress.\n- **This is a misconfigured or mislabeled resource** and the real ALB is not reflected in the data provided.\n\nEither way, a publicly-accessible EC2 instance with `0.0.0.0/0` inbound on 80 and 443, and `0.0.0.0/0` egress, sitting in the same VPC as the database, needs to be understood clearly. Its egress should not be `0.0.0.0/0` \u2014 it should be scoped to the web-frontend SG on port 8080 only.\n\n---\n\n### HIGH\n\n#### 3. Web-frontend ingress is a CIDR, not a security group reference \u2014 and the CIDR is unverified\n\n**What the data shows:**\n\n```\nsg-02web00000000000002 IpPermissions:\n  tcp/8080 from 10.40.10.0/24  (\"alb target subnet\")\n```\n\nEvery other internal tier-to-tier rule in this configuration uses `UserIdGroupPairs` (SG references), which is the correct pattern for EC2-to-EC2 controls. The web tier breaks this pattern by using a subnet CIDR instead.\n\n**Why this matters:**\n\nA SG reference means \"traffic from an instance carrying that SG\". A CIDR means \"traffic from any IP in that range, regardless of what instance it is or what SG it carries\". If any host in `10.40.10.0/24` is compromised \u2014 or if any new instance is placed in that subnet for any reason \u2014 it can reach port 8080 on every web-frontend instance. The control is subnet-scoped, not workload-scoped.\n\nIf this is intended to accept traffic from a managed AWS ALB (which uses dynamic IPs in its target subnets), the correct approach is to reference the ALB's SG directly. If this is a self-managed proxy (per finding #2), the correct approach is still to reference `sg-02lb00000000000001` in a `UserIdGroupPairs` entry.\n\n**Fix:** Replace the CIDR rule with a `UserIdGroupPairs` reference to `sg-02lb00000000000001`.\n\n---\n\n#### 4. All security groups have `0.0.0.0/0` egress \u2014 including database, Redis, and log aggregator\n\n**What the data shows:**\n\nEvery single SG in this fleet, including `sg-02db00000000000005`, `sg-02cache00000000009`, and `sg-02logs000000000011`, has:\n\n```\nIpPermissionsEgress:\n  protocol: -1\n  CidrIp: 0.0.0.0/0\n```\n\nFor a database instance, there is no legitimate reason to allow unrestricted outbound to the internet. Permitted outbound from a database should be:\n- Responses to MySQL clients (handled statelessly by EC2's stateful connection tracking \u2014 no explicit egress rule needed for return traffic)\n- Potentially: SSM endpoints (443 to the SSM endpoint SG) if managed via SSM\n- Potentially: DNS (UDP/TCP 53 to a resolver)\n\nUnrestricted egress on the DB instance means that if the database process or OS is compromised, the attacker can exfiltrate data, download tools, or beacon out to C2 infrastructure without any network-layer impediment.\n\nSame applies to Redis (`sg-02cache00000000009`) and the log aggregator (`sg-02logs000000000011`), both of which hold potentially sensitive data.\n\n---\n\n#### 5. SSM endpoints SG allows `10.0.0.0/8` \u2014 a /8 is a very broad internal trust boundary\n\n**What the data shows:**\n\n```\nsg-02ssm0000000000010 IpPermissions:\n  tcp/443 from 10.0.0.0/8\n```\n\n`10.0.0.0/8` covers 16.7 million addresses. If this VPC is peered with other VPCs, connected via Transit Gateway, or has a Direct Connect/VPN \u2014 all of which would use RFC 1918 space \u2014 any host in any of those connected networks can reach the SSM interface endpoints. SSM endpoints are the mechanism by which IAM-authenticated sessions are established to instances. Overly permissive access to SSM endpoints increases the attack surface for session hijacking or unauthorized instance access from peered environments.\n\nThe rule should reference the specific SGs or subnets of the instances that legitimately use SSM.\n\n---\n\n### MEDIUM\n\n#### 6. Monitoring SG (`sg-02mon0000000000007`) has no inbound from the tiers it presumably scrapes\n\n**What the data shows:**\n\n`sg-02mon0000000000007` accepts inbound on 3000 (Grafana) from `10.80.1.0/24`. There are no rules permitting the monitoring instance to scrape metrics from other tiers (Prometheus pull model typically requires the monitoring host to initiate connections to targets on their metrics port, e.g., 9090, 9100). There are also no rules on `sg-02app00000000000003`, `sg-02svc00000000000004`, or `sg-02db00000000000005` permitting inbound from a monitoring SG.\n\nThis is either:\n- A real gap: Prometheus cannot currently reach its scrape targets, meaning monitoring is silently broken or incomplete\n- Handled via a push model (agents push to Prometheus pushgateway) \u2014 but there's no SG for a pushgateway visible here\n- Handled by rules that exist on resources not shown (e.g., node exporters on their own SGs)\n\nA broken monitoring setup is a security concern because it means the fleet may be running without effective alerting.\n\n---\n\n#### 7. CI runner SG accepts inbound on 443 \u2014 runners typically only need outbound\n\n**What the data shows:**\n\n```\nsg-02ci00000000000008 IpPermissions:\n  tcp/443 from 10.80.7.0/24\n```\n\nStandard CI runner architectures (GitLab runners, GitHub Actions self-hosted, Jenkins agents) poll their coordinator outbound. They do not typically accept inbound connections. An inbound 443 rule on a CI runner is unusual and warrants explanation. CI runners frequently have elevated permissions (access to deployment credentials, secrets, artifact stores). An unnecessarily open inbound rule on a CI runner is a meaningful risk if the runner subnet has any compromised host.\n\n---\n\n#### 8. Single AZ deployment\n\n**What the data shows:**\n\nEvery single resource \u2014 all 11 SGs and all 6 instances \u2014 is tagged `az: eu-west-1a`. There is no instance or SG tagged for `eu-west-1b` or `eu-west-1c`.\n\nThis is not a direct security vulnerability but it is a resilience risk that intersects with security: a single-AZ architecture means a zonal event, a targeted attack causing an AZ-level disruption, or a deployment error in `eu-west-1a` takes down the entire application including any logging or monitoring that might detect an ongoing incident
      • judge_prompt.md 3.2 KB 
        # Judge prompt (reference)

This file documents the judge contract. The runner (`run_eval.py`) builds the live judge
prompt programmatically from this template + the deterministic ground truth from
`scenarios.expected_deep()`. Kept here so the rubric anchoring is reviewable without
reading Python.

## System

> You are an expert AWS / cloud-security evaluator grading a fleet security review against
> a 7-item rubric. Each item is binary: 1 (pass) or 0 (fail). Be strict but fair; no partial
> credit. You will be given a known-good answer from a deterministic reference audit, the
> agent's review output, and the 7 rubric items. Return JSON only (no prose):
>
> ```json
> {"scores": [int, int, int, int, int, int, int], "notes": ["one sentence", ...]}
> ```

## User (per trial)

Provides:

- **FIXTURE** id and **CONTEXT** (the generic fleet description the agent saw).
- **KNOWN-GOOD ANSWER** from the reused deterministic engine:
  - ground-truth finding codes (P1 path / B1 blast radius / H1 hub), top severity, clean flag,
    SG count, SG-to-SG edge count;
  - the shortest path entry -> crown jewel (or "(no path)"), with its hop count;
  - the blast radius (reachable tiers from the entry);
  - the expected headline (what the buried defect actually is, or that it is clean);
  - the correct top fix; and
  - the boundary the audit cannot cross.
- **AGENT REVIEW OUTPUT** (verbatim).
- The **7 RUBRIC ITEMS** (see `rubric.md`), with the two load-bearing items called out:
  item 2 (surfaces the buried LONG 4-6 hop needle as a primary finding, hops named end to
  end, not only the obvious surface items) and item 3 (does not over-flag the benign /
  neutralised bait, esp. the orphaned deep chain in `01`, the intended public ALB in `02`,
  the disconnected data island in `03`, and the broken chain in `04`).

## Anchoring rules for the judge

- The agent prompt was GENERIC ("review this for problems"); it did NOT name lateral
  movement, reachability, multi-hop depth, or the crown-jewel path. Reward the agent only
  for what it surfaces on its own. Do not penalize phrasing; grade substance.
- The needle is the composed LONG (4-6 hop) SG-to-SG path. Reading each rule as locally
  fine and stopping, or stopping at a single hop, is an item-2 and item-4 fail even if every
  individual observation is true. Naming only some hops but not assembling the full chain to
  the crown jewel does not satisfy item 2.
- On the deceptive-clean fixtures (`01`, `02`, `03`, `04`), the correct answer is "no
  reachable path to the crown jewel." Inventing a path from an orphaned deep chain (`01`),
  headlining the intended public-ALB `0.0.0.0/0` as the lateral exposure (`02`), splicing two
  disconnected islands into one route (`03`), or treating the broken chain as reachable across
  the cut (`04`), fails items 2/3/5. The agent must reason about reachability — note that the
  front tier accepts an internal CIDR rather than the ALB SG, or that nothing references the
  ALB SG, or that the islands have no joining edge — not just list the rules.
- Match the top fix and the boundary by substance, not wording. On a clean fixture, the
  correct top fix is "no change beyond confirming the boundary," NOT a fix for a path that
  does not exist.

Return JSON only.
      • README.md 4.3 KB 
        # Eval: sg-deceptive-reachability-auditor (control-only screening)

Decides whether the `sg-deceptive-reachability-auditor` skill is worth writing. There is
**no SKILL.md** and no treatment arm. We only run the CONTROL condition and read the verdict
off the control means.

## The experiment

The base model is strong on a directed question and on short SG paths. Earlier screening
showed it ACING short 2-3 hop SG paths (hub-pivot, flat mesh = 7.0) under a generic prompt,
but MISSING a 4-hop bastion chain (2.33, needle pass 0) and OVER-FLAGGING clean segmented
fleets (2.33). This harness is scoped ENTIRELY to that empirically-located weak region: the
**VOLUME + GENERIC-PROMPT + LONG-NEEDLE / DECEPTIVE-CLEAN** condition. A cold agent gets the
full `describe-security-groups` + `describe-instances` JSON for a 10-13 SG fleet and a
GENERIC "review this for problems" prompt that does **not** name lateral movement,
reachability, multi-hop chains, or the crown-jewel path. The question is whether it composes
the quiet LONG (4-6 hop) SG-reference needle out of the haystack unprompted — and, on the
deceptive-clean fleets, whether it correctly stays quiet instead of fabricating an
internet->db path that the segmentation actually neutralises.

This harness leans hard on the second failure mode. Four of the seven fixtures are
**deceptive-clean**: a deep chain that is orphaned because the front tier accepts an internal
CIDR rather than the public ALB SG (`01`), a loud public ALB that nothing SG-references
(`02`), two disjoint islands with the data island VPN-only (`03`), and a broken-segment fleet
where the chain is cut one hop in (`04`). On all four the engine returns CLEAN, and the cold
agent is expected to FABRICATE a critical path. The other three are **buried-deep needles**
(5-6 hops) with NO short/obvious public-to-db hop.

There are **no short, obvious 2-3 hop direct paths** here, and **no fixture where the issue
is a single obvious 0.0.0.0/0 -> db rule** — the base model already aces those, so they would
lift the aggregate above the screening threshold and defeat the purpose.

| Fixture | Verdict | Needle / why clean |
|---|---|---|
| `01-orphaned-front-internal-cidr` | CLEAN | deep chain orphaned (web accepts the internal mesh CIDR, not the ALB SG) |
| `02-public-alb-no-sg-ref` | CLEAN | loud public ALB intended; nothing references the ALB SG |
| `03-disjoint-public-vpn-islands` | CLEAN | two disconnected islands; data island is VPN-only |
| `04-broken-segment-midchain` | CLEAN | visible chain cut one hop in (web accepts the mesh CIDR, not the edge SG) |
| `05-six-hop-cdn-waf-gw-app-svc-db` | P1 + B1 + H1 | internet -> cdn -> waf -> gw -> app -> svc -> db (6 hops) |
| `06-compromised-ci-runner-deep` | P1 + B1 | ci -> build -> artifact -> deploy -> app -> db (5 hops, compromised-host entry) |
| `07-five-hop-ingress-mesh-broker-db` | P1 + B1 + H1 | internet -> ingress -> mesh -> app -> broker -> db (5 hops) |

Mix: 4 deceptive/segmented-clean, 3 buried-deep needles. Each fixture is a high-volume fleet
(10-13 SGs).

## Run

```bash
export ANTHROPIC_API_KEY=...
pip install anthropic
python tests/eval/run_eval.py --trials 3                 # full screening (~42 LLM calls)
python tests/eval/run_eval.py --trials 1 --fixtures 01,02 # smoke test
python tests/eval/run_eval.py --trials 3 --fresh          # ignore prior results
```

Defaults: `--trials 3`, agent + judge `claude-sonnet-4-6`, results in `eval_results.json`.
Each trial is persisted atomically; re-run the same command to resume after an interrupt.

## Ground truth offline (no key)

```bash
python tests/eval/scenarios.py   # prints needle path + hop count / clean / blast radius per fixture
```

The judge is anchored to `scenarios.expected_deep()`, which runs the reused deterministic
engine. The two load-bearing rubric items are **item 2** (surfaces the buried long needle as
a primary finding, hops named end to end, not only the obvious surface items) and **item 3**
(does not over-flag the benign / neutralised bait — the orphaned deep chain in `01`, the
intended public ALB in `02`, the disconnected data island in `03`, the broken chain in `04`).
See `rubric.md` and `judge_prompt.md`.

## Verdict (from control means only)

- aggregate `< 4.0/7` or a majority of fixtures `< 4.0` -> **BUILD**
- aggregate `< 5.5/7` -> **MAYBE**
- otherwise -> **SKIP**
      • rubric.md 4.7 KB 
        # Rubric: sg-deceptive-reachability-auditor (control-only screening)

Seven binary items. Each is 1 (pass) or 0 (fail); no partial credit. The judge grades the
cold agent's audit of a high-volume fleet (10-13 security groups) against the deterministic
ground truth from the reused reachability engine (`_deep.py` -> `_reach_engine.py`).

The agent was given a GENERIC prompt: the raw `describe-security-groups` +
`describe-instances` JSON for the whole fleet, the named entry point and crown-jewel tier,
and "review this for security/risk problems, rank by severity, give the top fix, and say
what you cannot determine." The prompt did NOT name lateral movement, reachability, the
SG-to-SG chain, multi-hop depth, or the crown-jewel path. The screening question is whether
the cold agent composes the non-obvious LONG (4-6 hop) needle in the haystack unprompted, and
whether it stays quiet on the DECEPTIVE/segmented-clean fleets instead of fabricating a path.

Every fixture is from the model's empirically-located WEAK region: there are NO short,
obvious 2-3 hop direct paths and NO single obvious 0.0.0.0/0 -> db rule (those the base model
already aces). Three fixtures are 5-6 hop needles; four are deceptive-clean fleets that bait
with a loud public CIDR or a visible-but-orphaned / broken deep SG-ref chain that does NOT
reach the crown jewel.

1. **Parse.** Recognises `UserIdGroupPairs` as SG-to-SG edges (SG B accepting SG A means a
   host in A can reach B), reads `0.0.0.0/0` / `::/0` as the internet entry (or the named
   compromised host as the entry, fixture `06`), reads the instance->SG membership map, and
   does not skip the SG-reference arrays in the volume.

2. **Surfaces the buried non-obvious issue (LOAD-BEARING).** On a needle fixture, the agent
   surfaces the buried LONG (4-6 hop) SG-to-SG reachable path to the crown jewel as A (the)
   PRIMARY finding, with the hops named end to end, rather than only listing the obvious
   surface items (a public `0.0.0.0/0` rule, a broad port range, a single hop) and stopping.
   On a deceptive-clean fixture, it correctly reports no real reachable path to the crown
   jewel. This is the item the cold agent most often misses: in a 10-13 SG haystack it tends
   to flag the loud CIDR exposure, clear each rule locally, and never compose the quiet long
   SG-ref chain.

3. **Does not over-flag the benign / neutralised bait (LOAD-BEARING).** Does not assert a
   path or exposure the ground truth lacks. Specifically: does NOT report the ORPHANED deep
   chain in `01` as a reachable path; does NOT call the public ALB's intended `0.0.0.0/0` in
   `02` the lateral path / the headline; does NOT splice the disconnected data island in `03`
   into a manufactured internet->db route; does NOT treat the BROKEN chain in `04` as
   reachable across the cut; does NOT manufacture any internet->db route on a clean fleet; and
   does NOT drown the real finding (or the clean verdict) in a wall of low-value nitpicks
   about correctly-scoped tiers (bastion, monitoring, ci, ssm).

4. **Composition / multi-hop (LONG).** Traces the SG-to-SG edges into a transitive LONG
   (4-6 hop) path rather than clearing each rule / SG in isolation. The agent must assemble
   entry -> ... -> crown jewel across many separate SGs end to end, not list each SG as
   locally fine and stop, and not stop at a single hop. On the deceptive-clean fleets, the
   same composition discipline is what reveals the chain is orphaned / broken / cross-island,
   so the agent must reason about reachability, not just rule locality.

5. **Criticality.** Ranks the long path that reaches the crown jewel as the headline
   (critical), the blast radius as high. Does NOT headline the directly-internet-facing
   front-door tier, nor (in `02`/`04`) the loud public-ALB / edge-proxy rule, over the quiet
   real long path. On a deceptive-clean fixture, does not invent a critical.

6. **Boundary.** Names at least one thing it cannot determine from the SG graph + membership
   alone, matching the ground-truth join: reachability is not exploitability -- live SG
   membership / running hosts, route tables, NACLs, or app-layer auth.

7. **Recommendation.** The top fix matches the ground truth in substance: break the
   offending edge/hop on the long chain (or interpose a broker/bastion), or (on a
   deceptive-clean fixture) no change beyond confirming the boundary — explicitly NOT
   "fix" a path that does not exist.

## Verdict (computed from CONTROL means only)

- Aggregate control mean **< 4.0/7**, or a **majority** of fixtures below 4.0 -> **BUILD**
  (cold agent is weak here; the skill is worth writing).
- Aggregate **< 5.5/7** -> **MAYBE** (mixed; inspect per-fixture, especially items 2 and 4).
- Otherwise -> **SKIP** (cold agent already strong; the skill adds little).
      • run_eval.py 19.8 KB 
        """
Lift eval for the sg-deceptive-reachability-auditor skill (control + treatment arms).

This started as a SCREENING harness (control only); now that SKILL.md exists, it also runs a
TREATMENT arm (same fixture, the agent additionally given SKILL.md as the methodology to
follow) and reports the lift = treatment - control per fixture. The control cells from the
original screening run are reused as-is (resume by (fixture, condition, trial)), so a lift
run only pays for the treatment trials.

Before writing a SKILL.md, the screening wanted to know
whether a cold agent (no skill, just domain expertise) finds a DEEP, buried multi-hop
SG-to-SG lateral path in a HIGH-VOLUME fleet when asked a GENERIC question -- and whether it
correctly reports NOTHING on deceptive/segmented-clean fleets. The thing under test is the
VOLUME + GENERIC-PROMPT + LONG-NEEDLE / DECEPTIVE-CLEAN condition, scoped ENTIRELY to the
model's empirically-located weak region: ~10-14 security groups where most rules are
ordinary and fine, and the real issue (when there is one) is a LONG 4-6 hop quiet
SG-reference chain from an internet-facing (or compromised-host) tier to a crown-jewel DB.
There are NO short, obvious 2-3 hop direct paths here; earlier screening showed the base
model ACING those (7.0) but MISSING a 4-hop bastion chain (2.33) and OVER-FLAGGING clean
segmented fleets (2.33). This harness replicates only that hard region.

So this runner only runs the CONTROL condition: for each fixture, N trials of a cold agent
given the raw describe-security-groups + describe-instances JSON for the WHOLE fleet and a
GENERIC "review this for security/risk problems" prompt that does NOT name lateral movement,
chains, reachability, the SG-to-SG path, or the crown-jewel target. Each output is graded
against the 7-item rubric (rubric.md) by an LLM judge, anchored to the deterministic
reference result (_deep.py, via scenarios.py) as ground truth.

The treatment arm reads SKILL.md (repo root) and prepends it as the methodology. The judge
is condition-agnostic: it grades any output against the same deterministic ground truth, so
control and treatment are scored on identical terms. print_summary reports control mean,
treatment mean, and lift per fixture.

Requirements:
- ANTHROPIC_API_KEY environment variable.
- `pip install anthropic` (the only non-stdlib dependency in the repo; isolated to tests/eval/).

Usage:
    python tests/eval/run_eval.py --trials 3
    python tests/eval/run_eval.py --trials 1 --fixtures 05,06   # smoke test
    python tests/eval/run_eval.py --trials 3 --fresh            # ignore prior results

Resume: each completed trial is persisted immediately (atomic temp+rename), and a re-run
reloads what is already on disk and fills only the missing (fixture, trial) cells. A crash,
an interrupt, or an API overload mid-run therefore never throws away completed work -- just
re-run the same command to finish. Pass --fresh to start clean.

Cost note: 7 fixtures x 3 trials, plus a judge call per output, is ~21 agent calls +
~21 judge calls = ~42 LLM calls. Expect a few dollars on Sonnet.
"""

from __future__ import annotations

import argparse
import json
import os
import statistics
import sys
import time
from pathlib import Path

try:
    from anthropic import Anthropic
except ImportError:
    print("ERROR: anthropic SDK not installed. Run: pip install anthropic", file=sys.stderr)
    sys.exit(1)

sys.path.insert(0, str(Path(__file__).parent))
from scenarios import SCENARIOS, load_fixture_text, expected_deep  # noqa: E402

DEFAULT_AGENT_MODEL = os.environ.get("EVAL_AGENT_MODEL", "claude-sonnet-4-6")
DEFAULT_JUDGE_MODEL = os.environ.get("EVAL_JUDGE_MODEL", "claude-sonnet-4-6")
MAX_TOKENS = 4096

# SKILL.md lives at the skill root (three levels up from tests/eval/).
SKILL_MD = Path(__file__).resolve().parents[2] / "SKILL.md"


def build_control_prompt(scenario: dict) -> str:
    """A GENERIC cloud-security-engineer prompt: the raw full-fleet SG + instance JSON and
    a plain "review this for problems" ask. No skill.

    This is the whole point of the screening: the prompt deliberately does NOT name lateral
    movement, reachability, the SG-to-SG reference graph, the transitive closure, the
    crown-jewel path, multi-hop chains, or the depth of any path. It does not even tell the
    agent to look for a path. It asks the agent to review the configuration, rank issues by
    severity, give the single most important fix, and state what it cannot determine. Whether
    the cold agent composes the buried LONG (4-6 hop) SG-ref needle out of a 10-14 SG haystack
    -- rather than fixating on the obvious public-CIDR exposure and stopping -- and whether it
    correctly stays quiet on the deceptive/segmented-clean fleets, is exactly what we measure.
    """
    return f"""You are a cloud security engineer doing a review of an AWS account's network configuration.

Context: {scenario['context']}

Here is the configuration, exactly as returned by the EC2 API:

{load_fixture_text(scenario)}

Review this for security and risk problems. Rank what you find by severity, give the single
most important fix, and state clearly what you cannot determine from this configuration alone.
Be specific and concrete about anything you flag."""


def build_treatment_prompt(scenario: dict, skill_md_text: str) -> str:
    """The treatment arm: the SAME generic review ask and the SAME fixture JSON as control,
    plus SKILL.md prepended as the methodology to follow. The ask is left identical so the
    only variable between the arms is the skill; the lift attributable to SKILL.md is what we
    measure. The agent is told to apply the methodology, not merely to read it."""
    return f"""You are a cloud security engineer doing a review of an AWS account's network configuration. Apply the methodology below to the configuration that follows it.

==== METHODOLOGY (SKILL.md) ====
{skill_md_text}
==== END METHODOLOGY ====

Context: {scenario['context']}

Here is the configuration, exactly as returned by the EC2 API:

{load_fixture_text(scenario)}

Apply the methodology above. Rank what you find by severity, give the single most important
fix, and state clearly what you cannot determine from this configuration alone. Be specific
and concrete about anything you flag."""


JUDGE_SYSTEM ="""You are an expert AWS / cloud-security evaluator grading a fleet security review against a 7-item rubric. Each item is binary: 1 (pass) or 0 (fail). Be strict but fair; no partial credit.

You will be given a known-good answer from a deterministic reference audit, the agent's review output, and the 7 rubric items. The agent was given a GENERIC prompt ("review this for problems") that did NOT name lateral movement, reachability, multi-hop chains, or the crown-jewel path -- reward the agent only for what it surfaces on its own, and grade substance, not phrasing.

Return JSON only (no prose), with this exact schema:

{
  "scores": [<int>, <int>, <int>, <int>, <int>, <int>, <int>],
  "notes": ["<one sentence>", ...]
}"""


def build_judge_prompt(scenario: dict, agent_output: str) -> str:
    exp = expected_deep(scenario)
    path = " -> ".join(exp["shortest_path"]) if exp["shortest_path"] else "(no path to the crown jewel)"
    hops = len(exp["shortest_path"]) - 1 if exp["shortest_path"] else 0
    return f"""FIXTURE: {scenario['id']}
CONTEXT (what the agent saw): {scenario['context']}

KNOWN-GOOD ANSWER (from the deterministic reference reachability engine):
- Findings (ground truth): {exp['codes']}  (top severity: {exp['top_severity']}; clean: {exp['clean']}; SGs: {exp['sg_count']}; SG-to-SG edges: {exp['edge_count']})
- Shortest path entry -> crown jewel: {path}  ({hops} hops)
- Blast radius (reachable tiers from the entry): {exp['blast_radius']}
- What the defect actually is: {scenario['expected_headline']}
- Correct top fix: {scenario['expected_top_fix']}
- Boundary the audit cannot cross: {scenario['expected_boundary_join']}

AGENT REVIEW OUTPUT:
{agent_output}

RUBRIC (score each 1 = pass, 0 = fail):
1. Parse: recognises UserIdGroupPairs as SG-to-SG edges (B accepting A means A can reach B), reads 0.0.0.0/0 / ::/0 as the internet entry (or the named compromised host as the entry), reads instance->SG membership, and does not skip the SG-reference arrays in the volume.
2. Surfaces the buried non-obvious issue (LOAD-BEARING): on a needle fixture, surfaces the buried LONG (4-6 hop) SG-to-SG reachable path to the crown jewel as A (the) PRIMARY finding with the hops named end to end, rather than only listing the obvious surface items (a public 0.0.0.0/0, a broad port range, a single hop) and stopping; on a clean fixture, correctly reports no real reachable path to the crown jewel.
3. Does not over-flag the benign / neutralised bait (LOAD-BEARING): does not assert a path/exposure the ground truth lacks -- not the ORPHANED deep chain in 05 as reachable, not the intended public-ALB 0.0.0.0/0 in 06 as the lateral path or headline, not the disconnected data island in 07 spliced into an internet->db route, no manufactured route on any clean fleet; and does not drown the real finding (or the clean verdict) in nitpicks about correctly-scoped tiers (bastion, monitoring, ci, ssm).
4. Composition / multi-hop (LONG): traces the SG-to-SG edges into a transitive LONG (4-6 hop) path rather than clearing each rule/SG in isolation; assembles entry -> ... -> crown jewel across many SGs end to end, not "each SG is locally fine" and stop, and not just a single hop.
5. Criticality: ranks the long path that reaches the crown jewel as the headline (critical), the blast radius as high; does not headline the directly-internet-facing front-door tier or (in 06) the loud public-ALB rule over the quiet real long path; does not invent a critical on a clean fixture.
6. Boundary: names at least one thing it cannot determine from the SG graph + membership alone, matching the ground-truth join (reachability is not exploitability: live SG membership / running hosts, route tables, NACLs, app-layer auth).
7. Recommendation: top fix matches the ground-truth fix in substance (break the offending edge/hop on the long chain, interpose a broker, or no change on a clean fixture beyond confirming the boundary).

Return JSON only."""


RETRYABLE_STATUS = {408, 409, 429, 500, 502, 503, 529}
MAX_RETRIES = 6


def _with_retries(fn, *args, **kwargs):
    """Call fn with exponential backoff on transient API errors (429/5xx/529/overloaded).

    The Anthropic SDK already retries a couple of times; this widens the window so a
    multi-minute overload spell drops far fewer trials. Re-raises on non-retryable errors
    or once retries are exhausted.

    Returns (result, call_seconds) where call_seconds is the wall-time of the SUCCESSFUL
    attempt only -- backoff sleeps and failed attempts are excluded.
    """
    delay = 2.0
    last_exc = None
    for attempt in range(MAX_RETRIES):
        try:
            t_call = time.time()
            return fn(*args, **kwargs), time.time() - t_call
        except Exception as e:  # noqa: BLE001 - inspect, then decide retryable
            status = getattr(e, "status_code", None)
            msg = str(e).lower()
            retryable = status in RETRYABLE_STATUS or "overloaded" in msg or "rate" in msg or "timeout" in msg
            if not retryable:
                raise
            last_exc = e
            if attempt < MAX_RETRIES - 1:
                time.sleep(delay)
                delay = min(delay * 2, 60.0)
    raise last_exc


def run_agent(client: Anthropic, model: str, prompt: str) -> tuple[str, float]:
    """Returns (agent_output_text, review_seconds). Seconds excludes retry backoff."""
    def _call():
        return client.messages.create(
            model=model,
            max_tokens=MAX_TOKENS,
            messages=[{"role": "user", "content": prompt}],
        )
    resp, call_s = _with_retries(_call)
    return "".join(block.text for block in resp.content if block.type == "text"), call_s


def run_judge(client: Anthropic, model: str, scenario: dict, agent_output: str) -> dict:
    def _call():
        return client.messages.create(
            model=model,
            max_tokens=1024,
            system=JUDGE_SYSTEM,
            messages=[{"role": "user", "content": build_judge_prompt(scenario, agent_output)}],
        )
    resp, _ = _with_retries(_call)
    raw = "".join(block.text for block in resp.content if block.type == "text").strip()
    if raw.startswith("```"):
        raw = raw.split("```", 2)[1]
        if raw.startswith("json"):
            raw = raw[4:]
        raw = raw.rsplit("```", 1)[0]
    return json.loads(raw.strip())


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--trials", type=int, default=3, help="Trials per fixture (control only)")
    parser.add_argument("--fixtures", default="", help="Comma-separated fixture IDs (prefix match); empty = all")
    parser.add_argument("--agent-model", default=DEFAULT_AGENT_MODEL)
    parser.add_argument("--judge-model", default=DEFAULT_JUDGE_MODEL)
    parser.add_argument("--output", default="eval_results.json", help="Where to write the raw results")
    parser.add_argument("--fresh", action="store_true", help="Ignore an existing results file and start clean (default: resume/fill gaps)")
    # Which arms to run. Default is treatment only, because the control cells are already on
    # disk from the screening run and are reused as-is -- a lift run should not re-pay for them.
    # Pass --conditions control,treatment to (re)run both.
    parser.add_argument("--conditions", default="treatment",
                        help="Comma-separated arms to run: control, treatment, or both (default: treatment)")
    args = parser.parse_args()

    conditions = [c.strip() for c in args.conditions.split(",") if c.strip()]
    bad = [c for c in conditions if c not in ("control", "treatment")]
    if bad:
        print(f"ERROR: unknown condition(s) {bad}; valid: control, treatment", file=sys.stderr)
        return 2

    if "ANTHROPIC_API_KEY" not in os.environ:
        print("ERROR: ANTHROPIC_API_KEY not set", file=sys.stderr)
        return 1

    skill_md_text = ""
    if "treatment" in conditions:
        if not SKILL_MD.exists():
            print(f"ERROR: treatment arm needs a SKILL.md at {SKILL_MD}", file=sys.stderr)
            return 1
        skill_md_text = SKILL_MD.read_text()

    client = Anthropic()

    to_run = SCENARIOS
    if args.fixtures:
        filters = [f.strip() for f in args.fixtures.split(",")]
        to_run = [s for s in SCENARIOS if any(s["id"].startswith(f) for f in filters)]

    n_cells = len(to_run) * len(conditions) * args.trials
    print(f"LIFT eval [{', '.join(conditions)}]: {len(to_run)} fixtures x {len(conditions)} conditions x {args.trials} trials = {n_cells} agent calls")
    print(f"Agent model: {args.agent_model}, Judge model: {args.judge_model}\n")

    # Resume: reload any completed trials from a prior run so a re-run fills ONLY the gaps
    # (e.g. trials dropped to a transient overload, or the control arm from screening),
    # never redoing finished work. Pass --fresh to ignore an existing results file.
    results: list[dict] = []
    completed: set[tuple[str, str, int]] = set()
    out_path = Path(args.output)
    if out_path.exists() and not args.fresh:
        try:
            results = json.loads(out_path.read_text())
            completed = {(r["fixture"], r.get("condition", "control"), r["trial"]) for r in results}
            print(f"Resuming from {args.output}: {len(completed)} trials already complete; filling gaps only.\n")
        except (json.JSONDecodeError, KeyError, OSError):
            results, completed = [], set()

    for scenario in to_run:
        for condition in conditions:
            for trial in range(args.trials):
                if (scenario["id"], condition, trial) in completed:
                    continue  # already have this cell from a prior run
                t_start = time.time()
                prompt = (build_treatment_prompt(scenario, skill_md_text) if condition == "treatment"
                          else build_control_prompt(scenario))
                try:
                    agent_output, agent_s = run_agent(client, args.agent_model, prompt)  # agent_s excludes retry backoff
                    judge_result = run_judge(client, args.judge_model, scenario, agent_output)
                    score = sum(judge_result["scores"])
                except Exception as e:
                    print(f"  ERROR on {scenario['id']} [{condition}] trial {trial}: {e}", file=sys.stderr)
                    continue
                elapsed = time.time() - t_start  # agent + judge, for cost/wall-clock accounting
                results.append({
                    "fixture": scenario["id"],
                    "condition": condition,
                    "trial": trial,
                    "score": score,
                    "scores_by_item": judge_result["scores"],
                    "notes": judge_result.get("notes", []),
                    "agent_output": agent_output,
                    "agent_chars": len(agent_output),
                    "agent_s": round(agent_s, 1),
                    "elapsed_s": round(elapsed, 1),
                })
                # Crash-safe: persist after every trial via atomic temp+rename so an
                # overload-induced death never throws away completed work.
                tmp = Path(str(args.output) + ".tmp")
                tmp.write_text(json.dumps(results, indent=2))
                tmp.replace(args.output)
                print(f"  {scenario['id']:<34} | {condition:<9} | trial {trial} | score {score}/7 | review {agent_s:.0f}s", flush=True)

    Path(args.output).write_text(json.dumps(results, indent=2))
    print(f"\nRaw results: {args.output}\n")
    print_summary(results, to_run)
    return 0


def print_summary(results: list[dict], to_run: list[dict]) -> None:
    ctrl: dict[str, list[int]] = {}
    treat: dict[str, list[int]] = {}
    for r in results:
        bucket = ctrl if r.get("condition") == "control" else treat
        bucket.setdefault(r["fixture"], []).append(r["score"])

    print(f"{'Fixture':<34} {'Control':>8} {'Treat':>8} {'Lift':>8} {'Nc':>4} {'Nt':>4}")
    print("-" * 74)
    c_means: list[float] = []
    t_means: list[float] = []
    lifts: list[float] = []
    for scenario in to_run:
        cs = ctrl.get(scenario["id"], [])
        ts = treat.get(scenario["id"], [])
        if not cs and not ts:
            continue
        c = statistics.mean(cs) if cs else float("nan")
        t = statistics.mean(ts) if ts else float("nan")
        c_str = f"{c:>8.2f}" if cs else f"{'n/a':>8}"
        t_str = f"{t:>8.2f}" if ts else f"{'n/a':>8}"
        if cs and ts:
            lift = t - c
            lifts.append(lift)
            c_means.append(c)
            t_means.append(t)
            l_str = f"{lift:>+8.2f}"
            flag = "  <- treat still <5" if t < 5.0 else ("  <- no lift" if lift <= 0 else "")
        else:
            l_str = f"{'-':>8}"
            flag = ""
        print(f"{scenario['id']:<34} {c_str} {t_str} {l_str} {len(cs):>4} {len(ts):>4}{flag}")
    print("-" * 74)

    if not lifts:
        print("\nNo paired control/treatment fixtures to summarize "
              f"(control: {sum(len(v) for v in ctrl.values())} cells, "
              f"treatment: {sum(len(v) for v in treat.values())} cells).")
        return

    c_agg = statistics.mean(c_means)
    t_agg = statistics.mean(t_means)
    print(f"\nAggregate: control {c_agg:.2f}/7  ->  treatment {t_agg:.2f}/7   (lift {t_agg - c_agg:+.2f})")
    print(f"  Fixtures improved: {sum(1 for l in lifts if l > 0)} / {len(lifts)};  "
          f"treatment >= 6/7: {sum(1 for t in t_means if t >= 6.0)} / {len(t_means)};  "
          f"treatment >= 5/7: {sum(1 for t in t_means if t >= 5.0)} / {len(t_means)}")
    weakest = min(zip(t_means, [s['id'] for s in to_run if ctrl.get(s['id']) and treat.get(s['id'])]))
    print(f"  Weakest treatment fixture: {weakest[1]} at {weakest[0]:.2f}/7 "
          "(the next one to close with a SKILL.md edit)")


if __name__ == "__main__":
    sys.exit(main())
      • scenarios.py 13.2 KB 
        """
Per-fixture contexts and expected answers for the sg-deceptive-reachability-auditor
screening eval.

The "expected_*" fields are the deterministic answers from the reused engine (_deep.py,
which delegates to the validated _reach_engine.py) run against each high-volume fleet
fixture. They are the source of truth the LLM judge compares the agent's output against,
so the findings are computed here by importing the engine rather than hand-copied (which
would drift).

Every fixture here is from the model's empirically-located WEAK region: LONG (4-6 hop)
lateral chains buried in a 10-13 SG fleet, plus DECEPTIVE / segmented-clean fleets where a
loud public exposure or a visible-but-orphaned deep SG-ref chain must NOT be reported as a
reachable path. There are NO short, obvious 2-3 hop direct paths -- those the base model
already aces, so they would lift the aggregate above the screening threshold and defeat the
purpose. The skill's value region IS the hard cases: four deceptive-clean fleets where the
cold agent tends to FABRICATE a critical path, and three buried-deep needles where it tends
to flag the loud surface and stop.

load_fixture_text renders the FULL volume the agent sees: every security group, every
instance, and the named entry + crown jewel -- a 10-13 SG haystack per fixture. The
control prompt (in run_eval.py) is deliberately GENERIC and does NOT name lateral movement,
chains, or reachability; these expected fields exist only for the judge, never for the agent.

Stdlib only. No external dependencies. `python scenarios.py` prints ground truth, no key.
"""

from __future__ import annotations

import json
import sys
from pathlib import Path

TESTS_DIR = Path(__file__).resolve().parent.parent
FIXTURES_DIR = TESTS_DIR.parent / "fixtures"

sys.path.insert(0, str(TESTS_DIR))
from _deep import run_deep  # noqa: E402

# Each entry pairs a fixture with the human-readable context the eval feeds the agent
# (generic -- a fleet description, no vector hint), plus the headline / fix / boundary the
# deterministic engine grounds (the judge's anchor only). Keep aligned with replay_*.py.
SCENARIOS = [
    {
        "id": "01-orphaned-front-internal-cidr",
        "context": "An 11-security-group production fleet. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "CLEAN. The config SHOWS a deep SG-reference chain web -> app -> session cache -> db, which looks like a buried lateral path, but the web frontend accepts only the internal service-mesh CIDR (10.50.0.0/16), NOT the public ALB SG, so that chain is ORPHANED from the internet entry. The internet reaches only the directly-exposed public ALB and stops. The database is NOT reachable. Do not mistake the visible-but-disconnected chain for a reachable path, and do not over-flag the busy but correctly-scoped ruleset.",
        "expected_top_fix": "None. Do not invent a lateral path from the orphaned deep chain. The segmentation is correct. Still report the boundary (route tables, NACLs, live membership) the SG graph cannot confirm.",
        "expected_boundary_join": "the SG graph alone cannot confirm even the bounded reach is live (route tables, NACLs, instance membership); a deep chain that is disconnected on paper is still only on paper.",
    },
    {
        "id": "02-public-alb-no-sg-ref",
        "context": "An 11-security-group production fleet. A public ALB carries the only 0.0.0.0/0 (and ::/0) ingress on 443/80. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "CLEAN. The loud 0.0.0.0/0 on the public ALB is the INTENDED public entry (that is what a public load balancer is for). A deep service chain web -> app -> internal services -> db is wired downstream, but NOTHING references the public ALB SG (it forwards to its target group by registered IP), so no SG-to-SG edge composes from the internet into the fleet. The database is NOT reachable. Do not headline the intended public ALB as a lateral chain and do not over-flag the busy but correctly-scoped ruleset.",
        "expected_top_fix": "None. The fleet is segmented and the public ALB exposure is intended. Do not invent a lateral path from the orphaned downstream chain. Still report the boundary (route tables, NACLs, live membership) the SG graph cannot confirm.",
        "expected_boundary_join": "the SG graph alone cannot confirm the bounded reach is live (route tables, NACLs, instance membership); the public ALB is a path to nothing further without an SG reference.",
    },
    {
        "id": "03-disjoint-public-vpn-islands",
        "context": "An 11-security-group production fleet across two AZs. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "CLEAN. The fleet is two disconnected islands. The internet-facing island (public ALB -> web -> web-app) is wired by CIDR, not SG refs, and dead-ends. The data island carries a juicy-looking deep SG-ref chain admin plane -> data processor -> db (+ analytics cache), but the admin plane accepts only the corp VPN CIDR and has no inbound SG reference, so the data island is reachable only from the VPN, never from the internet. No edge joins the islands. The crown-jewel database is NOT reachable from the internet. Do not splice the two islands into a manufactured internet -> db route.",
        "expected_top_fix": "None. The two islands are correctly disconnected and the data island is VPN-only. Do not invent a cross-island lateral path. Still report the boundary (route tables, NACLs, live membership) the SG graph cannot confirm.",
        "expected_boundary_join": "the SG graph alone cannot confirm the bounded reach is live (route tables, NACLs, instance membership); the data island's deep chain is reach-on-paper and is not joined to any internet entry.",
    },
    {
        "id": "04-broken-segment-midchain",
        "context": "A 10-security-group production fleet. A public edge proxy carries the only 0.0.0.0/0 (and ::/0) ingress on 443. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "CLEAN. A long service chain is VISIBLE (public edge proxy -> web -> app -> session cache -> db) and the edge proxy is the intended public entry, but the chain is CUT at the first internal hop: the web frontend accepts only the internal mesh CIDR (10.70.0.0/16), NOT the edge proxy SG, so the edge -> web link is not an SG edge and the rest of the chain is orphaned. The internet reaches only the directly-exposed edge proxy and stops. The database is NOT reachable. Do not mistake the visible-but-broken chain for a reachable path, and do not over-flag the busy but correctly-scoped ruleset.",
        "expected_top_fix": "None. The chain is broken at the edge -> web hop and the public edge exposure is intended. Do not invent a lateral path across the cut. Still report the boundary (route tables, NACLs, live membership) the SG graph cannot confirm.",
        "expected_boundary_join": "the SG graph alone cannot confirm the bounded reach is live (route tables, NACLs, instance membership); the orphaned tail of the chain is reach-on-paper and is not joined to the public edge proxy.",
    },
    {
        "id": "05-six-hop-cdn-waf-gw-app-svc-db",
        "context": "A 13-security-group production fleet. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "Buried in the 13-SG fleet, the SG-to-SG edges compose into a SIX-hop reachable path internet -> cdn -> waf -> gw -> app -> svc -> db to the crown-jewel database -- the deepest chain in the set, with the billing-service settlement-writer as the final link. Each rule accepts exactly one upstream tier and is textbook in isolation; the depth is the point. The scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue, logs) are not the finding.",
        "expected_top_fix": "Confirm each of the six hops is an intended trust relationship; the database should not be transitively reachable from the public CDN origin. Break the chain at the hop that crosses a trust boundary (e.g. the svc->db settlement writer or gw->app).",
        "expected_boundary_join": "live membership of each SG on the six-hop chain, plus route tables / NACLs / app-auth along the path; a long chain is reach-on-paper.",
    },
    {
        "id": "06-compromised-ci-runner-deep",
        "context": "A 12-security-group production fleet. The entry point is a COMPROMISED CI runner host (instance i-06ci0001), not the internet. The crown jewel is the database tier.",
        "expected_headline": "From the compromised CI runner, the SG-to-SG edges compose into a FIVE-hop internal build-pipeline path ci -> build -> artifact -> deploy -> app -> db to the crown-jewel database. The separate public ALB -> web -> web-cache front door does not touch the data plane, so the only path to the crown jewel is this internal pipeline chain. The depth + the non-internet entry are the point; a per-rule read clears each pipeline rule in isolation and misses the composed path.",
        "expected_top_fix": "Confirm each hop from the CI runner is intended; the database should not be transitively reachable from a compromised build host five hops away. Break the chain at the hop that crosses a trust boundary (e.g. deploy->app or app->db).",
        "expected_boundary_join": "live membership of each SG on the chain, plus route tables / NACLs / app-auth along the path; reachability from a foothold is reach-on-paper until confirmed.",
    },
    {
        "id": "07-five-hop-ingress-mesh-broker-db",
        "context": "A 12-security-group production fleet. Entry is the internet; the crown jewel is the database tier.",
        "expected_headline": "Buried in the 12-SG fleet, the SG-to-SG edges compose into a FIVE-hop reachable path internet -> ingress -> mesh -> app -> broker -> db to the crown-jewel database, with the event-broker journal-writer as the offbeat final link. Each tier accepts exactly one upstream and is fine in isolation; the chain only appears when all five hops are composed. The service-fed leaves (cache, queue, logs) and scoped tiers (bastion, monitoring, ci, ssm) are not the finding.",
        "expected_top_fix": "Break the chain at the hop that should not exist; the database should not be transitively reachable from the public ingress controller five hops away. Confirm each SG-reference edge along the path is an intended trust relationship.",
        "expected_boundary_join": "which instances are live members of each referenced SG along the five hops, plus route tables / NACLs / app-auth -- reachability-on-paper is not exploitability.",
    },
]


def fixture_dir(scenario: dict) -> Path:
    return FIXTURES_DIR / scenario["id"]


def load_fixture_text(scenario: dict) -> str:
    """The raw security-group + instance JSON the agent is given for this scenario.

    Renders the FULL volume: every SG and every instance in the fleet, so the agent
    genuinely sees the 10-13 SG haystack (not a pre-filtered slice). The named entry
    point and crown-jewel tier come from meta.json. NOTE: the agent prompt itself
    (run_eval.py) is generic and does not mention lateral movement / chains / reachability.
    """
    d = fixture_dir(scenario)
    sgs = json.loads((d / "security-groups.json").read_text())
    parts = ["aws ec2 describe-security-groups output:", json.dumps(sgs, indent=2)]
    inst_path = d / "instances.json"
    if inst_path.exists():
        instances = json.loads(inst_path.read_text())
        parts += ["", "aws ec2 describe-instances output (instance -> SG membership):", json.dumps(instances, indent=2)]
    meta_path = d / "meta.json"
    if meta_path.exists():
        meta = json.loads(meta_path.read_text())
        entry = meta.get("entry", "internet")
        crown = meta.get("crown_jewel", "(unspecified)")
        parts += ["", f"Entry point: {entry}", f"Crown-jewel tier: {crown}"]
    return "\n".join(parts)


def expected_deep(scenario: dict) -> dict:
    """Run the reused deterministic engine to get the ground-truth findings for the judge.

    The engine computes ONE fleet-wide transitive closure over the whole SG set (the
    aggregation across all sub-items), yielding the long needle path + blast radius, or clean.
    """
    r = run_deep(fixture_dir(scenario))
    return {
        "codes": sorted(r.codes()),
        "top_severity": r.top_severity,
        "clean": r.clean,
        "shortest_path": r.shortest_path,
        "blast_radius": r.reachable,
        "edge_count": len(r.edges),
        "boundary_count": len(r.boundary),
        "sg_count": r.sg_count,
    }


# Alias kept for parity with the sibling harnesses' expected_reach() / expected_needle() naming.
expected_reach = expected_deep
expected_needle = expected_deep


if __name__ == "__main__":
    # `python tests/eval/scenarios.py` prints the ground-truth answers, no API key needed.
    for s in SCENARIOS:
        exp = expected_deep(s)
        path = " -> ".join(exp["shortest_path"]) if exp["shortest_path"] else "(no path)"
        hops = len(exp["shortest_path"]) - 1 if exp["shortest_path"] else 0
        print(f"{s['id']:<34} codes={str(exp['codes']):<20} top={str(exp['top_severity']):<8} "
              f"clean={exp['clean']!s:<5} sgs={exp['sg_count']} edges={exp['edge_count']}")
        print(f"{'':<34} path={path}  ({hops} hops)")
        print(f"{'':<34} blast_radius={exp['blast_radius']}")
    • README.md 3.1 KB 
      # Tests: sg-deceptive-reachability-auditor

Deterministic ground-truth + replay tests for the deceptive-reachability screening fixtures.
No API key, no network, stdlib only.

## Engine (reused, not re-derived)

- `_reach_engine.py` — a **verbatim byte-for-byte copy** of the validated engine from the
  sibling skill `lateral-movement-reachability-auditor/tests/_reach.py` (carried in via the
  `sg-deep-lateral-auditor` screening harness). The graph build, BFS transitive closure,
  shortest-path, and articulation-hub logic are unchanged, so the ground truth here is
  provably the same computation.
- `_deep.py` — thin wrapper that re-exports the engine and exposes
  `run_deep(fixture_dir) -> Reachability`. It does NOT change engine logic. The
  `Reachability` result exposes `.findings` / `.codes()` / `.top_severity` / `.clean` /
  `.boundary` / `.shortest_path` / `.reachable`. For this fleet/estate harness, the
  "aggregation across sub-items" is the single fleet-wide transitive closure the engine
  already computes over the whole SG set (every SG a node, every UserIdGroupPair an edge).

## Scope: the model's WEAK region only

Every fixture is from the empirically-located weak region: LONG (4-6 hop) lateral chains
buried in a 10-13 SG haystack, plus DECEPTIVE / segmented-clean fleets where a loud public
exposure or a visible-but-orphaned deep SG-ref chain must NOT be reported as a reachable
path. There are **no short, obvious 2-3 hop direct paths** and **no fixture where the issue
is a single obvious public-to-db rule** — the base model trivially nails those (7.0), and
including them would lift the screening aggregate above threshold and defeat the purpose.

## Replay tests

One per fixture; each asserts the ground-truth verdict (long needle path / deceptive-clean):

```bash
for f in tests/replay_*.py; do python3 "$f"; done
```

| Fixture | Verdict | Needle / why clean |
|---|---|---|
| `01-orphaned-front-internal-cidr` | CLEAN | deep web->app->cache->db chain orphaned (web accepts the internal mesh CIDR, not the ALB SG) |
| `02-public-alb-no-sg-ref` | CLEAN | loud public ALB intended; nothing references the ALB SG (forwards by IP) |
| `03-disjoint-public-vpn-islands` | CLEAN | two disconnected islands; the data island's deep chain is VPN-only |
| `04-broken-segment-midchain` | CLEAN | visible edge->web->app->cache->db chain cut one hop in (web accepts the mesh CIDR, not the edge SG) |
| `05-six-hop-cdn-waf-gw-app-svc-db` | P1 + B1 + H1 | internet -> cdn -> waf -> gw -> app -> svc -> db (6 hops) |
| `06-compromised-ci-runner-deep` | P1 + B1 | ci -> build -> artifact -> deploy -> app -> db (5 hops, compromised-host entry) |
| `07-five-hop-ingress-mesh-broker-db` | P1 + B1 + H1 | internet -> ingress -> mesh -> app -> broker -> db (5 hops) |

Mix: 4 deceptive/segmented-clean, 3 buried-deep needles. Each fixture is a high-volume fleet
(10-13 SGs).

If a fixture and the engine disagree, **fix the fixture, never the engine** — the engine is
the validated oracle.

## Eval

`tests/eval/` holds the control-vs-treatment lift eval that measures the `SKILL.md`. See `tests/eval/README.md`.
    • replay_01_orphaned_front_internal_cidr.py 1.9 KB 
      """
Replay test for fixtures/01-orphaned-front-internal-cidr.

A clean control with full fleet volume (11 SGs) and a DECEPTIVE deep chain. The config
shows a long SG-reference chain web -> app -> session cache -> db, which looks like a
buried lateral path. But the web frontend accepts only the internal service-mesh CIDR,
NOT the public ALB SG, so that chain is orphaned from the internet entry: the internet
reaches only the directly-exposed public ALB and stops. The database is NOT reachable.
The audit must report no path, must NOT mistake the visible-but-disconnected chain for a
reachable one, and must still name the boundary.

Stdlib only. Run with: `python tests/replay_01_orphaned_front_internal_cidr.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "01-orphaned-front-internal-cidr"


def main() -> int:
    r = run_deep(FIXTURE_DIR)

    assertions = [
        (r.clean, f"expected a clean audit, got {sorted(r.codes())}"),
        (r.top_severity is None, "a clean audit has no top severity"),
        (r.shortest_path == [], f"no path to the crown jewel should exist, got {r.shortest_path}"),
        ("db" not in r.reachable, f"the database must NOT be in the blast radius, got {r.reachable}"),
        # Only the directly-exposed public ALB is reachable; the deep chain is orphaned.
        (set(r.reachable) == {"lb"}, f"only the public ALB tier is reachable, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
        (len(r.boundary) >= 3, "even a clean graph reports the joins it cannot make"),
    ]
    return report("replay_01_orphaned_front_internal_cidr", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_02_public_alb_no_sg_ref.py 1.9 KB 
      """
Replay test for fixtures/02-public-alb-no-sg-ref.

A clean control with full fleet volume (11 SGs) and a loud-looking surface. A public ALB
carries the only 0.0.0.0/0 (and ::/0) ingress on 443/80 -- the intended public entry. A
deep service chain web -> app -> internal services -> db is wired with SG references
downstream, but NOTHING references the public ALB SG (it forwards to its target group by
registered IP), so no SG-to-SG edge composes from the internet into the fleet. The
database is NOT reachable. The audit must report no path, must NOT headline the intended
public ALB as a lateral chain, and must still name the boundary.

Stdlib only. Run with: `python tests/replay_02_public_alb_no_sg_ref.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "02-public-alb-no-sg-ref"


def main() -> int:
    r = run_deep(FIXTURE_DIR)

    assertions = [
        (r.clean, f"expected a clean audit, got {sorted(r.codes())}"),
        (r.top_severity is None, "a clean audit has no top severity"),
        (r.shortest_path == [], f"no path to the crown jewel should exist, got {r.shortest_path}"),
        ("db" not in r.reachable, f"the database must NOT be in the blast radius, got {r.reachable}"),
        # Only the directly-exposed public ALB is reachable; no SG-ref composes onward.
        (set(r.reachable) == {"lb"}, f"only the public ALB tier is reachable, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
        (len(r.boundary) >= 3, "even a clean graph reports the joins it cannot make"),
    ]
    return report("replay_02_public_alb_no_sg_ref", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_03_disjoint_public_vpn_islands.py 2 KB 
      """
Replay test for fixtures/03-disjoint-public-vpn-islands.

A clean control with full fleet volume (11 SGs) and a DECEPTIVE two-island layout. One
island is internet-facing (public ALB -> web -> web-app, wired by CIDR, not SG refs, and
a dead end). The other is a data island wired with a juicy-looking deep SG-ref chain
admin plane -> data processor -> db (+ analytics cache), but the admin plane accepts only
the corp VPN CIDR and has no inbound SG reference, so the data island is reachable only
from the VPN, never from the internet. No edge joins the two islands. The crown-jewel
database is NOT reachable from the internet entry. The audit must report no path and must
NOT splice the two islands into a manufactured internet -> db route.

Stdlib only. Run with: `python tests/replay_03_disjoint_public_vpn_islands.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "03-disjoint-public-vpn-islands"


def main() -> int:
    r = run_deep(FIXTURE_DIR)

    assertions = [
        (r.clean, f"expected a clean audit, got {sorted(r.codes())}"),
        (r.top_severity is None, "a clean audit has no top severity"),
        (r.shortest_path == [], f"no path to the crown jewel should exist, got {r.shortest_path}"),
        ("db" not in r.reachable, f"the database must NOT be in the blast radius, got {r.reachable}"),
        # The data island (admin/dataproc/db/cache) must NOT be reachable from the internet.
        (set(r.reachable) == {"lb"}, f"only the public ALB tier is reachable, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
        (len(r.boundary) >= 3, "even a clean graph reports the joins it cannot make"),
    ]
    return report("replay_03_disjoint_public_vpn_islands", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_04_broken_segment_midchain.py 2 KB 
      """
Replay test for fixtures/04-broken-segment-midchain.

A clean control with full fleet volume (10 SGs) and a DECEPTIVE broken chain. A long
service chain is visible (public edge proxy -> web -> app -> session cache -> db), and the
edge proxy carries the only 0.0.0.0/0 ingress. But the chain is cut at the first internal
hop: the web frontend accepts only the internal mesh CIDR, NOT the edge proxy SG, so the
edge -> web link is not an SG edge and the rest of the chain is orphaned. The internet
reaches only the directly-exposed edge proxy and stops. The database is NOT reachable. The
audit must report no path, must NOT mistake the visible-but-broken chain for a reachable
one, and must still name the boundary.

Stdlib only. Run with: `python tests/replay_04_broken_segment_midchain.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "04-broken-segment-midchain"


def main() -> int:
    r = run_deep(FIXTURE_DIR)

    assertions = [
        (r.clean, f"expected a clean audit, got {sorted(r.codes())}"),
        (r.top_severity is None, "a clean audit has no top severity"),
        (r.shortest_path == [], f"no path to the crown jewel should exist, got {r.shortest_path}"),
        ("db" not in r.reachable, f"the database must NOT be in the blast radius, got {r.reachable}"),
        # Only the directly-exposed public edge proxy is reachable; the chain is cut after it.
        (set(r.reachable) == {"edge"}, f"only the public edge proxy tier is reachable, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
        (len(r.boundary) >= 3, "even a clean graph reports the joins it cannot make"),
    ]
    return report("replay_04_broken_segment_midchain", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_05_six_hop_cdn_waf_gw_app_svc_db.py 1.8 KB 
      """
Replay test for fixtures/05-six-hop-cdn-waf-gw-app-svc-db.

A 13-SG fleet whose core is a SIX-hop SG-reference chain: internet -> cdn -> waf -> gw ->
app -> svc -> db. Every rule accepts exactly one upstream tier and is textbook in
isolation; the depth is the point -- the full internet-to-database path spans six separate
security groups, with the billing-service settlement-writer as the last link. Surrounding
it are scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue, logs)
that pad the haystack and fan out from app (so app is also a bridging hub).

Stdlib only. Run with: `python tests/replay_05_six_hop_cdn_waf_gw_app_svc_db.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "05-six-hop-cdn-waf-gw-app-svc-db"


def main() -> int:
    r = run_deep(FIXTURE_DIR)
    p1 = next((f for f in r.findings if f.code == "P1"), None)

    assertions = [
        ("P1" in r.codes() and "B1" in r.codes(), f"expected P1 + B1, got {sorted(r.codes())}"),
        (r.top_severity == "critical", "a path to the crown jewel must be critical"),
        (r.shortest_path == ["internet", "cdn", "waf", "gw", "app", "svc", "db"],
         f"expected internet->cdn->waf->gw->app->svc->db, got {r.shortest_path}"),
        (p1 is not None and len(p1.path) - 1 == 6, "the needle is a 6-hop chain across seven SGs"),
        ("db" in r.reachable, f"the crown jewel must be in the blast radius, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
    ]
    return report("replay_05_six_hop_cdn_waf_gw_app_svc_db", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_06_compromised_ci_runner_deep.py 2 KB 
      """
Replay test for fixtures/06-compromised-ci-runner-deep.

A 12-SG fleet where the entry point is a COMPROMISED CI runner host (instance i-06ci0001),
not the internet. From that foothold a five-hop internal build-pipeline chain composes:
ci -> build -> artifact -> deploy -> app -> db. A separate public ALB -> web -> web-cache
front door exists but never touches the data plane, so the only path to the crown jewel is
the internal pipeline chain. The depth + non-internet entry is the point: the chain only
appears when the SG-to-SG hops are composed from the compromised host outward.

Stdlib only. Run with: `python tests/replay_06_compromised_ci_runner_deep.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "06-compromised-ci-runner-deep"


def main() -> int:
    r = run_deep(FIXTURE_DIR)
    p1 = next((f for f in r.findings if f.code == "P1"), None)

    assertions = [
        ("P1" in r.codes() and "B1" in r.codes(), f"expected P1 + B1, got {sorted(r.codes())}"),
        (r.top_severity == "critical", "a path to the crown jewel must be critical"),
        (r.shortest_path == ["ci", "build", "artifact", "deploy", "app", "db"],
         f"expected ci->build->artifact->deploy->app->db, got {r.shortest_path}"),
        (p1 is not None and len(p1.path) - 1 == 5, "the needle is a 5-hop chain from the compromised CI runner"),
        ("db" in r.reachable, f"the crown jewel must be in the blast radius, got {r.reachable}"),
        # The public web/cache island must NOT be reachable from the CI-runner entry.
        ("web" not in r.reachable and "cache" not in r.reachable,
         f"the public front door is a separate island, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
    ]
    return report("replay_06_compromised_ci_runner_deep", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • replay_07_five_hop_ingress_mesh_broker_db.py 1.8 KB 
      """
Replay test for fixtures/07-five-hop-ingress-mesh-broker-db.

A 12-SG fleet whose core is a FIVE-hop SG-reference chain: internet -> ingress -> mesh ->
app -> broker -> db. Each tier accepts exactly one upstream and is fine in isolation; the
chain only appears when all five hops are composed, with the event-broker journal-writer
as the offbeat last link into the database. The app-fed leaves (cache, queue, logs) and
scoped tiers (bastion, monitoring, ci, ssm) pad the haystack and fan out from app (so app
is also a bridging hub).

Stdlib only. Run with: `python tests/replay_07_five_hop_ingress_mesh_broker_db.py`.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _deep import run_deep  # noqa: E402
from _replay import report  # noqa: E402

FIXTURE_DIR = Path(__file__).parent.parent / "fixtures" / "07-five-hop-ingress-mesh-broker-db"


def main() -> int:
    r = run_deep(FIXTURE_DIR)
    p1 = next((f for f in r.findings if f.code == "P1"), None)

    assertions = [
        ("P1" in r.codes() and "B1" in r.codes(), f"expected P1 + B1, got {sorted(r.codes())}"),
        (r.top_severity == "critical", "a path to the crown jewel must be critical"),
        (r.shortest_path == ["internet", "ingress", "mesh", "app", "broker", "db"],
         f"expected internet->ingress->mesh->app->broker->db, got {r.shortest_path}"),
        (p1 is not None and len(p1.path) - 1 == 5, "the needle is a 5-hop chain across six SGs"),
        ("db" in r.reachable, f"the crown jewel must be in the blast radius, got {r.reachable}"),
        (r.sg_count >= 10, f"this is a high-volume fleet, got {r.sg_count} SGs"),
    ]
    return report("replay_07_five_hop_ingress_mesh_broker_db", r, assertions)


if __name__ == "__main__":
    sys.exit(main())
    • _deep.py 3 KB 
      """
Deterministic ground-truth engine for the sg-deep-lateral-auditor screening harness.

This module REUSES the already-validated reachability engine from the sibling skill
`lateral-movement-reachability-auditor` verbatim. The proven graph/BFS/articulation
logic lives in `_reach_engine.py` (a byte-for-byte copy of that skill's `tests/_reach.py`),
so ground truth here is provably the same computation. We do NOT change the engine.

The only thing this file adds is a thin alias `run_deep(fixture_dir) -> Reachability`
so the screening harness can speak its own verb while computing the identical result.
The returned object exposes the same surface the sibling engine returns:

    .findings        list[Finding]            (P1 path / B1 blast radius / H1 hub)
    .codes()         set[str]
    .top_severity    "critical"|"high"|... | None
    .clean           bool
    .boundary        list[str]
    .shortest_path   list[str]
    .reachable       list[str]   (blast radius, entry excluded)
    .edges           list[tuple[str,str]]

This screening harness is scoped ENTIRELY to the model's empirically-located WEAK
region: LONG (4-6 hop) lateral chains buried in a 10-14 SG fleet, plus deceptive /
segmented-clean fleets where a loud public exposure or a visible-but-orphaned deep
SG-ref chain must NOT be reported as a reachable path to the crown jewel. There are no
short, obvious 2-3 hop direct paths -- those the base model already aces. The "needle"
is the COMPOSITION of many ordinary single-upstream SG-to-SG edges into one long path
that only appears when the whole chain is assembled; a per-rule read never sees it.

The aggregation across the whole fleet is exactly what the engine already does: one
transitive closure over the entire SG set (every SG a node, every UserIdGroupPair an
edge), not a per-SG verdict.

Stdlib only. No external dependencies. No credentials. Python 3.10+.
"""

from __future__ import annotations

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parent))

# Re-export the proven engine's public surface unchanged.
from _reach_engine import (  # noqa: E402
    Finding,
    Reachability,
    build_graph,
    load_instances,
    load_security_groups,
    run_reach,
)

__all__ = ["run_deep", "Finding", "Reachability", "build_graph", "load_instances", "load_security_groups"]


def run_deep(fixture_dir: Path) -> Reachability:
    """Run the validated reachability engine over one fleet fixture and return the
    same Reachability result shape. This is the ground-truth oracle the screening
    harness anchors its LLM judge against.

    For this estate/fleet harness the "aggregation across sub-items" is the single
    transitive closure the engine already computes over the WHOLE security-group set:
    every SG in the fleet is a node, every UserIdGroupPair an edge, and the result is
    one fleet-wide reachability verdict (the long needle path + blast radius, or
    clean), not a per-SG list. We do not re-implement that; we delegate to the proven
    engine unchanged.
    """
    return run_reach(Path(fixture_dir))
    • _reach_engine.py 20.9 KB 
      """
Reference implementation of the lateral-movement-reachability-auditor methodology.

This module is a deterministic stand-in for what an AI agent does when it follows
SKILL.md. It exists so replay tests can assert that the methodology, applied to a
known set of security groups, produces the expected reachability findings -- and so
the control-only screening eval has a ground truth to anchor its LLM judge against.

The job is a GRAPH problem, and the whole point of the skill is the composition the
graph makes visible. The input is a set of security groups whose ingress rules
reference OTHER security groups (UserIdGroupPairs) plus an instance->SG membership
map. A per-rule read sees "app accepts from web" and "db accepts from app" as two
individually fine rules and misses that internet -> web -> app -> db is one reachable
path. This module composes those SG-to-SG edges into multi-hop paths.

Model
-----
Build a DIRECTED graph over security groups. An edge A -> B exists if SG B has an
INGRESS rule whose UserIdGroupPairs includes SG A (B accepts traffic FROM A), meaning
a host in A can reach a host in B. The synthetic node "internet" has an edge
internet -> X for every SG X with a 0.0.0.0/0 (or ::/0) ingress rule.

From the entry point named in meta.json ("internet" or a compromised instance id,
which resolves to that instance's SGs), compute the transitive closure with BFS
(visited set -> cycles terminate). Findings:

  P1 (critical) -- a path entry -> ... -> crown-jewel tier exists; report the SHORTEST
     path as an explicit ordered hop list.
  B1 (high)     -- the full reachable set (blast radius) from the entry, when it spans
     more than the entry tier itself.
  H1 (high)     -- a "pivot/hub" SG that bridges two otherwise-isolated reachable
     regions: removing it disconnects part of the blast radius (an articulation point
     in the reachable subgraph).

A SEGMENTED graph where no path reaches the crown jewel is CLEAN: the audit reports
the blast radius is bounded and does NOT include the crown jewel (and still reports
the boundary it cannot cross).

Stdlib only. No external dependencies. No external credentials. Python 3.10+.
"""

from __future__ import annotations

import json
from collections import deque
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

_SEVERITY_RANK = {"critical": 0, "high": 1, "medium": 2, "low": 3}

INTERNET = "internet"


@dataclass
class Finding:
    """One reachability finding, derived from the SG graph + membership alone."""

    code: str           # P1 (path to crown jewel) | B1 (blast radius) | H1 (hub/pivot)
    severity: str       # critical | high | medium | low
    attribute: str      # the SG(s) / edge / path the finding is grounded in
    title: str
    detail: str
    recommendation: str
    # For path-based findings, the ordered hop list (e.g. ["internet","web","app","db"]).
    path: list[str] = field(default_factory=list)


@dataclass
class Reachability:
    """Structured output of the methodology, one per audited graph."""

    entry: str                                  # the entry-point label (resolved)
    crown_jewel: str | None                     # the crown-jewel tier label, if named
    sg_count: int = 0
    findings: list[Finding] = field(default_factory=list)
    edges: list[tuple[str, str]] = field(default_factory=list)   # directed A->B (reach)
    reachable: list[str] = field(default_factory=list)           # blast radius (labels), entry excluded
    shortest_path: list[str] = field(default_factory=list)       # entry..crown jewel, if any
    boundary: list[str] = field(default_factory=list)

    @property
    def clean(self) -> bool:
        return len(self.findings) == 0

    @property
    def top_severity(self) -> str | None:
        if not self.findings:
            return None
        return min(self.findings, key=lambda f: _SEVERITY_RANK[f.severity]).severity

    def codes(self) -> set[str]:
        return {f.code for f in self.findings}


# --- Loading ------------------------------------------------------------------------


def _as_list(value: Any) -> list:
    if value is None:
        return []
    return value if isinstance(value, list) else [value]


def load_security_groups(path: Path) -> list[dict]:
    with path.open() as f:
        doc = json.load(f)
    if isinstance(doc, dict) and "SecurityGroups" in doc:
        return list(doc["SecurityGroups"])
    if isinstance(doc, list):
        return doc
    return [doc]


def load_instances(path: Path) -> list[dict]:
    if not path.exists():
        return []
    with path.open() as f:
        doc = json.load(f)
    if isinstance(doc, dict) and "Reservations" in doc:
        out: list[dict] = []
        for r in doc["Reservations"]:
            out.extend(r.get("Instances", []))
        return out
    if isinstance(doc, dict) and "Instances" in doc:
        return list(doc["Instances"])
    if isinstance(doc, list):
        return doc
    return [doc]


# --- Label helpers ------------------------------------------------------------------


def _sg_label(sg: dict) -> str:
    """A human label for an SG, preferring a `tier` tag, then GroupName, then GroupId."""
    for tag in _as_list(sg.get("Tags")):
        if isinstance(tag, dict) and tag.get("Key") in ("tier", "Name"):
            if tag.get("Value"):
                return str(tag["Value"])
    return sg.get("GroupName") or sg.get("GroupId") or "<unknown>"


def _rule_is_internet_facing(rule: dict) -> bool:
    for r in _as_list(rule.get("IpRanges")):
        if isinstance(r, dict) and r.get("CidrIp") == "0.0.0.0/0":
            return True
    for r in _as_list(rule.get("Ipv6Ranges")):
        if isinstance(r, dict) and r.get("CidrIpv6") == "::/0":
            return True
    return False


# --- Graph construction -------------------------------------------------------------


def build_graph(sgs: list[dict]) -> tuple[dict[str, set[str]], set[str]]:
    """Build the directed reachability graph keyed by GroupId.

    Returns (adjacency, internet_facing) where adjacency[A] is the set of GroupIds a
    host in A can reach in one hop, and internet_facing is the set of GroupIds with a
    0.0.0.0/0 (or ::/0) ingress rule. An edge A -> B is created when SG B has an INGRESS
    rule whose UserIdGroupPairs names A (B accepts FROM A).
    """
    ids = {sg.get("GroupId") for sg in sgs if sg.get("GroupId")}
    adjacency: dict[str, set[str]] = {gid: set() for gid in ids}
    internet_facing: set[str] = set()

    for sg in sgs:
        b = sg.get("GroupId")
        if not b:
            continue
        for rule in _as_list(sg.get("IpPermissions")):
            if _rule_is_internet_facing(rule):
                internet_facing.add(b)
            for pair in _as_list(rule.get("UserIdGroupPairs")):
                a = pair.get("GroupId") if isinstance(pair, dict) else None
                if a and a in ids:
                    adjacency.setdefault(a, set()).add(b)  # A can reach B

    if internet_facing:
        adjacency[INTERNET] = set(internet_facing)
    return adjacency, internet_facing


def _bfs_closure(adjacency: dict[str, set[str]], start: set[str]) -> dict[str, int]:
    """BFS from a set of start nodes. Returns {node: hop-distance}. Cycles terminate
    because a node is enqueued at most once (visited == keys of the dist map)."""
    dist: dict[str, int] = {s: 0 for s in start}
    queue: deque[str] = deque(start)
    while queue:
        node = queue.popleft()
        for nxt in adjacency.get(node, set()):
            if nxt not in dist:
                dist[nxt] = dist[node] + 1
                queue.append(nxt)
    return dist


def _shortest_path(adjacency: dict[str, set[str]], start: set[str], target: str) -> list[str]:
    """Shortest path (BFS, so fewest hops) from any start node to target, or []."""
    if target in start:
        return [target]
    prev: dict[str, str] = {}
    seen: set[str] = set(start)
    queue: deque[str] = deque(start)
    while queue:
        node = queue.popleft()
        for nxt in adjacency.get(node, set()):
            if nxt in seen:
                continue
            seen.add(nxt)
            prev[nxt] = node
            if nxt == target:
                path = [nxt]
                while path[-1] in prev:
                    path.append(prev[path[-1]])
                path.reverse()
                # Prepend the start node the path emerged from (already the head).
                return path
            queue.append(nxt)
    return []


def _undirected_components(nodes: set[str], adjacency: dict[str, set[str]]) -> int:
    """Number of connected components among `nodes`, treating edges as undirected and
    only counting edges whose BOTH endpoints are in `nodes`. Used to decide whether the
    set a candidate gates is one region (a linear continuation) or several (a fan-out)."""
    if not nodes:
        return 0
    adj: dict[str, set[str]] = {n: set() for n in nodes}
    for a, succ in adjacency.items():
        if a not in nodes:
            continue
        for b in succ:
            if b in nodes:
                adj[a].add(b)
                adj[b].add(a)
    seen: set[str] = set()
    comps = 0
    for n in nodes:
        if n in seen:
            continue
        comps += 1
        stack = [n]
        seen.add(n)
        while stack:
            x = stack.pop()
            for y in adj[x]:
                if y not in seen:
                    seen.add(y)
                    stack.append(y)
    return comps


def _articulation_hubs(
    adjacency: dict[str, set[str]], start: set[str], reachable: set[str]
) -> list[str]:
    """SGs that, if removed, disconnect TWO OR MORE otherwise-isolated reachable regions
    -- a true pivot/hub, not just any node on a linear chain.

    For each candidate node (reachable, not a start node), recompute the closure with the
    node deleted and find the set it gated (`lost`). A node is a hub only when `lost`
    forms two or more mutually-isolated regions (a fan-out bridge), distinguishing a
    shared-services SG that joins separate tiers from an ordinary intermediate hop on a
    single linear path (which gates only its own downstream continuation -- one region).

    Returns hubs ordered by how much they bridge (most regions, then most nodes, gated).
    """
    base = reachable - set(start)
    scored: list[tuple[int, int, str]] = []
    for cand in sorted(base):
        pruned: dict[str, set[str]] = {
            n: {m for m in succ if m != cand}
            for n, succ in adjacency.items()
            if n != cand
        }
        still = set(_bfs_closure(pruned, start)) - set(start)
        lost = base - still - {cand}
        if not lost:
            continue
        regions = _undirected_components(lost, adjacency)
        if regions >= 2:  # a fan-out bridge, not a linear continuation
            scored.append((regions, len(lost), cand))
    scored.sort(key=lambda t: (-t[0], -t[1], t[2]))
    return [c for _, _, c in scored]


# --- Boundary -----------------------------------------------------------------------


def _boundary_notes(entry_is_internet: bool, has_crown: bool) -> list[str]:
    notes = [
        "Reachability-on-paper is not exploitability. An edge means an SG accepts the "
        "referenced SG; it does not mean a live host is listening, nor that the route "
        "actually carries traffic. Join: SG graph to the live ENIs/instances actually "
        "in each SG.",
        "Subnet route tables decide whether two SGs are even on a routable path. An SG "
        "edge across unrouted subnets reaches nothing. Join: SG graph to the route tables.",
        "Network ACLs are a stateless layer below security groups and can deny traffic "
        "the SG graph would allow. Join: SG graph to the subnet NACLs.",
        "Application-layer authentication (a database password, an mTLS handshake, an "
        "app token) can stop a network-reachable hop from becoming access. Join: "
        "network reachability to the app-layer auth on each tier.",
    ]
    if entry_is_internet:
        notes.append(
            "The internet edge assumes the 0.0.0.0/0 SG is on a host with a public IP and "
            "an internet route. Without that, the entry point itself is unreachable. Join: "
            "internet-facing SG to its host's public IP + route table."
        )
    if has_crown:
        notes.append(
            "Whether the crown-jewel tier currently has a running host is a membership "
            "question the SG graph cannot answer; an empty SG is a path to nothing. Join: "
            "crown-jewel SG to its current instance membership."
        )
    return notes


# --- Orchestration ------------------------------------------------------------------


def run_reach(fixture_dir: Path) -> Reachability:
    """End-to-end: load the SG graph + instances, resolve the entry point and crown
    jewel from meta.json, compute the transitive closure, and return the Reachability.

    meta.json shape:
      {"entry": "internet" | "<instance-id>", "crown_jewel": "<tier-name-or-GroupId>"}
    The crown jewel is matched against SG tier/Name tags, GroupName, or GroupId.
    """
    sgs = load_security_groups(fixture_dir / "security-groups.json")
    instances = load_instances(fixture_dir / "instances.json")

    meta: dict = {}
    meta_path = fixture_dir / "meta.json"
    if meta_path.exists():
        with meta_path.open() as f:
            meta = json.load(f)
    entry_spec = meta.get("entry", "internet")
    crown_spec = meta.get("crown_jewel")

    adjacency, internet_facing = build_graph(sgs)

    id_to_label = {sg["GroupId"]: _sg_label(sg) for sg in sgs if sg.get("GroupId")}
    id_to_label[INTERNET] = "internet"

    def label(gid: str) -> str:
        return id_to_label.get(gid, gid)

    # Resolve the crown-jewel SG id from its spec (tier tag / GroupName / GroupId).
    crown_id: str | None = None
    if crown_spec:
        for sg in sgs:
            gid = sg.get("GroupId")
            if gid == crown_spec or _sg_label(sg) == crown_spec or sg.get("GroupName") == crown_spec:
                crown_id = gid
                break

    # Resolve the entry point to a set of start GroupIds.
    entry_is_internet = entry_spec == INTERNET
    if entry_is_internet:
        start_ids: set[str] = {INTERNET}
        entry_label = "internet"
    else:
        start_ids = set()
        entry_label = entry_spec
        for inst in instances:
            if inst.get("InstanceId") == entry_spec:
                for s in _as_list(inst.get("SecurityGroups")):
                    gid = s.get("GroupId") if isinstance(s, dict) else None
                    if gid:
                        start_ids.add(gid)
                for tag in _as_list(inst.get("Tags")):
                    if isinstance(tag, dict) and tag.get("Key") == "Name" and tag.get("Value"):
                        entry_label = tag["Value"]
        if not start_ids:
            # Fall back to treating the spec as an SG label/id directly.
            for sg in sgs:
                if _sg_label(sg) == entry_spec or sg.get("GroupId") == entry_spec:
                    start_ids.add(sg["GroupId"])

    dist = _bfs_closure(adjacency, start_ids)
    reachable_ids = set(dist) - set(start_ids)
    if INTERNET in reachable_ids:
        reachable_ids.discard(INTERNET)

    # Directed reach edges, excluding the synthetic internet node from the count surface
    # but keeping internet->X edges visible in the edge list.
    edges: list[tuple[str, str]] = []
    for a, succ in adjacency.items():
        for b in succ:
            edges.append((label(a), label(b)))
    edges.sort()

    shortest = _shortest_path(adjacency, start_ids, crown_id) if crown_id else []
    shortest_labels = [label(n) for n in shortest]

    # Lateral reach is the set of tiers reached by composing at least one SG-to-SG hop
    # BEYOND the entry-adjacent tier (distance >= 2 from the entry). A tier that is only
    # directly internet-facing (distance 1, the expected public exposure) is not lateral
    # movement; it is a different auditor's finding. The blast-radius finding fires only
    # when the closure genuinely composes edges into multi-hop reach -- so a segmented
    # graph where the chain is broken after the first hop stays clean here.
    lateral_ids = {g for g in reachable_ids if dist.get(g, 0) >= 2}

    findings: list[Finding] = []

    # P1 -- a path from the entry to the crown jewel.
    if crown_id and shortest:
        hops = len(shortest) - 1
        findings.append(Finding(
            code="P1", severity="critical",
            attribute=" -> ".join(shortest_labels),
            title=f"Reachable path from {entry_label} to the crown-jewel tier ({hops} hops)",
            detail=(
                f"Composing the SG-to-SG edges yields a {hops}-hop path from {entry_label} to "
                f"the crown-jewel tier: {' -> '.join(shortest_labels)}. No single ingress rule "
                "is alarming -- each tier accepting the tier in front of it is routine -- but the "
                "edges compose into one reachable path that a per-rule read never assembles. This "
                "is the lateral-movement chain the audit exists to surface."
            ),
            recommendation=(
                "Break the chain at the hop that should not exist: a low-trust tier should not be "
                "able to reach the crown jewel even transitively. Re-scope the offending ingress "
                "(remove the SG reference, or interpose a broker/bastion tier), and confirm each "
                "edge on the path is an intended trust relationship."
            ),
            path=shortest_labels,
        ))

    # B1 -- the blast radius, when the closure composes at least one lateral hop
    # (distance >= 2) beyond the directly-exposed entry-adjacent tier.
    if lateral_ids:
        radius_labels = sorted(label(g) for g in reachable_ids)
        crown_in = crown_id in reachable_ids if crown_id else False
        findings.append(Finding(
            code="B1", severity="high",
            attribute=f"{len(radius_labels)} SG(s) reachable from {entry_label}",
            title=f"Blast radius from {entry_label}: {len(radius_labels)} reachable tier(s)",
            detail=(
                f"From {entry_label}, the transitive closure of the SG graph reaches "
                f"{len(radius_labels)} other tier(s): {', '.join(radius_labels)}. "
                + ("The crown-jewel tier is inside this radius. " if crown_in else
                   "The crown-jewel tier is NOT inside this radius. ")
                + "This is the set of tiers a foothold at the entry can pivot to without any "
                "further misconfiguration -- it is bounded by the edges, not by any single rule."
            ),
            recommendation=(
                "Confirm every tier in the blast radius is intended to be reachable from the "
                "entry. Each unintended tier in the set is an edge to re-scope; minimize the "
                "transitive reach, not just the direct rules."
            ),
        ))

    # H1 -- a hub/pivot SG bridging two otherwise-isolated reachable regions.
    hubs = _articulation_hubs(adjacency, start_ids, reachable_ids | set(start_ids))
    # A hub is an INTERMEDIATE bridge, not the entry node nor the directly-exposed
    # front-door tier (distance 1, which is the expected public ingress, a different
    # auditor's finding). Restrict to nodes at least two hops in -- the quiet shared SG
    # that joins regions, not the obvious internet-facing edge.
    hubs = [h for h in hubs if h not in start_ids and h != INTERNET and dist.get(h, 0) >= 2]
    if hubs:
        hub = hubs[0]
        findings.append(Finding(
            code="H1", severity="high",
            attribute=f"hub SG {label(hub)}",
            title=f"Pivot/hub SG bridges otherwise-isolated regions ({label(hub)})",
            detail=(
                f"The SG '{label(hub)}' is a pivot: it is the only reachable bridge between two "
                "regions of the graph that are otherwise isolated from the entry. Remove it and "
                "part of the blast radius disconnects. A shared-services SG (monitoring, CI, a "
                "jump tier) that everything references is exactly this shape -- it quietly joins "
                "tiers that were never meant to reach each other."
            ),
            recommendation=(
                f"Treat '{label(hub)}' as a high-value chokepoint: minimize what references it and "
                "what it can reach, since compromising it (or a host in it) unlocks both regions it "
                "bridges. Split a shared-services SG per consumer rather than one SG every tier trusts."
            ),
        ))

    findings.sort(key=lambda f: (_SEVERITY_RANK[f.severity], f.code))

    return Reachability(
        entry=entry_label,
        crown_jewel=(label(crown_id) if crown_id else crown_spec),
        sg_count=len(sgs),
        findings=findings,
        edges=edges,
        reachable=sorted(label(g) for g in reachable_ids),
        shortest_path=shortest_labels,
        boundary=_boundary_notes(entry_is_internet=entry_is_internet, has_crown=crown_id is not None),
    )
    • _replay.py 881 B 
      """
Shared reporting helper for the replay tests. Stdlib only.

Each replay_NN_*.py loads one fleet fixture, runs the reachability engine, and hands a
list of (ok, message) assertion tuples to `report`. Keeps the per-test files focused on
the ground-truth verdict that matters for that fixture (long needle path / clean).
"""

from __future__ import annotations


def report(name: str, result, assertions) -> int:
    failed = [msg for ok, msg in assertions if not ok]
    if failed:
        print(f"FAIL: {name}")
        for msg in failed:
            print(f"  - {msg}")
        return 1
    print(f"PASS: {name} ({len(assertions)} assertions)")
    codes = sorted(result.codes()) or ["none"]
    print(f"  findings: {codes} (top severity: {result.top_severity})")
    if result.shortest_path:
        print(f"  shortest path: {' -> '.join(result.shortest_path)}")
    return 0
  • FAILURE_MODES.md 3.4 KB 
    # Failure modes: sg-deceptive-reachability-auditor

This skill composes a reachability graph from security-group references. It is correct
for what that graph can express and wrong in the predictable ways below. Read this before
acting on a finding.

## 1. Reachability is not exploitability

A P1 path means every SG on the path accepts the SG in front of it. It does **not** mean
the path carries traffic. Each of these breaks a graph-reachable path without changing a
single ingress rule:

- **No live host.** An SG with no running instance is a path to nothing. The closure
  reaches the SG; the breach reaches an empty tier. Confirm membership before treating P1
  as live.
- **Unrouted subnets.** Two SGs whose subnets have no route to each other are not on a
  routable path. The SG edge is real; the route is absent.
- **A denying NACL.** Network ACLs are a stateless layer below security groups and can
  deny what the SG graph allows. The skill does not read them.
- **App-layer auth.** A database password, an mTLS handshake, or an app token can stop a
  network-reachable hop from becoming access.

The boundary section of every audit names these. A P1 path is a hypothesis to confirm
against the live estate, not a proven breach.

## 2. The entry point and crown jewel are caller-supplied

The path is computed *from* the named entry (`internet` or a compromised instance id)
*to* the named crown-jewel tier. A wrong entry, a mislabelled crown jewel, or a crown
jewel that resolves to no SG changes or empties the result. The skill resolves the crown
jewel by `tier`/`Name` tag, then `GroupName`, then `GroupId`; an untagged or misnamed
tier will not match.

## 3. Only SG references and the internet edge are in the graph

The graph has exactly two edge sources: SG-to-SG `UserIdGroupPairs` and `0.0.0.0/0` /
`::/0` ingress. A route that exists via a **VPC peering connection, a transit gateway, a
VPC endpoint, a load balancer target group, or a CIDR-based rule naming a specific
internal range** does not appear as an SG reference and is outside this graph. A fleet
that looks segmented to this skill may be connected by one of those.

## 4. The hub (H1) is structural, not behavioural

H1 fires when removing a node disconnects two or more isolated regions of the blast
radius. That is a property of the *graph*, not of how the SG is used. A shared-services
SG that is referenced widely but whose host runs nothing exploitable is still reported as
a hub. The finding says "this is the chokepoint if the path is live," which inherits the
caveat in section 1.

## 5. Clean is "no path in this graph," not "safe"

A clean verdict (no P1, no lateral B1, no H1) means the SG graph composes no path from the
entry to the crown jewel. It does **not** prove the fleet is safe: a peering edge
(section 3) may connect it, or the segmentation may rely on a NACL or route table this
skill cannot see and therefore cannot confirm. The clean verdict always ships with the
boundary, for exactly this reason. Do not read "clean" as "audited and proven isolated."

## 6. Direction matters, and it is easy to invert

The edge is `A -> B` when **B accepts A** (B's ingress names A). Reading the rule as "A
accepts B" inverts every edge and produces a mirror-image path that does not exist. The
reference engine encodes the direction once (`adjacency[A].add(B)` when B's ingress names
A); a hand audit that gets it backwards will fabricate paths and miss real ones.
  • SKILL.md 14.5 KB 
    ---
name: sg-deceptive-reachability-auditor
description: Audit a fleet of AWS security groups for the multi-hop lateral-movement path that no single ingress rule reveals. Builds a directed reachability graph from the SG-to-SG references (an ingress rule on SG B naming SG A means a host in A can reach B), adds an internet edge for every 0.0.0.0/0 rule, then composes those edges into the transitive closure from a named entry point (the internet, or a compromised host). Reports the shortest reachable path to the crown-jewel tier, the blast radius, and any pivot/hub SG that bridges otherwise-isolated regions, each ranked by severity with a fix. Its discipline is symmetric: on a segmented or orphaned fleet where the chain does NOT reach the crown jewel, it reports clean and names the boundary instead of fabricating a path. Then it states what the SG graph alone cannot answer (live host membership, route tables, NACLs, app-layer auth). Use when asked to review a security-group fleet for lateral movement, blast radius, or whether the internet can reach a sensitive tier. Vendor-neutral; runs offline against describe-security-groups + describe-instances JSON with no Anyshift account.
---

# sg-deceptive-reachability-auditor

Reachability-audit skill for a fleet of AWS security groups. Takes the
`describe-security-groups` output for the fleet (plus `describe-instances` for the
instance-to-SG membership), composes the SG-to-SG references into a directed graph,
and answers one question a per-rule read cannot: from a named entry point, what can
actually be reached, and does any path reach the crown-jewel tier. It returns the
shortest path, the blast radius, and any pivot hub, ranked by severity, then names
exactly where the SG graph stops being able to answer the question.

The job is a graph problem, and the whole point of the skill is the composition the
graph makes visible. A per-rule read sees "app accepts from web" and "db accepts from
app" as two individually fine rules and never assembles that `internet -> web -> app
-> db` is one reachable path. In a fleet of 10-13 security groups, that chain is buried
among scoped tiers (bastion, monitoring, ci, ssm) and app-fed leaves (cache, queue,
logs), and the loud `0.0.0.0/0` rule on the front door draws the eye away from it. This
skill composes the edges instead of clearing each rule in isolation.

## When to invoke

- An agent is asked to review a security-group fleet for lateral movement, blast
  radius, or "can the internet reach the database."
- A fleet is being shipped or changed and the question is whether a low-trust tier can
  reach a sensitive one transitively, not just directly.
- An incident assumes a host is compromised and the question is what that foothold can
  pivot to.
- A fleet *looks* segmented and the claim "the database is isolated" needs to be
  confirmed against the actual edges rather than taken on trust.

## What this skill reads, and what it does not

It reads the static configuration of a **fleet of security groups** plus the
**instance-to-SG membership**. Both are EC2 control-plane reads
(`describe-security-groups`, `describe-instances`). That is the entire input. The audit
is correct and complete *for what the SG graph can tell you*, and it is explicit about
the rest. Reachability-on-paper is not exploitability, and every audit ends by naming
the joins it cannot make:

- It does **not** confirm a live host is listening. An edge means an SG accepts the
  referenced SG; it does not mean an instance in that SG is running and serving. An
  empty SG is a path to nothing. Join: SG graph to the live ENIs/instances in each SG.
- It does **not** read route tables. Two SGs on unrouted subnets are not on a routable
  path no matter what the ingress rules allow. Join: SG graph to the subnet route tables.
- It does **not** read network ACLs. A NACL is a stateless layer below security groups
  and can deny traffic the SG graph would allow. Join: SG graph to the subnet NACLs.
- It does **not** read app-layer auth. A database password, an mTLS handshake, or an
  app token can stop a network-reachable hop from becoming access. Join: network
  reachability to the app-layer auth on each tier.

Every audit ends by naming these. A clean (segmented) fleet still gets a boundary
section, because a network-segmented fleet is not a proven-safe system.

## The model

Build a **directed graph over security groups**. An edge `A -> B` exists when SG B has
an **ingress** rule whose `UserIdGroupPairs` includes SG A (B accepts traffic *from* A),
meaning a host in A can reach a host in B. A synthetic node `internet` has an edge
`internet -> X` for every SG X with a `0.0.0.0/0` (or `::/0`) ingress rule.

From the entry point named for the audit (`internet`, or a compromised instance id that
resolves to that instance's SGs), compute the transitive closure with BFS. A visited set
makes cycles terminate. The findings fall out of the closure.

## The methodology, in order

### 1. Parse the fleet into edges

Before any judgment, turn the JSON into the graph:

- For each SG, read its `IpPermissions` (ingress). Every `UserIdGroupPairs` entry naming
  another SG in the fleet is an **incoming** edge: `referenced-SG -> this-SG`. This is
  the step a naive read skips. The SG-reference arrays are where the chain lives.
- Every `0.0.0.0/0` / `::/0` `IpRanges` / `Ipv6Ranges` entry makes the SG
  internet-facing: `internet -> this-SG`.
- Read `describe-instances` for the instance-to-SG membership, so the entry point (a
  compromised host) resolves to a set of start SGs, and so a tier with no running host
  can be flagged as a path to nothing at the boundary.
- Label each SG by its `tier` / `Name` tag, then `GroupName`, then `GroupId`, so the path
  reads as `internet -> web -> app -> db`, not as a list of `sg-` ids.

### 2. Compute the closure and the path (P1)

Run BFS from the entry's start set. The reachable set is the closure minus the start.
If the crown-jewel tier is in the closure, compute the **shortest path** (fewest hops)
to it and report it as the headline:

- **P1 (critical) — a reachable path from the entry to the crown jewel.** Report the
  shortest path as an explicit ordered hop list (`internet -> cdn -> waf -> gw -> app ->
  svc -> db`). No single ingress rule is alarming; each tier accepting the tier in front
  of it is routine. The edges compose into one path a per-rule read never assembles.
  This is the lateral-movement chain the audit exists to surface, and on a needle fleet
  it is *the* primary finding, named end to end, not a footnote under the loud public
  rule.

### 3. Report the blast radius (B1)

- **B1 (high) — the blast radius.** When the closure composes at least one lateral hop
  (distance >= 2 from the entry, i.e. beyond the directly-exposed front-door tier), report
  the full reachable set: the tiers a foothold at the entry can pivot to with no further
  misconfiguration. State explicitly whether the crown jewel is inside the radius. A
  fleet whose chain breaks after the first hop has no lateral reach and does not fire
  B1 — that distinction is load-bearing for the clean fleets.

### 4. Find the pivot hub (H1)

- **H1 (high) — a pivot/hub SG bridging otherwise-isolated regions.** For each
  intermediate reachable SG, recompute the closure with that node removed. If its removal
  disconnects **two or more** mutually-isolated regions of the blast radius, it is a true
  pivot (an articulation point), not just an ordinary hop on a linear chain. A
  shared-services SG (monitoring, CI, a jump tier) that every tier references is exactly
  this shape: it quietly joins tiers that were never meant to reach each other. Do not
  report the entry node or the directly-internet-facing front door as a hub; those are a
  different auditor's finding.

### 5. Stay quiet on the deceptive-clean fleet

This is the half of the skill that the naive read gets wrong in the other direction.
A segmented, orphaned, or broken fleet where **no path reaches the crown jewel is
CLEAN**, and the audit must say so instead of manufacturing a path. The same composition
discipline is what proves it. Specifically:

- An **orphaned** deep chain (the deep tiers reference each other, but the front tier
  accepts only an internal service-mesh CIDR, not the public SG) is not a reachable path.
  Do not report it as one.
- An intended **public ALB** taking `0.0.0.0/0` is the expected ingress, not the lateral
  path and not the headline.
- A **disjoint** data island (a public region and an unconnected private region) must not
  be spliced into a manufactured `internet -> db` route.
- A **broken** mid-chain segment (the chain is cut at one hop) is not reachable across
  the cut.
- Do not drown the real finding, or the clean verdict, in a wall of low-value nitpicks
  about correctly-scoped tiers (bastion, monitoring, ci, ssm).

On a clean fleet the audit reports: no reachable path to the crown jewel, the bounded
blast radius (and the boundary it cannot cross), and the join checks that would confirm
the segmentation holds. It does **not** invent a critical.

### 6. Rank and report, then name the boundary

Order findings by severity (critical, high). For each: the path/SG it is grounded in,
what it means, and the fix. Then list the boundary from step "What this skill reads."
A clean fleet still gets a boundary section.

## Severity model

| Severity | Meaning |
|---|---|
| **critical** | A composed path from the entry reaches the crown-jewel tier. P1. |
| **high** | A foothold at the entry can pivot laterally, or a single SG bridges isolated regions. B1, H1. |

There is no low band here: a reachability finding is grounded in the composed graph, not
in a heuristic. The uncertainty lives entirely in the boundary (is a host live, is the
subnet routed, does a NACL deny, is there app auth), which is why the boundary section is
mandatory rather than a footnote.

## Rule reference

| Code | Rule | Severity | Grounded in |
|---|---|---|---|
| P1 | Reachable path from the entry to the crown-jewel tier | critical | shortest path in the SG closure |
| B1 | Blast radius spans a lateral hop (distance >= 2) beyond the front door | high | transitive closure from the entry |
| H1 | Pivot/hub SG bridges two or more otherwise-isolated reachable regions | high | articulation point in the reachable subgraph |

The matching half of every rule is the clean verdict: P1 absent (no path), B1 absent
(no lateral hop), H1 absent (no bridge) on a segmented fleet is the correct, complete
output, not a failure to find something.

## Output format

The agent's final message in any invocation must include:

1. **Fleet**: SG count, the entry point, the crown-jewel tier.
2. **Findings**: ranked by severity, each with the code, the path/SG it is grounded in,
   what it means, and the fix. The P1 path named hop by hop. Or "no reachable path to the
   crown jewel" for a segmented fleet, with the bounded blast radius stated.
3. **Boundary**: the joins this audit could not make (live membership, route tables,
   NACLs, app-layer auth), stated explicitly so the gap is visible instead of silent.

## Worked examples

Seven end-to-end fixtures are committed under `fixtures/`, each a fleet of 10-13 security
groups with a runnable replay test. They are split between buried needles and
deceptive-clean fleets, with no short obvious 2-3 hop path in either set:

- [`05-six-hop-cdn-waf-gw-app-svc-db`](./fixtures/05-six-hop-cdn-waf-gw-app-svc-db/): a
  six-hop service chain (CDN to WAF to gateway to app to billing service to db) wired with
  ordinary single-upstream references; the P1 needle (critical).
- [`06-compromised-ci-runner-deep`](./fixtures/06-compromised-ci-runner-deep/): the entry
  is a compromised CI host, not the internet; the path composes from the foothold inward.
- [`07-five-hop-ingress-mesh-broker-db`](./fixtures/07-five-hop-ingress-mesh-broker-db/):
  a five-hop ingress-to-mesh-to-broker-to-db chain.
- [`01-orphaned-front-internal-cidr`](./fixtures/01-orphaned-front-internal-cidr/): the
  deep chain exists but the front tier accepts only the internal mesh CIDR, so it is
  orphaned from the internet entry. Clean.
- [`02-public-alb-no-sg-ref`](./fixtures/02-public-alb-no-sg-ref/): an intended public ALB
  on `0.0.0.0/0` with no onward SG reference. Clean (the public rule is not the headline).
- [`03-disjoint-public-vpn-islands`](./fixtures/03-disjoint-public-vpn-islands/): a public
  island and an unconnected private island; no route between them. Clean.
- [`04-broken-segment-midchain`](./fixtures/04-broken-segment-midchain/): a deep chain cut
  at one mid-chain hop, so it does not reach the crown jewel. Clean.

## Replay tests

Every fixture has a replay test in `tests/` that runs the methodology (via the
deterministic reference engine `tests/_reach_engine.py`, wrapped by `tests/_deep.py`)
against the committed JSON, with no external credentials. Run from the skill directory:

```bash
for t in tests/replay_*.py; do python "$t" || exit 1; done
```

The seven tests cover the three needle paths (P1/B1/H1 present, hops correct) and the
four deceptive-clean fleets (no path fabricated). Tests exit non-zero if the audit
composes the wrong path or invents one on a clean fleet. See
[`tests/README.md`](./tests/README.md) for the fixture schema.

## Failure modes

This skill is wrong in predictable ways. Read [`FAILURE_MODES.md`](./FAILURE_MODES.md)
before relying on it. Highlights:

- It audits reachability, not exploitability. A path that passes every edge can reach a
  tier with no live host, an unrouted subnet, a denying NACL, or app-layer auth that stops
  the hop. Reachability-on-paper is a hypothesis to confirm, not a breach.
- The crown-jewel tier and the entry point are supplied by the caller. A wrong entry or a
  mislabelled crown jewel changes the path.
- It reasons over the SG references and the internet edge only. A reachable route via a
  peering connection, a transit gateway, or a VPC endpoint that does not appear as an SG
  reference is outside the graph this skill builds.

## Anyshift integration (opt-in)

The audit above runs end-to-end against the `describe-security-groups` +
`describe-instances` output the user already has. No Anyshift dependency.

Every boundary note in this skill is a join: SG graph to live instance membership, SG
graph to the route tables, SG graph to the subnet NACLs, network reachability to the
app-layer auth on each tier. The Anyshift MCP can act as a context primer by resolving
those joins from a versioned resource graph, so a P1 path can be confirmed (the host is
live, the subnet is routed, no NACL denies) instead of left as a hypothesis at the
boundary. A measured "with vs without" delta will be published here once the integration
has been exercised against the replay fixtures.

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related

Claude Prompt
Tighten-my-prose editing prompt
Claude Skill
Adversarial code-review skill for Claude
Claude Workflow
Deep-research workflow (fan-out + verify)
Claude Skill
Conventional-commit writer for Claude