interface:
  display_name: "Semgrep Analysis"
  short_description: "Scan code for security issues with Semgrep"
  icon_small: "assets/trail-of-bits-mark.svg"
  icon_large: "assets/trail-of-bits-mark.svg"
  brand_color: "#D83A34"

# Semgrep Rulesets Reference

## Complete Ruleset Catalog

### Security-Focused Rulesets

| Ruleset | Description | Use Case |
|---------|-------------|----------|
| `p/security-audit` | Comprehensive vulnerability detection, higher false positives | Manual audits, security reviews |
| `p/secrets` | Hardcoded credentials, API keys, tokens | Always include |
| `p/owasp-top-ten` | OWASP Top 10 web application vulnerabilities | Web app security |
| `p/cwe-top-25` | CWE Top 25 most dangerous software weaknesses | General security |
| `p/sql-injection` | SQL injection patterns and tainted data flows | Database security |
| `p/insecure-transport` | Ensures code uses encrypted channels | Network security |
| `p/gitleaks` | Hard-coded credentials detection (gitleaks port) | Secrets scanning |
| `p/findsecbugs` | FindSecBugs rule pack for Java | Java security |
| `p/phpcs-security-audit` | PHP security audit rules | PHP security |

### CI/CD Rulesets

| Ruleset | Description | Use Case |
|---------|-------------|----------|
| `p/default` | Default ruleset, balanced coverage | First-time users |
| `p/ci` | High-confidence security + logic bugs, low FP | CI pipelines |
| `p/r2c-ci` | Low false positives, CI-safe | CI/CD blocking |
| `p/r2c` | Community favorite, curated by Semgrep (618k+ downloads) | General scanning |
| `p/auto` | Auto-selects rules based on detected languages/frameworks | Quick scans |
| `p/comment` | Comment-related rules | Code review |

### Third-Party Rulesets

| Ruleset | Description | Maintainer |
|---------|-------------|------------|
| `p/gitlab` | GitLab-maintained security rules | GitLab |

---

## Ruleset Selection Algorithm

Follow this algorithm to select rulesets based on detected languages and frameworks.

### Step 1: Always Include Security Baseline

```json
{
  "baseline": ["p/security-audit", "p/secrets"]
}
```

- `p/security-audit` - Comprehensive vulnerability detection (always include)
- `p/secrets` - Hardcoded credentials, API keys, tokens (always include)

### Step 2: Add Language-Specific Rulesets

For each detected language, add the primary ruleset. If a framework is detected, add its ruleset too.

**GA Languages (production-ready):**

| Detection | Primary Ruleset | Framework Rulesets | Pro Rule Count |
|-----------|-----------------|-------------------|----------------|
| `.py` | `p/python` | `p/django`, `p/flask`, `p/fastapi` | 710+ |
| `.js`, `.jsx` | `p/javascript` | `p/react`, `p/nodejs`, `p/express`, `p/nextjs`, `p/angular` | 250+ (JS), 70+ (JSX) |
| `.ts`, `.tsx` | `p/typescript` | `p/react`, `p/nodejs`, `p/express`, `p/nextjs`, `p/angular` | 230+ |
| `.go` | `p/golang` | `p/go` (alias) | 80+ |
| `.java` | `p/java` | `p/spring`, `p/findsecbugs` | 190+ |
| `.kt` | `p/kotlin` | `p/spring` | 60+ |
| `.rb` | `p/ruby` | `p/rails` | 40+ |
| `.php` | `p/php` | `p/symfony`, `p/laravel`, `p/phpcs-security-audit` | 50+ |
| `.c`, `.cpp`, `.h` | `p/c` | - | 150+ |
| `.rs` | `p/rust` | - | 40+ |
| `.cs` | `p/csharp` | - | 170+ |
| `.scala` | `p/scala` | - | Community |
| `.swift` | `p/swift` | - | 60+ |

**Beta Languages (Pro recommended):**

| Detection | Primary Ruleset | Notes |
|-----------|-----------------|-------|
| `.ex`, `.exs` | `p/elixir` | Requires Pro for best coverage |
| `.cls`, `.trigger` | `p/apex` | Salesforce; requires Pro |

**Experimental Languages:**

| Detection | Primary Ruleset | Notes |
|-----------|-----------------|-------|
| `.sol` | No official ruleset | Use Decurity third-party rules |
| `Dockerfile` | `p/dockerfile` | Limited rules |
| `.yaml`, `.yml` | `p/yaml` | K8s, GitHub Actions, docker-compose patterns |
| `.json` | `r/json.aws` | AWS IAM policies; use `r/json.*` for specific rules |
| Bash scripts | - | Community support |
| Cairo, Circom | - | Experimental, smart contracts |

**Framework detection hints:**

| Framework | Detection Signals | Ruleset |
|-----------|------------------|---------|
| Django | `settings.py`, `urls.py`, `django` in requirements | `p/django` |
| Flask | `flask` in requirements, `@app.route` | `p/flask` |
| FastAPI | `fastapi` in requirements, `@app.get/post` | `p/fastapi` |
| React | `package.json` with react dependency, `.jsx`/`.tsx` files | `p/react` |
| Next.js | `next.config.js`, `pages/` or `app/` directory | `p/nextjs` |
| Angular | `angular.json`, `@angular/` dependencies | `p/angular` |
| Express | `express` in package.json, `app.use()` patterns | `p/express` |
| NestJS | `@nestjs/` dependencies, `@Controller` decorators | `p/nodejs` |
| Spring | `pom.xml` with spring, `@SpringBootApplication` | `p/spring` |
| Rails | `Gemfile` with rails, `config/routes.rb` | `p/rails` |
| Laravel | `composer.json` with laravel, `artisan` | `p/laravel` |
| Symfony | `composer.json` with symfony, `config/packages/` | `p/symfony` |

### Step 3: Add Infrastructure Rulesets

| Detection | Ruleset | Description |
|-----------|---------|-------------|
| `Dockerfile` | `p/dockerfile` | Container security, best practices |
| `.tf`, `.hcl` | `p/terraform` | IaC misconfigurations, CIS benchmarks, AWS/Azure/GCP |
| k8s manifests | `p/kubernetes` | K8s security, RBAC issues |
| CloudFormation | `p/cloudformation` | AWS infrastructure security |
| GitHub Actions | `p/github-actions` | CI/CD security, secrets exposure |
| `.yaml`, `.yml` | `p/yaml` | Generic YAML patterns (K8s, docker-compose) |
| AWS IAM JSON | `r/json.aws` | IAM policy misconfigurations (use `--config r/json.aws`) |

### Step 4: Add Third-Party Rulesets

These are **NOT optional**. Include automatically when language matches:

| Languages | Source | Why Required |
|-----------|--------|--------------|
| Python, Go, Ruby, JS/TS, Terraform, HCL | [Trail of Bits](https://github.com/trailofbits/semgrep-rules) | Security audit patterns from real engagements (AGPLv3) |
| C, C++ | [0xdea](https://github.com/0xdea/semgrep-rules) | Memory safety, low-level vulnerabilities |
| Solidity, Cairo, Rust | [Decurity](https://github.com/Decurity/semgrep-smart-contracts) | Smart contract vulnerabilities, DeFi exploits |
| Go | [dgryski](https://github.com/dgryski/semgrep-go) | Additional Go-specific patterns |
| Android (Java/Kotlin) | [MindedSecurity](https://github.com/mindedsecurity/semgrep-rules-android-security) | OWASP MASTG-derived mobile security rules |
| Java, Go, JS/TS, C#, Python, PHP | [elttam](https://github.com/elttam/semgrep-rules) | Security consulting patterns |
| Dockerfile, PHP, Go, Java | [kondukto](https://github.com/kondukto-io/semgrep-rules) | Container and web app security |
| PHP, Kotlin, Java | [dotta](https://github.com/federicodotta/semgrep-rules) | Pentest-derived web/mobile app rules |
| Terraform, HCL | [HashiCorp](https://github.com/hashicorp-forge/semgrep-rules) | HashiCorp infrastructure patterns |
| Swift, Java, Cobol | [akabe1](https://github.com/akabe1/akabe1-semgrep-rules) | iOS and legacy system patterns |
| Java | [Atlassian Labs](https://github.com/atlassian-labs/atlassian-sast-ruleset) | Atlassian-maintained Java rules |
| Python, JS/TS, Java, Ruby, Go, PHP | [Apiiro](https://github.com/apiiro/malicious-code-ruleset) | Malicious code detection, supply chain |

### Step 5: Verify Rulesets

Before finalizing, verify official rulesets load:

```bash
# Quick validation (exits 0 if valid)
semgrep --config p/python --validate --metrics=off 2>&1 | head -3
```

Or browse the [Semgrep Registry](https://semgrep.dev/explore).

### Output Format

```json
{
  "baseline": ["p/security-audit", "p/secrets"],
  "python": ["p/python", "p/django"],
  "javascript": ["p/javascript", "p/react", "p/nodejs"],
  "docker": ["p/dockerfile"],
  "third_party": ["https://github.com/trailofbits/semgrep-rules"]
}
```

# Scan Modes Reference

## Mode: Run All

Full scan with all rulesets and severity levels. Current default behavior. No filtering applied — all findings are reported and triaged.

## Mode: Important Only

Focused on high-confidence security vulnerabilities. Excludes code quality, best practices, and low-confidence audit findings.

### Pre-Filter: CLI Severity Flag

Add these flags to every `semgrep` command:

```bash
--severity WARNING --severity ERROR
```

This excludes INFO findings at scan time, reducing output volume before post-filtering.

`--severity` takes `INFO`, `WARNING`, or `ERROR`, and nothing else. Anything else exits 2 before
scanning, with no output written. The `LOW`/`MEDIUM`/`HIGH`/`CRITICAL` scale in the table below
belongs to the rule metadata, which the post-filter reads. The two are not interchangeable.

The two scales do not nest. A registry rule can carry CLI severity `INFO` and metadata
`impact: HIGH`, and this flag drops it at scan time before the post-filter sees it. The volume
reduction is why the pre-filter runs at scan time, but it makes important-only "WARNING and
above, then filtered on metadata" rather than "everything the metadata filter would keep".
Check a missing finding against a run-all scan before concluding the rule did not fire.

### Post-Filter: Metadata Criteria

After scanning, filter each JSON result file to keep only findings matching ALL of:

| Metadata Field | Accepted Values | Rationale |
|---|---|---|
| `extra.metadata.category` | `"security"` | Excludes correctness, best-practice, maintainability, performance |
| `extra.metadata.confidence` | `"MEDIUM"`, `"HIGH"` | Excludes low-precision rules (high false positive rate) |
| `extra.metadata.impact` | `"MEDIUM"`, `"HIGH"` | Excludes low-impact informational findings |

**Third-party rules** (Trail of Bits, 0xdea, Decurity, etc.) may not have `confidence`/`impact`/`category` metadata. Findings **without** these metadata fields are **kept by the post-filter** — we cannot filter what is not annotated, and third-party rules are typically security-focused.

This exemption applies only to findings that reach the post-filter. The pre-filter above runs first and on every command, including the cross-language unit that carries the cloned third-party repos, so an unannotated rule with CLI `severity: INFO` is dropped at scan time and never becomes a finding the exemption can keep. A third-party rule is exempt from the metadata filter, not from `--severity`.

### Semgrep Metadata Background

Semgrep security rules have these metadata fields (required for `category: security` in the official registry):

| Field | Purpose | Metadata values (never CLI `--severity` values) |
|---|---|---|
| `severity` (top-level) | Overall rule severity, derived from likelihood × impact | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` |
| `category` | Rule category | `security`, `correctness`, `best-practice`, `maintainability`, `performance` |
| `confidence` | True positive rate of the rule (precision) | `LOW`, `MEDIUM`, `HIGH` |
| `impact` | Potential damage if vulnerability is exploited | `LOW`, `MEDIUM`, `HIGH` |
| `likelihood` | How likely the vulnerability is exploitable | `LOW`, `MEDIUM`, `HIGH` |
| `subcategory` | Finding type | `vuln`, `audit`, `secure default` |

Key relationship: `severity = f(likelihood, impact)` while `confidence` is independent (describes rule quality, not vulnerability severity).

### Post-Filter jq Command

Apply to each JSON result file after scanning:

```bash
# Filter a single result file
jq '{
  results: [.results[] |
    ((.extra.metadata.category // "security") | ascii_downcase) as $cat |
    ((.extra.metadata.confidence // "HIGH") | ascii_upcase) as $conf |
    ((.extra.metadata.impact // "HIGH") | ascii_upcase) as $imp |
    select(
      ($cat == "security") and
      ($conf == "MEDIUM" or $conf == "HIGH") and
      ($imp == "MEDIUM" or $imp == "HIGH")
    )
  ],
  errors: .errors,
  paths: .paths
}' "$f" > "${f%.json}-important.json"
```

Default values (`// "security"`, `// "HIGH"`) handle third-party rules without metadata — they pass all filters by default.

### Filter All Result Files in a Directory

Raw scan output lives in `$OUTPUT_DIR/raw/`. The filter creates `*-important.json` files alongside the originals — the raw files are preserved unmodified.

```bash
# Apply important-only filter to all scan result JSON files in raw/
filter_failed=0
for f in "$OUTPUT_DIR/raw"/*-*.json; do
  [[ "$f" == *-triage.json || "$f" == *-important.json ]] && continue
  out="${f%.json}-important.json"
  # The redirect creates $out before jq runs, so a jq failure leaves a zero-byte file sitting
  # there. merge_sarif.py --important reads that as a corrupt filter and aborts the whole
  # merge, losing every other scan's findings to one bad file. Delete it and name the file.
  if ! jq '{
    results: [.results[] |
      ((.extra.metadata.category // "security") | ascii_downcase) as $cat |
      ((.extra.metadata.confidence // "HIGH") | ascii_upcase) as $conf |
      ((.extra.metadata.impact // "HIGH") | ascii_upcase) as $imp |
      select(
        ($cat == "security") and
        ($conf == "MEDIUM" or $conf == "HIGH") and
        ($imp == "MEDIUM" or $imp == "HIGH")
      )
    ],
    errors: .errors,
    paths: .paths
  }' "$f" >"$out"; then
    rm -f "$out"
    filter_failed=$((filter_failed + 1))
    echo "post-filter failed on $f" >&2
    continue
  fi
  BEFORE=$(jq '.results | length' "$f")
  AFTER=$(jq '.results | length' "$out")
  echo "$f: $BEFORE → $AFTER findings (filtered $(( BEFORE - AFTER )))"
done
[ "$filter_failed" -eq 0 ] ||
  echo "$filter_failed file(s) failed to filter; fix them before merging" >&2
```

### The Filter Does Not Apply to SARIF

Both filters above are written for semgrep's JSON shape and cannot be pointed at a `.sarif` file.
SARIF has no top-level `.results` and no `extra.metadata` — `category`, `confidence` and `impact`
are simply not in the format — so the filter exits with `Cannot iterate over null`, and
redirecting it over its own input truncates the file first.

The merged SARIF is filtered by `scripts/merge_sarif.py --important` instead, which keeps the
findings these filters kept by matching `(check_id, path, start.line)` against SARIF's
`(ruleId, uri, region.startLine)`. Run the JSON filter over every file in `raw/` first: the merge
fails rather than filtering if any scan has no `*-important.json` beside it, since a partial key
set would drop real findings from the primary deliverable with nothing downstream to notice.

### Scanner Command Modifications

`scripts/run-scans.sh` puts these on every command it generates; they are not yours to add.

- **Run all**: no severity flags
- **Important only**: `--severity WARNING --severity ERROR`

That pre-filter is applied by semgrep at scan time, before the metadata post-filter above. A
rule shipping with CLI severity INFO is dropped by the flag and never reaches the filter that
would have kept it.

# /// script
# requires-python = ">=3.11"
# dependencies = []
# ///
"""Merge SARIF files into a single consolidated output.

Usage:
    uv run --no-project merge_sarif.py RAW_DIR OUTPUT_FILE [--important] [--scans scans.json]

Reads *.sarif files from RAW_DIR (e.g., $OUTPUT_DIR/raw), produces
OUTPUT_FILE (e.g., $OUTPUT_DIR/results/results.sarif) containing all
findings merged and deduplicated.

--scans names the scans.json run-scans.sh wrote, and drops the SARIF files
belonging to scans recorded under .failed. Those files exist because a scan
that died part-way may still have written one, so without this a single dead
scan takes the whole run's deliverable with it: under --important its output
has no post-filter beside it, and that is an error rather than an empty
filter. Pass it whenever scans.json exists.

--important restricts the merged output to the findings that survived the
important-only post-filter. That filter reads semgrep's JSON metadata
(category/confidence/impact), which SARIF does not carry, so it cannot be
re-run against SARIF. Finding identity can be matched across the two formats
though, and that is what this flag does. Without it the merged SARIF in
important-only mode keeps every finding the mode exists to exclude.

The merge is pure Python and shells out to nothing. It used to try
`npx @microsoft/sarif-multitool` first, which made the output depend on
whether that package happened to be in the npx cache: the two backends do not
agree. Only this one dedups results on (ruleId, uri, startLine), which is the
identity --important matches against and the reason the report is told to
count from the merged file rather than sum per-scan totals. Multitool also
normalizes artifactLocation.uri, which would leave --important matching
nothing and blaming a semgrep format change that never happened.
"""

from __future__ import annotations

import json
import sys
from pathlib import Path

Key = tuple[str, str, int]


def sarif_key(result: dict) -> Key:
    """Identity of one SARIF result: rule, file, line.

    The same triple in semgrep's JSON is (check_id, path, start.line), verified
    field-for-field against semgrep output. Both the merge dedup and the
    --important filter read it from here so the two cannot drift apart. The
    filter is only correct while its keys are the keys the merge produced.
    """
    locations = result.get("locations", [])
    uri = ""
    start_line = 0
    if locations:
        phys = locations[0].get("physicalLocation", {})
        uri = phys.get("artifactLocation", {}).get("uri", "")
        start_line = phys.get("region", {}).get("startLine", 0)
    return (result.get("ruleId", ""), uri, start_line)


def json_key(result: dict) -> Key:
    """The same identity read out of a semgrep JSON result."""
    return (
        result.get("check_id", ""),
        result.get("path", ""),
        result.get("start", {}).get("line", 0),
    )


def failed_sarifs(scans_json: Path) -> set[Path]:
    """Resolved paths of the SARIF files belonging to scans that did not succeed.

    run-scans.sh records a failed scan carrying the same paths a success does, because a scan
    that crashed part-way may still have written a file. What it wrote is not a scan result:
    it is whatever semgrep produced before it stopped. Those files must not be held to the
    post-filter requirement, or one dead process denies every healthy scan a deliverable.
    """
    data = json.loads(scans_json.read_text())
    failed = data.get("failed")
    if not isinstance(failed, list):
        raise ValueError(
            f"{scans_json} has no .failed array; it is not a scans.json written by run-scans.sh"
        )
    return {
        Path(entry["sarif"]).resolve()
        for entry in failed
        if isinstance(entry, dict) and entry.get("sarif")
    }


def surviving_keys(sarif_files: list[Path]) -> set[Key]:
    """Keys kept by the important-only post-filter, one *-important.json per SARIF.

    Derived from the SARIF files going into the merge rather than by globbing
    *-important.json, so a post-filter that ran on only some of them is an error
    here instead of a merged SARIF quietly missing whole rulesets.

    Raises ValueError when a filter file is missing or unreadable. Filtering
    against a partial key set drops real findings from the primary deliverable
    and there is nothing downstream that could notice. That is stricter than
    the merge itself, which warns and skips a SARIF file it cannot parse: an
    unparseable SARIF contributes no findings either way, while an unparseable
    filter file silently removes findings the SARIF does contain.
    """
    keys: set[Key] = set()
    missing: list[str] = []
    for sarif_file in sarif_files:
        filtered = sarif_file.with_name(f"{sarif_file.stem}-important.json")
        if not filtered.is_file():
            missing.append(filtered.name)
            continue
        try:
            data = json.loads(filtered.read_text())
        except json.JSONDecodeError as e:
            raise ValueError(f"{filtered} is not valid JSON: {e}") from e
        results = data.get("results")
        if not isinstance(results, list):
            raise ValueError(
                f"{filtered} has no .results array; it is not a filtered semgrep result file"
            )
        for result in results:
            keys.add(json_key(result))

    if missing:
        raise ValueError(
            f"{len(missing)} of {len(sarif_files)} scans have no post-filtered JSON "
            f"({', '.join(sorted(missing)[:5])}"
            f"{', ...' if len(missing) > 5 else ''}). Run the important-only post-filter "
            "over every file in the raw directory first."
        )
    return keys


def filter_to_keys(merged: dict, keys: set[Key]) -> tuple[int, int]:
    """Keep only results whose identity survived the post-filter. Returns (kept, dropped)."""
    kept = 0
    dropped = 0
    for run in merged.get("runs", []):
        keeping = []
        for result in run.get("results", []):
            if sarif_key(result) in keys:
                keeping.append(result)
                kept += 1
            else:
                dropped += 1
        run["results"] = keeping
    return kept, dropped


def merge_sarif_pure_python(sarif_files: list[Path]) -> tuple[dict, list[str]]:
    """Merge every SARIF into one run, deduplicating results by sarif_key.

    Returns (merged, unparseable). The unparseable list is returned rather than just
    warned about: a scan can exit 0, write a valid .json that puts it in .scans with a
    finding count, and still leave a truncated .sarif. Dropping that file silently makes
    results.sarif short by exactly those findings with nothing anywhere pointing at it.

    The dedup is what makes the merged total meaningful: one finding flagged by two
    rulesets is one row here and two in a sum of per-scan counts.
    """
    merged = {
        "version": "2.1.0",
        "$schema": "https://json.schemastore.org/sarif-2.1.0.json",
        "runs": [],
    }

    seen_rules: dict[str, dict] = {}
    all_results: list[dict] = []
    seen_results: set[Key] = set()
    tool_info: dict | None = None
    skipped_files: list[str] = []

    for sarif_file in sorted(sarif_files):
        try:
            data = json.loads(sarif_file.read_text())
        except json.JSONDecodeError as e:
            print(f"Warning: Failed to parse {sarif_file}: {e}", file=sys.stderr)
            skipped_files.append(str(sarif_file))
            continue

        for run in data.get("runs", []):
            if tool_info is None and run.get("tool"):
                tool_info = run["tool"]

            driver = run.get("tool", {}).get("driver", {})
            for rule in driver.get("rules", []):
                rule_id = rule.get("id", "")
                if rule_id and rule_id not in seen_rules:
                    seen_rules[rule_id] = rule

            for result in run.get("results", []):
                dedup_key = sarif_key(result)
                if dedup_key in seen_results:
                    continue
                seen_results.add(dedup_key)
                all_results.append(result)

    if all_results:
        merged_run = {
            "tool": tool_info or {"driver": {"name": "semgrep", "rules": []}},
            "results": all_results,
        }
        merged_run["tool"]["driver"]["rules"] = list(seen_rules.values())
        merged["runs"].append(merged_run)

    return merged, skipped_files


def main() -> int:
    argv = sys.argv[1:]
    important = "--important" in argv
    argv = [a for a in argv if a != "--important"]

    scans_json: Path | None = None
    if "--scans" in argv:
        i = argv.index("--scans")
        if i + 1 >= len(argv):
            print("--scans needs the path to scans.json", file=sys.stderr)
            return 1
        scans_json = Path(argv[i + 1])
        del argv[i : i + 2]

    if len(argv) != 2:
        print(
            f"Usage: {sys.argv[0]} RAW_DIR OUTPUT_FILE [--important] [--scans scans.json]",
            file=sys.stderr,
        )
        return 1

    raw_dir = Path(argv[0])
    output_file = Path(argv[1])

    if not raw_dir.is_dir():
        print(f"Error: {raw_dir} is not a directory", file=sys.stderr)
        return 1

    # Collect SARIF files from raw directory only
    sarif_files = sorted(raw_dir.glob("*.sarif"))
    print(f"Found {len(sarif_files)} SARIF files to merge in {raw_dir}")

    if not sarif_files:
        print("No SARIF files found, nothing to merge", file=sys.stderr)
        return 1

    # Before the post-filter requirement below, so a dead scan is dropped rather than held to it.
    if scans_json is not None:
        try:
            excluded = failed_sarifs(scans_json)
        except (OSError, json.JSONDecodeError, ValueError) as e:
            print(f"Error: {e}", file=sys.stderr)
            return 1
        kept = [p for p in sarif_files if p.resolve() not in excluded]
        if len(kept) != len(sarif_files):
            names = sorted(p.name for p in sarif_files if p.resolve() in excluded)
            print(
                f"excluding {len(sarif_files) - len(kept)} SARIF file(s) from failed scans: "
                f"{', '.join(names)}"
            )
        sarif_files = kept
        # Every scan failed. Writing an empty merge here would report a clean run over nothing.
        if not sarif_files:
            print(
                f"Error: every scan in {scans_json} failed, so there is nothing to merge",
                file=sys.stderr,
            )
            return 1

    # Resolved before the merge, not after: a post-filter that did not run is a broken run,
    # and finding that out first costs nothing while finding it out afterwards means either
    # a wrong results.sarif on disk or a merge thrown away.
    keys: set[Key] = set()
    if important:
        try:
            keys = surviving_keys(sarif_files)
        except ValueError as e:
            print(f"Error: {e}", file=sys.stderr)
            return 1
        print(f"important-only: {len(keys)} findings survived the post-filter")

    # Ensure output directory exists
    output_file.parent.mkdir(parents=True, exist_ok=True)

    merged, unparseable = merge_sarif_pure_python(sarif_files)
    if unparseable:
        # stdout, alongside the --scans exclusions, because that is the stream the Report phase
        # reads and it must give these their own section. On stderr this was invisible to the
        # summary: the scan sits in .scans as a success carrying its finding count, so nothing
        # in scans.json or results.sarif shows that those findings went missing.
        names = ", ".join(sorted(Path(p).name for p in unparseable))
        print(
            f"unparseable: {len(unparseable)} of {len(sarif_files)} SARIF files could not be "
            f"parsed and are missing from the merge: {names}"
        )
    # Nothing was read at all, so "0 findings" would be a clean run rather than a broken one.
    if unparseable and len(unparseable) == len(sarif_files):
        print(
            f"Error: none of the {len(sarif_files)} SARIF files could be parsed; "
            "there is nothing to merge",
            file=sys.stderr,
        )
        return 1

    if important:
        before = sum(len(run.get("results", [])) for run in merged.get("runs", []))
        kept, dropped = filter_to_keys(merged, keys)
        print(f"important-only: kept {kept} of {before} merged findings, dropped {dropped}")
        # Unreachable while the two formats agree: every key came from the *-important.json
        # sibling of a SARIF in this merge, so at least one must match something. Reaching it
        # means sarif_key and json_key are no longer reading one identity out of two shapes —
        # semgrep changed an output format — and every finding was dropped for that reason
        # rather than by the filter. Writing anyway ships an empty results.sarif that reads as
        # a clean important-only run, which is the failure this whole flag exists to prevent.
        # Guarded on keys: a filter that legitimately kept nothing is a real zero, not drift.
        if keys and before and not kept:
            print(
                f"Error: the post-filter kept {len(keys)} findings and the merge read {before}, "
                "but none of them matched. The JSON and SARIF records of a finding no longer "
                "reduce to the same (rule, file, line), so refusing to write a zero-finding "
                f"{output_file}. Compare a raw *.json against its *.sarif in {raw_dir}.",
                file=sys.stderr,
            )
            return 1

    result_count = sum(len(run.get("results", [])) for run in merged.get("runs", []))
    print(f"Merged SARIF contains {result_count} findings")

    # Write output
    output_file.write_text(json.dumps(merged, indent=2))
    print(f"Written to {output_file}")

    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env bash
# Step 4 of the semgrep skill: run the selected rulesets and report what each scan produced.
# Generating every command here is what keeps --metrics=off, --include and --exclude out of a
# model's hands. Exit codes and counts come from the processes and the JSON they wrote.
# No `declare -A` and no `wait -n`, so this runs on macOS's bash 3.2.
set -euo pipefail

readonly METRICS_OFF="--metrics=off"
# semgrep --severity accepts INFO, WARNING and ERROR only, and rejects anything else with exit 2
# before scanning. The metadata thresholds LOW/MEDIUM/HIGH are applied by the post-filter in
# references/scan-modes.md, not here.
SEVERITY_FLAGS=(--severity WARNING --severity ERROR)
readonly DEFAULT_JOBS=4

# A semgrep join rule carries `mode: join` and the `join:` block that mode needs, and requiring
# both is what keeps the prune below off a rule that merely mentions the words. Each is anchored
# as a YAML key on its own line — optionally the first key of a list item, optionally quoted,
# with a trailing comment allowed — because one file holds many rules and deleting it on a
# `message:` that quotes the docs would take every sibling rule with it. A rule that names the
# mode without the block is not a valid join rule; semgrep rejects it per-rule, which the exit-2
# handling below now survives, so leaving it in place costs nothing.
readonly JOIN_MODE_RE="^[[:space:]]*(-[[:space:]]+)?mode:[[:space:]]*(join|\"join\"|'join')[[:space:]]*(#.*)?\$"
readonly JOIN_BLOCK_RE="^[[:space:]]*join:[[:space:]]*(#.*)?\$"

usage() {
  cat <<'USAGE'
Usage: run-scans.sh --target DIR --output-dir DIR --mode MODE --rulesets FILE [options]

  --target DIR       absolute path to the tree to scan
  --output-dir DIR   absolute path for results; raw/ and repos/ are created under it
  --mode MODE        run-all | important-only
  --rulesets FILE    JSON: {"baseline":[...], "<language>":[...], "third_party":["https://..."]}
  --pro              add --pro to every command (Semgrep Pro engine)
  --jobs N           concurrent semgrep processes (default 4)
  --dry-run          print the commands that would run, then exit; clones nothing

Writes OUTPUT_DIR/scans.json:
  {scans:[{lang,ruleset,json,sarif,findings,filesScanned,partial,exitCode}],
   failed:[...], skipped:[...], unscoped:[lang], alsoShared:["lang/ruleset"]}
  partial is a scan that wrote complete output while some of its rules failed to compile.
USAGE
}

die() {
  echo "run-scans.sh: $*" >&2
  exit 1
}

# --include globs per language. Checked against semgrep by scanning one file per extension;
# re-run that when bumping it, because a type omitted here is excluded with no signal and the
# ruleset reads clean rather than incomplete. Absent because semgrep does not parse them:
# .mts, .cts, .C, Containerfile, Dockerfile.prod. javascript carries the TypeScript globs
# because one detection category covers both.
includes_for() {
  case "$1" in
    python) echo '*.py *.pyi' ;;
    javascript) echo '*.js *.jsx *.mjs *.cjs *.ts *.tsx' ;;
    typescript) echo '*.ts *.tsx' ;;
    go) echo '*.go' ;;
    ruby) echo '*.rb' ;;
    java) echo '*.java *.jsp' ;;
    kotlin) echo '*.kt *.kts' ;;
    php) echo '*.php *.phtml' ;;
    c) echo '*.c *.h' ;;
    cpp) echo '*.c *.cc *.cpp *.cxx *.h *.hh *.hpp *.hxx' ;;
    csharp) echo '*.cs' ;;
    rust) echo '*.rs' ;;
    scala) echo '*.scala' ;;
    swift) echo '*.swift' ;;
    elixir) echo '*.ex *.exs' ;;
    solidity) echo '*.sol' ;;
    docker) echo 'Dockerfile *.dockerfile' ;;
    terraform) echo '*.tf *.tfvars *.hcl' ;;
    json) echo '*.json' ;;
    apex) echo '*.cls *.trigger' ;;
    cloudformation) echo '*.yaml *.yml *.json' ;;
    github-actions) echo '*.yml *.yaml' ;;
    kubernetes) echo '*.yaml *.yml' ;;
    yaml) echo '*.yaml *.yml' ;;
    *) echo '' ;;
  esac
}

# Folds plan spellings onto the keys includes_for knows, so `js` and `javascript` do not
# become two units and scan the same ruleset twice.
canonical_lang() {
  local k
  k=$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]')
  case "$k" in
    js | jsx | node | nodejs | 'js/ts' | 'javascript/typescript') echo javascript ;;
    ts | tsx) echo typescript ;;
    golang) echo go ;;
    'c/c++' | 'c++' | cxx) echo cpp ;;
    dockerfile) echo docker ;;
    k8s) echo kubernetes ;;
    'c#' | dotnet) echo csharp ;;
    sol) echo solidity ;;
    tf | hcl) echo terraform ;;
    cfn) echo cloudformation ;;
    'github actions' | githubactions | gha) echo github-actions ;;
    salesforce) echo apex ;;
    *) echo "$k" ;;
  esac
}

# Filenames, not identifiers: a registry id and a clone path must both reduce to something safe
# to concatenate into a shell-quoted path.
slug() {
  case "$1" in
    *://*) repo_dir_name "$1" ;;
    *) printf '%s' "${1##*/}" | sed 's/[^A-Za-z0-9._-]\{1,\}/-/g; s/^-\{1,\}//; s/-\{1,\}$//' ;;
  esac
}

# The clone directory carries the owner: several orgs publish a repo named semgrep-rules, and
# keying on the basename alone would collide them into one directory.
repo_dir_name() {
  local u=$1 repo owner
  u=${u%.git}
  u=${u%/}
  repo=${u##*/}
  u=${u%/*}
  owner=${u##*/}
  printf '%s' "${owner:-unknown}-${repo:-rules}" | sed 's/[^A-Za-z0-9._-]\{1,\}/-/g'
}

# Deletes every .yaml/.yml under $1 that the remaining arguments select, printing what it took.
# The status is returned rather than swallowed: the prunes that use this keep files semgrep
# cannot handle out of a --config directory, and one that quietly stops matching puts the bug it
# exists to prevent straight back, with nothing on stderr to say so.
prune_yaml() { # prune_yaml <dir> <find-predicate...>
  local dir=$1
  shift
  find "$dir" \( -name '*.yaml' -o -name '*.yml' \) -type f "$@" -print -delete
}

# `wc -l` reads one line in the empty string, so the zero case comes from the string itself.
count_lines() {
  if [ -z "$1" ]; then
    echo 0
  else
    printf '%s\n' "$1" | wc -l | tr -d ' '
  fi
}

TARGET="" OUTPUT_DIR="" MODE="" RULESETS_FILE="" PRO="" DRY_RUN=""
JOBS=$DEFAULT_JOBS
while [ $# -gt 0 ]; do
  case "$1" in
    --target)
      TARGET=${2:-}
      shift 2
      ;;
    --output-dir)
      OUTPUT_DIR=${2:-}
      shift 2
      ;;
    --mode)
      MODE=${2:-}
      shift 2
      ;;
    --rulesets)
      RULESETS_FILE=${2:-}
      shift 2
      ;;
    --jobs)
      JOBS=${2:-}
      shift 2
      ;;
    --pro)
      PRO=1
      shift
      ;;
    --dry-run)
      DRY_RUN=1
      shift
      ;;
    -h | --help)
      usage
      exit 0
      ;;
    *) die "unknown argument: $1" ;;
  esac
done

command -v jq >/dev/null 2>&1 || die "jq is required"
[ -n "$DRY_RUN" ] || command -v semgrep >/dev/null 2>&1 || die "semgrep is required"

# Each check fails the run rather than degrading it: a scan with no rulesets, or against the
# wrong path, produces a clean-looking empty report.
case "$TARGET" in /*) ;; *) die "--target must be an absolute path, got '${TARGET}'" ;; esac
case "$OUTPUT_DIR" in /*) ;; *) die "--output-dir must be an absolute path, got '${OUTPUT_DIR}'" ;; esac
[ -d "$TARGET" ] || die "--target is not a directory: $TARGET"
case "$MODE" in run-all | important-only) ;; *) die "--mode must be run-all or important-only, got '${MODE}'" ;; esac
if [ -z "$RULESETS_FILE" ] || [ ! -r "$RULESETS_FILE" ]; then
  die "--rulesets file is missing or unreadable: ${RULESETS_FILE}"
fi
jq -e . "$RULESETS_FILE" >/dev/null 2>&1 || die "--rulesets is not valid JSON: $RULESETS_FILE"
case "$JOBS" in '' | *[!0-9]*) die "--jobs must be a positive integer, got '${JOBS}'" ;; esac
[ "$JOBS" -ge 1 ] || die "--jobs must be at least 1"

# Both paths resolve the same way before being compared. Resolving only one makes the equality
# and inside-the-target checks miss on any symlinked path (every /var path on macOS), and the
# run then scans its own cloned rules. The output dir may not exist yet, so the deepest
# existing ancestor is resolved and the rest appended.
resolve_path() {
  local p=$1 tail=""
  while [ ! -d "$p" ]; do
    tail="/${p##*/}$tail"
    p=${p%/*}
    [ -n "$p" ] || p=/
  done
  printf '%s' "$(cd "$p" && pwd)$tail"
}

TARGET=$(resolve_path "$TARGET")
TARGET_ROOT=${TARGET%/}
OUTPUT_ROOT=$(resolve_path "${OUTPUT_DIR%/}")
[ "$OUTPUT_ROOT" != "$TARGET_ROOT" ] ||
  die "--output-dir is the scan target; the run would scan its own output and its cloned rules"

# A denylist, not an allowlist: real directories contain spaces and parentheses, which are
# inert. These four stay live inside double quotes and a control character can smuggle in a
# newline. One pattern each, since one bracket expression cannot hold both quote styles.
has_unsafe_char() {
  case "$1" in
    *'"'*) return 0 ;;
    *'$'*) return 0 ;;
    *'`'*) return 0 ;;
    *[\\]*) return 0 ;;
    *[[:cntrl:]]*) return 0 ;;
  esac
  return 1
}

for p in "$TARGET" "$OUTPUT_ROOT"; do
  ! has_unsafe_char "$p" || die "path contains a character that is not safe to splice: $p"
done

# jq -e over the whole file, so a bad entry stops the run before anything is fetched.
jq -e '
  to_entries
  | all(.value | type == "array")
' "$RULESETS_FILE" >/dev/null 2>&1 || die "every value in --rulesets must be an array"

jq -e '
  to_entries
  | map(select(.key != "third_party") | .value[])
  | all(type == "string" and test("^[A-Za-z0-9._-]+(/[A-Za-z0-9._-]+)*$")
        and (split("/") | index("..") | not))
' "$RULESETS_FILE" >/dev/null 2>&1 ||
  die "ruleset entries must be registry identifiers like p/python; repository URLs go under third_party"

jq -e '
  (.third_party // [])
  | all(type == "string" and test("^https://[A-Za-z0-9.-]+(:[0-9]+)?(/[A-Za-z0-9._-]+)+(\\.git)?/?$"))
' "$RULESETS_FILE" >/dev/null 2>&1 || die "third_party entries must be https git URLs"

RAW_DIR="$OUTPUT_ROOT/raw"
REPOS_DIR="$OUTPUT_ROOT/repos"

# The default output dir lands inside the target, so without --exclude every scan also reads
# the cloned rule repos (full of literal example secrets) and the raw JSON siblings are still
# writing. --exclude takes a pattern, not a rooted path: `out` also excludes `vendor/out`, and
# there is no anchored form, so it is announced rather than applied quietly.
EXCLUDE_ARG=""
EXCLUDE_PATTERN=""
case "$OUTPUT_ROOT/" in
  "$TARGET_ROOT"/*)
    rel=${OUTPUT_ROOT#"$TARGET_ROOT"/}
    EXCLUDE_ARG="--exclude=$rel"
    # Recorded in scans.json, not just announced here. This drops files from every scan, and an
    # unanchored pattern drops more than the output directory: --exclude=out also skips
    # src/out/. On stderr alone the report has nothing to show, so the gap looks like clean
    # coverage — the same silent truncation `coveredNothing` and the unparseable list exist to
    # prevent.
    EXCLUDE_PATTERN="$rel"
    echo "note: output directory is inside the target; excluding '$rel' from every scan." >&2
    echo "      semgrep matches that pattern anywhere in the tree." >&2
    ;;
esac

WORK=$(mktemp -d "${TMPDIR:-/tmp}/run-scans.XXXXXX")
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT

SCAN_LIST="$WORK/scans.tsv"
SKIPPED="$WORK/skipped.tsv"
ALSO_SHARED="$WORK/also-shared.txt"
UNSCOPED="$WORK/unscoped.txt"
# A ruleset whose --include globs matched no file exits 0 with an empty result, which is
# indistinguishable in scans.json from a ruleset that ran and found nothing. That is how a plan
# naming the wrong languages reads as a clean audit: p/python on a Go tree reports 0 findings
# exactly like p/gosec would have. semgrep counts what it opened in .paths.scanned, so the two
# can be told apart and the ones that covered nothing named.
COVERED_NOTHING="$WORK/covered-nothing.txt"
: >"$SCAN_LIST"
: >"$SKIPPED"
: >"$ALSO_SHARED"
: >"$UNSCOPED"
: >"$COVERED_NOTHING"

# Stems are the filename half of every output path and the key results are matched on, so a
# collision would let one scan's result be read as another's.
USED_STEMS="$WORK/stems.txt"
: >"$USED_STEMS"
unique_stem() {
  local base=$1 n=2 candidate=$1
  while grep -Fxq "$candidate" "$USED_STEMS" 2>/dev/null; do
    candidate="$base-$n"
    n=$((n + 1))
  done
  printf '%s\n' "$candidate" >>"$USED_STEMS"
  printf '%s' "$candidate"
}

add_scan() {
  local lang=$1 ruleset=$2 config=$3 includes=$4 stem
  stem=$(unique_stem "$(printf '%s' "$lang" | sed 's/[^A-Za-z0-9._-]\{1,\}/-/g')-$(slug "$ruleset")")
  printf '%s\t%s\t%s\t%s\t%s\n' "$stem" "$lang" "$ruleset" "$config" "$includes" >>"$SCAN_LIST"
}

mapfile_compat() { # read newline-separated stdin into a named array, bash 3.2 safe
  local __name=$1 __line
  eval "$__name=()"
  while IFS= read -r __line; do
    [ -n "$__line" ] || continue
    eval "$__name+=(\"\$__line\")"
  done
}

BASELINE=()
THIRD_PARTY=()
LANG_KEYS=()
mapfile_compat BASELINE < <(jq -r '(.baseline // []) | unique[]' "$RULESETS_FILE")
mapfile_compat THIRD_PARTY < <(jq -r '(.third_party // [])[]' "$RULESETS_FILE")
mapfile_compat LANG_KEYS < <(jq -r 'to_entries[] | select(.key != "baseline" and .key != "third_party" and (.value | length) > 0) | .key' "$RULESETS_FILE")

# Deduplicated by clone directory rather than by exact string: two spellings of one repository
# (with and without .git) survive a string comparison but collide on one destination, and the
# second clone would fail into a non-empty tree.
CLONE_URLS=()
CLONE_NAMES=()
for url in ${THIRD_PARTY[@]+"${THIRD_PARTY[@]}"}; do
  name=$(repo_dir_name "$url")
  seen=""
  for existing in ${CLONE_NAMES[@]+"${CLONE_NAMES[@]}"}; do
    [ "$existing" = "$name" ] && seen=1 && break
  done
  [ -n "$seen" ] && continue
  CLONE_URLS+=("$url")
  CLONE_NAMES+=("$name")
done

if [ -z "$DRY_RUN" ]; then
  # Cleared for the same reason each clone destination is. merge_sarif.py globs every *.sarif
  # in here, so into a reused output directory a run that drops a ruleset still merges the
  # previous run's output for it: results.sarif and its total then cover a ruleset this run's
  # scans.json never mentions. Only this script's own output lives here.
  rm -rf "$RAW_DIR"
  mkdir -p "$RAW_DIR"
  [ ${#CLONE_URLS[@]} -eq 0 ] || mkdir -p "$REPOS_DIR"
fi

# ------------------------------------------------------------------ clone phase
CLONED_URLS=()
CLONED_PATHS=()
i=0
while [ $i -lt ${#CLONE_URLS[@]} ]; do
  url=${CLONE_URLS[$i]}
  name=${CLONE_NAMES[$i]}
  i=$((i + 1))
  dest="$REPOS_DIR/$name"
  if [ -n "$DRY_RUN" ]; then
    CLONED_URLS+=("$url")
    CLONED_PATHS+=("$dest")
    continue
  fi
  # Cleared first: git clone refuses a non-empty directory, so a reused output directory would
  # drop an approved ruleset while usable rules sat on disk.
  rm -rf "$dest"
  if ! err=$(git clone --depth 1 "$url" "$dest" 2>&1); then
    printf '%s\t%s\n' "$url" "$(printf '%s' "$err" | tail -n 3 | tr '\n' ' ')" >>"$SKIPPED"
    continue
  fi
  # semgrep parses EVERY .yaml/.yml under a --config directory as a rule file, and a single
  # unparseable one aborts the whole scan with exit 7 — no findings from any of the rules that
  # were fine. Rule repos ship their own CI config alongside their rules, and a workflow's
  # `on: pull_request:` is a null value, which semgrep rejects outright. Observed killing both
  # trailofbits/semgrep-rules (.github/workflows/semgrep-rules-format.yml) and
  # elttam/semgrep-rules (perf-templates/benchmark-tests.yml) — two required rulesets silently
  # contributing nothing.
  #
  # A semgrep rule file always has a top-level `rules:` key; nothing else here does. Pruning on
  # that also drops the `*.test.yaml` fixtures, which are rule test inputs rather than rules.
  # Measured on this prune alone: keeps 118/145 files for trailofbits and 80/94 for elttam,
  # losing no real rule. The join prune below then takes elttam's one join rule, leaving 79.
  #
  # `mode: join` rules are the second thing semgrep 1.173 cannot take: they crash it outright
  # (AttributeError in join_rule.py), which kills the whole batch and writes no output at all —
  # a hard process failure rather than the rule-level error a bad rule produces. elttam's
  # rules/generic/jsp-likely-xss.yaml is one. Only .yaml/.yml are considered, so a README
  # quoting the docs is not a candidate, and the pair of keys above is what makes it a rule.
  #
  # A prune that deletes nothing is a legitimate outcome — a repo may ship rules and nothing
  # else — but a prune that cannot run is not, so its status is checked and the ruleset is
  # skipped with a reason rather than scanned from a half-pruned tree.
  if ! non_rules=$(prune_yaml "$dest" ! -exec grep -q '^rules:' {} \;) ||
    ! join_rules=$(prune_yaml "$dest" -exec grep -qE "$JOIN_MODE_RE" {} \; \
      -exec grep -qE "$JOIN_BLOCK_RE" {} \;); then
    printf '%s\t%s\n' "$url" "could not prune unscannable YAML from the clone" >>"$SKIPPED"
    rm -rf "$dest"
    continue
  fi
  # Reported every time, including as zeroes. These two prunes are the difference between a
  # ruleset that runs and one that contributes nothing, and a run that stopped pruning would
  # otherwise look exactly like a run with nothing to prune.
  printf '%s: pruned %s non-rule YAML file(s) and %s join-mode rule file(s)\n' \
    "$name" "$(count_lines "$non_rules")" "$(count_lines "$join_rules")" >&2

  # A repository that cloned but carries no rules scans nothing, and reporting it as fine would
  # show a completed scan against a ruleset that never ran.
  #
  # -print -quit rather than `find … | head -1`: under pipefail, head exits after the first line
  # and find dies on SIGPIPE with 141 once its output passes the pipe buffer. pipefail makes the
  # pipeline non-zero, so a repository with enough rules to fill 64 KiB of pathnames — which is
  # every real rule repo, trailofbits/semgrep-rules included — was recorded as carrying none.
  # The required third-party rules then silently did not run. -quit stops at the first match
  # with no reader to race.
  if [ -z "$(find "$dest" \( -name '*.yaml' -o -name '*.yml' \) -type f -print -quit)" ]; then
    printf '%s\t%s\n' "$url" "cloned but contains no rule files" >>"$SKIPPED"
    continue
  fi
  CLONED_URLS+=("$url")
  CLONED_PATHS+=("$dest")
done

# ------------------------------------------------------------- build the scan list
# Cross-language rulesets scan the whole target unscoped, so they run once for the run rather
# than once per language, and they never take --include: they carry rules for every language
# and a filter would drop findings in the files it does not match.
for ruleset in ${BASELINE[@]+"${BASELINE[@]}"}; do
  add_scan "all" "$ruleset" "$ruleset" ""
done
i=0
while [ $i -lt ${#CLONED_URLS[@]} ]; do
  add_scan "all" "${CLONED_URLS[$i]}" "${CLONED_PATHS[$i]}" ""
  i=$((i + 1))
done

is_baseline() {
  for b in ${BASELINE[@]+"${BASELINE[@]}"}; do
    [ "$b" = "$1" ] && return 0
  done
  return 1
}

# Language keys are folded onto their canonical name before any unit exists, so `js` and
# `javascript` in one plan produce one unit rather than two with identical globs.
CANON_SEEN="$WORK/canon.txt"
: >"$CANON_SEEN"
for key in ${LANG_KEYS[@]+"${LANG_KEYS[@]}"}; do
  # The key reaches the scan command as the language half of the output filename, and it
  # arrives as free-form generated text, so it gets the same treatment the paths do.
  ! has_unsafe_char "$key" || die "ruleset key reaches the scan command as a filename; '$key' is not a language"
  lang=$(canonical_lang "$key")
  [ "$lang" != "all" ] || die "ruleset key '$key' names the reserved language 'all' used by the cross-language unit"
  printf '%s\n' "$lang" >>"$CANON_SEEN"
done

# Redirected rather than piped: a `for … in $(…)` splits on every whitespace character, and a
# pipeline would put the loop in a subshell.
sort -u "$CANON_SEEN" >"$WORK/langs.txt"
while IFS= read -r lang; do
  [ -n "$lang" ] || continue
  # jq cannot express canonical_lang, so the union happens here: every key folding onto this
  # language contributes its rulesets, deduplicated. A ruleset listed twice for one language
  # would otherwise scan twice, and the second copy's failure would read as the first's success.
  own=""
  for key in ${LANG_KEYS[@]+"${LANG_KEYS[@]}"}; do
    [ "$(canonical_lang "$key")" = "$lang" ] || continue
    own="$own$(jq -r --arg k "$key" '.[$k][]' "$RULESETS_FILE")
"
  done
  own=$(printf '%s' "$own" | grep -v '^$' | sort -u || true)
  [ -n "$own" ] || continue

  globs=$(includes_for "$lang")
  had_own=""
  while IFS= read -r ruleset; do
    [ -n "$ruleset" ] || continue
    # A ruleset already running unscoped over the whole target does not need a narrower second
    # run. The merged SARIF dedups the copies; a per-scan sum of findings does not.
    if is_baseline "$ruleset"; then
      printf '%s/%s\n' "$lang" "$ruleset" >>"$ALSO_SHARED"
      continue
    fi
    add_scan "$lang" "$ruleset" "$ruleset" "$globs"
    had_own=1
  done <<EOF
$own
EOF
  # Reported only when the language actually got a unit. An unrecognized name costs the
  # --include optimization, not coverage: the rules run against every file and fail to match
  # what they do not understand. A language emptied by the baseline dedup ran nothing at all,
  # so naming it here would point at a scan that never happened.
  if [ -n "$had_own" ] && [ -z "$globs" ]; then
    printf '%s\n' "$lang" >>"$UNSCOPED"
  fi
done <"$WORK/langs.txt"

[ -s "$SCAN_LIST" ] || die "the ruleset plan produced no scans; there is nothing to run"

# ---------------------------------------------------------------------- commands
# Populates the global ARGV with the command for one scan. An array executed directly rather
# than a string run through eval: `eval "$cmd" &` reports exit status 1 whatever the command
# actually exited with, which would mark every failed scan as a success, and building argv also
# leaves no shell-quoting surface for a ruleset or path to escape through.
ARGV=()
build_argv() {
  local config=$1 includes=$2 json=$3 sarif=$4 g
  ARGV=(semgrep)
  [ -z "$PRO" ] || ARGV+=(--pro)
  ARGV+=("$METRICS_OFF")
  [ "$MODE" != "important-only" ] || ARGV+=("${SEVERITY_FLAGS[@]}")
  # Unquoted on purpose: includes is a space-separated glob list and must word-split here.
  # shellcheck disable=SC2086
  for g in $includes; do ARGV+=("--include=$g"); done
  # On every command including the unscoped cross-language ones: those are precisely the
  # rulesets that would otherwise read the cloned rule repositories.
  [ -z "$EXCLUDE_ARG" ] || ARGV+=("$EXCLUDE_ARG")
  ARGV+=(--config "$config" --json -o "$json" "--sarif-output=$sarif" "$TARGET")
}

# A pasteable rendering of ARGV for --dry-run. Anything outside a plainly safe set is wrapped
# whole, so a glob reaches semgrep rather than being expanded by the shell it is pasted into.
render_argv() {
  local out="" a
  for a in "${ARGV[@]}"; do
    case "$a" in
      *[!A-Za-z0-9._=/:-]*) out="$out \"$a\"" ;;
      *) out="$out $a" ;;
    esac
  done
  printf '%s' "${out# }"
}

if [ -n "$DRY_RUN" ]; then
  while IFS=$'\t' read -r stem lang ruleset config includes; do
    build_argv "$config" "$includes" "$RAW_DIR/$stem.json" "$RAW_DIR/$stem.sarif"
    render_argv
    printf '\n'
  done <"$SCAN_LIST"
  exit 0
fi

# ------------------------------------------------------------------- run the scans
# Batched rather than a rolling slot count, because `wait -n` needs bash 4.3 and this has to run
# on the bash 3.2 that ships with macOS. semgrep holds the rules and the scanned ASTs in memory,
# so an unbounded fan-out gets processes OOM-killed and those come back as ordinary scan
# failures with nothing pointing at memory as the cause.
total=$(wc -l <"$SCAN_LIST" | tr -d ' ')
echo "running $total scan(s), $JOBS at a time" >&2

pids=()
stems=()
run_batch() {
  local idx=0
  for idx in "${!pids[@]}"; do
    if wait "${pids[$idx]}"; then
      echo 0 >"$WORK/rc.${stems[$idx]}"
    else
      echo $? >"$WORK/rc.${stems[$idx]}"
    fi
  done
  pids=()
  stems=()
}

while IFS=$'\t' read -r stem lang ruleset config includes; do
  json="$RAW_DIR/$stem.json"
  sarif="$RAW_DIR/$stem.sarif"
  build_argv "$config" "$includes" "$json" "$sarif"
  "${ARGV[@]}" >"$WORK/out.$stem" 2>"$WORK/err.$stem" &
  pids+=("$!")
  stems+=("$stem")
  [ ${#pids[@]} -lt "$JOBS" ] || run_batch
done <"$SCAN_LIST"
[ ${#pids[@]} -eq 0 ] || run_batch

# ---------------------------------------------------------------------- assemble
: >"$WORK/scans.jsonl"
: >"$WORK/failed.jsonl"
while IFS=$'\t' read -r stem lang ruleset config includes; do
  json="$RAW_DIR/$stem.json"
  sarif="$RAW_DIR/$stem.sarif"
  rc=$(cat "$WORK/rc.$stem" 2>/dev/null || echo 127)
  findings=-1
  scanned=-1
  ok=""
  # Exit 0 covers both "found nothing" and "found plenty", so it says nothing about findings.
  # Exit 1 is a successful scan on older versions.
  #
  # Exit 2 is not only "no scan happened". semgrep also returns 2 when individual rules fail
  # to compile while the rest of the run completes and writes complete output — e.g. 12 Java
  # rules in elttam/semgrep-rules that current semgrep cannot parse, alongside 107 that ran
  # fine, and apiiro/malicious-code-ruleset whose own log read "Scan completed successfully
  # • Findings: 51". Both were thrown away. Exit 2 also still covers a bad argument, where
  # nothing is written at all; the artifact checks below tell the two apart, so 2 is allowed
  # through and flagged partial rather than trusted outright.
  #
  # Anything outside 0/1/2 stays fatal however plausible the artifacts look, exit 7 (config
  # would not load) included. Verified on semgrep 1.173: the exit code for an unloadable
  # config depends on the OUTPUT FLAGS, which is worth knowing before trusting either number.
  # Same rules directory, same target, back to back:
  #
  #   semgrep --config rules target                              -> 7, nothing written
  #   semgrep --config rules -o out.json --sarif-output=out.sarif -> 2, nothing written
  #
  # This script uses the second form, so a config that will not load reaches here as 2 with
  # no artifacts, and the -s checks below reject it on their own. The fatal branch is
  # therefore belt-and-braces rather than the thing doing the work — but it costs nothing,
  # it is what the suite pins, and it means a future semgrep that writes an empty result set
  # alongside a hard failure cannot be read as a clean scan.
  partial=""
  fatal=""
  case "$rc" in
    0 | 1) ;;
    2) partial=1 ;;
    *) fatal=1 ;;
  esac
  if [ -z "$fatal" ] && [ -s "$json" ] && [ -s "$sarif" ]; then
    if findings=$(jq -e '.results | length' "$json" 2>/dev/null); then
      ok=1
      # null and empty are different answers: a semgrep that does not report .paths gives -1,
      # "not known", while a present-but-empty list is a scan that genuinely opened no file.
      # Collapsing them with `// []` would report every scan as covering nothing.
      scanned=$(jq 'if .paths.scanned == null then -1 else (.paths.scanned | length) end' \
        "$json" 2>/dev/null || echo -1)
    else
      findings=-1
    fi
  fi
  if [ -n "$ok" ]; then
    [ "$scanned" -ne 0 ] || printf '%s/%s\n' "$lang" "$ruleset" >>"$COVERED_NOTHING"
    # `partial` marks a run whose results are real but incomplete — some rules failed to
    # compile. Reporting it as an unqualified success would overstate coverage; dropping it
    # entirely (the old behaviour) understated it far worse.
    jq -nc --arg lang "$lang" --arg ruleset "$ruleset" --arg json "$json" \
      --arg sarif "$sarif" --argjson findings "$findings" --argjson scanned "$scanned" \
      --argjson partial "$([ -n "$partial" ] && echo true || echo false)" --arg rc "$rc" \
      '{lang:$lang, ruleset:$ruleset, json:$json, sarif:$sarif, findings:$findings,
        filesScanned:$scanned, partial:$partial, exitCode:($rc|tonumber)}' \
      >>"$WORK/scans.jsonl"
  else
    # Carries the same paths a success does: a scan that crashed part-way may still have
    # written a partial file, and the report needs to be able to name it.
    err=$(tail -c 800 "$WORK/err.$stem" 2>/dev/null || true)
    [ -n "$err" ] || err="semgrep exited $rc"
    jq -nc --arg lang "$lang" --arg ruleset "$ruleset" --arg json "$json" \
      --arg sarif "$sarif" --arg error "$err" \
      '{lang:$lang, ruleset:$ruleset, json:$json, sarif:$sarif, error:$error}' \
      >>"$WORK/failed.jsonl"
  fi
done <"$SCAN_LIST"

PRO_JSON=false
[ -z "$PRO" ] || PRO_JSON=true

jq -n \
  --arg outputDir "$OUTPUT_ROOT" \
  --arg rawDir "$RAW_DIR" \
  --arg reposPath "$REPOS_DIR" \
  --arg mode "$MODE" \
  --arg excludePattern "$EXCLUDE_PATTERN" \
  --argjson pro "$PRO_JSON" \
  --slurpfile scans "$WORK/scans.jsonl" \
  --slurpfile failed "$WORK/failed.jsonl" \
  --rawfile skippedRaw "$SKIPPED" \
  --rawfile alsoSharedRaw "$ALSO_SHARED" \
  --rawfile unscopedRaw "$UNSCOPED" \
  --rawfile coveredNothingRaw "$COVERED_NOTHING" \
  '{
     outputDir: $outputDir, rawDir: $rawDir, reposPath: $reposPath, mode: $mode, pro: $pro,
     excludePattern: $excludePattern,
     scans: $scans, failed: $failed,
     skipped: ($skippedRaw | split("\n") | map(select(length > 0)) | map(split("\t"))
               | map({ruleset: .[0], reason: (.[1] // "clone failed")})),
     alsoShared: ($alsoSharedRaw | split("\n") | map(select(length > 0)) | unique),
     unscoped: ($unscopedRaw | split("\n") | map(select(length > 0)) | unique),
     coveredNothing: ($coveredNothingRaw | split("\n") | map(select(length > 0)) | unique)
   }' >"$OUTPUT_ROOT/scans.json"

n_ok=$(jq '.scans | length' "$OUTPUT_ROOT/scans.json")
n_failed=$(jq '.failed | length' "$OUTPUT_ROOT/scans.json")
n_skipped=$(jq '.skipped | length' "$OUTPUT_ROOT/scans.json")
echo "$n_ok scan(s) succeeded, $n_failed failed, $n_skipped skipped" >&2
echo "$OUTPUT_ROOT/scans.json"

# A run where nothing succeeded is a failed run, not a clean one. Reporting zero findings from
# it is the failure this whole script is shaped to avoid.
[ "$n_ok" -gt 0 ] || exit 1

# /// script
# requires-python = ">=3.11"
# dependencies = ["pytest>=8"]
# ///
"""Tests for merge_sarif.py, with the weight on --important.

The important-only merge rests on one claim: a finding's identity in semgrep's JSON output
(check_id, path, start.line) is the same triple SARIF carries as (ruleId, uri,
region.startLine). test_key_contract pins the field names this script reads, but builds both
halves itself, so only test_key_contract_against_real_semgrep can notice semgrep changing
either shape. That one runs semgrep and compares the two records of one real finding.

The negatives matter as much: a post-filter that ran over only some scans, or wrote a file
that will not parse, must fail the merge. Filtering against a partial key set drops real
findings from the primary deliverable and nothing downstream could notice. The exception is a
scan recorded under .failed in scans.json, whose output is whatever a dying process wrote:
--scans drops those, so one crashed scan cannot deny every healthy scan a deliverable.
"""

from __future__ import annotations

import json
import shutil
import subprocess
import sys
from pathlib import Path

import pytest
from merge_sarif import (
    filter_to_keys,
    json_key,
    merge_sarif_pure_python,
    sarif_key,
    surviving_keys,
)

SCRIPT = Path(__file__).with_name("merge_sarif.py")

RULE = "python.lang.security.insecure-hash-algorithms-md5.insecure-hash-algorithm-md5"
OTHER = "python.lang.security.audit.subprocess-shell-true.subprocess-shell-true"


def sarif_result(rule: str, uri: str, line: int) -> dict:
    return {
        "ruleId": rule,
        "message": {"text": "finding"},
        "locations": [
            {
                "physicalLocation": {
                    "artifactLocation": {"uri": uri},
                    "region": {"startLine": line, "startColumn": 1},
                }
            }
        ],
    }


def sarif_doc(*results: dict) -> dict:
    return {
        "version": "2.1.0",
        "runs": [{"tool": {"driver": {"name": "semgrep", "rules": []}}, "results": list(results)}],
    }


def json_result(rule: str, path: str, line: int) -> dict:
    return {"check_id": rule, "path": path, "start": {"line": line, "col": 1}, "extra": {}}


def write_scan(raw: Path, stem: str, sarif: list[dict], filtered: list[dict] | None) -> None:
    """One scan's output: the SARIF the merge reads, and optionally its post-filtered JSON."""
    (raw / f"{stem}.sarif").write_text(json.dumps(sarif_doc(*sarif)))
    if filtered is not None:
        (raw / f"{stem}-important.json").write_text(
            json.dumps({"results": filtered, "errors": [], "paths": {}})
        )


def run_merge(raw: Path, out: Path, *flags: str) -> subprocess.CompletedProcess:
    return subprocess.run(
        [sys.executable, str(SCRIPT), str(raw), str(out), *flags],
        capture_output=True,
        text=True,
    )


def count(sarif_file: Path) -> int:
    data = json.loads(sarif_file.read_text())
    return sum(len(run.get("results", [])) for run in data.get("runs", []))


# --------------------------------------------------------------- the cross-format contract


def test_key_contract():
    """The shapes this script expects, pinned.

    Both halves are built by this file from one literal path, so this fixes the field names
    `sarif_key` and `json_key` read and nothing more. It cannot notice semgrep changing either
    output — test_key_contract_against_real_semgrep below is the one that can.
    """
    from_json = json_result(RULE, "src/app.py", 5)
    from_sarif = sarif_result(RULE, "src/app.py", 5)
    assert json_key(from_json) == sarif_key(from_sarif) == (RULE, "src/app.py", 5)


MD5_RULE = """\
rules:
  - id: insecure-md5
    pattern: hashlib.md5(...)
    message: md5 is insecure
    languages: [python]
    severity: WARNING
"""


def semgrep_bin() -> str:
    """Fail rather than skip, the same reason run_workflow_tests.sh fails without node.

    A skip here reads as a clean run while the only check that could catch cross-format drift
    silently did not execute. semgrep is this plugin's own dependency and CI installs it.
    """
    found = shutil.which("semgrep")
    if not found:
        raise AssertionError(
            "semgrep is not installed, so the JSON/SARIF contract went unverified. "
            "Install it (uv tool install semgrep) — this suite must not pass without it."
        )
    return found


@pytest.mark.parametrize("absolute", [True, False])
def test_key_contract_against_real_semgrep(tmp_path, absolute):
    """The contract read out of semgrep itself, over one real finding.

    --important rests entirely on the claim that (check_id, path, start.line) in the JSON is
    (ruleId, uri, region.startLine) in the SARIF. Only this test can see that claim break: if
    semgrep changes either shape, the keys stop matching, every finding is dropped and the
    deliverable goes empty. run-scans.sh always passes an absolute target; the relative case is
    parametrized because a path shape is exactly where the two formats would diverge first.
    """
    src = tmp_path / "src"
    src.mkdir()
    (src / "app.py").write_text(
        "import hashlib\ndef f(x):\n    return hashlib.md5(x).hexdigest()\n"
    )
    rule = tmp_path / "md5.yaml"
    rule.write_text(MD5_RULE)
    json_out = tmp_path / "out.json"
    sarif_out = tmp_path / "out.sarif"

    proc = subprocess.run(
        [
            semgrep_bin(),
            "--metrics=off",
            "--config",
            str(rule),
            "--json",
            "-o",
            str(json_out),
            f"--sarif-output={sarif_out}",
            str(src) if absolute else "src",
        ],
        cwd=tmp_path,
        capture_output=True,
        text=True,
    )
    assert json_out.is_file(), f"semgrep wrote no JSON: {proc.stderr}"
    assert sarif_out.is_file(), f"semgrep wrote no SARIF: {proc.stderr}"

    json_results = json.loads(json_out.read_text())["results"]
    sarif_results = json.loads(sarif_out.read_text())["runs"][0]["results"]
    assert len(json_results) == 1, f"expected one JSON finding, got {len(json_results)}"
    assert len(sarif_results) == 1, f"expected one SARIF finding, got {len(sarif_results)}"

    # The whole contract in one line. A mismatch here is the empty-deliverable bug, found at
    # its source rather than as a zero-finding results.sarif nobody can explain.
    assert json_key(json_results[0]) == sarif_key(sarif_results[0])


def test_keys_differ_on_line():
    assert json_key(json_result(RULE, "src/app.py", 5)) != sarif_key(
        sarif_result(RULE, "src/app.py", 6)
    )


def test_sarif_key_tolerates_a_result_with_no_location():
    assert sarif_key({"ruleId": RULE}) == (RULE, "", 0)


# ------------------------------------------------------------------------- surviving_keys


def test_surviving_keys_reads_every_scan(tmp_path):
    raw = tmp_path
    write_scan(raw, "py", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)])
    write_scan(raw, "secrets", [sarif_result(OTHER, "b.py", 9)], [json_result(OTHER, "b.py", 9)])
    keys = surviving_keys(sorted(raw.glob("*.sarif")))
    assert keys == {(RULE, "a.py", 5), (OTHER, "b.py", 9)}


def test_surviving_keys_is_empty_when_the_filter_kept_nothing(tmp_path):
    """A real outcome, distinct from a filter that never ran: the files exist and are empty."""
    write_scan(tmp_path, "python-python", [sarif_result(RULE, "a.py", 5)], [])
    assert surviving_keys(sorted(tmp_path.glob("*.sarif"))) == set()


def test_a_scan_with_no_filtered_json_fails_the_merge(tmp_path):
    """The silent-omission case: without this, that scan's findings vanish from results.sarif."""
    raw = tmp_path
    write_scan(raw, "py", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)])
    write_scan(raw, "all-secrets", [sarif_result(OTHER, "b.py", 9)], None)
    with pytest.raises(ValueError, match="all-secrets-important.json"):
        surviving_keys(sorted(raw.glob("*.sarif")))


def test_an_unparseable_filter_file_fails_the_merge(tmp_path):
    write_scan(tmp_path, "python-python", [sarif_result(RULE, "a.py", 5)], [])
    (tmp_path / "python-python-important.json").write_text("not json")
    with pytest.raises(ValueError, match="not valid JSON"):
        surviving_keys(sorted(tmp_path.glob("*.sarif")))


def test_a_filter_file_with_no_results_array_fails_the_merge(tmp_path):
    """Catches a SARIF file handed in where a filtered JSON belongs; it would filter to nothing."""
    write_scan(tmp_path, "python-python", [sarif_result(RULE, "a.py", 5)], [])
    (tmp_path / "python-python-important.json").write_text(json.dumps(sarif_doc()))
    with pytest.raises(ValueError, match="no .results array"):
        surviving_keys(sorted(tmp_path.glob("*.sarif")))


# -------------------------------------------------------------------------- filter_to_keys


def test_filter_keeps_only_surviving_findings():
    merged = sarif_doc(sarif_result(RULE, "a.py", 5), sarif_result(OTHER, "a.py", 3))
    kept, dropped = filter_to_keys(merged, {(RULE, "a.py", 5)})
    assert (kept, dropped) == (1, 1)
    assert [r["ruleId"] for r in merged["runs"][0]["results"]] == [RULE]


def test_filter_against_an_empty_key_set_empties_the_results():
    merged = sarif_doc(sarif_result(RULE, "a.py", 5))
    assert filter_to_keys(merged, set()) == (0, 1)
    assert merged["runs"][0]["results"] == []


# ------------------------------------------------------------------------------ end to end


def test_important_merge_filters_the_deliverable(tmp_path):
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw,
        "python-python",
        [sarif_result(RULE, "a.py", 5), sarif_result(OTHER, "a.py", 3)],
        [json_result(RULE, "a.py", 5)],
    )
    out = tmp_path / "results" / "results.sarif"
    proc = run_merge(raw, out, "--important")
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1
    assert json.loads(out.read_text())["runs"][0]["results"][0]["ruleId"] == RULE


def test_run_all_merge_keeps_everything(tmp_path):
    """The default path must be unchanged by the flag's existence."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw,
        "python-python",
        [sarif_result(RULE, "a.py", 5), sarif_result(OTHER, "a.py", 3)],
        [json_result(RULE, "a.py", 5)],
    )
    out = tmp_path / "results" / "results.sarif"
    assert run_merge(raw, out).returncode == 0
    assert count(out) == 2


def test_important_without_a_post_filter_fails_and_writes_nothing(tmp_path):
    """The whole point of resolving keys before the merge: no half-right file on disk."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    out = tmp_path / "results" / "results.sarif"
    proc = run_merge(raw, out, "--important")
    assert proc.returncode == 1
    assert "post-filtered JSON" in proc.stderr
    assert not out.exists()


def test_a_total_key_mismatch_fails_the_merge(tmp_path):
    """Cross-format drift, forced: the filter kept a finding no merged result matches.

    Without the guard this writes a zero-finding results.sarif and exits 0, so important-only
    reports a clean run that found nothing while the JSON side has findings.
    """
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw,
        "python-python",
        [sarif_result(RULE, "src/app.py", 5)],
        [json_result(RULE, "/abs/proj/src/app.py", 5)],
    )
    out = tmp_path / "results" / "results.sarif"
    proc = run_merge(raw, out, "--important")
    assert proc.returncode == 1, proc.stdout
    assert "none of them matched" in proc.stderr
    assert not out.exists()


def test_a_filter_that_kept_nothing_is_not_a_mismatch(tmp_path):
    """The guard is conditioned on the key set, not on the kept count.

    A post-filter that legitimately excluded every finding is a real zero. If this goes red,
    important-only can no longer report an honest empty result.
    """
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], [])
    out = tmp_path / "results" / "results.sarif"
    proc = run_merge(raw, out, "--important")
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 0


def test_a_partial_key_mismatch_still_merges(tmp_path):
    """One matching key is enough to prove the formats still agree; the rest is the filter."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw,
        "python-python",
        [sarif_result(RULE, "a.py", 5), sarif_result(OTHER, "a.py", 3)],
        [json_result(RULE, "a.py", 5), json_result(OTHER, "/elsewhere/a.py", 3)],
    )
    out = tmp_path / "results" / "results.sarif"
    proc = run_merge(raw, out, "--important")
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1


def test_important_leaves_an_existing_deliverable_alone_when_it_fails(tmp_path):
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    out = tmp_path / "results.sarif"
    out.write_text(json.dumps(sarif_doc(sarif_result(RULE, "a.py", 5))))
    before = out.read_text()
    assert run_merge(raw, out, "--important").returncode == 1
    assert out.read_text() == before


# --------------------------------------------------------------------- unparseable SARIF


def test_an_unparseable_sarif_is_named_on_stdout(tmp_path):
    """The silent-omission case this whole flag set exists to prevent.

    A scan can exit 0, write a valid .json — so it lands in .scans with a finding count — and
    still leave a truncated .sarif. The merge drops it and the total is short by exactly those
    findings. On stderr that was invisible to the report; it has to be in the same stream the
    Report phase reads, and named, or the run presents as clean.
    """
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    (raw / "python-broken.sarif").write_text('{"runs":[{"results":[')
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out)
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1, "the healthy scan must still merge"
    assert "unparseable" in proc.stdout
    assert "python-broken.sarif" in proc.stdout, "the file must be named, not just counted"


def test_every_sarif_unparseable_is_an_error(tmp_path):
    """Zero findings from zero readable files is a broken run, not a clean one."""
    raw = tmp_path / "raw"
    raw.mkdir()
    (raw / "python-broken.sarif").write_text('{"runs":[{"results":[')
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out)
    assert proc.returncode == 1
    assert "nothing to merge" in proc.stderr
    assert not out.exists()


def test_the_merge_returns_what_it_could_not_read(tmp_path):
    """The list is a return value, so a caller cannot forget to look at it."""
    write_scan(tmp_path, "ok", [sarif_result(RULE, "a.py", 5)], None)
    (tmp_path / "bad.sarif").write_text("{{{")
    merged, unparseable = merge_sarif_pure_python(sorted(tmp_path.glob("*.sarif")))
    assert [Path(p).name for p in unparseable] == ["bad.sarif"]
    assert sum(len(r["results"]) for r in merged["runs"]) == 1


# ------------------------------------------------------------------------- one merge backend


def test_the_merge_shells_out_to_nothing(tmp_path):
    """Run with an empty PATH, so any external merge tool is unreachable.

    The merge used to try `npx @microsoft/sarif-multitool` first and fall back to Python, which
    made the result depend on whether that package sat in the npx cache. Only the Python merge
    dedups on sarif_key, and multitool rewrites artifactLocation.uri, so on a machine that had
    it cached --important would match nothing and blame a semgrep format change. One backend,
    same answer everywhere.
    """
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw, "python-python", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)]
    )
    out = tmp_path / "results.sarif"
    proc = subprocess.run(
        [sys.executable, str(SCRIPT), str(raw), str(out), "--important"],
        capture_output=True,
        text=True,
        env={"PATH": "", "HOME": str(tmp_path)},
    )
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1
    assert "multitool" not in proc.stdout.lower(), "no external merge tool may be consulted"

    # The empty PATH above proves the merge survives without a backend, not that it stopped
    # looking for one: a reintroduced optional branch would just fall back and pass. This is
    # the assertion that fails if one comes back.
    assert "import subprocess" not in SCRIPT.read_text(), (
        "merge_sarif.py must not shell out; a second merge backend disagrees with this one "
        "on dedup and on artifactLocation.uri, and which one runs would depend on the machine"
    )


# ------------------------------------------------------------------------------------ --scans


def write_scans_json(path: Path, succeeded: list[Path], failed: list[Path]) -> Path:
    """A scans.json in run-scans.sh's shape. Both lists carry the same `sarif` key."""
    path.write_text(
        json.dumps(
            {
                "scans": [
                    {"lang": "python", "ruleset": "p/python", "sarif": str(p)} for p in succeeded
                ],
                "failed": [
                    {"lang": "python", "ruleset": "p/x", "sarif": str(p), "error": "exited 7"}
                    for p in failed
                ],
                "skipped": [],
            }
        )
    )
    return path


def dead_scan(raw: Path, stem: str) -> Path:
    """A scan recorded under .failed: its SARIF is on disk with no post-filter beside it."""
    sarif = raw / f"{stem}.sarif"
    sarif.write_text(json.dumps(sarif_doc(sarif_result(OTHER, "b.py", 9))))
    return sarif


def test_a_failed_scan_no_longer_denies_every_other_scan_a_deliverable(tmp_path):
    """The point of the flag: one dead scan must not take the whole important-only merge."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw, "python-python", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)]
    )
    dead = dead_scan(raw, "python-broken")
    scans = write_scans_json(tmp_path / "scans.json", [raw / "python-python.sarif"], [dead])
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out, "--important", "--scans", str(scans))
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1
    assert "python-broken.sarif" in proc.stdout, "the excluded file must be named for the report"


def test_the_same_run_without_scans_json_still_fails(tmp_path):
    """Pins that the flag is what makes it survivable, not a change in the merge's strictness."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw, "python-python", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)]
    )
    dead_scan(raw, "python-broken")
    out = tmp_path / "results.sarif"
    assert run_merge(raw, out, "--important").returncode == 1
    assert not out.exists()


def test_a_succeeded_scan_missing_its_filter_still_fails(tmp_path):
    """Only failed scans are exempt.

    A healthy scan with no post-filter beside it still aborts the merge: its findings are real,
    and filtering against a key set that never saw them drops them from the deliverable with
    nothing downstream able to notice.
    """
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw, "python-python", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)]
    )
    healthy = dead_scan(raw, "python-other")  # same shape, but recorded as a success below
    scans = write_scans_json(tmp_path / "scans.json", [raw / "python-python.sarif", healthy], [])
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out, "--important", "--scans", str(scans))
    assert proc.returncode == 1
    assert "python-other-important.json" in proc.stderr
    assert not out.exists()


def test_run_all_also_drops_a_failed_scan_output(tmp_path):
    """A dead process's file is not a scan result in either mode."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    dead = dead_scan(raw, "python-broken")
    scans = write_scans_json(tmp_path / "scans.json", [raw / "python-python.sarif"], [dead])
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out, "--scans", str(scans))
    assert proc.returncode == 0, proc.stderr
    assert count(out) == 1, "the failed scan's finding must not reach the merge"


def test_every_scan_failed_is_an_error_not_an_empty_merge(tmp_path):
    """Excluding everything would otherwise write an empty SARIF and report a clean run."""
    raw = tmp_path / "raw"
    raw.mkdir()
    dead = dead_scan(raw, "python-broken")
    scans = write_scans_json(tmp_path / "scans.json", [], [dead])
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out, "--scans", str(scans))
    assert proc.returncode == 1
    assert "nothing to merge" in proc.stderr
    assert not out.exists()


def test_a_scans_json_with_no_failed_array_is_rejected(tmp_path):
    """Catches the wrong file being passed; treating it as 'nothing failed' would be silent."""
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(
        raw, "python-python", [sarif_result(RULE, "a.py", 5)], [json_result(RULE, "a.py", 5)]
    )
    bad = tmp_path / "scans.json"
    bad.write_text(json.dumps({"scans": []}))
    out = tmp_path / "results.sarif"
    proc = run_merge(raw, out, "--important", "--scans", str(bad))
    assert proc.returncode == 1
    assert "not a scans.json" in proc.stderr
    assert not out.exists()


def test_scans_flag_without_a_path_is_rejected(tmp_path):
    raw = tmp_path / "raw"
    raw.mkdir()
    write_scan(raw, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    proc = run_merge(raw, tmp_path / "results.sarif", "--scans")
    assert proc.returncode == 1
    assert "--scans needs" in proc.stderr


def test_an_empty_raw_directory_is_an_error(tmp_path):
    raw = tmp_path / "raw"
    raw.mkdir()
    assert run_merge(raw, tmp_path / "o.sarif").returncode == 1


# ------------------------------------------------------------------------------ merge dedup


def test_merge_dedups_one_finding_flagged_by_two_rulesets(tmp_path):
    """The reason the report counts from the merge and never sums per-scan counts."""
    write_scan(tmp_path, "python-python", [sarif_result(RULE, "a.py", 5)], None)
    write_scan(tmp_path, "all-audit", [sarif_result(RULE, "a.py", 5)], None)
    merged, _ = merge_sarif_pure_python(sorted(tmp_path.glob("*.sarif")))
    assert sum(len(run["results"]) for run in merged["runs"]) == 1

# Semgrep Scan Workflow

Complete 5-step scan execution process. Read from start to finish and follow each step in order.

## Task System Enforcement

On invocation, create these tasks with dependencies:

```
TaskCreate: "Detect languages and Pro availability" (Step 1)
TaskCreate: "Select scan mode and rulesets" (Step 2) - blockedBy: Step 1
TaskCreate: "Present plan with rulesets, get approval" (Step 3) - blockedBy: Step 2
TaskCreate: "Execute scans with approved rulesets and mode" (Step 4) - blockedBy: Step 3
TaskCreate: "Merge results and report" (Step 5) - blockedBy: Step 4
```

### Mandatory Gate

| Task | Gate Type | Cannot Proceed Until |
|------|-----------|---------------------|
| Step 3 | **HARD GATE** | User explicitly approves rulesets + plan |

Mark Step 3 as `completed` ONLY after user says "yes", "proceed", "approved", or equivalent.

---

## Step 1: Resolve Output Directory, Detect Languages and Pro Availability

> **Entry:** User has specified or confirmed the target directory.
> **Exit:** `OUTPUT_DIR` resolved and created; language list with file counts produced; Pro availability determined.

### Resolve Output Directory

If the user specified an output directory in their prompt, use it as `OUTPUT_DIR`. Otherwise, auto-increment. In both cases, **always `mkdir -p`** to ensure the directory exists.

```bash
if [ -n "$USER_SPECIFIED_DIR" ]; then
  OUTPUT_DIR="$USER_SPECIFIED_DIR"
else
  BASE="static_analysis_semgrep"
  N=1
  while [ -e "${BASE}_${N}" ]; do
    N=$((N + 1))
  done
  OUTPUT_DIR="${BASE}_${N}"
fi
mkdir -p "$OUTPUT_DIR/raw" "$OUTPUT_DIR/results"

# Absolute from here on. run-scans.sh rejects a relative path, and that rejection lands
# *after* the user has passed the hard gate, so a path this skill generated itself would send
# them back through approval.
OUTPUT_DIR=$(cd "$OUTPUT_DIR" && pwd)
# The -d test first: `cd ""` returns 0, so a TARGET that was never bound would pass a bare
# `cd || exit` and silently resolve to the session's CWD, scanning whatever happens to be there.
[ -n "$TARGET" ] && [ -d "$TARGET" ] || { echo "ERROR: TARGET is unset or not a directory"; exit 1; }
TARGET=$(cd "$TARGET" && pwd)
echo "Output directory: $OUTPUT_DIR"
echo "Target: $TARGET"
```

Pass `$TARGET` and `$OUTPUT_DIR` to Step 4 exactly as resolved here. Do not re-derive either.

`$OUTPUT_DIR` is used by all subsequent steps. Raw per-scan output goes to `$OUTPUT_DIR/raw/`; merged and filtered results go to `$OUTPUT_DIR/results/`.

**Detect Pro availability** (requires Bash):

```bash
if ! command -v semgrep >/dev/null 2>&1; then
  echo "ERROR: semgrep is not installed. Install from https://semgrep.dev/docs/getting-started/"
  exit 1
fi
semgrep --version
# --metrics=off applies here too. This is the first semgrep invocation of the run and it
# resolves p/default against the registry, so without the flag an audit phones home before
# the user has approved anything. Principle 1 has no exceptions.
semgrep --pro --validate --metrics=off --config p/default 2>/dev/null && echo "Pro: AVAILABLE" || echo "Pro: NOT AVAILABLE"
```

**Detect languages** using Glob (not Bash). Run these patterns against the target directory and count matches:

`**/*.py`, `**/*.pyi`, `**/*.js`, `**/*.jsx`, `**/*.mjs`, `**/*.cjs`, `**/*.ts`, `**/*.tsx`, `**/*.go`, `**/*.rb`, `**/*.java`, `**/*.jsp`, `**/*.kt`, `**/*.kts`, `**/*.php`, `**/*.phtml`, `**/*.c`, `**/*.cc`, `**/*.cpp`, `**/*.cxx`, `**/*.h`, `**/*.hh`, `**/*.hpp`, `**/*.hxx`, `**/*.cs`, `**/*.rs`, `**/*.scala`, `**/*.swift`, `**/*.ex`, `**/*.exs`, `**/*.cls`, `**/*.trigger`, `**/*.sol`, `**/Dockerfile`, `**/*.dockerfile`, `**/*.tf`, `**/*.tfvars`, `**/*.hcl`, `**/*.yaml`, `**/*.yml`, `**/*.json`

Step 2 can only select a ruleset for a category this step detected, so an extension missing here removes its ruleset from the scan with no signal — the report then reads clean rather than incomplete. The list is the union of the `includes_for` globs in [run-scans.sh](../scripts/run-scans.sh); keep the two in sync when either changes. `.mts`, `.cts`, `.C`, `Containerfile`, and `Dockerfile.prod` are absent from both, because semgrep does not parse them.

**Two extensions are matched by glob but assigned by content, not by extension.** `.yaml`/`.yml` and `.json` each feed several categories, and both are common in repositories that have no infrastructure to scan at all — nearly every project carries `package.json`, `tsconfig.json` and a lockfile. Assigning a category from the extension alone would attach an AWS IAM ruleset to every scan and report a JSON "language" for a project that has none. Assigning nothing would leave `r/json.aws` and JSON-format CloudFormation unreachable, which is worse: an unselected category never enters `rulesets.json`, so it cannot appear in `coveredNothing`, `failed` or `skipped` either, and the report reads clean. Glob for both, then read a sample and assign on the markers below.

Also check for framework markers: `**/package.json`, `**/pyproject.toml`, `**/requirements.txt`, `**/Gemfile`, `**/composer.json`, `**/go.mod`, `**/Cargo.toml`, `**/pom.xml`. Use Read to inspect these files for framework dependencies (e.g., read `package.json` to detect React, Express, Next.js; read `pyproject.toml` for Django, Flask, FastAPI). The `**/` prefix is required, not cosmetic: a bare `package.json` matches only the target root, so a monorepo with `packages/*/package.json` or `services/*/go.mod` gets no framework rulesets at all.

Map findings to categories:

| Detection | Category |
|-----------|----------|
| `.py`, `.pyi`, `pyproject.toml`, `requirements.txt` | Python |
| `.js`, `.jsx`, `.mjs`, `.cjs`, `.ts`, `.tsx`, `package.json` | JavaScript/TypeScript |
| `.go`, `go.mod` | Go |
| `.rb`, `Gemfile` | Ruby |
| `.java`, `.jsp`, `pom.xml` | Java |
| `.kt`, `.kts` | Kotlin |
| `.php`, `.phtml`, `composer.json` | PHP |
| `.c`, `.cc`, `.cpp`, `.cxx`, `.h`, `.hh`, `.hpp`, `.hxx` | C/C++ |
| `.cs` | C# |
| `.rs`, `Cargo.toml` | Rust |
| `.scala` | Scala |
| `.swift` | Swift |
| `.ex`, `.exs` | Elixir |
| `.cls`, `.trigger` | Apex |
| `.sol` | Solidity |
| `Dockerfile`, `.dockerfile` | Docker |
| `.tf`, `.tfvars`, `.hcl` | Terraform |
| `.yaml`, `.yml` | YAML, Kubernetes, GitHub Actions, or CloudFormation — disambiguate below |
| `.json` | CloudFormation or JSON, or no category at all — disambiguate below |

**Disambiguating YAML.** One `.yaml`/`.yml` match feeds four categories, so Read a sample of the matches before assigning:

- path under `.github/workflows/` → GitHub Actions
- `apiVersion:` together with `kind:` → Kubernetes
- `AWSTemplateFormatVersion:`, or `Resources:` with a `Type: AWS::` member → CloudFormation
- anything else → YAML

These are not exclusive; assign every category that matches. Include the generic YAML category whenever any YAML is present, since `p/yaml` carries patterns the specific rulesets do not.

**Disambiguating JSON.** Unlike YAML, `.json` has no catch-all: most JSON in a repository is build configuration that no ruleset covers, so the default is to assign nothing. Read a sample and assign only on these markers:

- `"AWSTemplateFormatVersion"`, or `"Resources"` whose members carry a `"Type": "AWS::…"` → CloudFormation
- a `"Statement"` array whose elements have `"Effect"` → JSON (this is the IAM policy shape `r/json.aws` targets)
- anything else, including `package.json`, `tsconfig.json`, `composer.json`, lockfiles and editor settings → **no category**

Do not report a `json` category because JSON files exist. Report it when a sampled file has the IAM policy shape. Prefer sampling files whose path suggests infrastructure — `iam/`, `policies/`, `cloudformation/`, `infra/`, `*template*.json` — since a repository with thousands of JSON files will have its IAM policies outnumbered by build configuration, and a sample drawn without regard to path is likely to miss them.

---

## Step 2: Select Scan Mode and Rulesets

> **Entry:** Step 1 complete — languages detected, Pro status known.
> **Exit:** Scan mode selected; structured rulesets JSON compiled for all detected languages.

**First, select scan mode** using `AskUserQuestion`:

```
header: "Scan Mode"
question: "Which scan mode should be used?"
multiSelect: false
options:
  - label: "Run all (Recommended)"
    description: "Full coverage — all rulesets, all severity levels"
  - label: "Important only"
    description: "Security vulnerabilities only — medium-high confidence and impact, no code quality"
```

Record the selected mode. It affects Steps 4 and 5.

**Then, select rulesets.** Using the detected languages and frameworks from Step 1, follow the **Ruleset Selection Algorithm** in [rulesets.md](../references/rulesets.md).

The algorithm covers:
1. Security baseline (always included)
2. Language-specific rulesets
3. Framework rulesets (if detected)
4. Infrastructure rulesets
5. **Required** third-party rulesets (Trail of Bits, 0xdea, Decurity — NOT optional)
6. Registry verification

**Output:** Structured JSON passed to Step 3 for user review:

```json
{
  "baseline": ["p/security-audit", "p/secrets"],
  "python": ["p/python", "p/django"],
  "javascript": ["p/javascript", "p/react", "p/nodejs"],
  "docker": ["p/dockerfile"],
  "third_party": ["https://github.com/trailofbits/semgrep-rules"]
}
```

---

## Step 3: CRITICAL GATE — Present Plan and Get Approval

> **Entry:** Step 2 complete — scan mode and rulesets selected.
> **Exit:** User has explicitly approved the plan (quoted confirmation).

> **⛔ MANDATORY CHECKPOINT — DO NOT SKIP**
>
> This step requires explicit user approval before proceeding.
> User may modify rulesets before approving.

Present plan to user with **explicit ruleset listing**:

```
## Semgrep Scan Plan

**Target:** /path/to/codebase
**Output directory:** $OUTPUT_DIR
**Engine:** Semgrep Pro (cross-file analysis) | Semgrep OSS (single-file)
**Scan mode:** Run all | Important only (security vulns, medium-high confidence/impact)
[in important-only mode, add:] Note: important-only passes --severity WARNING --severity ERROR
to every command, including the third-party repos. Trail of Bits / 0xdea / Decurity rules that
ship with CLI severity INFO are dropped at scan time, before the metadata filter that would
otherwise keep them. Choose "Run all" if you want those.

### Detected Languages/Technologies:
- Python (1,234 files) - Django framework detected
- JavaScript (567 files) - React detected
- Dockerfile (3 files)

### Rulesets to Run:

**Security Baseline (always included):**
- [x] `p/security-audit` - Comprehensive security rules
- [x] `p/secrets` - Hardcoded credentials, API keys

**Python (1,234 files):**
- [x] `p/python` - Python security patterns
- [x] `p/django` - Django-specific vulnerabilities

**JavaScript (567 files):**
- [x] `p/javascript` - JavaScript security patterns
- [x] `p/react` - React-specific issues
- [x] `p/nodejs` - Node.js server-side patterns

**Docker (3 files):**
- [x] `p/dockerfile` - Dockerfile best practices

**Third-party (auto-included for detected languages):**
- [x] Trail of Bits rules - https://github.com/trailofbits/semgrep-rules

**Want to modify rulesets?** Tell me which to add or remove.
**Ready to scan?** Say "proceed" or "yes".
```

**⛔ STOP: Await explicit user approval.**

1. **If user wants to modify rulesets:** Add/remove as requested, re-present the updated plan, return to waiting.
2. **Use AskUserQuestion** if user hasn't responded:
   ```
   "I've prepared the scan plan with N rulesets (including Trail of Bits). Proceed with scanning?"
   Options: ["Yes, run scan", "Modify rulesets first"]
   ```
3. **Valid approval:** "yes", "proceed", "approved", "go ahead", "looks good", "run it"
4. **NOT approval:** User's original request ("scan this codebase"), silence, questions about the plan

### Pre-Scan Checklist

Before marking Step 3 complete:
- [ ] Target directory shown to user
- [ ] Engine type (Pro/OSS) displayed
- [ ] Languages detected and listed
- [ ] **All rulesets explicitly listed with checkboxes**
- [ ] User given opportunity to modify rulesets
- [ ] User explicitly approved (quote their confirmation)
- [ ] **Final ruleset list captured for Step 4**

### Log Approved Rulesets

After approval, write the approved plan to `$OUTPUT_DIR/rulesets.json`. This is the same file
Step 4 hands to the scanner: what the user approved and what runs are one artifact, so there is
no second copy to transcribe and no way for the two to disagree.

Fill in the plan that was just approved. Every value is an array, even a single ruleset:

```bash
cat > "$OUTPUT_DIR/rulesets.json" << 'RULESETS'
{
  "baseline": [<the always-on rulesets from Step 2>],
  "<each detected language>": [<its approved rulesets>],
  "third_party": [<approved repository URLs>]
}
RULESETS
```

One key per language *detected in Step 1*, using the lowercase names from that step. A language
key for a language the target does not contain scans nothing: its `--include` globs match no
file, semgrep exits 0 with an empty result, and the report shows the ruleset with 0 findings
exactly as it would for a ruleset that ran and found nothing. The script counts the files each
scan opened and lists any that covered nothing under `coveredNothing` in `scans.json`, but
getting the languages right here is what stops it happening.

Repository URLs go under `third_party` and nowhere else. Registry identifiers like `p/python`
go under a language key; a `https://…` there fails the identifier check and the script exits
without scanning.

---

## Step 4: Run the Scans

> **Entry:** Step 3 approved — user explicitly confirmed the plan.
> **Exit:** `$OUTPUT_DIR/scans.json` exists; result files exist in `$OUTPUT_DIR/raw/`.

Run the script against the plan Step 3 already wrote. One Bash call; there is no subagent in
this step, and no second copy of the ruleset list to compose here.

```bash
{baseDir}/scripts/run-scans.sh \
  --target "$TARGET" \
  --output-dir "$OUTPUT_DIR" \
  --mode run-all \
  --rulesets "$OUTPUT_DIR/rulesets.json"
```

Do not rewrite `rulesets.json` here. It is the plan the user approved at the Step 3 gate, and
regenerating it at this point is how a ruleset nobody agreed to reaches the scanner. If it needs
to change, go back to Step 3 and get the change approved.

`--mode` is `run-all` or `important-only`. Add `--pro` only when Step 1
printed `Pro: AVAILABLE`; it puts `--pro` on every command, so passing it without a licence
fails every scan in the run. `--jobs N` sets how many semgrep processes run at once (default 4);
semgrep holds the rules and the scanned ASTs in memory, so raising it on a large tree trades
memory for wall-clock.

Repository URLs go under `third_party` and nowhere else. A `https://…` under a language key
fails the registry-identifier check and the script exits without scanning.

The script clones each third-party repo once, generates every `semgrep` command, and runs them
in batches. `--metrics=off`, the `--include` scoping rule, `--exclude` for the output directory
and the severity flags are all its job, not yours. It writes `$OUTPUT_DIR/scans.json`:

| Field | Meaning |
|-------|---------|
| `scans` | Rulesets that ran, with `json`, `sarif`, `findings`, `filesScanned`, `partial` and `exitCode` for each. `findings` is counted from the JSON the scan wrote; `filesScanned` is how many files semgrep opened, or `-1` when it did not say; `exitCode` is what semgrep exited with |
| `scans[].partial` | `true` when the scan wrote complete output while some of its rules failed to compile — semgrep exits 2 and reports the rest of the run normally. The findings are real and in the merge; the rules that never compiled found nothing and cannot say so, so this reads as an unqualified success unless it is called out. **Must be shown.** |
| `coveredNothing` | Rulesets that ran against zero files, because their `--include` globs matched nothing in the target. They report 0 findings exactly like a ruleset that ran and found nothing, so a plan naming a language the target does not contain reads as a clean audit. **Must be shown.** |
| `failed` | Rulesets that ran and did not produce usable output, with the `json` and `sarif` paths they may have partly written, and the stderr excerpt. **Must be shown to the user.** |
| `skipped` | Rulesets dropped before scanning, mostly repos that would not clone. **Must be shown.** |
| `unscoped` | Languages with no `--include` globs, which ran against every file |
| `alsoShared` | Rulesets dropped from a language because the same ruleset is already running unscoped over the whole target. Coverage is unaffected; report them so a per-ruleset accounting adds up |
| `excludePattern` | Set when the output directory sits inside the target: the pattern passed as `--exclude` to every scan, or `""`. semgrep matches it anywhere in the tree, so `out` also drops `src/out/`. **Must be shown when non-empty.** |
| `reposPath` | The clone directory Step 5 deletes |

**A non-zero exit means no scan succeeded.** The script exits 1 when `scans` is empty, so a run
that produced nothing fails loudly rather than handing Step 5 an empty result to report as zero
findings. Read the message, say that no scan ran, and stop; do not retry with adjusted
arguments, because the approved plan is what produced them.

**If `failed` or `skipped` is non-empty**, carry both into the Step 5 report. A run that covered
four of nine rulesets reads exactly like one that covered four of four unless you say otherwise.
The same line is why any scan with `partial: true` is carried across as well: it is in `scans` as
a success, so the rules of it that never ran are invisible in every count the report otherwise
prints.

---

## Step 5: Merge Results and Report

> **Entry:** Step 4 complete — the workflow returned.
> **Exit:** `results.sarif` exists in `$OUTPUT_DIR/results/` and is valid JSON; `repos/` deleted.

Read the result with `jq` from `$OUTPUT_DIR/scans.json`. Every entry there was written after
the script checked the exit code and confirmed both output files were non-empty, so the entries
do not need re-verifying.

**Important-only mode: Post-filter before merge.** Apply the filter from [scan-modes.md](../references/scan-modes.md) ("Filter All Result Files in a Directory" section) to each result JSON in `$OUTPUT_DIR/raw/`. The filter creates `*-important.json` files alongside the originals — the originals are preserved unmodified.

**Generate merged SARIF** using the merge script. The resolved path is in SKILL.md's "Merge command" section — use that exact path:

```bash
# run-all
uv run --no-project {baseDir}/scripts/merge_sarif.py "$OUTPUT_DIR/raw" "$OUTPUT_DIR/results/results.sarif" \
  --scans "$OUTPUT_DIR/scans.json"

# important-only, once the post-filter above has run over every file in raw/
uv run --no-project {baseDir}/scripts/merge_sarif.py "$OUTPUT_DIR/raw" "$OUTPUT_DIR/results/results.sarif" \
  --important --scans "$OUTPUT_DIR/scans.json"
```

- **Run-all mode:** The script merges all `*.sarif` files from `$OUTPUT_DIR/raw/`.
- **Important-only mode:** `--important` is not optional. The JSON post-filter does not touch the
  SARIF files the merge reads, so without that flag `results.sarif` keeps every finding the mode
  exists to exclude while the JSON side is correctly filtered, and the Total findings counted from
  it is the run-all total.

  Do **not** try to run the jq filter from scan-modes.md against a `.sarif` file. It reads
  `.results[].extra.metadata`, which SARIF does not have — there is no top-level `.results` at
  all — so it exits with `Cannot iterate over null` and, if redirected over its own input,
  truncates the merged SARIF to nothing. `--important` matches findings across the two formats on
  `(rule, file, line)`, the same key the merge dedups on, and fails rather than filtering if any
  scan in `raw/` has no `*-important.json` beside it.

**Verify merged SARIF is valid:**

```bash
python -c "import json; d=json.load(open('$OUTPUT_DIR/results/results.sarif')); print(f'{sum(len(r.get(\"results\",[]))for r in d.get(\"runs\",[]))} findings in merged SARIF')"
```

If verification fails, the merge script produced invalid output — investigate before reporting.

**Delete the cloned rulesets** once the merge has succeeded. The workflow clones each
third-party repo into `repos/` and leaves it there for the scanners; this is the only place
the deletion happens, and nothing that reads it is still running by now.

```bash
[ -n "$OUTPUT_DIR" ] && rm -rf "$OUTPUT_DIR/repos"
```

**Report to user:**

```
## Semgrep Scan Complete

**Scanned:** 1,804 files
**Rulesets used:** 9 (including Trail of Bits)
**Total findings:** 156   [count this from results.sarif, never by summing scans[].findings:
one finding flagged by two rulesets is one row in the merge and two in that sum]

### By Severity:
- ERROR: 5
- WARNING: 18
- INFO: 9

### By Category:
- SQL Injection: 3
- XSS: 7
- Hardcoded secrets: 2
- Insecure configuration: 12
- Code quality: 8

### Did Not Run:
[omit this section only when failed and skipped are both empty]
- Skipped: <ruleset> — <reason from the workflow>
- Failed: <ruleset> — <error from the workflow>

### Ran Partially:
[omit when no scan has partial: true]
- <ruleset> — ran and wrote full output, but some of its rules failed to compile (semgrep exit
  <exitCode>). Its findings are in the total below; the rules that did not compile scanned
  nothing, so this ruleset's coverage is narrower than its entry in the scan count suggests

### Also Covered Unscoped:
[omit when alsoShared is empty]
- <ruleset> — already running over the whole target from the baseline, so it was not scanned
  again under <language>. Coverage is unaffected; this is why the ruleset count and the scan
  count differ

### Ran Unscoped:
[omit when unscoped is empty]
- <language> — no --include map, so its rulesets ran against every file

### Covered Nothing:
[omit when coveredNothing is empty]
- <language>/<ruleset> — matched no file in the target, so it reports 0 findings without having
  looked at anything. Check the plan against the languages Step 1 detected: this is what a
  ruleset for a language the target does not contain looks like

### Missing From The Merge:
[omit when the merge printed no "unparseable:" line]
- <file> — the scan succeeded and is counted in scans.json, but its SARIF could not be parsed,
  so its findings are not in results.sarif. The total below is short by that scan's `findings`
  count from scans.json

### Excluded From Every Scan:
[omit when excludePattern is empty]
- <excludePattern> — the output directory sits inside the target, so this pattern was excluded
  from every scan. semgrep matches it anywhere in the tree, so any other directory with that
  name was skipped too. Move the output directory outside the target to scan those files

Results written to:
- $OUTPUT_DIR/results/results.sarif (merged SARIF)
- $OUTPUT_DIR/raw/ (per-scan raw results, unfiltered)
- $OUTPUT_DIR/rulesets.json (the approved plan, as passed to the scanner)
```

**Verify** before reporting: confirm `results.sarif` exists and is valid JSON.