trailmark-review-gate
Runs a Trailmark structural review gate over a branch, pull request, fix commit, release diff, or git ref range to detect new entrypoints, new tainted paths, removed validation or authorization calls, privilege-boundary drift, blast-radius growth, complexity growth, and newly rea
Install
npx skills add https://github.com/trailofbits/skills/tree/main/plugins/trailmark/skills/trailmark-review-gate
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install trailofbits-skills@llmmart
git clone https://github.com/trailofbits/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole trailofbits/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Trailmark Review Gate
Apply deterministic security gate rules to Trailmark structural diff evidence. This skill does not replace line-level review. It produces a compact structural packet reviewers can cite while they inspect the code.
When to Use
- Reviewing a branch, pull request, release diff, or fix commit
- Checking whether a change expands attack surface
- Looking for removed validation or authorization on reachable paths
- Comparing before/after taint, privilege-boundary, blast-radius, or complexity signals
- Producing graph evidence for a differential review
When NOT to Use
- Single-snapshot analysis. Use
trailmarkortrailmark-structural. - Text-diff review only. Use
differential-review. - Full vulnerability discovery. Use an audit or bug-finding workflow.
- One static finding. Use
trailmark-finding-triage. - Tooling is unavailable and the user wants manual review only.
Rationalizations to Reject
| Rationalization | Why It Is Wrong | Required Action |
|---|---|---|
| "The line diff is small, so no graph gate is needed" | Small changes can create new call paths | Compare before/after graphs |
| "Graph gate passed, so the PR is secure" | The gate only checks structural regressions | Still perform line-level review |
| "Trailmark failed, so pass the gate" | Tool failure is unknown risk, not success | Emit UNKNOWN |
| "Tests pass, so removed validation is fine" | Tests may miss affected entrypoint paths | Review the removed path manually |
| "Only new code matters" | Removed auth, validation, and callers can be higher risk than additions | Review removals and path changes |
Workflow
Review Gate Progress:
- [ ] Step 1: Resolve before/after inputs
- [ ] Step 2: Build graph-evolution evidence
- [ ] Step 3: Normalize structural changes
- [ ] Step 4: Apply gate rules
- [ ] Step 5: Emit review packet and actions
Step 1: Resolve Inputs
Accept two refs, a branch name, a commit range, or before/after directories.
Do not check out branches unnecessarily. Prefer git diff, git show, and
git worktrees, following the graph-evolution snapshot workflow.
Step 2: Build Graph Evidence
Run graph-evolution or equivalent Trailmark before/after graph analysis.
Both snapshots must run engine.preanalysis() so taint, privilege-boundary,
blast-radius, complexity, and entrypoint signals are available.
Record Trailmark version and any feature probes. If graph construction fails,
emit UNKNOWN.
Step 3: Normalize Changes
Normalize evidence into:
- added, removed, and modified nodes
- added and removed edges
- entrypoint set changes
- taint membership changes
- privilege-boundary membership changes
- blast-radius changes
- complexity changes
- newly reachable sensitive sinks
- unresolved, proxy, or dynamic edge changes
Step 4: Apply Gate Rules
Apply the rules in references/gate-rules.md. Gate verdicts are:
| Verdict | Meaning |
|---|---|
FAIL |
A high-risk structural regression needs review before acceptance |
WARN |
A meaningful graph change needs reviewer attention |
PASS |
No configured structural gate fired |
UNKNOWN |
Trailmark failed or evidence is too incomplete |
Step 5: Emit Packet
Write the packet using
references/output-format.md, then hand it to
the branch reviewer. Use
references/review-integration.md when
combining this packet with differential-review or another PR review process.
Requirements
- Never mutate the user's working branch while comparing refs.
- Never report
PASSwhen Trailmark failed. - Separate graph evidence from manual security judgment.
- Include exact changed nodes or paths for every
FAILandWARN. - Include limitations when parser, proxy, unresolved-call, or dynamic-dispatch uncertainty affects the verdict.
Files (skills)
-
agents
-
openai.yaml 246 B
interface: display_name: "Trailmark Review Gate" short_description: "Check code changes for structural security regressions" icon_small: "assets/trail-of-bits-mark.svg" icon_large: "assets/trail-of-bits-mark.svg" brand_color: "#D83A34"
-
-
assets
-
trail-of-bits-mark.svg 3 KB · in bundle
-
-
references
-
gate-rules.md 2.2 KB
# Gate Rules Start with deterministic, conservative rules. A triggered rule creates a review obligation; it does not prove a vulnerability. | Rule | Verdict | Why it matters | |---|---|---| | New untrusted entrypoint | `FAIL` | Expands external attack surface | | New path from untrusted entrypoint to sensitive sink | `FAIL` | Creates a candidate exploit path | | Removed auth, validation, or sanitization call on reachable path | `FAIL` | Common regression in fixes and feature PRs | | Newly tainted privilege-boundary node | `FAIL` | Trust transition now handles untrusted data | | Blast radius growth above threshold | `WARN` | A bug may now affect more code | | Complexity growth on tainted or boundary node | `WARN` | Risky logic became harder to review | | New unresolved, proxy, or dynamic call on reachable path | `WARN` | Graph uncertainty increased in a risky area | | Dead security function removed | `WARN` | May be cleanup or accidental security removal | ## Default Thresholds Use these defaults unless the repository has stricter local rules: | Signal | Default threshold | |---|---| | Blast radius growth | `+5` downstream reachable nodes or `+25%`, whichever is larger | | Complexity growth | cyclomatic complexity `+3` on tainted, boundary, or entrypoint-reachable node | | Sensitive sink path | any new path from untrusted entrypoint to sink | | Unresolved/proxy growth | any new unresolved/proxy edge on an entrypoint-reachable path | Thresholds are intentionally conservative. They reduce noise while still catching structural changes that line diffs often understate. ## Sensitive Sink Categories Flag new reachable paths to: - value transfer - authorization or role decisions - persistence or state writes - parsing or deserialization - cryptographic keys, sessions, or signatures - external process, network, or file operations - upgrade, plugin, hook, or dynamic dispatch mechanisms ## Rule Precedence Use the most severe triggered verdict: 1. `UNKNOWN` if Trailmark cannot produce adequate evidence 2. `FAIL` if any fail rule triggers 3. `WARN` if any warn rule triggers 4. `PASS` only if evidence is adequate and no rule triggers If both `UNKNOWN` and `FAIL` seem applicable, emit `UNKNOWN` and list the suspected fail condition as a manual review target. -
output-format.md 1 KB
# Output Format Use Markdown unless the user asks for JSON. ```markdown # Trailmark Review Gate ## Verdict Gate: PASS | WARN | FAIL | UNKNOWN Confidence: High | Medium | Low ## Triggered Rules | Rule | Verdict | Evidence | |---|---|---| ## Structural Changes | Change | Before | After | Review target | |---|---|---|---| ## Entrypoint And Reachability Changes ## Privilege And Taint Changes ## Blast Radius And Complexity Changes ## Limitations ## Recommended Reviewer Actions ``` ## Evidence Requirements For each triggered rule, include: - changed node or edge identifier - source file or symbol when available - entrypoint path when relevant - before/after metric when metric-based - manual review target ## Wording Requirements - Say "gate fired" instead of "vulnerability found". - Say "review target" instead of "exploit path" unless exploitability is separately established. - Say `UNKNOWN` when Trailmark fails or parser support is too incomplete. - Do not claim a `PASS` means the change is secure. -
review-integration.md 1.4 KB
# Review Integration Use the review gate packet as supporting evidence for a human branch review. It should be attached to, pasted into, or summarized alongside line-level review notes. ## With Differential Review Use `differential-review` for line-level analysis and this skill for structural signals. Recommended order: 1. Run `differential-review` to identify risky changed files and functions. 2. Run `graph-evolution` and `trailmark-review-gate` on the same before/after range. 3. Cross-reference `FAIL` and `WARN` rules with changed source lines. 4. Add the gate packet to the review notes. 5. Treat `PASS` as "no configured graph rule fired", not as approval. ## With PR Review Processes When an engagement has a separate PR review workflow, include: - gate verdict - triggered rules table - exact changed nodes and paths - manual reviewer actions - limitations Do not use GitHub write actions unless the review process explicitly asks for them. The packet is review evidence, not an automatic merge decision. ## With Remediation Review For a fix commit: - compare vulnerable base to the proposed fix - check that affected reachable paths changed as expected - check that no new entrypoint or sensitive-sink path appeared - report `UNKNOWN` if graph evidence cannot confirm the structural change This does not replace semantic verification that the original finding was fixed.
-
-
SKILL.md 4.3 KB
--- name: trailmark-review-gate description: "Runs a Trailmark structural review gate over a branch, pull request, fix commit, release diff, or git ref range to detect new entrypoints, new tainted paths, removed validation or authorization calls, privilege-boundary drift, blast-radius growth, complexity growth, and newly reachable sensitive sinks. Use when reviewing a PR, branch, remediation commit, or release diff where graph-level security regressions should be checked before merge." allowed-tools: - Bash - Read - Grep - Glob - Write --- # Trailmark Review Gate Apply deterministic security gate rules to Trailmark structural diff evidence. This skill does not replace line-level review. It produces a compact structural packet reviewers can cite while they inspect the code. ## When to Use - Reviewing a branch, pull request, release diff, or fix commit - Checking whether a change expands attack surface - Looking for removed validation or authorization on reachable paths - Comparing before/after taint, privilege-boundary, blast-radius, or complexity signals - Producing graph evidence for a differential review ## When NOT to Use - Single-snapshot analysis. Use `trailmark` or `trailmark-structural`. - Text-diff review only. Use `differential-review`. - Full vulnerability discovery. Use an audit or bug-finding workflow. - One static finding. Use `trailmark-finding-triage`. - Tooling is unavailable and the user wants manual review only. ## Rationalizations to Reject | Rationalization | Why It Is Wrong | Required Action | |---|---|---| | "The line diff is small, so no graph gate is needed" | Small changes can create new call paths | Compare before/after graphs | | "Graph gate passed, so the PR is secure" | The gate only checks structural regressions | Still perform line-level review | | "Trailmark failed, so pass the gate" | Tool failure is unknown risk, not success | Emit `UNKNOWN` | | "Tests pass, so removed validation is fine" | Tests may miss affected entrypoint paths | Review the removed path manually | | "Only new code matters" | Removed auth, validation, and callers can be higher risk than additions | Review removals and path changes | ## Workflow ``` Review Gate Progress: - [ ] Step 1: Resolve before/after inputs - [ ] Step 2: Build graph-evolution evidence - [ ] Step 3: Normalize structural changes - [ ] Step 4: Apply gate rules - [ ] Step 5: Emit review packet and actions ``` ### Step 1: Resolve Inputs Accept two refs, a branch name, a commit range, or before/after directories. Do not check out branches unnecessarily. Prefer `git diff`, `git show`, and git worktrees, following the `graph-evolution` snapshot workflow. ### Step 2: Build Graph Evidence Run `graph-evolution` or equivalent Trailmark before/after graph analysis. Both snapshots must run `engine.preanalysis()` so taint, privilege-boundary, blast-radius, complexity, and entrypoint signals are available. Record Trailmark version and any feature probes. If graph construction fails, emit `UNKNOWN`. ### Step 3: Normalize Changes Normalize evidence into: - added, removed, and modified nodes - added and removed edges - entrypoint set changes - taint membership changes - privilege-boundary membership changes - blast-radius changes - complexity changes - newly reachable sensitive sinks - unresolved, proxy, or dynamic edge changes ### Step 4: Apply Gate Rules Apply the rules in [references/gate-rules.md](references/gate-rules.md). Gate verdicts are: | Verdict | Meaning | |---|---| | `FAIL` | A high-risk structural regression needs review before acceptance | | `WARN` | A meaningful graph change needs reviewer attention | | `PASS` | No configured structural gate fired | | `UNKNOWN` | Trailmark failed or evidence is too incomplete | ### Step 5: Emit Packet Write the packet using [references/output-format.md](references/output-format.md), then hand it to the branch reviewer. Use [references/review-integration.md](references/review-integration.md) when combining this packet with `differential-review` or another PR review process. ## Requirements - Never mutate the user's working branch while comparing refs. - Never report `PASS` when Trailmark failed. - Separate graph evidence from manual security judgment. - Include exact changed nodes or paths for every `FAIL` and `WARN`. - Include limitations when parser, proxy, unresolved-call, or dynamic-dispatch uncertainty affects the verdict.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.