generate-cve-json
Generate a CVE 5.x JSON document from an <tracker> tracking issue, ready to paste into the Vulnogram `#source` tab of the ASF CVE tool at https://cveprocess.apache.org/cve5/<CVE-ID>#source. The conversion is deterministic: same issue in, same JSON bytes out. Handles multiple cred
Install
npx skills add https://github.com/apache/magpie/tree/main/tools/cve-tool-vulnogram/generate-cve-json
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install apache-magpie@llmmart
git clone https://github.com/apache/magpie.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole apache/magpie collection as a plugin from our marketplace. Git is the plain clone.
README
Table of Contents generated with DocToc
generate-cve-json
Small Python project that generates a CVE 5.x JSON record from an
<tracker> tracking issue, ready to paste into the Vulnogram
#source tab of the ASF CVE tool.
The behavioural contract and the security-process context live in
SKILL.md. This README covers the local-setup and test
workflow for the project itself.
Run
From the framework's root (this repository when running standalone;
the .apache-magpie/ snapshot path inside an adopting tracker repo):
uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <ISSUE-NUMBER> [options]
Skill files reference the same invocation via the <framework>
placeholder so the path resolves in either context:
uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <ISSUE-NUMBER>
<framework> substitutes to .apache-magpie/apache-magpie in
adopting projects and to . (the repository root) in framework
standalone — see the placeholder convention in
AGENTS.md.
Equivalent forms:
# as a module
uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json python -m generate_cve_json <ISSUE-NUMBER>
# from inside the project dir
cd <framework>/tools/cve-tool-vulnogram/generate-cve-json
uv run generate-cve-json <ISSUE-NUMBER>
Flags are documented in generate-cve-json --help and in SKILL.md.
Test
cd tools/cve-tool-vulnogram/generate-cve-json
uv run --group dev pytest
Lint / type-check
cd tools/cve-tool-vulnogram/generate-cve-json
uv run --group dev ruff check src tests
uv run --group dev ruff format --check src tests
uv run --group dev mypy
The prek hooks configured in .pre-commit-config.yaml at the
repository root run ruff check, ruff format --check, and mypy
on the project files automatically on every commit that touches them.
Skill manifest
generate-cve-json
This skill produces a CVE 5.x JSON document from a tracking issue in
<tracker>, ready to
paste into the Vulnogram "#source" tab of the ASF CVE tool. The goal is
to eliminate the manual "copy each field from the issue into the right
Vulnogram form input" step when you are preparing to publish an advisory.
Project-agnostic by design. All project-specific values (vendor, top-level product / package name, project display map, CNA org id, generator tag, …) are loaded from a TOML config the adopting project ships at
<project-config>/tools/cve-tool-vulnogram/cve-json-config.toml. Concreteapache-foo-project-*strings appearing in this document are illustrative examples of how a project with a project-style package layout would configure things; replace them mentally with the adopter's own package taxonomy. The schema is documented in the package README.
Golden rule: the script generates a proposal JSON document. It parses a handful of structured fields from the issue body, but it cannot read the security team member's mind. Always review the generated JSON before pasting, and always do the final review inside Vulnogram before moving the CVE from DRAFT → REVIEW → READY → PUBLIC.
Release-vote gating (opt-in, recommended for ASF projects). The
emitted CNA_private.state follows a tri-state state machine:
DRAFT— the CNA is incomplete or the project has opted into release-vote gating and no vote is in progress yet.REVIEW— the CNA is review-ready (CVE ID + title + description + affected versions + CWE + severity + ≥ 1 credit + ≥ 1 reference) and either the project hasn't opted into gating (legacy: ready ⇒ REVIEW) or an RC vote is in progress (signalled by the configured tracker label or a--reviewCLI flag).PUBLIC— the CNA is review-ready and the public advisory has shipped (avendor-advisoryreference is present).
Projects opt into gating by setting [workflow].release_vote_gating = true in their cve-json-config.toml and choosing the label name
via [workflow].rc_voting_label (default "rc voting"). The sync
skill is responsible for detecting [VOTE] threads on the project's
dev list (e.g. dev@<project>.apache.org) and proposing the label
add/remove; the generator only reads the label on the tracker. Non-
ASF adopters who publish advisories without a separate release-vote
step typically leave gating off — the legacy "ready ⇒ REVIEW"
behaviour is the right default for that workflow.
Determinism: the same input issue body produces exactly the same JSON bytes on every run. The script uses only the Python standard library, has no timestamps or machine-dependent values in its output, sorts JSON keys, and sorts references alphabetically. This lets you paste the result into Vulnogram, tweak fields in the tool, re-run the script later, and cleanly diff the two to see what the tool has added / what you changed by hand.
Inputs
- Issue number (required) — e.g.
232. - Optional CLI overrides:
--cve-id CVE-YYYY-NNNN+— override the CVE ID if the issue body'sCVE tool linkfield has not yet been filled in, or retarget the JSON to a different CVE ID.--title "<vendor>: <product>: …"— override the CVE title. Default is the GitHub issue title with the project's<vendor>: <product>:prefix (sourced from the TOML config) when it does not already start with that phrase.--version-start X.Y.Z— override the start of the affected version range (theaffected[].versions[].versionfield). Default is the lower bound parsed from the Affected versions field when it uses>= X, < Ysyntax, otherwise"0".--remediation-developer "Name"— append atype: "remediation developer"credit on top of whatever the body's Remediation developer field already lists (auto-populated by thesecurity-issue-syncskill from the linked PR's author). Repeat the flag to add multiple developers; duplicates between the body field and CLI flags are dropped silently. The reporter credit(s) from the Reporter credited as field are always emitted withtype: "finder".--vendor/--product/--package-name/--collection-url— override the product identity fields. The defaults come from the project's TOML config (product.vendor,product.default_product,product.default_package_name,product.default_collection_url). They are used as the identity for Affected versions lines that don't start with a recognisable per-package directory name (see the multi-product note below).--collection-urlreaches the record only through the purl it is used to derive (seeproduct.purl_typebelow) — except for a product that opts out of purls, the one case a record still carries acollectionURL.Helm charts —
purl_type = "helm", and you must know where the chart is published. A chart is identified by its repository, not by its name: two projects may both ship a chart calledsuperset. So the generator emitspkg:helm/<chart>?repository_url=<base URL>taking the base URL from
product.default_collection_url(or the package'scollection_urloverride), and refuses to emit a purl at all without one rather than producing a chart identifier that matches somebody else's chart.If the base URL is not recorded anywhere, ask whoever publishes the chart. It is whatever a user would put in
helm repo add— thehttps://…site servingindex.yaml, or theoci://…registry path for a chart published as an OCI artifact. Either is accepted verbatim; a trailing slash is dropped so the same repository does not produce two identifiers. Do not infer it from the project's homepage: charts are routinely served from a different host than the project site.Note that
helmis not one of the types the purl specification registers. It is used because the scanners that would match this advisory to a deployed chart emit it, and because the two spec-correct encodings —pkg:oci/for an OCI registry andpkg:generic/for a classic repository — would give the same chart two different identities depending on how it happened to be published. A strict validator may object to the type; being invisible to every scanner is the worse outcome.product.purl_type(config only) — the Package URL type for the project's packages (pypi,npm,cargo, …). Everyaffected[]entry carries apackageURL(pkg:<type>/<packageName>). Per the CVE Record Format the purl never includes a version — the entry'sversions[]carries the range.The purl is the only package identifier the record carries. The ASF CVE tool treats the Package URL as the recommended identifier and derives the legacy
collectionURL/packageNamepair from it when the record is serialised for publication, so the generator emits the purl alone. Writing the pair as well would hand the tool two identifiers that can disagree — which it reports rather than silently reconciles.A purl is required, not optional. A record needs one to be promoted, so the generator refuses to emit a record it cannot build a purl for rather than producing one the CNA will reject later. Leaving the key unset is fine when the type can be derived from
product.default_collection_url— PyPI, npm, crates.io, RubyGems and NuGet hosts are recognised — and only hosts whose type is unambiguous are mapped, because a guessed type yields a valid-looking identifier pointing at the wrong ecosystem.Set
purl_type = "none"to state deliberately that a product has no package host — a source-only release published todist.apache.organd nowhere else, for instance. That is an explicit decision rather than an omission, which is the distinction the previous optional behaviour lost. Those entries — and only those — carrycollectionURL/packageNameinstead, because there is no purl for the CVE tool to derive them from. Name normalisation follows the package-url spec per type, and is implemented for the types whose rules have been read from it:pypi(lowercased,_becomes-) andnpm(scope becomes the namespace, so@angular/animationrenders aspkg:npm/%40angular/animation; names keep their case, since mixed-case npm packages were grandfathered in). Any other type has its name passed through unchanged, and a name needing namespace semantics is covered byproduct.purl_namespacebelow, and without one such a type is an error rather than a guessed purl.product.purl_namespace(optional, config only) — the namespace for purl types that require one: a MavengroupId, a Composer vendor, a Go module prefix. Types the spec gives no namespace (pypi,cargo,gem,nuget) ignore it, and an npm scope carried in the package name itself wins over it. A type that requires a namespace and has none is an error naming this key — a guessed Maven groupId would point at another organisation's artifact, and silently omitting the purl produces an unpromotable record.[packages.overrides."<packageName>"](config only) — per-package distribution identity, for a project that ships to more than one ecosystem. Acceptsproduct,collection_urlandpurl_type; anything omitted falls through to theproduct.*default, so a package that differs only in purl type need not restate its collection URL.The case this exists for: a project whose packages are on PyPI and which also ships, say, a Helm chart from its own site.
product.*describes only the majority ecosystem, so without an override the odd-one-out inherits it and the record claimspkg:pypi/<chart>— a package that host does not carry. Since purls are what scanners match on, a wrong one is acted upon, unlike the wrong free-text product name it replaced.[packages.overrides."apache-example-helm-chart"] product = "Apache Example Helm Chart" collection_url = "https://example.apache.org/" purl_type = "none"The key is the resolved
packageName, so the package must be one the configuredpackage_patternmatches — extending that pattern is a prerequisite, not an extra.--product-forstill wins over theproductset here.--product-for PACKAGE=PRODUCT— override the CVE product display name for a specificpackageName. Repeat to override multiple packages. Useful when a package is not in the project'sproject_display_mapconfig, or when an acronym needs different casing from the title-cased fallback. Example:--product-for apache-foo-project-baz='Apache Foo Project Baz'.--org-id <uuid>— override the CNA assigner org id (defaults to the ASF org id).--discovery <word>— overridesource.discovery(default"UNKNOWN"; valid CVE 5.x values includeUNKNOWN,INTERNAL,EXTERNAL,USER).--no-envelope— emit only the innercnacontainer instead of the full CVE 5.x record (envelope is the default).--review/--draft(mutually exclusive) — force the emittedCNA_private.statetoREVIEWorDRAFTregardless of the tracker's labels. Useful in two cases:--reviewlets a release manager nudge a record forward by hand when therc votinglabel is not yet set on the tracker.--draftwalks a record back when an RC vote was cancelled or failed and the label is still around. Both flags only matter when release-vote gating is enabled in the project's TOML config (see below); otherwise the state is derived from the CNA's readiness alone and these flags have no effect beyond what the legacy logic produces.
--attach— after generating the JSON, embed it at the end of the tracking issue's body (after the CVE tool link field), wrapped in a collapsible<details>block. The block is bracketed by HTML-comment markers (<!-- generate-cve-json: cve=CVE-YYYY-NNNN+ version=v1 -->…<!-- generate-cve-json:end cve=CVE-YYYY-NNNN+ version=v1 -->) that the script uses on later runs to find the existing block and replace it in place, so re-runs update the embedded attachment instead of duplicating it or breaking other body fields. The attachment lives in the body — not as a comment — so it stays above every status-change comment in the timeline (effectively "pinned" without needing any pin mechanism). Requires the positional issue argument; incompatible with--stdin.
Prerequisites on the tracking issue
For the generated JSON to be useful, the issue body should already be
filled in through a prior security-issue-sync run. In particular:
Short public summary for publish — becomes the CVE description.
Affected versions — becomes the CVE
affected[]list. The script understands the common version-expression shapes (< 3.2.2,>= 2.0.0, < 3.2.2,<= 3.2.1, a bare version like3.1.5, and a bare lower bound like>= 2.0.0). Multi-product CVEs are supported — put one package per line, prefixing each with the package directory name as it appears in the adopter's repo, and the script emits oneaffected[]entry per line with the rightproductand package identity. Example (illustrative — using a hypotheticalapache-fooproject's sub-project layout):apache-foo-project-alpha <=6.5.0 apache-foo-project-beta <=1.9.0Known package directory names are resolved to the vendor-preferred display casing via the project's
packages.project_display_mapconfig table; unknown packages fall back to title-cased dash-split and can be overridden with--product-for. A line without a package prefix (or a single-line field) falls back to the--product/--package-namedefaults, which preserves the single-product behaviour.< NEXT VERSIONplaceholder — multi-package trackers don't know which package version will ship the fix until the wave's release manager picks it during a release cut. Until then, the Affected versions lines use the literal tokenNEXT VERSIONas the upper bound, e.g.:apache-foo-project-alpha < NEXT VERSION apache-foo-project-beta < NEXT VERSIONThe generator strips
< NEXT VERSIONbefore parsing each line and emits aversions[]entry withoutlessThan(open-ended upper bound — "affected from <low> onwards, no fix released yet"). When the wave ships and the version is known, thesecurity-issue-syncskill replaces eachNEXT VERSIONwith the actual< X.Y.Zand the next regen produces a fully-bounded entry. Case-insensitive; combines with a lower bound (e.g.>= 2.0.0, < NEXT VERSIONbecomes{version: "2.0.0", status: "affected"}).Security mailing list thread — internal navigation reference only; the script does not export URLs from this field into
references[]. Keep whatever the reporter or triager put there.Public advisory URL — each URL in this field is extracted and added to
references[]withtags: ["vendor-advisory"]. Populated by the release manager (or thesecurity-issue-syncskill) once the advisory is archived on<users-list>. The--advisory-urlCLI flag still exists for ad-hoc overrides.PR with the fix — each URL in this field becomes a reference URL. Multiple URLs are supported: paste them on separate lines, as a bullet list, or comma-separated — the script extracts every
https?://…token it finds.Reporter credited as — each line becomes one CVE credit entry with
type: "finder". Multiple credits are supported: put each person on their own line.Full Name, Affiliationon a single line is treated as one credit, not two, so the commonJed Cunningham, Astronomerpattern works as expected. Bullets (-,*,1.) are stripped. Blank lines are ignored. If you need to credit many people::Jed Cunningham Saurabh Banawar selen (Huntr bounty 3e88d364-5047-4768-a52c-6568f21ef35b)Remediation developer — each line becomes one CVE credit entry with
type: "remediation developer". Same parsing rules as Reporter credited as (newline-separated,Full Name, Affiliationis one credit, bullets stripped). Auto-populated by thesecurity-issue-syncskill from the linked PR's author the first time PR with the fix is set; manual edits survive subsequent syncs (the skill only proposes appending names that aren't already there). The--remediation-developerCLI flag adds further names on top of whatever the body already lists.
Bot / AI credit policy. This generator is intentionally neutral on credit content: whatever a tracker's Reporter credited as or Remediation developer field carries is what lands in
credits[]. The filtering of obvious bot / AI accounts (e.g.dependabot[bot],*-scanner,automated-*) happens upstream in the skills at extraction time — seebot-credits-policy.mdfor the detection rule and the per-skill enforcement sites. Keeping the filter upstream means an intentional human override (typed directly into the field) survives every JSON regeneration without needing a special bypass flag here.
- CWE —
CWE-285: Improper Authorizationstyle works; so does a bareCWE-285or a plain sentence. The script extracts theCWE-\d+token for thecweIdfield and uses the rest as the human-readable description. - Severity —
None,Low,Medium,High,Critical(case-insensitive) are emitted as the text content of ametrics[].otherblock. Vulnogram lets you replace this with a CVSS vector in its form if you want a numeric score. - CVE tool link — the ASF CVE tool URL, e.g.
https://cveprocess.apache.org/cve5/CVE-2026-40913. The script extracts theCVE-YYYY-NNNN+token from this field. If the field is still_No response_, pass the CVE ID with--cve-id.
If one of these fields is missing, the JSON still generates, but the reviewer will need to fill the gap in Vulnogram. The skill surfaces any empty field in the proposal so nothing is silently skipped.
Prerequisites
ghCLI authenticated with collaborator access to<tracker>— the script reads the tracker viagh.uvinstalled — the script is a smalluv-managed Python project and is invoked asuv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <N>.
See
Prerequisites for running the agent skills
in README.md.
Step 0 — Pre-flight check
Before reading the tracker:
gh api repos/<tracker> --jq .namereturns the adopter's tracker repo name (per<project-config>/project.md), anduv --versionreturns.
If either fails, stop and tell the user what to install or log in to.
Step 1 — Verify the issue has the required fields
Fetch the issue body and check every template field the script reads. If
a field is missing or still _No response_, either run
security-issue-sync first to fill it
in, or override it on the command line.
gh issue view <N> --repo <tracker> --json body --jq .body \
| grep -E '^###|^_No response_'
Ask the user whether to proceed if any critical field is empty (description, affected versions, CVE tool link, credits). Do not silently generate a JSON with placeholder values.
Step 2 — Run the generator
Run the project's console script through uv run --project, which
prepares the (cached) virtualenv on first use and reuses it on later
runs:
uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <N> \
--output /tmp/<CVE-ID>.json \
--version-start <earliest-affected-version>
--version-start is the one flag the tracking issue body almost never
contains and that Vulnogram expects filled in (the body field usually
encodes only the upper bound). The remediation developer credit comes
from the body's Remediation developer field, populated by the
security-issue-sync skill from the linked PR's author — no CLI flag
needed in the normal flow. For a fix that landed in 3.2.2 and was
first introduced in 3.0.0, for example:
uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \
--output /tmp/CVE-2026-40913.json \
--version-start 3.0.0
Pass --remediation-developer "Name" only when you need to add a
developer credit on top of (or in place of) what the body already
contains — for example a co-author who didn't end up as the PR's
GitHub author.
Additional flags, all optional:
--cve-id CVE-YYYY-NNNN+— override the CVE ID if the CVE tool link field is empty.--title "<vendor>: <product>: …"— override the title.--vendor/--product/--package-name/--collection-url— override product identity (defaults sourced from the project's TOML config under[product]).--org-id <uuid>— override the CNA assigner org id (defaults to the ASF org id).--discovery UNKNOWN|INTERNAL|EXTERNAL|USER— overridesource.discovery.--no-envelope— emit only thecnacontainer (nocveMetadata, nodataType/dataVersionwrapper). Use this if Vulnogram's#sourcetab is in "inner block only" mode.--stdin— read the issue body from stdin instead of callinggh. Useful for offline iteration and for drafting by hand.
The script is deterministic — re-running it with the same flags and the same tracking-issue body produces the same JSON bytes.
Output shape (in brief)
The generated record matches what Vulnogram exports after a save, minus editor cruft. Notable fields:
containers.cna.affected[]—vendor,product,packageURL(the version-less purl) and aversions[]entry withversion,lessThan,status: "affected",versionType: "semver". An entry for a product configured withpurl_type = "none"carriescollectionURL/packageNamein place of the purl.containers.cna.descriptions[]— both a plainvalueand an HTMLsupportingMediaalternative (Vulnogram's WYSIWYG mode needs both).containers.cna.problemTypes[].descriptions[]—cweId, human-readabledescription,type: "CWE".containers.cna.metrics[].other—type: "Textual description of severity"andcontent.text= the severity word.containers.cna.credits[]— one entry per Reporter credited as line (type"finder"), plus one entry per Remediation developer body line and per--remediation-developerCLI override (type"remediation developer"); duplicates between the body field and CLI flags are dropped silently.containers.cna.references[]— URLs with auto-taggedtags:- GitHub
pull/orcommit/URLs →["patch"]; lists.apache.org/security.apache.org→["vendor-advisory"];- everything else → no tag.
- GitHub
containers.cna.source.discovery—"UNKNOWN"by default.containers.cna.providerMetadata.orgId— ASF assigner org id.cveMetadata—assignerOrgId,cveId,serial,state: "PUBLISHED".
Step 3 — Surface the output to the user
After the script finishes, print these three things in order:
The output file path, with a one-line
catsuggestion so the user can review the JSON in the terminal:``` Wrote /tmp/cve-CVE-2026-40913.json cat /tmp/cve-CVE-2026-40913.json ```A clipboard-copy command appropriate to the user's platform. On Linux with
xclipinstalled:``` xclip -selection clipboard < /tmp/cve-CVE-2026-40913.json ```On Wayland:
wl-copy < /tmp/cve-...json. On macOS:pbcopy < …. Ifxclip/wl-copy/pbcopyis not on PATH, skip the clipboard command and tell the user to copy manually.The Vulnogram
#sourcepaste URL, as a clickable link rendered per the "Linking CVEs" rule inAGENTS.md:``` Paste the JSON into the Vulnogram #source tab: [CVE-2026-40913](https://cveprocess.apache.org/cve5/CVE-2026-40913#source) ```
The #source tab on the ASF CVE tool is the direct "paste raw JSON" view of the Vulnogram form. The page loads the current record, you paste the script output over the top, click Save, and the form view reflects the new values.
Optional: --attach to embed (or refresh) the JSON in the issue body
If the user also wants the JSON attached to the tracking issue itself
(so it is discoverable from the issue without needing the local file),
add --attach to the invocation:
uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \
--output /tmp/CVE-2026-40913.json \
--version-start 3.0.0 \
--attach
What --attach does:
- After generating the JSON (exactly the same bytes as without
--attach), edits the tracking issue's body to embed the full JSON inside a four-backtick fenced code block, collapsed behind a<details>disclosure so long records don't bloat the issue view. The block is appended after the existing template fields, right after the CVE tool link field, so it lives at the end of the body. - Brackets the attachment with a pair of hidden HTML-comment markers
(
<!-- generate-cve-json: cve=CVE-YYYY-NNNN+ version=v1 -->…<!-- generate-cve-json:end cve=CVE-YYYY-NNNN+ version=v1 -->) so subsequent runs can find the existing embedded block and replace it in place, without spawning duplicates and without touching any other text in the body. - Re-running with
--attachis safe and idempotent: same issue body → same JSON → the script patches the body, leaving you with one and only one embedded attachment per CVE id. If the current body already matches what the script would write, the PATCH is skipped entirely (no no-op timestamp on the issue). - The script prints
Embedded CVE JSON in issue body on <tracker>#NNNon first run andReplaced CVE JSON in issue body on <tracker>#NNNon subsequent runs, followed by a URL that deep-links to the## CVE JSON — paste-ready for …heading anchor inside the body.
Why embedded in the body and not as a comment? Two reasons:
- Natural "pinning" without an API for it. GitHub has no pin-comment API. A separate comment ends up buried below every status-change comment — so a newcomer looking at the issue sees a long comment timeline with no obvious way to find the CVE JSON. The issue body always renders above the entire comment timeline, so anything embedded in the body is effectively pinned.
- One place to read the tracker. The reporter-template fields, the CVE metadata, and the paste-ready JSON are all in one place — no hunting through the timeline to reconstruct the current state.
(GitHub also does not expose its user-attachments file-upload
pipeline to the REST API — only the web UI drag-and-drop uses it — so
a real file attachment isn't available to automation anyway. Embedding
as body text is the closest automatable equivalent and is directly
visible without a download round-trip.)
Confidentiality. The embedded block lives inside the private repo,
so it inherits the repo-wide confidentiality rules. Linking CVE
references inside the block follows the "Linking CVEs" rule in
AGENTS.md: before publication the block links
the ASF CVE tool; after publication, re-running the script includes a
cve.org link as well.
Step 4 — Propose an issue comment recording the update
Per the "Keeping the reporter informed" rule in
README.md, any status change on an issue must be
recorded in an issue comment. Pasting a new version of the CVE record is
a status change. Propose (and, on confirmation, post) a short comment
like:
CVE entry regenerated from the tracking issue — generated paste-ready JSON for
CVE-2026-40913from the current body fields (description, affected versions< 3.2.2, CWE-285, Low severity, N credits, M references). Pasted into the Vulnogram#sourcetab; the record is now in sync with the tracking issue.
Include a count of credits and references so a later reviewer can sanity-check that nothing was dropped.
Step 5 — Never edit the JSON in place after pasting
Once the JSON has been pasted into Vulnogram and saved, do not edit
the local JSON file to match tool-side changes. Re-run the script instead
(it is deterministic — you will get a clean baseline), diff the new
output against the current Vulnogram state, and paste the merged JSON
back. This keeps the tracking issue as the single source of truth: if
Vulnogram shows something different from the generated JSON, either the
issue body is out of date and needs a security-issue-sync run, or the
tool-side difference is intentional and the reviewer will keep it.
Guardrails
- Confidentiality. The script deliberately drops any URL that
points at
cveprocess.apache.orgor the project's<tracker>repo from the references list before serialising. Those URLs are private ASF-internal links and should not appear in a published CVE record. See the "Confidentiality of<tracker>" section ofAGENTS.md. - CVE IDs are always linked per the "Linking CVEs" rule in
AGENTS.md. When the skill mentions the CVE in proposals, recaps or comments on the<tracker>issue, it must render the ID as a markdown link — before publication to the ASF CVE tool, and additionally tocve.orgafter publication. <tracker>references are always linked per the "Linking<tracker>issues and PRs" rule inAGENTS.md. When the skill mentions the tracking issue in its own comments, render it as a markdown link.- Deterministic output is a feature. Do not introduce timestamps, random UUIDs, ordering dependencies on dict iteration, or other sources of non-determinism into the script. If you need to add a new field, make sure the output still hashes the same across runs on the same input.
- Multi-entry fields. Credits are split on newlines only to
preserve the
Full Name, Affiliationpattern. References are extracted from URL tokens in the field value. Do not reintroduce comma-splitting on credits. - No envelope means no metadata.
--no-envelopedrops thecveMetadatablock which includes the CVE ID; the JSON is pure CNA content. Make sure the user knows they will have to set the CVE ID by hand in Vulnogram in that mode.
References
AGENTS.md— repo-wide conventions (confidentiality, Linking CVEs, Linking<tracker>issues and PRs, release-branch defaults).README.md— handling process, in particular step 13 (fill in CVE tool fields and send advisory from the tool) and step 15 (paste the attached JSON into Vulnogram's #source tab, move the CVE to PUBLIC, close the issue).security-issue-sync— the sibling skill that populates the tracking issue fields this skill consumes.security-issue-fix— the other sibling skill that opens a public PR and updates the tracking issue with the fix URL.
Files (magpie)
-
src
-
generate_cve_json
-
cve_json.py 105.1 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Generate a CVE 5.x JSON record from an airflow-s/airflow-s issue. This script reads a tracking issue via ``gh issue view``, parses the standard template fields in the issue body, and emits a JSON document that can be pasted straight into the Vulnogram ``#source`` tab of the ASF CVE tool at https://cveprocess.apache.org/cve5/<CVE-ID>#source The output shape is modelled on a real CVE record exported from Vulnogram so that a paste-and-save round-trip does not mangle the record. In particular, the script produces: * ``affected[]`` entries with ``vendor``, ``product``, a version-less ``packageURL`` (``pkg:pypi/apache-airflow``) and a semver range. The purl is the CNA's recommended package identifier: Vulnogram derives the legacy ``collectionURL`` / ``packageName`` pair from it when the record is serialised for publication, so the generator emits that pair only for a product with no purl at all (``product.purl_type = "none"``); * a ``descriptions[]`` entry with both a plain ``value`` and an HTML ``supportingMedia`` alternative (Vulnogram's WYSIWYG mode expects both); * ``metrics[].other`` with ``type`` = *"Textual description of severity"* and ``content.text`` = the severity word; * ``problemTypes[].descriptions[]`` with ``cweId``, ``description`` and ``type`` = *"CWE"*; * ``credits[]`` entries from the *Reporter credited as* field (``type: "finder"`` by default; ``type: "tool"`` when the credit matches the bot/AI policy in ``tools/cve-tool-vulnogram/bot-credits-policy.md`` — Dependabot, Renovate, Snyk, scanners, ``*-bot`` / ``*[bot]`` handles, etc.) and the *Remediation developer* field (``type: "remediation developer"``) — both newline-separated, with the ``Full Name, Affiliation`` pattern preserved as one credit; the ``--remediation-developer`` CLI flag still works as an additional / override mechanism on top of the body field; * ``references[]`` with automatic ``tags`` (``patch`` for ``github.com/.../pull/...`` URLs, ``vendor-advisory`` for ``lists.apache.org``/``security.apache.org``). URLs from the tracking-issue's *Public advisory URL* body field are picked up automatically and tagged as ``vendor-advisory``; URLs from the *Security mailing list thread* field are **deliberately not** exported because that field holds the internal ``security@airflow.apache.org`` thread URL, which 404s for anyone outside the security team; * ``providerMetadata.orgId`` and ``cveMetadata.assignerOrgId`` set to the ASF org id; * an ``x_generator`` annotation so the origin of the JSON is visible. The output is **deterministic**: the same issue body and the same CLI flags always produce the same JSON bytes. Keys are sorted, references are sorted alphabetically, credits preserve the order of first occurrence, and no timestamps or machine-dependent data are included. Usage:: # Read the issue from GitHub and write the JSON to stdout: uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 # Write to a file and show the Vulnogram paste URL: uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \\ --output /tmp/CVE-2026-40913.json # Attach an explicit "remediation developer" credit: uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \\ --remediation-developer "Kevin Yang" # Test mode: read a prepared issue body from stdin so you can # iterate on the parser without touching the network: cat /tmp/issue232-body.md | \\ uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json \\ --stdin --cve-id CVE-2026-40913 \\ --title "Apache Airflow: ..." The runtime uses only the Python standard library; the ``pyproject.toml`` declares no runtime dependencies, so ``uv run`` resolves the environment in milliseconds. Dev tooling (pytest, ruff, mypy) lives in the ``dev`` dependency group. """ from __future__ import annotations import argparse import contextlib import html import json import os import re import subprocess import sys import tempfile # ----------------------------------------------------------------------------- # Configuration loading. # ----------------------------------------------------------------------------- # # All project-specific values (vendor, top-level product/package, project # display map, package-name regex, CNA org id, generator tag, …) are read # at module load time from a TOML config file the adopting project ships # in its tracker repo at: # # <project-config>/tools/cve-tool-vulnogram/cve-json-config.toml # # (where `<project-config>` is the adopting project's # `.apache-magpie-overrides/` directory, per the apache/magpie # placeholder convention). # # Resolution order for the config path: # 1. The `--config <path>` CLI flag (or the `config_path=` argument to # `_load_config()`). # 2. The `CVE_JSON_CONFIG` environment variable. # 3. `<cwd>/.apache-magpie-overrides/tools/cve-tool-vulnogram/cve-json-config.toml`. # Back-compat: if only the pre-rename `tools/vulnogram/` copy exists at # this default location, it is used with a deprecation warning. # # Schema: see the README in this package. import tomllib import urllib.parse from pathlib import Path _DEFAULT_CONFIG_RELPATH = ".apache-magpie-overrides/tools/cve-tool-vulnogram/cve-json-config.toml" # Pre-rename location (`tools/vulnogram` was renamed to `tools/cve-tool-vulnogram`). # Used as a fallback for adopters who upgraded across the rename — see `_load_config`. _LEGACY_CONFIG_RELPATH = ".apache-magpie-overrides/tools/vulnogram/cve-json-config.toml" _CONFIG_PATH_ENV = "CVE_JSON_CONFIG" # Cached, lazily-loaded config. `_populate_constants()` reads this and # fills the module globals below; tests that want to use a different # config can call `_set_config_path(path)` before calling any # generator function. _CONFIG: dict | None = None def _resolve_config_path(config_path: Path | str | None = None) -> tuple[Path, bool]: """Resolve the config file path following the documented order. Returns ``(path, is_default)`` — ``is_default`` is True only when the path came from the default location (no ``--config``, no ``$CVE_JSON_CONFIG``), the single case the legacy-path fallback in ``_load_config`` applies to. Keeping the precedence decision here means the fallback keys off one source of truth instead of re-deriving it. """ if config_path is not None: return Path(config_path), False env = os.environ.get(_CONFIG_PATH_ENV) if env: return Path(env), False return Path.cwd() / _DEFAULT_CONFIG_RELPATH, True def _load_config(config_path: Path | str | None = None) -> dict: """Load TOML config, raising ``FileNotFoundError`` with usage hints if it cannot be located. """ path, is_default = _resolve_config_path(config_path) if not path.exists(): # Back-compat: `tools/vulnogram` was renamed to `tools/cve-tool-vulnogram`. # An adopter who upgraded across the rename still keeps their committed # config at the pre-rename path. When resolving from the default location # (no `--config`, no `$CVE_JSON_CONFIG`) and only the legacy copy exists, # use it and warn — so the upgrade doesn't hard-break mid-run. legacy = Path.cwd() / _LEGACY_CONFIG_RELPATH if is_default and legacy.is_file(): print( f"generate-cve-json: WARNING reading config from the legacy path " f"{_LEGACY_CONFIG_RELPATH}. The tool dir was renamed " f"`tools/vulnogram` -> `tools/cve-tool-vulnogram`; move the file to " f"{_DEFAULT_CONFIG_RELPATH} to silence this warning.", file=sys.stderr, ) path = legacy else: raise FileNotFoundError( f"generate-cve-json: config file not found at {path}.\n" f" Pass --config <path>, set ${_CONFIG_PATH_ENV}, or place a config\n" f" at {_DEFAULT_CONFIG_RELPATH} relative to the cwd.\n" f" Schema: see the generate-cve-json package README." ) return tomllib.loads(path.read_text(encoding="utf-8")) def _set_config_path(config_path: Path | str) -> None: """Set / override the config path (used by `--config` CLI and tests).""" global _CONFIG _CONFIG = _load_config(config_path) _populate_constants() def _populate_constants() -> None: """(Re)populate module globals from the loaded config.""" global _CONFIG if _CONFIG is None: _CONFIG = _load_config() cfg = _CONFIG global DEFAULT_REPO, DEFAULT_VENDOR, DEFAULT_PRODUCT global DEFAULT_PACKAGE_NAME, DEFAULT_COLLECTION_URL, DEFAULT_PURL_TYPE, DEFAULT_PURL_NAMESPACE global DEFAULT_ASF_ORG_ID, GENERATOR_TAG, SKILL_SOURCE_URL global PROJECT_DISPLAY_MAP, PACKAGE_RE global TOP_LEVEL_NAME, TOP_LEVEL_PRODUCT, PROJECT_PRODUCT_TEMPLATE global CNA_PRIVATE_PROJECT_URL, CNA_PRIVATE_OWNER, CNA_PRIVATE_USERS_LIST global PACKAGE_OVERRIDES global TITLE_STRIP_RE, TRACKER_FILTER_TOKEN global RELEASE_VOTE_GATING, RC_VOTING_LABEL, FORWARD_STATE_LABELS DEFAULT_REPO = cfg["meta"]["tracker_repo"] DEFAULT_VENDOR = cfg["product"]["vendor"] DEFAULT_PRODUCT = cfg["product"]["default_product"] DEFAULT_PACKAGE_NAME = cfg["product"]["default_package_name"] DEFAULT_COLLECTION_URL = cfg["product"]["default_collection_url"] DEFAULT_PURL_TYPE = cfg["product"].get("purl_type", "") DEFAULT_PURL_NAMESPACE = cfg["product"].get("purl_namespace", "") DEFAULT_ASF_ORG_ID = cfg["cna"]["org_id"] GENERATOR_TAG = cfg["meta"]["generator_tag"] SKILL_SOURCE_URL = cfg["meta"].get( "skill_source_url", f"https://github.com/{cfg['meta']['tracker_repo']}" ) PROJECT_DISPLAY_MAP = dict(cfg["packages"]["project_display_map"]) PACKAGE_OVERRIDES = {name: dict(values) for name, values in cfg["packages"].get("overrides", {}).items()} PACKAGE_RE = re.compile(cfg["packages"]["package_pattern"]) TOP_LEVEL_NAME = cfg["packages"]["top_level_name"] TOP_LEVEL_PRODUCT = cfg["packages"]["top_level_product"] PROJECT_PRODUCT_TEMPLATE = cfg["packages"]["project_product_template"] # Per-project CVE 5.x `CNA_private` envelope fields. These end up # in every CVE record this tool generates for the project, and # used to be hardcoded to Airflow values — now config-driven so # adopters can populate them with their project's URLs / lists. cna_private_cfg = cfg.get("cna_private", {}) CNA_PRIVATE_PROJECT_URL = cna_private_cfg.get("project_url", "") CNA_PRIVATE_OWNER = cna_private_cfg.get("owner", "") CNA_PRIVATE_USERS_LIST = cna_private_cfg.get("users_list", "") # Workflow gating around the DRAFT → REVIEW transition. The ASF CVE # tool's REVIEW state means "this record is about to be published" # — so for ASF projects with a release-vote cadence, advancing to # REVIEW only makes sense once an RC is being voted on. Non-ASF # adopters (which often don't have a separate release-vote step # before publishing the advisory) keep the original behavior: a # fully-populated record auto-advances to REVIEW. # # Opt in by setting `[workflow].release_vote_gating = true` and the # tracker label name in `[workflow].rc_voting_label`. The sync skill # is responsible for detecting active [VOTE] threads on the project's # dev mailing list (e.g. `dev@<project>.apache.org`) and proposing # the label add/remove; the generator only reads the label. workflow_cfg = cfg.get("workflow", {}) RELEASE_VOTE_GATING = bool(workflow_cfg.get("release_vote_gating", False)) RC_VOTING_LABEL = workflow_cfg.get("rc_voting_label", "rc voting") # Forward-state labels — when the tracker carries any of these the # release has already shipped (the vote, if there was one, passed) and # the rc-voting gate is moot. Treat them as "vote completed", which # means a ready CNA stays at REVIEW (or advances to PUBLIC on # vendor-advisory) regardless of whether the `rc voting` label is # still on the tracker. Without this, removing `rc voting` at the # `pr merged → fix released` transition (per the sync skill) walks # the state back to DRAFT — the wrong direction for a record that's # about to be published. FORWARD_STATE_LABELS = set( workflow_cfg.get( "forward_state_labels", ["fix released", "announced - emails sent", "announced", "vendor-advisory ready"], ) ) # Title-strip regex for `resolve_title` — built from the configured # top-level product so the *"<vendor>: <product>:"* prefix is # stripped without hardcoding a specific project's name. _stripped_product = re.escape(TOP_LEVEL_PRODUCT.strip()) TITLE_STRIP_RE = re.compile( rf"^\s*{_stripped_product}\s*[:\-–—]?\s*", # noqa: RUF001 — en-/em-dash deliberate re.IGNORECASE, ) # Substring used by `build_references` to filter out tracker URLs # before serialising. We use the tracker repo's owner segment # (the part before `/`) — distinctive enough for any URL that # points at the tracker (issues, PRs, raw blobs all carry it) # without false positives on URLs that mention the project name # in a path component. TRACKER_FILTER_TOKEN = DEFAULT_REPO.split("/", 1)[0] # Module-level constants — populated by `_populate_constants()`. The # initial values are placeholders; the call below populates them at # import time. Tests can override by calling `_set_config_path()` # before importing any generator function. DEFAULT_REPO: str = "" DEFAULT_VENDOR: str = "" DEFAULT_PRODUCT: str = "" DEFAULT_PACKAGE_NAME: str = "" DEFAULT_COLLECTION_URL: str = "" # The purl type for the project's packages. Empty means "derive it from # `product.default_collection_url`"; `"none"` opts the product out of purls # and falls its entries back to the legacy `collectionURL` / `packageName` # pair. DEFAULT_PURL_TYPE: str = "" # Optional: the namespace for types that require one — a Maven groupId, a # Composer vendor, a Go module prefix. Ignored for types that have none. DEFAULT_PURL_NAMESPACE: str = "" DEFAULT_ASF_ORG_ID: str = "" GENERATOR_TAG: str = "" SKILL_SOURCE_URL: str = "" PROJECT_DISPLAY_MAP: dict[str, str] = {} PACKAGE_RE: re.Pattern[str] = re.compile("") TOP_LEVEL_NAME: str = "" TOP_LEVEL_PRODUCT: str = "" PROJECT_PRODUCT_TEMPLATE: str = "" CNA_PRIVATE_PROJECT_URL: str = "" CNA_PRIVATE_OWNER: str = "" CNA_PRIVATE_USERS_LIST: str = "" TITLE_STRIP_RE: re.Pattern[str] = re.compile("") TRACKER_FILTER_TOKEN: str = "" RELEASE_VOTE_GATING: bool = False RC_VOTING_LABEL: str = "" FORWARD_STATE_LABELS: set[str] = set() # CVE 5.x convention values that are not project-specific. DEFAULT_CREDIT_TYPE = "finder" TOOL_CREDIT_TYPE = "tool" DEFAULT_LANG = "en" DEFAULT_DISCOVERY = "UNKNOWN" # Bot / AI / automation detection for the *Reporter credited as* # field. When a credit row matches any of these rules, the CVE 5.x # ``credits[]`` entry is emitted with ``type: "tool"`` instead of # ``type: "finder"``. The matching rules mirror # ``tools/cve-tool-vulnogram/bot-credits-policy.md`` (single source of truth); # update both together. Detection is intentionally broad — false # positives are cheap (the user can edit the JSON afterwards or set # the credit name to a clearer human-style string), false negatives # put a bot handle into the ``finder`` credit class in a public CVE # record, which is what this policy exists to prevent. # # Known bot / automation names matched as a case-insensitive # whole-word against the credit string (e.g. ``"Dependabot"``, # ``"discovered by Automated Scanner v3"``). BOT_CREDIT_KNOWN_NAMES: tuple[str, ...] = ( "dependabot", "renovate", "snyk-bot", "snyk", "copilot", "ghsa-probot", "github-actions", "mend-bot", "mend", "whitesource", "sonatype-lift", "lift-bot", "codecov", "mergify", "mergifyio", "allcontributors", "fossabot", "imgbotapp", "pre-commit-ci", "claude", "chatgpt", "gpt-bot", "gpt", "anthropic", "openai", "automated", "automation", "scanner", "auto-scanner", "vulnerability-scanner", "security-scanner", "sast", "dast", ) # Handle-shaped patterns matched as case-insensitive regexes against # the credit string. ``\b`` anchors keep them from firing on # unrelated human names (``Joe Bot`` does not match ``*-bot`` because # the space breaks the word boundary, but ``joe-bot`` does). BOT_CREDIT_PATTERNS: tuple[re.Pattern[str], ...] = ( re.compile(r"\b[\w]*-bot\b", re.IGNORECASE), re.compile(r"\b[\w]+bot\b", re.IGNORECASE), re.compile(r"\b[\w]*-ai\b", re.IGNORECASE), re.compile(r"\b[\w-]+\.ai\b", re.IGNORECASE), re.compile(r"\b[\w]*-agent\b", re.IGNORECASE), re.compile(r"\b[\w]*-gpt\b", re.IGNORECASE), re.compile(r"\b[\w]*scanner[\w]*\b", re.IGNORECASE), re.compile(r"\b[\w]*automat[\w]*\b", re.IGNORECASE), ) def is_bot_credit(name: str) -> bool: """Return True if *name* matches the bot/AI credit policy. Matches any of three rules: 1. Literal GitHub ``[bot]`` suffix — handle ends in ``[bot]`` (e.g. ``dependabot[bot]``). 2. Known-bot / automation-name list — case-insensitive whole-word occurrence of any of ``BOT_CREDIT_KNOWN_NAMES`` in the credit string. 3. Suffix / contains-pattern handles — any of ``BOT_CREDIT_PATTERNS`` matches. See ``tools/cve-tool-vulnogram/bot-credits-policy.md`` for the canonical rule set and rationale. """ cleaned = name.strip() if not cleaned: return False if cleaned.endswith("[bot]"): return True lowered = cleaned.lower() for known in BOT_CREDIT_KNOWN_NAMES: if re.search(rf"\b{re.escape(known)}\b", lowered): return True return any(pattern.search(cleaned) for pattern in BOT_CREDIT_PATTERNS) # Populate at import time. If the config is not present, defer the # error to first actual use so test harnesses can call # `_set_config_path()` before exercising the module. with contextlib.suppress(FileNotFoundError): _populate_constants() NO_RESPONSE = "_No response_" # Sentinel token for an as-yet-unreleased upper bound in # ``Affected versions`` entries. Used predominantly for projects # trackers where the wave milestone (date-based, e.g. ``Projects # 2026-04-21``) does not reveal which exact project package version # will ship the fix. The token is stripped before parsing and the # resulting CVE 5.x ``versions[]`` entry omits ``lessThan`` — # downstream consumers (Vulnogram, cve.org) read that as # *"affected from <low> onwards, no fix released yet"*. Once the # release ships and the version is known the sync skill replaces the # token with the real upper bound (``< X.Y.Z``). NEXT_VERSION_TOKEN_RE = re.compile(r"<\s*NEXT\s+VERSION\b", re.IGNORECASE) # Map a CVE ``affected[].collectionURL`` to the canonical project-page # URL template used to render a clickable per-package link in the # attachment table. The PyPI ``collectionURL`` is the legacy # ``https://pypi.python.org`` host, but the canonical project page lives # on ``pypi.org/project/<name>/``; the table renders that. # Only collection URLs Airflow actually emits today need entries here — # unknown URLs fall through to ``None`` and the attachment table # renders ``—`` for them. # Entries normally carry a purl rather than the legacy pair, and the table # reads their link out of :data:`PURL_TYPE_TO_PROJECT_URL_TEMPLATE`; this map # serves the ``purl_type = "none"`` entries that still carry a collection URL. COLLECTION_URL_TO_PROJECT_URL_TEMPLATE: dict[str, str] = { "https://pypi.python.org": "https://pypi.org/project/{package}/", "https://pypi.org": "https://pypi.org/project/{package}/", } # HTML-comment marker prefix used to identify the single "CVE JSON # attachment" comment on a tracking issue, so ``--attach`` can update # the existing comment in place instead of posting a duplicate on every # re-run. The full marker includes the CVE id and a version tag. ATTACHMENT_MARKER_BEGIN_PREFIX = "<!-- generate-cve-json:" ATTACHMENT_MARKER_END_PREFIX = "<!-- generate-cve-json:end" ATTACHMENT_MARKER_VERSION = "v1" # Back-compat alias — the `v1` comment-based attachments used a single # marker. New body-embedded attachments use a begin/end pair. ATTACHMENT_MARKER_PREFIX = ATTACHMENT_MARKER_BEGIN_PREFIX # ----------------------------------------------------------------------------- # Input: fetch the issue body from GitHub (or read from stdin in test mode). # ----------------------------------------------------------------------------- def fetch_issue(issue_number: int | str, repo: str) -> tuple[str, str, list[str]]: """Return ``(title, body, labels)`` for the given issue. Calls ``gh issue view <N> --repo <repo> --json title,body,labels``. The ``labels`` list contains label names (strings); the gh JSON objects' ``.name`` field is unwrapped here so callers can do a simple ``"rc voting" in labels`` check. Raises a ``RuntimeError`` with the stderr output if ``gh`` is unavailable or the issue does not exist. """ try: result = subprocess.run( [ "gh", "issue", "view", str(issue_number), "--repo", repo, "--json", "title,body,labels", ], check=True, capture_output=True, text=True, ) except FileNotFoundError as exc: raise RuntimeError( "`gh` CLI not found on PATH. Install it from https://cli.github.com/ " "or run with --stdin to bypass the network call." ) from exc except subprocess.CalledProcessError as exc: raise RuntimeError(f"`gh issue view {issue_number} --repo {repo}` failed:\n{exc.stderr}") from exc data = json.loads(result.stdout) labels = [ label.get("name", "") for label in (data.get("labels") or []) if isinstance(label, dict) and label.get("name") ] return data.get("title", ""), data.get("body", ""), labels # ----------------------------------------------------------------------------- # Parse the issue body into individual template fields. # ----------------------------------------------------------------------------- def extract_field(body: str, heading: str) -> str: """Return the text under ``### <heading>`` up to the next ``###``. Trailing whitespace is stripped. The literal placeholder ``_No response_`` is normalised to an empty string so callers only have to check truthiness. """ pattern = rf"^###\s+{re.escape(heading)}\s*\n+(?P<value>.*?)(?=\n###\s|\Z)" match = re.search(pattern, body, re.MULTILINE | re.DOTALL) if not match: return "" value = match.group("value").strip() if value == NO_RESPONSE: return "" return value def parse_credits_from_field(value: str) -> list[str]: """Split the ``Reporter credited as`` field into a list of credit names. Credits are split on **newlines only**, deliberately *not* on commas: the common ``Full Name, Affiliation`` pattern (for example ``Jed Cunningham, Astronomer``) is one credit, not two. If you need to credit multiple people, put each one on its own line. Bullets (``-``, ``*``, ``+``, ``1.``) are stripped. Blank lines are ignored. De-duplicated preserving order of first occurrence. """ if not value: return [] names: list[str] = [] seen: set[str] = set() for raw_line in value.splitlines(): line = raw_line.strip() if not line: continue line = re.sub(r"^(?:[-*+]\s+|\d+[.)]\s+)", "", line) line = line.strip(" `\t") if line and line not in seen: seen.add(line) names.append(line) return names def combine_remediation_developers(field_value: str, cli_overrides: list[str]) -> list[str]: """Merge body-field developers with CLI overrides; preserve order, drop dupes. The body's *Remediation developer* field is the source of truth in the normal flow (auto-populated by the sync skill from the PR's author). The ``--remediation-developer`` CLI flag is additive: any name passed there is appended after the body entries unless it already appears in the body. Duplicates are dropped silently so a triager can pass ``--remediation-developer "Alice"`` without worrying about whether Alice is already in the body. """ combined: list[str] = [] seen: set[str] = set() for name in [*parse_credits_from_field(field_value), *cli_overrides]: cleaned = name.strip() if cleaned and cleaned not in seen: combined.append(cleaned) seen.add(cleaned) return combined def parse_url_list(value: str) -> list[str]: """Return every URL mentioned in ``value`` in order of first occurrence. Handles bullet lists, plain-text lines and markdown-link syntax (``[text](url)``) uniformly. Non-URL tokens are ignored, which lets a reviewer annotate the field with notes without breaking parsing. """ if not value: return [] urls: list[str] = [] seen: set[str] = set() for match in re.finditer(r"https?://[^\s)>\]]+", value): url = match.group(0).rstrip(").,;>") if url not in seen: seen.add(url) urls.append(url) return urls def parse_cve_id(value: str) -> str: """Extract the first ``CVE-YYYY-NNNN+`` token from ``value``.""" match = re.search(r"CVE-\d{4}-\d{4,}", value) return match.group(0) if match else "" def parse_cwe(value: str) -> tuple[str, str]: """Return ``(cweId, human-readable description)`` from a CWE field. Accepts ``CWE-285``, ``CWE-285: Improper Authorization``, ``CWE-285 (Improper Authorization)``, and free-form text. If no ``CWE-\\d+`` token is found, ``cweId`` is an empty string and the full value is returned as the description. When the title following the id is wrapped in a single layer of parentheses or brackets (e.g. ``CWE-285 (Improper Authorization)`` — the form most projects' CWE pickers serialise to), the outer wrapper is stripped before the description is assembled. Without the strip the result reads ``"CWE-285: (Improper Authorization)"`` — both a colon *and* parens, which the ASF CVE-CNA reviewers flag as cluttered. The accepted shapes are ``"CWE-285: Title"`` (colon separator, no wrapper) or ``"CWE-285 Title"`` (no separator); the parser emits the colon-separated form. Only the **outer** wrapper is stripped: ``CWE-285 (Foo) Bar`` stays ``"CWE-285: (Foo) Bar"`` because the parens are not the outermost shape. """ value = value.strip() id_match = re.search(r"CWE-\d+", value) cwe_id = id_match.group(0) if id_match else "" if cwe_id: # Use "CWE-285: Improper Authorization" as the description when # a title follows the id; otherwise just the id. title_match = re.match(r"^CWE-\d+\s*[:\-]?\s*(?P<rest>.*)", value) rest = (title_match.group("rest") if title_match else "").strip() # Strip a single layer of outer wrapping parens / brackets. # The inner content must contain no unbalanced wrappers; this # avoids stripping ``(Foo) Bar (Baz)`` to ``Foo) Bar (Baz``. wrapper_match = re.match(r"^\((?P<paren>[^()]+)\)$|^\[(?P<bracket>[^\[\]]+)\]$", rest) if wrapper_match: rest = (wrapper_match.group("paren") or wrapper_match.group("bracket")).strip() description = f"{cwe_id}: {rest}" if rest else cwe_id else: description = value return cwe_id, description def parse_affected_versions(value: str, version_start_override: str | None) -> list[dict]: """Turn the ``Affected versions`` field into a CVE ``versions`` array. Supports a few common shapes: * ``< 3.2.2`` / ``<3.2.2`` → ``{version: "0", lessThan: "3.2.2", ...}`` * ``>= 2.0.0, < 3.2.2`` → ``{version: "2.0.0", lessThan: "3.2.2", ...}`` * ``<= 3.2.1`` / ``<=3.2.1`` → ``{version: "0", lessThanOrEqual: "3.2.1", ...}`` * A bare version like ``3.1.5`` → ``{version: "3.1.5", status: "affected"}`` * A bare lower bound like ``>=2.0.0`` → ``{version: "2.0.0", status: "affected"}`` (no upper bound — useful for trackers whose fix-shipped version is not yet known). The ``< NEXT VERSION`` sentinel signals "fix not yet released, upper bound unknown" — used predominantly on project trackers where the wave milestone is date-based and the package version that will carry the fix is decided by the release manager during the wave. The token is stripped before further parsing; the resulting entry has no ``lessThan``. Once the release ships, the sync skill replaces ``< NEXT VERSION`` with the actual ``< X.Y.Z`` upper bound and the next regen produces a fully-bounded entry. See the ``NEXT VERSION`` rule in ``.claude/skills/security-issue-sync/SKILL.md`` for the lifecycle. ``--version-start`` overrides the low bound unconditionally when set, because the issue body rarely specifies it and a reviewer usually knows the first affected release by heart. **Strict mode** — unparsable inputs raise ``ValueError``. The prior fallback that emitted ``{"version": <raw string>}`` produced invalid CVE 5.x records: the schema's ``version`` field is a literal version, never a range expression. Forcing the parser to fail loud catches reviewer-typo / parenthetical-comment inputs before they ship as malformed JSON to Vulnogram (for example, ``>= 3.0.0 (reporter verified...)`` falling through and serialising the parenthetical clause into the ``version`` field). **Bare lower bounds without an upper bound** emit a warning to stderr unless the ``< NEXT VERSION`` sentinel was used. The resulting record claims the bare version is the *sole* affected release, which is almost always wrong; the warning catches the case without blocking emission, since a small number of workflows legitimately want bare-version entries (single-point- fix records where only one release train is involved). """ raw_value_for_diagnostics = (value or "").strip() cleaned = value.strip().strip("`").strip() if value else "" low_bound = (version_start_override or "").strip() or "0" has_next_version_sentinel = bool(NEXT_VERSION_TOKEN_RE.search(cleaned)) if has_next_version_sentinel: cleaned = NEXT_VERSION_TOKEN_RE.sub("", cleaned).strip().rstrip(",").strip() if not cleaned: return [ { "status": "affected", "version": low_bound, "versionType": "semver", } ] range_match = re.search( r">=?\s*(?P<low>[0-9][0-9A-Za-z.\-+]*)\s*,?\s*<\s*(?P<high>[0-9][0-9A-Za-z.\-+]*)", cleaned, ) if range_match: return [ { "status": "affected", "version": version_start_override or range_match.group("low"), "lessThan": range_match.group("high"), "versionType": "semver", } ] less_than_or_equal_match = re.match(r"<=\s*(?P<high>[0-9][0-9A-Za-z.\-+]*)", cleaned) if less_than_or_equal_match: return [ { "status": "affected", "version": low_bound, "lessThanOrEqual": less_than_or_equal_match.group("high"), "versionType": "semver", } ] less_than_match = re.match(r"<\s*(?P<high>[0-9][0-9A-Za-z.\-+]*)", cleaned) if less_than_match: return [ { "status": "affected", "version": low_bound, "lessThan": less_than_match.group("high"), "versionType": "semver", } ] single_match = re.match(r"^(?P<v>[0-9][0-9A-Za-z.\-+]*)$", cleaned) if single_match: return [ { "status": "affected", "version": single_match.group("v"), "versionType": "semver", } ] ge_only_match = re.match(r">=?\s*(?P<low>[0-9][0-9A-Za-z.\-+]*)\s*$", cleaned) if ge_only_match: # Warn when the input was a bare lower bound AND no `< NEXT # VERSION` sentinel was given. The emitted entry has only a # `version` field (no `lessThan`), which CVE 5.x readers # interpret as "this version alone is affected" — almost # always misleading for a CVE that affects a continuous # range. The sentinel signals "fix not yet released, upper # bound unknown" and is the right shape for in-flight # trackers; without it, the reviewer should add an explicit # upper bound. if not has_next_version_sentinel: print( f"warning: `Affected versions` value {raw_value_for_diagnostics!r} " f"has a bare lower bound (no upper bound, no `NEXT VERSION` " f"sentinel). The emitted record will claim " f"{ge_only_match.group('low')!r} alone is affected. Add a " f"`< X.Y.Z` upper bound or the `< NEXT VERSION` sentinel " f"to silence this warning.", file=sys.stderr, ) return [ { "status": "affected", "version": version_start_override or ge_only_match.group("low"), "versionType": "semver", } ] # Fall-through: nothing matched. Refuse to emit a record whose # `version` field is a range expression — that JSON would be # rejected by Vulnogram's schema validation anyway, and silently # emitting it has shipped malformed records to reviewers (CVE- # 2026-46763 — `>= 3.0.0` ended up as a literal `version` string # because the parser couldn't strip a trailing parenthetical). raise ValueError( f"Could not parse `Affected versions` value {raw_value_for_diagnostics!r}. " f"Supported shapes: `< X.Y.Z`, `<= X.Y.Z`, `>= X.Y.Z`, " f"`>= X.Y.Z, < A.B.C`, and bare `X.Y.Z`. Strip parenthetical " f"comments, stray backticks, and markdown wrappers before " f"re-running. The `< NEXT VERSION` sentinel may be combined " f"with `>= X.Y.Z` when the fix-shipped version is not yet " f"known." ) def normalise_severity(value: str) -> str: """Return the severity as a lower-case word (``none`` / ``low`` / ``moderate`` / ``medium`` / ``high`` / ``important`` / ``critical``) or the original text if it doesn't match the standard set.""" lowered = value.strip().lower() if lowered in {"none", "low", "moderate", "medium", "high", "important", "critical"}: return lowered return value.strip() def to_html(text: str) -> str: """Convert a plain-text description to the minimal HTML shape that Vulnogram's WYSIWYG mode stores in ``supportingMedia``. The output is HTML-escaped and paragraph breaks (``\\n\\n``) are rendered as ``<br><br>``, single newlines as ``<br>``. No inline styles or span wrappers are emitted -- those are editor cruft that appears after a save round-trip and is not needed on paste-in. """ escaped = html.escape(text) escaped = escaped.replace("\r\n", "\n").replace("\n\n", "<br><br>") escaped = escaped.replace("\n", "<br>") return escaped # ----------------------------------------------------------------------------- # Reference tagging. # ----------------------------------------------------------------------------- def classify_reference(url: str) -> list[str]: """Return the CVE reference ``tags`` for ``url``. * ``github.com/.../pull/N`` and ``github.com/.../commit/<sha>`` → ``["patch"]``. * ``lists.apache.org/...`` and ``security.apache.org/...`` → ``["vendor-advisory"]``. * ``cve.org/CVERecord?id=...`` and ``nvd.nist.gov/vuln/detail/...`` → ``["related"]`` (links to other CVE records on the public CVE databases — used for incomplete-fix / sibling-CVE cross-references per ASF Security's request). * Anything else → no tags (empty list). """ if re.search(r"github\.com/[^/]+/[^/]+/(pull|commit)/", url): return ["patch"] # Match the host exactly, not a substring — `if "lists.apache.org" # in url` would also flag e.g. `https://evil.com/?q=lists.apache.org`, # which CodeQL flags as `py/incomplete-url-substring-sanitization`. try: host = (urllib.parse.urlparse(url).hostname or "").lower() except ValueError: return [] if host in ("lists.apache.org", "security.apache.org"): return ["vendor-advisory"] if host in ("cve.org", "www.cve.org", "nvd.nist.gov"): return ["related"] return [] # Match a complete CVE-YYYY-NNNNN identifier with word boundaries so # substrings inside larger tokens (e.g. ``CVE-2026-12345-extra``) do # not match. _CVE_ID_RE = re.compile(r"\bCVE-\d{4}-\d{4,7}\b") def extract_related_cve_ids(text: str, current_cve_id: str | None = None) -> list[str]: """Extract distinct CVE identifiers cited in ``text``, in order of first appearance. ``current_cve_id`` is excluded from the result so the generator never emits a self-reference. The check is case-insensitive. Typical inputs: * The tracker's *Short public summary for publish* body field — where Gate #3 (incomplete-fix cross-CVE clause) places a prior CVE identifier the current CVE is a follow-up to. * The tracker's *Security mailing list thread* field — when the report references a prior CVE in its body for context. Output is a list (not a set) to preserve first-appearance order so the emitted references list is deterministic across runs. """ seen: set[str] = set() ordered: list[str] = [] current_upper = (current_cve_id or "").upper() for match in _CVE_ID_RE.finditer(text): cve_id = match.group(0).upper() if cve_id == current_upper: continue if cve_id in seen: continue seen.add(cve_id) ordered.append(cve_id) return ordered def related_cve_url(cve_id: str) -> str: """Return the public ``cve.org`` record URL for a CVE identifier. Format matches ASF Security's preference per Arnout Engelen's 2026-05-29 review comment on CVE-2026-49298: ``https://cve.org/ CVERecord?id=<CVE-ID>``. """ return f"https://www.cve.org/CVERecord?id={cve_id}" def build_references( mailing_list_field: str, pr_field: str, extra_urls: list[str] | None = None, ) -> list[dict]: """Collect every URL from the relevant fields and emit a sorted, de-duplicated CVE ``references`` list with automatic tags. The ASF CVE tool URL (``cveprocess.apache.org``) and any ``airflow-s`` URLs are filtered out before serialising -- those are private tracker links and must not appear in a public CVE record. The ``mailing_list_field`` argument is **deliberately ignored** for URL extraction. The tracking-issue's "Security mailing list thread" field is an internal reference that, in our flow, almost always points at a non-publicly archived ``security@airflow.apache.org`` thread -- those URLs 404 for anyone outside the security team. The field is kept in the function signature for call-site compatibility; callers that have a real public advisory URL to include must pass it via ``extra_urls`` (typically the ``--advisory-url`` CLI flag). See the "CVE references must never point at non-public mailing-list threads" section of ``AGENTS.md`` for the full rationale. """ _ = mailing_list_field # intentionally unused; see docstring gathered: list[str] = list(parse_url_list(pr_field)) if extra_urls: gathered.extend(extra_urls) filtered = [ url for url in gathered if "cveprocess.apache.org" not in url and TRACKER_FILTER_TOKEN not in url ] # De-duplicate preserving order, then sort alphabetically. seen: set[str] = set() ordered: list[str] = [] for url in filtered: if url not in seen: seen.add(url) ordered.append(url) ordered.sort() references: list[dict] = [] for url in ordered: entry: dict = {"url": url} tags = classify_reference(url) if tags: entry["tags"] = tags references.append(entry) return references # ----------------------------------------------------------------------------- # Build the CVE 5.x CNA container dict. # ----------------------------------------------------------------------------- # Per-package distribution identity, keyed by packageName. A project may ship # to more than one ecosystem — a Python distribution on PyPI and a Helm chart # served from its own site, say — and `product.*` describes only one of them. # Without this an entry for the odd-one-out inherits the majority ecosystem and # claims a purl on a host that does not carry it. PACKAGE_OVERRIDES: dict[str, dict[str, str]] = {} def package_identity( package_name: str, *, collection_url: str, purl_type: str, ) -> tuple[str, str]: """Return ``(collection_url, purl_type)`` for one package. `[packages.overrides."<packageName>"]` wins over the `product.*` defaults. Anything the override omits falls through, so a package that differs only in purl type does not have to restate its collection URL. """ if not TOP_LEVEL_NAME: _populate_constants() override = PACKAGE_OVERRIDES.get(package_name, {}) return ( override.get("collection_url", collection_url), override.get("purl_type", purl_type), ) def _product_for_package( package_name: str, *, product_overrides: dict[str, str] | None = None, ) -> str: """Return the display product name for a project package directory name. The mapping is driven by the loaded config (`<project-config>/tools/ vulnogram/cve-json-config.toml`): * `package_name == TOP_LEVEL_NAME` → `TOP_LEVEL_PRODUCT` (e.g. `apache-airflow` → `Apache Airflow`). * `package_name` matches `PACKAGE_RE` and the regex captures a non-empty named `project` group → `PROJECT_PRODUCT_TEMPLATE.format(display=...)`, where `display` is `PROJECT_DISPLAY_MAP[<project>]` when the directory name is known, or a title-cased dash-split fallback otherwise (`foo-bar` → `Foo Bar`). The `project` group encodes the project's subpackage convention (`-project-<dir>`, `-providers-<dir>`, …) — whatever the regex declares — so the mapping works for any subpackage prefix the project ships. * `product_overrides` lets callers shadow either source by package name — used by the `--product-for` CLI flag for unknown subpackages or acronyms that don't round-trip through `title()`. Unknown `package_name` (neither `TOP_LEVEL_NAME` nor a project subpackage) is returned unchanged — callers that pass a non-project package name are opting into bring-your-own product naming. """ if not TOP_LEVEL_NAME: _populate_constants() overrides = product_overrides or {} if package_name in overrides: return overrides[package_name] configured = PACKAGE_OVERRIDES.get(package_name, {}).get("product") if configured: return configured if package_name == TOP_LEVEL_NAME: return TOP_LEVEL_PRODUCT # Anchor at both ends: a partial match (`match`) can capture an # arbitrary suffix into the `project` group for non-standard # package strings, producing a fabricated product name. If the # package name does not fully match the configured pattern we # fall through and treat it as opaque. match = PACKAGE_RE.fullmatch(package_name) if match is not None: project_dir = (match.groupdict().get("project") or "").strip() if project_dir: if project_dir in PROJECT_DISPLAY_MAP: display = PROJECT_DISPLAY_MAP[project_dir] else: display = " ".join(part.title() for part in project_dir.split("-") if part) return PROJECT_PRODUCT_TEMPLATE.format(display=display) return package_name def _split_affected_lines(affected_versions_value: str) -> list[tuple[str | None, str]]: """Split the *Affected versions* body field into per-package entries. Each non-empty line is inspected for a leading Airflow package directory name (``apache-airflow`` or ``apache-airflow-project-<dir>``); when detected, the line splits into ``(package_name, version_expression)``. Lines without a recognisable package prefix yield ``(None, <whole line>)`` so the caller can fall back to the default product/package. Bullets (``-``, ``*``, ``+``, ``1.``) are stripped so a reviewer can write the field as a bulleted list. When the input is empty, returns ``[(None, "")]`` so callers emit a single placeholder ``affected[]`` entry (the historical behaviour for an empty field). """ if not affected_versions_value.strip(): return [(None, "")] entries: list[tuple[str | None, str]] = [] for raw_line in affected_versions_value.splitlines(): line = raw_line.strip() if not line: continue line = re.sub(r"^(?:[-*+]\s+|\d+[.)]\s+)", "", line).strip() if not line: continue if not PACKAGE_RE.pattern: _populate_constants() match = PACKAGE_RE.match(line) if match: package = match.group("package") rest = (match.group("rest") or "").strip() entries.append((package, rest)) else: entries.append((None, line)) if not entries: return [(None, "")] return entries def build_affected( affected_versions_value: str, *, vendor: str, product: str, package_name: str, collection_url: str, version_start: str | None, product_overrides: dict[str, str] | None = None, purl_type: str | None = None, purl_namespace: str | None = None, ) -> list[dict]: """Return one ``affected[]`` entry per detected package. The *Affected versions* body field is parsed line by line. Each line that starts with an Airflow package directory name (``apache-airflow`` or ``apache-airflow-project-<dir>``) creates an entry with that package's derived product / package identity. Lines without a recognisable prefix — or an empty field — fall back to the explicit ``product`` / ``package_name`` arguments, which is the historical single-entry shape. ``product_overrides`` is consulted *before* :data:`PROJECT_DISPLAY_MAP` and lets callers shadow the resolved product name for any specific package. It is wired to the ``--product-for PACKAGE=PRODUCT`` CLI flag. """ entries: list[dict] = [] for detected_package, version_expr in _split_affected_lines(affected_versions_value): if detected_package is not None: entry_package = detected_package entry_product = _product_for_package( detected_package, product_overrides=product_overrides, ) version_source = version_expr else: entry_package = package_name entry_product = product version_source = version_expr or affected_versions_value # The purl is the CNA's recommended package identifier, and the CVE # tool derives `collectionURL` / `packageName` from it when both are # absent — while reporting a stored pair that disagrees with the purl. # So an entry carries one or the other and never both: the purl when # there is one, the legacy pair only for a product that opted out of # purls with `purl_type = "none"` because its ecosystem has none. entry_collection_url, entry_purl_type = package_identity( entry_package, collection_url=collection_url, purl_type=purl_type or "" ) resolved_type = resolve_purl_type(entry_purl_type, entry_collection_url) purl: str | None = None if resolved_type != PURL_TYPE_NONE: purl = compute_purl( resolved_type, entry_package, namespace=purl_namespace or "", # For a Helm chart the collection URL *is* the chart # repository, which is the only thing that identifies it. repository_url=entry_collection_url or "", ) if not purl: raise ValueError( f"Could not build a Package URL (purl) for {entry_package!r}. " f"A CVE record needs `affected[].packageURL` to be promoted, " f"so the record is not emitted without one.\n" f"Fix one of:\n" f' - set `product.purl_type` (e.g. "pypi", "npm", "maven") ' f"when it cannot be derived from " f"`product.default_collection_url` ({entry_collection_url!r}), " f'or `[packages.overrides."{entry_package}"].purl_type`;\n' f" - set `product.purl_namespace` when the type requires one " f"(maven groupId, composer vendor, go module prefix);\n" f" - set `product.default_collection_url` (or the package's " f"`collection_url` override) when the type requires a " f"repository. A Helm chart needs the base URL it is published " f"to — ask whoever publishes the chart if it is not recorded " f"anywhere; the chart name alone does not identify it;\n" f' - set `product.purl_type = "{PURL_TYPE_NONE}"` to state ' f"deliberately that this product has no package host." ) # Keys are inserted in alphabetical order. entry: dict = {} if purl is None: entry["collectionURL"] = entry_collection_url entry["defaultStatus"] = "unaffected" if purl is None: entry["packageName"] = entry_package else: entry["packageURL"] = purl entry["product"] = entry_product entry["vendor"] = vendor entry["versions"] = parse_affected_versions(version_source, version_start) entries.append(entry) return entries def build_descriptions(plain_text: str) -> list[dict]: if not plain_text: return [] return [ { "lang": DEFAULT_LANG, "supportingMedia": [ { "base64": False, "type": "text/html", "value": to_html(plain_text), } ], "value": plain_text, } ] def build_problem_types(cwe_value: str) -> list[dict]: if not cwe_value: return [] cwe_id, description = parse_cwe(cwe_value) desc: dict = { "lang": DEFAULT_LANG, "description": description, } if cwe_id: desc["cweId"] = cwe_id desc["type"] = "CWE" return [{"descriptions": [desc]}] def build_metrics(severity_value: str) -> list[dict]: if not severity_value: return [] return [ { "other": { "content": {"text": normalise_severity(severity_value)}, "type": "Textual description of severity", } } ] def build_credits( credited_as_value: str, *, remediation_developers: list[str], ) -> list[dict]: credits: list[dict] = [] for name in parse_credits_from_field(credited_as_value): credit_type = TOOL_CREDIT_TYPE if is_bot_credit(name) else DEFAULT_CREDIT_TYPE credits.append({"lang": DEFAULT_LANG, "type": credit_type, "value": name}) for name in remediation_developers: cleaned = name.strip() if cleaned: credits.append( { "lang": DEFAULT_LANG, "type": "remediation developer", "value": cleaned, } ) return credits def build_cna_container( *, title: str, description: str, affected_versions_value: str, cwe_value: str, severity_value: str, credits_value: str, mailing_list_value: str, pr_value: str, vendor: str, product: str, package_name: str, collection_url: str, org_id: str, version_start: str | None, discovery: str, remediation_developers: list[str], advisory_urls: list[str] | None = None, purl_type: str | None = None, purl_namespace: str | None = None, product_overrides: dict[str, str] | None = None, current_cve_id: str | None = None, ) -> dict: # Sibling-CVE cross-references — extract every distinct CVE-YYYY-NNNNN # mentioned in the description (the short public summary) and emit a # ``cve.org/CVERecord?id=<id>`` reference for each, tagged ``related`` # by :func:`classify_reference`. This satisfies ASF Security's request # (Arnout Engelen, 2026-05-29 review on CVE-2026-49298) that incomplete- # fix follow-ups carry a structured ``references[]`` link back to the # prior CVE. The current record's own CVE ID is excluded so the # generator never emits a self-reference. related_cve_urls = [related_cve_url(cid) for cid in extract_related_cve_ids(description, current_cve_id)] extra_urls = list(advisory_urls or []) + related_cve_urls cna: dict = { "affected": build_affected( affected_versions_value, vendor=vendor, product=product, package_name=package_name, collection_url=collection_url, version_start=version_start, product_overrides=product_overrides, purl_type=purl_type, purl_namespace=purl_namespace, ), "credits": build_credits(credits_value, remediation_developers=remediation_developers), "descriptions": build_descriptions(description), "metrics": build_metrics(severity_value), "problemTypes": build_problem_types(cwe_value), "providerMetadata": {"orgId": org_id}, "references": build_references(mailing_list_value, pr_value, extra_urls=extra_urls), "source": {"discovery": discovery}, "title": title, "x_generator": {"engine": GENERATOR_TAG}, } return cna def _has_vendor_advisory_reference(cna: dict) -> bool: """Return ``True`` when the CNA container has at least one ``references[]`` entry tagged ``vendor-advisory``. :func:`classify_reference` tags ``lists.apache.org/...`` and ``security.apache.org/...`` URLs as ``vendor-advisory`` automatically. When the tracking issue's *"Public advisory URL"* body field has been populated with the archived advisory URL — i.e. the advisory has actually shipped to users@ / announce@ and been archived on the public ponymail archive — that URL flows into ``references[]`` and flips this check to ``True``. That is the cue the Vulnogram workflow state should advance from ``REVIEW`` to ``PUBLIC``. The result lands in ``CNA_private.state`` in the emitted envelope (:func:`wrap_cve_record`); ``cveMetadata.state`` is the standard CVE 5.x schema field and always holds ``"PUBLISHED"``. """ references = cna.get("references") or [] for ref in references: tags = ref.get("tags") or [] if "vendor-advisory" in tags: return True return False def _is_cna_ready_for_review(cna: dict, cve_id: str) -> bool: """Return ``True`` when the CNA container has everything a release manager needs for the ASF CVE tool's ``REVIEW`` state. The fields checked here correspond to the minimum a release manager needs to send the public advisory: a CVE ID, a title, a non-empty description, at least one affected-versions entry, a CWE in ``problemTypes``, a severity in ``metrics``, at least one credit, and at least one reference (the fix PR). If any of those is missing, the record is a ``DRAFT`` -- still useful as the paste-ready scaffold, but not yet review-ready. """ if not cve_id: return False if not cna.get("title") or not str(cna.get("title")).strip(): return False descriptions = cna.get("descriptions") or [] if not any((d.get("value") or "").strip() for d in descriptions): return False affected = cna.get("affected") or [] if not any(a.get("versions") for a in affected): return False problem_types = cna.get("problemTypes") or [] has_cwe = any( (desc.get("cweId") or "").strip() for pt in problem_types for desc in (pt.get("descriptions") or []) ) if not has_cwe: return False metrics = cna.get("metrics") or [] has_severity = False for metric in metrics: other = metric.get("other") or {} content = other.get("content") or {} severity_text = (content.get("text") or "").strip().lower() if severity_text and severity_text not in {"unknown", "_no response_"}: has_severity = True break if not has_severity: return False credits = cna.get("credits") or [] if not any(c.get("value") for c in credits): return False references = cna.get("references") or [] return any(r.get("url") for r in references) def compute_cna_private_state( cna: dict, cve_id: str, *, release_vote_in_progress: bool | None = None, ) -> str: """Return the ``CNA_private.state`` value (DRAFT / REVIEW / PUBLIC). State machine: * ``DRAFT`` — the CNA is incomplete (some required field missing) *or* the caller signalled "release-vote gating is on but no vote is happening" (``release_vote_in_progress=False``). * ``REVIEW`` — the CNA is fully populated (``_is_cna_ready_for_review`` passes), no public advisory has shipped yet, AND either release-vote gating is disabled (``release_vote_in_progress=None``, the default — legacy behaviour for non-ASF adopters) or the caller signalled that an RC is being voted (``release_vote_in_progress=True``). * ``PUBLIC`` — the CNA is review-ready *and* the public advisory has shipped (a ``vendor-advisory`` -
__init__.py 3 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Public entry points for the ``generate-cve-json`` project. The implementation lives in :mod:`generate_cve_json.cve_json`; this package re-exports the names that callers (``generate-cve-json`` console script, ``python -m generate_cve_json``, and the test suite) depend on so they can keep using ``from generate_cve_json import X``. """ from __future__ import annotations from generate_cve_json.cve_json import ( COLLECTION_URL_TO_PROJECT_URL_TEMPLATE, NEXT_VERSION_TOKEN_RE, PROJECT_DISPLAY_MAP, PURL_TYPE_TO_PROJECT_URL_TEMPLATE, _build_attachment_body, _has_vendor_advisory_reference, _is_cna_ready_for_review, _product_for_package, attach_to_issue, build_affected, build_cna_container, build_credits, build_descriptions, build_metrics, build_problem_types, build_references, classify_reference, combine_remediation_developers, compute_cna_private_state, compute_package_url, compute_purl, emit_json, extract_field, fetch_issue, format_version_range, main, package_identity, parse_affected_versions, parse_args, parse_credits_from_field, parse_cve_id, parse_cwe, parse_url_list, resolve_purl_type, resolve_title, split_purl, wrap_cve_record, ) __all__ = [ "COLLECTION_URL_TO_PROJECT_URL_TEMPLATE", "NEXT_VERSION_TOKEN_RE", "PROJECT_DISPLAY_MAP", "PURL_TYPE_TO_PROJECT_URL_TEMPLATE", "_build_attachment_body", "_has_vendor_advisory_reference", "_is_cna_ready_for_review", "_product_for_package", "attach_to_issue", "build_affected", "build_cna_container", "build_credits", "build_descriptions", "build_metrics", "build_problem_types", "build_references", "classify_reference", "combine_remediation_developers", "compute_cna_private_state", "compute_package_url", "compute_purl", "emit_json", "extract_field", "fetch_issue", "format_version_range", "main", "package_identity", "parse_affected_versions", "parse_args", "parse_credits_from_field", "parse_cve_id", "parse_cwe", "parse_url_list", "resolve_purl_type", "resolve_title", "split_purl", "wrap_cve_record", ] -
__main__.py 951 B
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Entry point for ``python -m generate_cve_json``.""" from __future__ import annotations from generate_cve_json import main if __name__ == "__main__": main()
-
-
-
tests
-
fixtures
-
cve-json-config-providers.toml 2.5 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # generate-cve-json — TEST FIXTURE config (regression coverage for # subpackage prefixes other than `-project-`). # # This fixture mirrors a project that ships subpackages under a # `-providers-<dir>` convention (the shape used by, for example, # Apache Airflow's third-party-integration packages on PyPI). The # `_product_for_package` lookup must read the `project` group from # whatever the configured `package_pattern` declares — not assume # a literal `-project-` substring — so the display-map lookup # fires for any subpackage convention an adopter chooses. [product] vendor = "Apache Software Foundation" default_product = "Apache Example" default_package_name = "apache-example" default_collection_url = "https://pypi.python.org" # Package URL type for `affected[].packageURL`. Derived from # `default_collection_url` when unset; `"none"` opts out explicitly. purl_type = "pypi" [cna] org_id = "f0158376-9dc2-43b6-827c-5f631a4d8d09" [cna_private] project_url = "https://example.apache.org/" owner = "example" users_list = "users@example.apache.org" [meta] tracker_repo = "apache-example-s/apache-example-s" generator_tag = "apache-example-s/generate_cve_json.py" [packages] # Subpackages live under `apache-example-providers-<dir>` here, not # under `-project-<dir>`. The named `project` group is still required # and consumed the same way by the generator. package_pattern = '^(?P<package>apache-example(?:-providers-(?P<project>[a-z0-9][a-z0-9_-]*))?)(?:\s+(?P<rest>.*))?$' top_level_name = "apache-example" top_level_product = "Apache Example" project_product_template = "Apache Example {display} provider" [packages.project_display_map] "cncf-kubernetes" = "CNCF Kubernetes" "amazon" = "Amazon" "apache-spark" = "Apache Spark" -
cve-json-config.toml 5.8 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # generate-cve-json — TEST FIXTURE config. # # This file is a fixture for the tool's own pytest suite, NOT a # default that adopters should copy. It describes a fictional # **`apache-example`** project — top-level package `apache-example` # with a sub-project layout (`apache-example-project-<name>`). The # shape is realistic enough to exercise every code path in the tool # (top-level + sub-project package matching, project display-map # lookup, multi-product CVE handling, the `< NEXT VERSION` # placeholder, etc.) without tying the test suite to any real # adopter's taxonomy. # # Adopters MUST NOT copy this file. Write your own # `cve-json-config.toml` from scratch using the schema documented in # the package README. Nothing in the framework treats the values # below as defaults. [product] # CVE 5.x `affected[].vendor` — the CNA vendor name. All Apache # project CVEs share this vendor regardless of which project ships # the affected code. vendor = "Apache Software Foundation" # Top-level product display name and package name. Used as the default # `product` / `packageName` for *Affected versions* lines that don't # match a more specific entry in `[packages]` below. default_product = "Apache Example" default_package_name = "apache-example" # Where the package is distributed (`affected[].collectionURL`). default_collection_url = "https://pypi.python.org" [cna] # CNA assigner UUID (CVE 5.x `cveMetadata.assignerOrgId` / # `providerMetadata.orgId`). This is the ASF org id (shared across # every Apache project's CVEs). org_id = "f0158376-9dc2-43b6-827c-5f631a4d8d09" [cna_private] # Per-project fields that go into the CVE 5.x `CNA_private` # envelope. These are project-scoped values that ASF Vulnogram # uses internally; adopters fill them with their own URLs. project_url = "https://example.apache.org/" owner = "example" users_list = "users@example.apache.org" [workflow] # Opt-in: when true, the DRAFT → REVIEW transition is gated on an # active release vote (detected either via the rc_voting_label below # or via the --review CLI override). Default: false — non-ASF # adopters that publish advisories without a separate release-vote # step want the original behavior where a fully-populated record # auto-advances to REVIEW. ASF adopters set this to true to keep the # Vulnogram REVIEW state aligned with the actual "RC is being voted" # window. release_vote_gating = false # Tracker label that signals "an RC is currently being voted". Only # consulted when release_vote_gating = true; otherwise the label has # no effect on the emitted state. Default: "rc voting". rc_voting_label = "rc voting" [meta] # Tracker repo slug (org/name). Used for the `x_generator.engine` # tag in the CVE record and for the self-source-link below. tracker_repo = "apache-example-s/apache-example-s" # Stable identifier embedded in the CVE record's `x_generator.engine` # field so a reader can identify which tool produced the JSON. generator_tag = "apache-example-s/generate_cve_json.py" # Self-source URL the script can self-document with (e.g. in error # messages, on the `_print_skill_link` line of the generated comment). # Optional: defaults to f"https://github.com/{tracker_repo}". skill_source_url = "https://github.com/apache-example-s/apache-example-s/tree/main/tools/cve-tool-vulnogram/generate-cve-json" [packages] # Regex matching this project's package names. Required named groups: # `package` — the full package name as shipped on PyPI (whatever # collection_url declares). # `project` — optional; the sub-project directory name used to look # up a display name in `project_display_map` below. May # be empty when the line names the top-level package # only. # `rest` — optional; everything after the package name, used # by the script to consume the trailing version-range # expression. package_pattern = '^(?P<package>apache-example(?:-project-(?P<project>[a-z0-9][a-z0-9_-]*))?)(?:\s+(?P<rest>.*))?$' # Top-level package name + product. When the matched `package_pattern` # yields just the top-level name (no `project` group), the product is # `top_level_product`. When `project` is present, the product is # `project_product_template` with `{display}` substituted from # `project_display_map` (or a title-cased fallback). top_level_name = "apache-example" top_level_product = "Apache Example" project_product_template = "Apache Example Project {display}" # Sub-project directory-name → vendor-preferred display casing for # the CVE `product` field. Directory names are lowercase # (`foo`, `acme-xyz`); CVE product names follow vendor-preferred # casing (`Foo`, `Acme XYZ`). Extend this table when a new sub-project # appears in a CVE; unknown names fall back to a title-cased # dash-split of the directory name, which is correct for most # single-word sub-projects but may need an entry here for acronyms. [packages.project_display_map] "foo" = "Foo" "bar" = "Bar" "kerfluffle" = "Kerfluffle" "pop-corn" = "Pop Corn" "xyz" = "XYZ" "acme-xyz" = "Acme XYZ"
-
-
conftest.py 1.8 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Pytest conftest for generate-cve-json. The Python tool loads all project-specific values from a TOML config the adopting project ships at: <adopter-tracker>/.apache-magpie-overrides/tools/cve-tool-vulnogram/cve-json-config.toml The tool reads that config relative to ``cwd`` (or via the ``CVE_JSON_CONFIG`` environment variable / the ``--config`` CLI flag). For the framework's own pytest suite (run from this repository, no adopter present), we point ``CVE_JSON_CONFIG`` at the test fixture at ``tests/fixtures/cve-json-config.toml``. The fixture mirrors one adopter's configuration so the existing tests keep their assertions; **it is not the framework's default configuration** — adopters write their own per the schema in the package README. """ from __future__ import annotations import os from pathlib import Path _FIXTURE_CONFIG = Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml" assert _FIXTURE_CONFIG.exists(), f"missing test fixture: {_FIXTURE_CONFIG}" os.environ.setdefault("CVE_JSON_CONFIG", str(_FIXTURE_CONFIG)) -
test_cli.py 28.8 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Tests for the CLI surface and `gh`-shelling-out helpers. These cover the parts of `generate_cve_json.cve_json` that don't have existing unit tests in `test_generate_cve_json.py`: - `parse_args` (CLI argument shape); - `main` (orchestration, all error paths, --stdin / --output / --attach / --no-envelope flags); - `fetch_issue`, `_gh_api_json`, `_fetch_issue` (gh subprocess wrappers with mocked `subprocess.run`); - `_splice_attachment_into_body` (in-place attachment splicing); - `attach_to_issue` (idempotent attachment with mocked gh). """ from __future__ import annotations import io import json import subprocess from pathlib import Path from unittest.mock import MagicMock, patch import pytest from generate_cve_json import cve_json def _issue_body( *, summary: str = "An RCE in the FooOperator.", affected: str = ">= 3.0.0, < 3.1.0", cwe: str = "CWE-352", severity: str = "high", credited: str = "Alice <alice@example.com>", cve_tool: str = "CVE-2026-12345", extra: dict[str, str] | None = None, ) -> str: """Build a minimal issue body with the standard tracker fields.""" fields = { "Short public summary for publish": summary, "Affected versions": affected, "Security mailing list thread": "https://lists.example.org/thread.html/abc", "Public advisory URL": "", "Reporter credited as": credited, "PR with the fix": "https://github.com/upstream/repo/pull/4242", "Remediation developer": "", "CWE": cwe, "Severity": severity, "CVE tool link": cve_tool, } if extra: fields.update(extra) return "\n".join(f"### {k}\n{v}\n" for k, v in fields.items()) # --- parse_args ------------------------------------------------------------ class TestParseArgs: def test_minimal_positional(self): args = cve_json.parse_args(["123"]) assert args.issue == "123" assert args.stdin is False assert args.attach is False assert args.no_envelope is False assert args.advisory_url == [] assert args.remediation_developer == [] assert args.product_for == [] assert args.config is None def test_stdin_no_positional(self): args = cve_json.parse_args(["--stdin"]) assert args.issue is None assert args.stdin is True def test_repeatable_flags_accumulate(self): args = cve_json.parse_args( [ "123", "--advisory-url", "https://a.example/1", "--advisory-url", "https://a.example/2", "--remediation-developer", "Carol", "--product-for", "pkg-a=Display A", "--product-for", "pkg-b=Display B", ] ) assert args.advisory_url == ["https://a.example/1", "https://a.example/2"] assert args.remediation_developer == ["Carol"] assert args.product_for == ["pkg-a=Display A", "pkg-b=Display B"] def test_overrides(self, tmp_path): cfg = tmp_path / "c.toml" args = cve_json.parse_args( [ "42", "--config", str(cfg), "--repo", "owner/repo", "--cve-id", "CVE-2099-0001", "--title", "Example Title", "--vendor", "Foo", "--product", "Bar", "--package-name", "bar-py", "--collection-url", "https://pypi.org/project/bar", "--org-id", "org-1", "--version-start", "1.0.0", "--discovery", "INTERNAL", "--no-envelope", "--attach", "--output", str(tmp_path / "out.json"), ] ) assert args.config == cfg assert args.repo == "owner/repo" assert args.cve_id == "CVE-2099-0001" assert args.no_envelope is True assert args.attach is True assert args.output == tmp_path / "out.json" # --- _splice_attachment_into_body ----------------------------------------- class TestSpliceAttachment: cve_id = "CVE-2026-99999" def _markers(self): return ( cve_json._attachment_marker_begin(self.cve_id), cve_json._attachment_marker_end(self.cve_id), ) def _attachment(self): begin, end = self._markers() return f"{begin}\n## CVE JSON\n```json\n{{}}\n```\n{end}\n" def test_replaces_existing_attachment_block(self): begin, end = self._markers() body = f"### Body before\nstuff\n\n{begin}\nOLD CONTENT\n{end}\ntrailing line\n" result = cve_json._splice_attachment_into_body(body, self._attachment(), self.cve_id) assert "OLD CONTENT" not in result assert "## CVE JSON" in result # Markers are present and bracket the new content. assert begin in result assert end in result def test_legacy_single_marker_path(self): begin, _ = self._markers() body = f"### Body\n\n{begin}\nleftover legacy content with no end marker\n" result = cve_json._splice_attachment_into_body(body, self._attachment(), self.cve_id) assert "leftover legacy content" not in result assert "## CVE JSON" in result def test_appends_after_cve_tool_link_when_no_existing_attachment(self): body = _issue_body() result = cve_json._splice_attachment_into_body(body, self._attachment(), self.cve_id) assert "## CVE JSON" in result # The attachment lands after the CVE-tool-link section. cve_tool_idx = result.index("### CVE tool link") attach_idx = result.index("## CVE JSON") assert attach_idx > cve_tool_idx def test_appends_at_end_when_no_cve_tool_link_field(self): body = "### Body\nstuff\n" result = cve_json._splice_attachment_into_body(body, self._attachment(), self.cve_id) assert result.endswith(self._markers()[1] + "\n") # --- fetch_issue (gh subprocess wrapper) ---------------------------------- class TestFetchIssue: def test_returns_title_body_and_labels(self): completed = MagicMock( stdout=json.dumps( { "title": "T", "body": "B", "labels": [ {"name": "rc voting"}, {"name": "airflow"}, ], } ) ) with patch("generate_cve_json.cve_json.subprocess.run", return_value=completed) as run: title, body, labels = cve_json.fetch_issue(42, "owner/repo") assert (title, body) == ("T", "B") assert labels == ["rc voting", "airflow"] # Verify the gh call shape. cmd = run.call_args.args[0] assert cmd[:3] == ["gh", "issue", "view"] assert "--repo" in cmd assert "owner/repo" in cmd # Verify we ask gh for labels in addition to title and body. json_arg_index = cmd.index("--json") assert "labels" in cmd[json_arg_index + 1] def test_returns_empty_labels_when_field_missing(self): # gh returns no `labels` key when the issue has no labels — we # default to an empty list rather than blowing up. completed = MagicMock(stdout=json.dumps({"title": "T", "body": "B"})) with patch("generate_cve_json.cve_json.subprocess.run", return_value=completed): title, body, labels = cve_json.fetch_issue(42, "owner/repo") assert (title, body, labels) == ("T", "B", []) def test_gh_missing_raises_runtime_error(self): with patch( "generate_cve_json.cve_json.subprocess.run", side_effect=FileNotFoundError, ): with pytest.raises(RuntimeError, match="`gh` CLI not found"): cve_json.fetch_issue(1, "o/r") def test_gh_failure_raises_runtime_error(self): err = subprocess.CalledProcessError(returncode=1, cmd=["gh"], stderr="not found") with patch("generate_cve_json.cve_json.subprocess.run", side_effect=err): with pytest.raises(RuntimeError, match="not found"): cve_json.fetch_issue(1, "o/r") # --- _gh_api_json ---------------------------------------------------------- class TestGhApiJson: def test_returns_parsed_json(self): completed = MagicMock(stdout='{"id": 7}') with patch("generate_cve_json.cve_json.subprocess.run", return_value=completed) as run: result = cve_json._gh_api_json(["repos/o/r/issues/1"]) assert result == {"id": 7} cmd = run.call_args.args[0] assert cmd[:2] == ["gh", "api"] assert cmd[2] == "repos/o/r/issues/1" def test_returns_empty_dict_on_empty_stdout(self): completed = MagicMock(stdout=" \n") with patch("generate_cve_json.cve_json.subprocess.run", return_value=completed): assert cve_json._gh_api_json(["repos/o/r"]) == {} def test_writes_body_payload_to_temp_file_and_uses_input_flag(self, tmp_path): completed = MagicMock(stdout="{}") captured_args: dict = {} def fake_run(cmd, **_kw): captured_args["cmd"] = cmd # Read back the temp file we wrote, prove it has our payload. input_idx = cmd.index("--input") captured_args["payload"] = json.loads(Path(cmd[input_idx + 1]).read_text()) return completed with patch("generate_cve_json.cve_json.subprocess.run", side_effect=fake_run): cve_json._gh_api_json( ["-X", "PATCH", "repos/o/r/issues/1"], body_payload={"body": "new"}, ) assert captured_args["payload"] == {"body": "new"} assert "--input" in captured_args["cmd"] def test_gh_missing_raises_runtime_error(self): with patch( "generate_cve_json.cve_json.subprocess.run", side_effect=FileNotFoundError, ): with pytest.raises(RuntimeError, match="`gh` CLI not found"): cve_json._gh_api_json(["x"]) def test_gh_failure_raises_runtime_error(self): err = subprocess.CalledProcessError(returncode=22, cmd=["gh"], stderr="rate limited\n") with patch("generate_cve_json.cve_json.subprocess.run", side_effect=err): with pytest.raises(RuntimeError, match="rate limited"): cve_json._gh_api_json(["x"]) # --- _fetch_issue (REST wrapper) ------------------------------------------ class TestFetchIssueRest: def test_returns_dict(self): with patch( "generate_cve_json.cve_json._gh_api_json", return_value={"body": "x", "html_url": "u"}, ): assert cve_json._fetch_issue("o/r", "42") == { "body": "x", "html_url": "u", } def test_raises_when_response_not_a_dict(self): with patch("generate_cve_json.cve_json._gh_api_json", return_value=[]): with pytest.raises(RuntimeError, match="Unexpected response shape"): cve_json._fetch_issue("o/r", "42") # --- attach_to_issue ------------------------------------------------------- def _attach_kwargs() -> dict: """Common keyword arguments for `attach_to_issue` calls in this module.""" return { "issue_number": "42", "repo": "o/r", "cve_id": "CVE-2026-12345", "json_text": '{"a": 1}', "cna": {"title": "T"}, "cna_private_state": "DRAFT", } class TestAttachToIssue: def test_appends_attachment_when_body_has_no_existing_marker(self): with ( patch( "generate_cve_json.cve_json._fetch_issue", return_value={"body": _issue_body(), "html_url": "https://x/42"}, ), patch("generate_cve_json.cve_json._gh_api_json", return_value={}) as patch_call, ): url, was_update = cve_json.attach_to_issue(**_attach_kwargs()) assert was_update is False assert url.endswith("#cve-json--paste-ready-for-cve-2026-12345") # PATCH was called with a body that contains the begin marker. body_payload = patch_call.call_args.kwargs["body_payload"]["body"] assert cve_json._attachment_marker_begin("CVE-2026-12345") in body_payload def test_replaces_existing_attachment_idempotently(self): cve_id = "CVE-2026-12345" begin = cve_json._attachment_marker_begin(cve_id) end = cve_json._attachment_marker_end(cve_id) body_with_existing = _issue_body() + f"\n{begin}\nstale content\n{end}\n" with ( patch( "generate_cve_json.cve_json._fetch_issue", return_value={"body": body_with_existing, "html_url": "https://x/42"}, ), patch("generate_cve_json.cve_json._gh_api_json", return_value={}), ): url, was_update = cve_json.attach_to_issue(**_attach_kwargs()) assert was_update is True assert url.endswith("#cve-json--paste-ready-for-cve-2026-12345") def test_skips_patch_when_body_unchanged(self): # Pre-build the attachment that _splice would produce, so the # spliced result is byte-identical to the existing body. cve_id = "CVE-2026-12345" attachment = cve_json._build_attachment_body( cve_id=cve_id, json_text='{"a": 1}', cna={"title": "T"}, cna_private_state="DRAFT", ) existing = cve_json._splice_attachment_into_body(_issue_body(), attachment, cve_id) with ( patch( "generate_cve_json.cve_json._fetch_issue", return_value={"body": existing, "html_url": "https://x/42"}, ), patch("generate_cve_json.cve_json._gh_api_json") as patch_call, ): url, was_update = cve_json.attach_to_issue(**_attach_kwargs()) assert was_update is True # had_existing was True patch_call.assert_not_called() assert url == "https://x/42" # --- main: error paths ----------------------------------------------------- class TestMainErrors: def test_attach_with_stdin_returns_2(self, capsys): rc = cve_json.main(["--stdin", "--attach"]) assert rc == 2 assert "--attach cannot be combined with --stdin" in capsys.readouterr().err def test_attach_without_issue_returns_2(self, capsys): rc = cve_json.main(["--attach"]) assert rc == 2 assert "--attach requires the positional issue argument" in capsys.readouterr().err def test_missing_issue_without_stdin_returns_2(self, capsys): rc = cve_json.main([]) assert rc == 2 assert "issue number is required unless --stdin" in capsys.readouterr().err def test_fetch_issue_failure_returns_1(self, capsys): with patch( "generate_cve_json.cve_json.fetch_issue", side_effect=RuntimeError("gh boom"), ): rc = cve_json.main(["123"]) assert rc == 1 assert "gh boom" in capsys.readouterr().err def test_product_for_without_equals_returns_2(self, capsys, monkeypatch): monkeypatch.setattr("sys.stdin", io.StringIO(_issue_body())) rc = cve_json.main(["--stdin", "--product-for", "no-equals-sign"]) assert rc == 2 assert "PACKAGE=PRODUCT" in capsys.readouterr().err def test_product_for_with_empty_value_returns_2(self, capsys, monkeypatch): monkeypatch.setattr("sys.stdin", io.StringIO(_issue_body())) rc = cve_json.main(["--stdin", "--product-for", "pkg="]) assert rc == 2 assert "non-empty PACKAGE and PRODUCT" in capsys.readouterr().err def test_config_not_found_returns_2(self, tmp_path, capsys): rc = cve_json.main(["--stdin", "--config", str(tmp_path / "nope.toml")]) assert rc == 2 assert "error:" in capsys.readouterr().err def test_unparsable_affected_versions_returns_2(self, capsys, monkeypatch): # parse_affected_versions now raises ValueError on un-parseable # input rather than emitting a malformed `version` string. main # catches that and exits 2 with a clean error message — no # traceback shown to the user. body = _issue_body(affected=">= 3.0.0 (reporter verified against 3.2.1)") monkeypatch.setattr("sys.stdin", io.StringIO(body)) rc = cve_json.main(["--stdin"]) assert rc == 2 captured = capsys.readouterr() assert "Could not parse" in captured.err assert "Affected versions" in captured.err # --- main: happy paths ----------------------------------------------------- class TestMainHappyPath: def test_stdin_writes_full_envelope_to_stdout(self, monkeypatch, capsys): monkeypatch.setattr("sys.stdin", io.StringIO(_issue_body())) rc = cve_json.main(["--stdin"]) assert rc == 0 out = capsys.readouterr().out record = json.loads(out) # Full CVE 5.x record envelope: cveMetadata + containers. assert "cveMetadata" in record assert "containers" in record assert record["cveMetadata"]["cveId"] == "CVE-2026-12345" def test_stdin_no_envelope_emits_cna_only(self, monkeypatch, capsys): monkeypatch.setattr("sys.stdin", io.StringIO(_issue_body())) rc = cve_json.main(["--stdin", "--no-envelope"]) assert rc == 0 record = json.loads(capsys.readouterr().out) # Bare CNA container — no top-level cveMetadata. assert "cveMetadata" not in record assert "title" in record assert "affected" in record def test_output_to_file(self, monkeypatch, tmp_path, capsys): monkeypatch.setattr("sys.stdin", io.StringIO(_issue_body())) out_file = tmp_path / "cve.json" rc = cve_json.main(["--stdin", "--output", str(out_file)]) assert rc == 0 # File written, not stdout. record = json.loads(out_file.read_text()) assert "cveMetadata" in record # Friendly post-write print includes the byte count. assert "Wrote " in capsys.readouterr().out def test_fetch_path_uses_gh(self, capsys): with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), []), ) as fetch: rc = cve_json.main(["123"]) assert rc == 0 fetch.assert_called_once() record = json.loads(capsys.readouterr().out) assert record["cveMetadata"]["cveId"] == "CVE-2026-12345" def test_attach_happy_path(self, capsys): with ( patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), []), ), patch( "generate_cve_json.cve_json.attach_to_issue", return_value=("https://x/42#cve-json--paste-ready-for-cve-2026-12345", False), ) as attach, ): rc = cve_json.main(["123", "--attach"]) assert rc == 0 attach.assert_called_once() out = capsys.readouterr().out assert "Embedded CVE JSON" in out def test_attach_replace_path(self, capsys): with ( patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), []), ), patch( "generate_cve_json.cve_json.attach_to_issue", return_value=("https://x/42#anchor", True), ), ): rc = cve_json.main(["123", "--attach"]) assert rc == 0 assert "Replaced CVE JSON" in capsys.readouterr().out def test_attach_failure_returns_1(self, capsys): with ( patch( "generate_cve_json.cve_json.fetch_issue", return_value=("T", _issue_body(), []), ), patch( "generate_cve_json.cve_json.attach_to_issue", side_effect=RuntimeError("attach boom"), ), ): rc = cve_json.main(["123", "--attach"]) assert rc == 1 assert "attach boom" in capsys.readouterr().err # --- main: release-vote gating -------------------------------------------- class TestReleaseVoteGating: """End-to-end coverage for the [workflow].release_vote_gating switch and the --review / --draft CLI overrides. The state machine has three modes: 1. Gating off (default): fully-populated CNA ⇒ REVIEW. Legacy behaviour preserved for non-ASF adopters. 2. Gating on, no label / no override: CNA stays at DRAFT. 3. Gating on + rc-voting label *or* --review: CNA ⇒ REVIEW. The --review / --draft flags also work in gating-off mode (manual overrides remain available regardless of config). """ @staticmethod def _write_gating_on_config(tmp_path, label: str = "rc voting") -> str: """Write a config that mirrors the fixture but with release_vote_gating = true. Returns its path as a string suitable for the --config CLI flag. """ fixture = (Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml").read_text() # Flip the gating flag and (optionally) the label name. patched = fixture.replace( "release_vote_gating = false", "release_vote_gating = true", ).replace( 'rc_voting_label = "rc voting"', f'rc_voting_label = "{label}"', ) cfg_path = tmp_path / "cve-json-config-gating-on.toml" cfg_path.write_text(patched) return str(cfg_path) def test_default_gating_off_emits_review(self, capsys): # Default config has release_vote_gating = false, so a ready # CNA without any vote signal still advances to REVIEW. with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), []), ): rc = cve_json.main(["123"]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "REVIEW" def test_gating_on_without_label_stays_draft(self, capsys, tmp_path): cfg = self._write_gating_on_config(tmp_path) with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["airflow"]), ): rc = cve_json.main(["123", "--config", cfg]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "DRAFT" def test_gating_on_with_label_emits_review(self, capsys, tmp_path): cfg = self._write_gating_on_config(tmp_path) with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["airflow", "rc voting"]), ): rc = cve_json.main(["123", "--config", cfg]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "REVIEW" def test_review_flag_overrides_gating_with_no_label(self, capsys, tmp_path): cfg = self._write_gating_on_config(tmp_path) with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), []), ): rc = cve_json.main(["123", "--config", cfg, "--review"]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "REVIEW" def test_draft_flag_overrides_gating_with_label(self, capsys, tmp_path): # --draft beats a "rc voting" label — used to walk a record # back when the RC vote failed but the label is still on the # tracker. cfg = self._write_gating_on_config(tmp_path) with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["rc voting"]), ): rc = cve_json.main(["123", "--config", cfg, "--draft"]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "DRAFT" def test_forward_state_labels_keep_state_at_review_when_rc_voting_removed(self, capsys, tmp_path): # After the `pr merged → fix released` transition (per the sync # skill's convention), the `rc voting` label is removed and # `fix released` is added. Without the forward-state-label # check, the state would walk back from REVIEW to DRAFT — # exactly the wrong direction for a record that is about to # be published. Verify every default forward-state label # individually keeps the state at REVIEW. cfg = self._write_gating_on_config(tmp_path) for forward_label in [ "fix released", "announced - emails sent", "announced", "vendor-advisory ready", ]: with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["airflow", forward_label]), ): rc = cve_json.main(["123", "--config", cfg]) assert rc == 0, f"non-zero exit on label={forward_label!r}" record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "REVIEW", ( f"forward-state label {forward_label!r} should keep state at REVIEW, " f"got {record['CNA_private']['state']!r}" ) def test_custom_forward_state_labels_from_config(self, capsys, tmp_path): # Adopters can extend / replace the default forward-state-label # set via [workflow].forward_state_labels in the TOML config. fixture = (Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml").read_text() patched = fixture.replace( "release_vote_gating = false", 'release_vote_gating = true\nforward_state_labels = ["shipped", "advisory-published"]', ) cfg_path = tmp_path / "cve-json-config-custom-forward.toml" cfg_path.write_text(patched) # The default `fix released` is no longer in the set, so a # ready CNA with only `fix released` walks back to DRAFT. with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["airflow", "fix released"]), ): rc = cve_json.main(["123", "--config", str(cfg_path)]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "DRAFT" # `shipped` is in the configured set → REVIEW. with patch( "generate_cve_json.cve_json.fetch_issue", return_value=("Issue title", _issue_body(), ["airflow", "shipped"]), ): rc = cve_json.main(["123", "--config", str(cfg_path)]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "REVIEW" def test_review_and_draft_are_mutually_exclusive(self): # argparse raises SystemExit on mutually-exclusive group conflict; # the exit code is 2. with pytest.raises(SystemExit) as exc_info: cve_json.main(["--stdin", "--review", "--draft"]) assert exc_info.value.code == 2 def test_custom_rc_voting_label_from_config(self, capsys, tmp_path): # The label name is configurable; if a project sets a custom # rc_voting_label, only that label gates REVIEW. The default # "rc voting" label on a tracker for such a project is just # an unrelated tag. cfg = self._write_gating_on_config(tmp_path, label="release-vote-in-progress") with patch( "generate_cve_json.cve_json.fetch_issue", return_value=( "Issue title", _issue_body(), ["rc voting"], # wrong label name — should NOT trigger ), ): rc = cve_json.main(["123", "--config", cfg]) assert rc == 0 record = json.loads(capsys.readouterr().out) assert record["CNA_private"]["state"] == "DRAFT" -
test_config_resolution.py 5.2 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. from __future__ import annotations from pathlib import Path import pytest from generate_cve_json.cve_json import _load_config _DEFAULT_REL = ".apache-magpie-overrides/tools/cve-tool-vulnogram/cve-json-config.toml" _LEGACY_REL = ".apache-magpie-overrides/tools/vulnogram/cve-json-config.toml" def _write(root, relpath, vendor): p = root / relpath p.parent.mkdir(parents=True, exist_ok=True) p.write_text(f'vendor = "{vendor}"\n') return p @pytest.fixture(autouse=True) def _clean_env(monkeypatch, tmp_path): """Resolve config from the (empty) tmp cwd, with no env override.""" monkeypatch.chdir(tmp_path) monkeypatch.delenv("CVE_JSON_CONFIG", raising=False) def test_default_path_is_used_when_present(tmp_path, capsys): _write(tmp_path, _DEFAULT_REL, "New") assert _load_config() == {"vendor": "New"} assert "WARNING" not in capsys.readouterr().err def test_legacy_path_used_with_warning_when_default_missing(tmp_path, capsys): _write(tmp_path, _LEGACY_REL, "Legacy") assert _load_config() == {"vendor": "Legacy"} err = capsys.readouterr().err assert "legacy path" in err assert "tools/cve-tool-vulnogram" in err # points the adopter at the new path def test_default_wins_over_legacy_when_both_present(tmp_path, capsys): _write(tmp_path, _DEFAULT_REL, "New") _write(tmp_path, _LEGACY_REL, "Legacy") assert _load_config() == {"vendor": "New"} assert "WARNING" not in capsys.readouterr().err def test_missing_both_raises_pointing_at_the_new_path(): with pytest.raises(FileNotFoundError) as exc: _load_config() assert "cve-tool-vulnogram" in str(exc.value) def test_explicit_config_path_does_not_fall_back_to_legacy(tmp_path): # Legacy exists, but an explicit (missing) --config path must not silently # resolve to it — explicit intent wins and the error is surfaced. _write(tmp_path, _LEGACY_REL, "Legacy") with pytest.raises(FileNotFoundError): _load_config(tmp_path / "explicit-but-missing.toml") def test_env_config_path_does_not_fall_back_to_legacy(tmp_path, monkeypatch): # Same guarantee as --config: $CVE_JSON_CONFIG naming a missing path must # raise, even when the legacy pre-rename config is present on disk. _write(tmp_path, _LEGACY_REL, "Legacy") monkeypatch.setenv("CVE_JSON_CONFIG", str(tmp_path / "missing.toml")) with pytest.raises(FileNotFoundError): _load_config() @pytest.fixture def _restore_default_config(): """Re-point the module globals at the conftest fixture after the test. `_set_config_path` mutates module-level state, so a test that loads a different config leaks that config into every test that runs after it. """ yield from generate_cve_json import cve_json cve_json._set_config_path(Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml") def test_purl_type_is_optional_and_defaults_to_empty(_restore_default_config): """An adopter who has not opted in gets no `purl_type`, and no purl.""" from generate_cve_json import cve_json fixture = Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml" cve_json._set_config_path(fixture) assert cve_json.DEFAULT_PURL_TYPE == "" def test_purl_type_is_read_from_the_product_section(_restore_default_config): from generate_cve_json import cve_json fixture = Path(__file__).resolve().parent / "fixtures" / "cve-json-config-providers.toml" cve_json._set_config_path(fixture) assert cve_json.DEFAULT_PURL_TYPE == "pypi" def test_purl_namespace_is_optional_and_defaults_to_empty(_restore_default_config): from generate_cve_json import cve_json cve_json._set_config_path(Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml") assert cve_json.DEFAULT_PURL_NAMESPACE == "" def test_purl_namespace_is_read_from_the_product_section(tmp_path, _restore_default_config): from generate_cve_json import cve_json source = Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml" config = tmp_path / "cve-json-config.toml" config.write_text( source.read_text().replace( 'default_collection_url = "https://pypi.python.org"', 'default_collection_url = "https://pypi.python.org"\n' 'purl_type = "maven"\n' 'purl_namespace = "org.apache.example"', ) ) cve_json._set_config_path(config) assert cve_json.DEFAULT_PURL_TYPE == "maven" assert cve_json.DEFAULT_PURL_NAMESPACE == "org.apache.example" -
test_generate_cve_json.py 85 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. """Unit tests for pure helpers in ``generate_cve_json``. Everything that shells out to ``gh`` or touches the network lives in ``fetch_issue``, ``attach_to_issue``, ``_gh_api_json``, and ``_find_existing_attachment_comment_id``. Those are excluded from this test suite; they can be exercised against a live issue by running the CLI end-to-end. """ from __future__ import annotations import json from collections.abc import Iterator from pathlib import Path from typing import Any import pytest from generate_cve_json import ( _build_attachment_body, _is_cna_ready_for_review, _product_for_package, build_affected, build_cna_container, build_credits, build_descriptions, build_metrics, build_problem_types, build_references, classify_reference, combine_remediation_developers, compute_cna_private_state, compute_package_url, compute_purl, cve_json, emit_json, extract_field, format_version_range, package_identity, parse_affected_versions, parse_credits_from_field, parse_cve_id, parse_cwe, parse_url_list, resolve_purl_type, resolve_title, split_purl, wrap_cve_record, ) from generate_cve_json.cve_json import is_bot_credit, normalise_severity, to_html DEFAULT_AFFECTED_ARGS: dict[str, Any] = { "vendor": "Apache Software Foundation", "product": "Apache Example", "package_name": "apache-example", "collection_url": "https://pypi.python.org", "version_start": None, } # --------------------------------------------------------------------------- # Issue body field extraction # --------------------------------------------------------------------------- class TestExtractField: def test_returns_value_up_to_next_field(self): body = ( "### The issue description\nSummary text.\n\n### Affected versions\n>=3.0.0\n\n### CWE\nCWE-352\n" ) assert extract_field(body, "Affected versions") == ">=3.0.0" assert extract_field(body, "CWE") == "CWE-352" def test_missing_field_returns_empty_string(self): body = "### Short public summary for publish\n\nHello.\n" assert extract_field(body, "CVE tool link") == "" def test_no_response_placeholder_is_normalised_to_empty(self): body = "### Severity\n\n_No response_\n" assert extract_field(body, "Severity") == "" def test_trailing_whitespace_is_stripped(self): body = "### Short public summary for publish\n\nHello. \n\n### Affected versions\n\n\n" assert extract_field(body, "Short public summary for publish") == "Hello." def test_public_advisory_url_field_is_separate_from_security_mailing_list_thread(self): body = ( "### Security mailing list thread\n\n" "https://lists.apache.org/thread/fake-security-private-hash\n\n" "### Public advisory URL\n\n" "https://lists.apache.org/thread/real-users-archive-id?users@airflow.apache.org\n\n" "### Reporter credited as\n\nAlice\n" ) # The two fields are addressed independently by name; the private # security@ URL stays in the body but is never exported. assert ( extract_field(body, "Security mailing list thread") == "https://lists.apache.org/thread/fake-security-private-hash" ) assert ( extract_field(body, "Public advisory URL") == "https://lists.apache.org/thread/real-users-archive-id?users@airflow.apache.org" ) # --------------------------------------------------------------------------- # Credit parsing # --------------------------------------------------------------------------- class TestParseCreditsFromField: def test_newline_separated_credits_produce_multiple_entries(self): value = "Alice Smith\nBob Jones (Acme Corp)" assert parse_credits_from_field(value) == ["Alice Smith", "Bob Jones (Acme Corp)"] def test_comma_inside_a_credit_stays_in_the_same_credit(self): value = "Jane Doe, Acme Security" # Comma-splitting is deliberately off: "Name, Affiliation" is ONE credit. assert parse_credits_from_field(value) == ["Jane Doe, Acme Security"] def test_bullets_are_stripped(self): value = "- Alice\n* Bob\n+ Charlie\n1. Diana" assert parse_credits_from_field(value) == ["Alice", "Bob", "Charlie", "Diana"] def test_duplicates_are_removed_preserving_order(self): value = "Alice\nBob\nAlice" assert parse_credits_from_field(value) == ["Alice", "Bob"] def test_empty_field_returns_empty_list(self): assert parse_credits_from_field("") == [] # --------------------------------------------------------------------------- # Bot / AI credit detection (drives `type: "tool"` in build_credits) # --------------------------------------------------------------------------- class TestIsBotCredit: @pytest.mark.parametrize( "name", [ "dependabot[bot]", "github-actions[bot]", "renovate[bot]", "copilot[bot]", "ghsa-probot[bot]", ], ) def test_github_bot_suffix_matches(self, name: str): assert is_bot_credit(name) is True @pytest.mark.parametrize( "name", [ "Dependabot", "DEPENDABOT", "Snyk", "Renovate", "github-actions", "Claude", "ChatGPT", "Mend", "Whitesource", ], ) def test_known_name_list_matches_case_insensitively(self, name: str): assert is_bot_credit(name) is True @pytest.mark.parametrize( "name", [ "discovered by Automated Scanner v3", "reported via security-scanner", "found by Renovate during dependency sweep", ], ) def test_known_name_matches_inside_free_form_string(self, name: str): assert is_bot_credit(name) is True @pytest.mark.parametrize( "name", [ "claude-bot", "release-bot", "securitybot", "scan-ai", "bugbunny.ai", # domain-style AI-service handle "xbow.ai", "triage-agent", "secaudit-gpt", "securityscanner-7", "automated-triage", "automaton", ], ) def test_pattern_handles_match(self, name: str): assert is_bot_credit(name) is True @pytest.mark.parametrize( "name", [ "Alice Smith", "Bob Jones (Acme Corp)", "Jane Doe, Acme Security", "Joe Bot", # space breaks word boundary on *bot/-bot patterns "Botev Martinov", # word boundary stops the bot suffix at "Botev" "Renovation Engineering Inc", # 'renovate' substring, not whole word "AI Research Lab", # uppercase free-standing; no *-ai handle shape ], ) def test_human_names_do_not_match(self, name: str): assert is_bot_credit(name) is False def test_empty_string_does_not_match(self): assert is_bot_credit("") is False def test_whitespace_only_does_not_match(self): assert is_bot_credit(" ") is False class TestBuildCreditsBotTypeAssignment: def test_plain_human_credit_gets_finder_type(self): credits = build_credits("Alice Smith", remediation_developers=[]) assert credits == [{"lang": "en", "type": "finder", "value": "Alice Smith"}] def test_github_bot_suffix_credit_gets_tool_type(self): credits = build_credits("dependabot[bot]", remediation_developers=[]) assert credits == [{"lang": "en", "type": "tool", "value": "dependabot[bot]"}] def test_known_bot_name_credit_gets_tool_type(self): credits = build_credits("Dependabot", remediation_developers=[]) assert credits == [{"lang": "en", "type": "tool", "value": "Dependabot"}] def test_pattern_handle_credit_gets_tool_type(self): credits = build_credits("claude-bot", remediation_developers=[]) assert credits == [{"lang": "en", "type": "tool", "value": "claude-bot"}] def test_mixed_credits_get_per_row_types(self): credits = build_credits( "Alice Smith\nDependabot\nBob Jones (Acme Corp)", remediation_developers=[], ) assert credits == [ {"lang": "en", "type": "finder", "value": "Alice Smith"}, {"lang": "en", "type": "tool", "value": "Dependabot"}, {"lang": "en", "type": "finder", "value": "Bob Jones (Acme Corp)"}, ] def test_remediation_developer_type_unaffected_by_bot_policy(self): """Remediation-developer side is intentionally NOT routed through tool.""" credits = build_credits( "Alice Smith", remediation_developers=["dependabot[bot]"], ) # Bot lands as remediation developer with its original type — the # finder-side tool routing does not extend here. Skipping bots on # the remediation side is a separate (upstream-skill) concern. assert credits == [ {"lang": "en", "type": "finder", "value": "Alice Smith"}, { "lang": "en", "type": "remediation developer", "value": "dependabot[bot]", }, ] # --------------------------------------------------------------------------- # URL list parsing # --------------------------------------------------------------------------- class TestParseUrlList: def test_multiple_urls_extracted(self): value = ( "First:\nhttps://github.com/apache/airflow/pull/64114\n" "Second: https://github.com/apache/airflow/pull/65346" ) urls = parse_url_list(value) assert urls == [ "https://github.com/apache/airflow/pull/64114", "https://github.com/apache/airflow/pull/65346", ] def test_bare_text_with_no_urls_returns_empty(self): assert parse_url_list("No links here.") == [] # --------------------------------------------------------------------------- # CVE ID / CWE parsing # --------------------------------------------------------------------------- class TestParseCveId: def test_extracts_cve_from_asf_tool_link(self): assert parse_cve_id("https://cveprocess.apache.org/cve5/CVE-2026-40948") == "CVE-2026-40948" def test_extracts_cve_from_plain_id(self): assert parse_cve_id("CVE-2026-40948") == "CVE-2026-40948" def test_missing_returns_empty(self): assert parse_cve_id("") == "" class TestParseCwe: def test_cwe_with_title(self): cwe_id, description = parse_cwe("CWE-352: Cross-Site Request Forgery (CSRF)") assert cwe_id == "CWE-352" assert description == "CWE-352: Cross-Site Request Forgery (CSRF)" def test_cwe_without_title_emits_bare_id_as_description(self): cwe_id, description = parse_cwe("CWE-614") assert cwe_id == "CWE-614" assert description == "CWE-614" def test_no_cwe_keeps_original_text_as_description(self): cwe_id, description = parse_cwe("_No response_") assert cwe_id == "" assert description == "_No response_" def test_cwe_with_parens_around_title_strips_outer_wrapper(self): # `CWE-285 (Improper Authorization)` is what most projects' # CWE pickers serialise to. Without the strip, the result is # `CWE-285: (Improper Authorization)` — colon AND parens — # which reviewers flag as cluttered. This regression closes # the Arnout-reported issue on CVE-2026-46763 (2026-05-28). cwe_id, description = parse_cwe("CWE-285 (Improper Authorization)") assert cwe_id == "CWE-285" assert description == "CWE-285: Improper Authorization" def test_cwe_with_brackets_around_title_also_stripped(self): cwe_id, description = parse_cwe("CWE-352 [Cross-Site Request Forgery]") assert cwe_id == "CWE-352" assert description == "CWE-352: Cross-Site Request Forgery" def test_cwe_with_inner_parens_not_stripped(self): # The wrapper-strip only fires when the outer characters are # the wrapper. Inner parens stay; they are content. cwe_id, description = parse_cwe("CWE-285 Title (annotation)") assert cwe_id == "CWE-285" assert description == "CWE-285: Title (annotation)" def test_cwe_with_mismatched_wrappers_not_stripped(self): # A leading `(` without a trailing `)` is not a wrapper. cwe_id, description = parse_cwe("CWE-285 (Foo) Bar (Baz)") assert cwe_id == "CWE-285" assert description == "CWE-285: (Foo) Bar (Baz)" # --------------------------------------------------------------------------- # Affected-versions parsing # --------------------------------------------------------------------------- class TestParseAffectedVersions: def test_bare_less_than_creates_range(self): versions = parse_affected_versions("<3.2.0", None) assert versions == [ {"lessThan": "3.2.0", "status": "affected", "version": "0", "versionType": "semver"}, ] def test_inclusive_range(self): versions = parse_affected_versions(">=3.0.0, <3.2.0", None) assert versions == [ {"lessThan": "3.2.0", "status": "affected", "version": "3.0.0", "versionType": "semver"}, ] def test_less_or_equal_upper_bound(self): versions = parse_affected_versions("<=6.5.0", None) assert versions == [ { "lessThanOrEqual": "6.5.0", "status": "affected", "version": "0", "versionType": "semver", }, ] def test_version_start_override_sets_low_bound(self): versions = parse_affected_versions(">=3.0.0, <3.2.0", "2.10.5") assert versions[0]["version"] == "2.10.5" def test_unknown_string_raises_value_error(self): # The parser now refuses to emit garbage: an unparsable # `Affected versions` field is a hard error rather than a # bare-string fallback. This regression closes the issue on # CVE-2026-46763 (2026-05-28) where the field `>= 3.0.0 (...)` # serialised the parenthetical clause into the JSON `version` # field. with pytest.raises(ValueError, match="Could not parse"): parse_affected_versions("all versions", None) def test_unparsable_input_error_mentions_the_value(self): # The error message includes the offending value so the # reviewer can find it in the issue body. with pytest.raises(ValueError, match="all versions"): parse_affected_versions("all versions", None) def test_lower_bound_only_emits_open_ended_entry(self): # `>=2.0.0` (no upper) — useful for trackers whose fix-shipped # version isn't known yet (typically providers). Still emits # (does not raise) but issues a warning to stderr — see the # warning test below. versions = parse_affected_versions(">=2.0.0", None) assert versions == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] def test_lower_bound_only_with_space(self): versions = parse_affected_versions(">= 2.0.0", None) assert versions == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] def test_lower_bound_only_warns_about_missing_upper_bound(self, capsys): # A bare lower bound has the misleading shape where CVE 5.x # readers interpret it as "this version alone is affected". # Warn so a reviewer notices and fills in `< X.Y.Z` or the # `< NEXT VERSION` sentinel. parse_affected_versions(">= 2.0.0", None) captured = capsys.readouterr() assert "bare lower bound" in captured.err assert "'2.0.0'" in captured.err or '"2.0.0"' in captured.err # No stdout chatter; warnings go to stderr only. assert captured.out == "" def test_lower_bound_with_next_version_sentinel_does_not_warn(self, capsys): # `< NEXT VERSION` is the documented "fix not yet released, # upper bound unknown" form — emitting the bare-version entry # is intentional, no warning. parse_affected_versions(">= 2.0.0, < NEXT VERSION", None) captured = capsys.readouterr() assert captured.err == "" assert captured.out == "" class TestParseAffectedVersionsNextVersionPlaceholder: """The `< NEXT VERSION` sentinel: fix not yet released, upper bound unknown. Stripped before further parsing; resulting entry has no `lessThan`. Used predominantly for projects trackers where the wave milestone is date-based and the package version that ships the fix is decided by the release manager during the wave. """ def test_just_next_version_placeholder(self): versions = parse_affected_versions("< NEXT VERSION", None) assert versions == [ {"status": "affected", "version": "0", "versionType": "semver"}, ] def test_lowercase_token_also_accepted(self): versions = parse_affected_versions("< next version", None) assert versions == [ {"status": "affected", "version": "0", "versionType": "semver"}, ] def test_lower_bound_with_next_version_upper(self): versions = parse_affected_versions(">= 2.0.0 < NEXT VERSION", None) assert versions == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] def test_lower_bound_with_comma_and_next_version_upper(self): versions = parse_affected_versions(">= 2.0.0, < NEXT VERSION", None) assert versions == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] def test_real_version_replacing_placeholder_round_trips(self): # Once the release manager knows the fix-shipped version, the # sync skill replaces `< NEXT VERSION` with `< X.Y.Z`. The # parser must produce the standard fully-bounded shape. versions = parse_affected_versions(">= 2.0.0, < 5.6.0", None) assert versions == [ {"status": "affected", "version": "2.0.0", "lessThan": "5.6.0", "versionType": "semver"}, ] # --------------------------------------------------------------------------- # Product-name resolution for Airflow packages # --------------------------------------------------------------------------- class TestProductForPackage: def test_core_package_resolves_to_apache_airflow(self): assert _product_for_package("apache-example") == "Apache Example" def test_known_provider_uses_display_map_casing(self): assert _product_for_package("apache-example-project-foo") == "Apache Example Project Foo" assert ( _product_for_package("apache-example-project-kerfluffle") == "Apache Example Project Kerfluffle" ) assert _product_for_package("apache-example-project-acme-xyz") == "Apache Example Project Acme XYZ" # Confirms the user-cited mapping example from the Vulnogram form: # apache-example-project-bar → "Apache Example Project Bar". assert _product_for_package("apache-example-project-bar") == "Apache Example Project Bar" def test_unknown_provider_falls_back_to_title_case(self): assert ( _product_for_package("apache-example-project-madeup-widget") == "Apache Example Project Madeup Widget" ) def test_overrides_win_over_display_map(self): assert ( _product_for_package( "apache-example-project-foo", product_overrides={ "apache-example-project-foo": "Custom ES Display", }, ) == "Custom ES Display" ) def test_overrides_win_over_title_case_fallback(self): assert ( _product_for_package( "apache-example-project-madeup-widget", product_overrides={ "apache-example-project-madeup-widget": "Apache Example Project WIDGET", }, ) == "Apache Example Project WIDGET" ) # --------------------------------------------------------------------------- # Product-name resolution for projects whose subpackages use a # convention other than `-project-` (regression coverage for the bug # where the lookup was anchored on a hardcoded `-project-` prefix # instead of reading the `project` group from the configured # `package_pattern`). # --------------------------------------------------------------------------- class TestProductForPackageProvidersStyle: """Subpackage prefix is `-providers-`, not `-project-`. Mirrors the shape Apache Airflow ships on PyPI (`apache-airflow-providers-<dir>`). The lookup must read the `project` named group from whatever the configured `package_pattern` declares — the prefix is a project-level convention, not a generator constant. """ @pytest.fixture(autouse=True) def _providers_config(self) -> Iterator[None]: from pathlib import Path from generate_cve_json.cve_json import _set_config_path fixture = Path(__file__).resolve().parent / "fixtures" / "cve-json-config-providers.toml" default = Path(__file__).resolve().parent / "fixtures" / "cve-json-config.toml" _set_config_path(fixture) try: yield finally: _set_config_path(default) def test_known_provider_uses_display_map_casing(self): # `cncf-kubernetes` is captured by the `(?P<project>...)` # group of the configured pattern and looked up in the # display map — the resolver must not fall through to # returning the raw package name. assert ( _product_for_package("apache-example-providers-cncf-kubernetes") == "Apache Example CNCF Kubernetes provider" ) def test_unknown_provider_falls_back_to_title_case(self): assert ( _product_for_package("apache-example-providers-madeup-widget") == "Apache Example Madeup Widget provider" ) def test_top_level_still_resolves(self): assert _product_for_package("apache-example") == "Apache Example" def test_overrides_still_win(self): assert ( _product_for_package( "apache-example-providers-cncf-kubernetes", product_overrides={ "apache-example-providers-cncf-kubernetes": "CUSTOM", }, ) == "CUSTOM" ) # --------------------------------------------------------------------------- # Multi-product `build_affected` # --------------------------------------------------------------------------- class TestBuildAffectedSingleProduct: def test_empty_field_emits_one_placeholder_entry(self): entries = build_affected("", **DEFAULT_AFFECTED_ARGS) assert len(entries) == 1 assert entries[0]["packageURL"] == "pkg:pypi/apache-example" assert entries[0]["product"] == "Apache Example" def test_bare_version_range_uses_defaults(self): entries = build_affected("<3.2.2", **DEFAULT_AFFECTED_ARGS) assert len(entries) == 1 assert entries[0]["packageURL"] == "pkg:pypi/apache-example" assert entries[0]["product"] == "Apache Example" assert entries[0]["versions"][0]["lessThan"] == "3.2.2" def test_single_line_with_package_prefix_detects_provider(self): entries = build_affected( "apache-example-project-foo <=6.5.0", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 1 assert entries[0]["packageURL"] == "pkg:pypi/apache-example-project-foo" assert entries[0]["product"] == "Apache Example Project Foo" assert entries[0]["versions"][0]["lessThanOrEqual"] == "6.5.0" def test_single_line_with_core_package_prefix_detects_core(self): entries = build_affected( "apache-example <3.2.2", vendor="Apache Software Foundation", product="IGNORED", package_name="ignored", collection_url="https://pypi.python.org", version_start=None, ) assert len(entries) == 1 # Package prefix wins over the default argument because the body # has said the package name explicitly. assert entries[0]["packageURL"] == "pkg:pypi/apache-example" assert entries[0]["product"] == "Apache Example" class TestBuildAffectedMultiProduct: def test_two_providers_produce_two_entries(self): entries = build_affected( "apache-example-project-foo <=6.5.0\napache-example-project-kerfluffle <=1.9.0", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 2 es, os_ = entries assert es["packageURL"] == "pkg:pypi/apache-example-project-foo" assert es["product"] == "Apache Example Project Foo" assert es["versions"][0]["lessThanOrEqual"] == "6.5.0" assert os_["packageURL"] == "pkg:pypi/apache-example-project-kerfluffle" assert os_["product"] == "Apache Example Project Kerfluffle" assert os_["versions"][0]["lessThanOrEqual"] == "1.9.0" def test_bullet_prefixes_are_stripped(self): entries = build_affected( "- apache-example-project-foo <=6.5.0\n* apache-example-project-kerfluffle <=1.9.0", **DEFAULT_AFFECTED_ARGS, ) assert [e["packageURL"] for e in entries] == [ "pkg:pypi/apache-example-project-foo", "pkg:pypi/apache-example-project-kerfluffle", ] def test_blank_lines_between_entries_are_ignored(self): entries = build_affected( "apache-example-project-foo <=6.5.0\n\napache-example-project-kerfluffle <=1.9.0\n", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 2 def test_mixed_known_and_unknown_provider(self): entries = build_affected( "apache-example-project-foo <=6.5.0\napache-example-project-brand-new <=0.1.0", **DEFAULT_AFFECTED_ARGS, ) assert entries[0]["product"] == "Apache Example Project Foo" assert entries[1]["product"] == "Apache Example Project Brand New" def test_product_overrides_applied_per_entry(self): entries = build_affected( "apache-example-project-brand-new <=0.1.0", product_overrides={ "apache-example-project-brand-new": "Apache Example Project BRAND", }, **DEFAULT_AFFECTED_ARGS, ) assert entries[0]["product"] == "Apache Example Project BRAND" def test_line_without_prefix_falls_back_to_defaults(self): # A single line that does not carry a recognisable package # prefix stays in the legacy single-entry path and takes the # explicit product / packageName args. (The version-range # shape itself is incidental to what this test exercises; # use a parseable shape since the parser no longer accepts # arbitrary free-form text.) entries = build_affected( "<= 1.0.0", vendor="Apache Software Foundation", product="Apache Example Helm Chart", package_name="apache-example-project-helm-chart", collection_url="https://airflow.apache.org/", version_start=None, # A Helm chart served from the project's own site is not on a # package host the purl spec models, so it opts out explicitly. purl_type="none", ) assert len(entries) == 1 assert entries[0]["packageName"] == "apache-example-project-helm-chart" assert entries[0]["collectionURL"] == "https://airflow.apache.org/" assert entries[0]["product"] == "Apache Example Helm Chart" assert "packageURL" not in entries[0] def test_multi_line_is_deterministic(self): # Re-generating must produce byte-identical output, which is # what keeps the `--attach` idempotence guarantee intact. first = build_affected( "apache-example-project-foo <=6.5.0\napache-example-project-kerfluffle <=1.9.0", **DEFAULT_AFFECTED_ARGS, ) second = build_affected( "apache-example-project-foo <=6.5.0\napache-example-project-kerfluffle <=1.9.0", **DEFAULT_AFFECTED_ARGS, ) assert first == second def test_per_line_next_version_placeholder(self): # The providers-tracker pattern: one line per affected package, # all with `< NEXT VERSION` until the wave ships. entries = build_affected( "apache-example-project-foo < NEXT VERSION\napache-example-project-kerfluffle < NEXT VERSION", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 2 es, os_ = entries assert es["packageURL"] == "pkg:pypi/apache-example-project-foo" assert es["versions"] == [ {"status": "affected", "version": "0", "versionType": "semver"}, ] assert os_["packageURL"] == "pkg:pypi/apache-example-project-kerfluffle" assert os_["versions"] == [ {"status": "affected", "version": "0", "versionType": "semver"}, ] def test_per_line_lower_bound_only(self): # Lower-bound-only line (e.g. `apache-example-project-xyz >=2.0.0`) # — useful when affected versions are known to start at X but the # fix-shipped version isn't known yet. entries = build_affected( "apache-example-project-xyz >=2.0.0", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 1 assert entries[0]["packageURL"] == "pkg:pypi/apache-example-project-xyz" assert entries[0]["versions"] == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] def test_per_line_lower_bound_with_next_version_upper(self): entries = build_affected( "apache-example-project-xyz >= 2.0.0, < NEXT VERSION", **DEFAULT_AFFECTED_ARGS, ) assert len(entries) == 1 assert entries[0]["versions"] == [ {"status": "affected", "version": "2.0.0", "versionType": "semver"}, ] # --------------------------------------------------------------------------- # Reference tagging # --------------------------------------------------------------------------- class TestClassifyReference: def test_github_pr_tagged_as_patch(self): assert classify_reference("https://github.com/apache/airflow/pull/64114") == ["patch"] def test_github_commit_tagged_as_patch(self): assert classify_reference("https://github.com/apache/airflow/commit/abc123") == ["patch"] def test_lists_apache_tagged_as_vendor_advisory(self): assert classify_reference("https://lists.apache.org/thread/abc") == ["vendor-advisory"] def test_plain_doc_url_has_no_tags(self): assert classify_reference("https://airflow.apache.org/docs/") == [] def test_security_apache_tagged_as_vendor_advisory(self): assert classify_reference("https://security.apache.org/foo") == ["vendor-advisory"] def test_evil_substring_in_path_is_not_tagged(self): # Regression: substring match would have flagged this; CodeQL # `py/incomplete-url-substring-sanitization`. Hostname match # rejects it correctly. assert classify_reference("https://evil.example/?q=lists.apache.org") == [] assert classify_reference("https://evil.example/security.apache.org") == [] def test_subdomain_is_not_treated_as_apache(self): assert classify_reference("https://lists.apache.org.evil.example/x") == [] def test_malformed_url_returns_no_tags(self): assert classify_reference("not a url") == [] def test_cve_org_record_tagged_as_related(self): # ASF Security's preferred URL form for sibling/incomplete-fix CVE # cross-references (Arnout Engelen, 2026-05-29 review on # CVE-2026-49298). assert classify_reference("https://www.cve.org/CVERecord?id=CVE-2026-27173") == ["related"] assert classify_reference("https://cve.org/CVERecord?id=CVE-2025-68438") == ["related"] def test_nvd_record_tagged_as_related(self): # NVD is the same CVE database under a different URL form; treat # as related too. assert classify_reference("https://nvd.nist.gov/vuln/detail/CVE-2026-27173") == ["related"] class TestExtractRelatedCveIds: def test_extracts_single_prior_cve(self): from generate_cve_json.cve_json import extract_related_cve_ids summary = ( "This is a variant of CWE-200 previously addressed in CVE-2025-68438; " "that fix did not cover the nested sensitive-keyword allowlist." ) assert extract_related_cve_ids(summary) == ["CVE-2025-68438"] def test_extracts_multiple_distinct_in_order(self): from generate_cve_json.cve_json import extract_related_cve_ids summary = "Fix-bypass of CVE-2026-33858. Also related to CVE-2025-50213 and CVE-2025-27018." assert extract_related_cve_ids(summary) == [ "CVE-2026-33858", "CVE-2025-50213", "CVE-2025-27018", ] def test_excludes_current_cve_id(self): from generate_cve_json.cve_json import extract_related_cve_ids summary = "CVE-2026-42359 fixes a PATCH-path bypass of CVE-2026-33858." assert extract_related_cve_ids(summary, current_cve_id="CVE-2026-42359") == [ "CVE-2026-33858", ] def test_current_cve_id_match_is_case_insensitive(self): from generate_cve_json.cve_json import extract_related_cve_ids summary = "cve-2026-42359 fixes a PATCH-path bypass of CVE-2026-33858." assert extract_related_cve_ids(summary, current_cve_id="CVE-2026-42359") == [ "CVE-2026-33858", ] def test_deduplicates_repeated_mentions(self): from generate_cve_json.cve_json import extract_related_cve_ids summary = "CVE-2025-68438 was incomplete; this CVE follows CVE-2025-68438." assert extract_related_cve_ids(summary) == ["CVE-2025-68438"] def test_substring_in_larger_token_does_not_match(self): from generate_cve_json.cve_json import extract_related_cve_ids # Word-boundary regex must not match identifiers embedded in # larger tokens (defensive against accidental hits). assert extract_related_cve_ids("seeCVE-2026-33858trailing") == [] assert extract_related_cve_ids("CVE-2026-33858x") == [] def test_short_form_required_at_least_four_digits(self): from generate_cve_json.cve_json import extract_related_cve_ids # CVE-YYYY-NNNN minimum (matches MITRE's 4-7 digit constraint). assert extract_related_cve_ids("CVE-2026-123") == [] assert extract_related_cve_ids("CVE-2026-1234") == ["CVE-2026-1234"] def test_empty_string_returns_empty_list(self): from generate_cve_json.cve_json import extract_related_cve_ids assert extract_related_cve_ids("") == [] def test_no_cve_id_in_text_returns_empty_list(self): from generate_cve_json.cve_json import extract_related_cve_ids assert extract_related_cve_ids("no CVE here, just narrative.") == [] class TestRelatedCveUrl: def test_url_format_matches_cve_org(self): from generate_cve_json.cve_json import related_cve_url assert related_cve_url("CVE-2026-27173") == "https://www.cve.org/CVERecord?id=CVE-2026-27173" class TestBuildReferences: def test_mailing_list_field_urls_are_not_auto_included(self): refs = build_references( mailing_list_field="https://lists.apache.org/thread/fake-security-thread", pr_field="https://github.com/apache/airflow/pull/64114", ) urls = [r["url"] for r in refs] assert urls == ["https://github.com/apache/airflow/pull/64114"] def test_explicit_advisory_url_is_included(self): refs = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/64114", extra_urls=["https://lists.apache.org/thread/real-users-archive-url"], ) urls = {r["url"] for r in refs} assert "https://lists.apache.org/thread/real-users-archive-url" in urls # And it gets tagged correctly. by_url = {r["url"]: r for r in refs} advisory = by_url["https://lists.apache.org/thread/real-users-archive-url"] assert advisory.get("tags") == ["vendor-advisory"] def test_airflow_s_and_cveprocess_urls_are_filtered_out(self): refs = build_references( mailing_list_field="", pr_field="", extra_urls=[ "https://github.com/apache-example-s/apache-example-s/issues/256", "https://cveprocess.apache.org/cve5/CVE-2026-40948", "https://github.com/apache/airflow/pull/64114", ], ) urls = [r["url"] for r in refs] assert urls == ["https://github.com/apache/airflow/pull/64114"] # --------------------------------------------------------------------------- # Title handling # --------------------------------------------------------------------------- class TestResolveTitle: def test_strips_apache_airflow_prefix_from_issue_title(self): assert resolve_title("Apache Example: DAG auth bypass", "", None) == "DAG auth bypass" def test_override_wins(self): assert resolve_title("from-issue", "summary", "my override") == "my override" def test_falls_back_to_summary_when_empty(self): assert resolve_title("", "A summary. With more text.", None) == "A summary" # --------------------------------------------------------------------------- # Readiness helper + envelope state # --------------------------------------------------------------------------- def _ready_cna() -> dict: """Build a minimal CNA container that satisfies every readiness rule.""" return { "title": "Example vulnerability", "descriptions": [{"lang": "en", "value": "A description."}], "affected": build_affected( ">=3.0.0, <3.2.0", vendor="Apache Software Foundation", product="Apache Example", package_name="apache-example", collection_url="https://pypi.python.org", version_start=None, ), "problemTypes": build_problem_types("CWE-352: CSRF"), "metrics": build_metrics("Low"), "credits": build_credits("Alice Smith", remediation_developers=["Bob"]), "references": build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", ), } class TestIsCnaReadyForReview: def test_fully_populated_cna_is_ready(self): assert _is_cna_ready_for_review(_ready_cna(), "CVE-2026-00001") is True def test_missing_cve_id_blocks_review(self): assert _is_cna_ready_for_review(_ready_cna(), "") is False def test_missing_title_blocks_review(self): cna = _ready_cna() cna["title"] = "" assert _is_cna_ready_for_review(cna, "CVE-2026-00001") is False def test_missing_credit_blocks_review(self): cna = _ready_cna() cna["credits"] = [] assert _is_cna_ready_for_review(cna, "CVE-2026-00001") is False def test_unknown_severity_blocks_review(self): cna = _ready_cna() cna["metrics"] = build_metrics("Unknown") assert _is_cna_ready_for_review(cna, "CVE-2026-00001") is False def test_missing_cwe_blocks_review(self): cna = _ready_cna() cna["problemTypes"] = [] assert _is_cna_ready_for_review(cna, "CVE-2026-00001") is False def test_missing_reference_blocks_review(self): cna = _ready_cna() cna["references"] = [] assert _is_cna_ready_for_review(cna, "CVE-2026-00001") is False class TestWrapCveRecord: def test_cve_metadata_state_is_always_published(self): # `cveMetadata.state` is the CVE 5.x schema field — only valid # values are PUBLISHED / REJECTED for a submitted record. # The generator hard-codes PUBLISHED regardless of workflow. record = wrap_cve_record(_ready_cna(), cve_id="CVE-2026-00001", org_id="org") assert record["cveMetadata"]["state"] == "PUBLISHED" assert record["cveMetadata"]["cveId"] == "CVE-2026-00001" def test_cve_metadata_state_published_even_when_draft(self): cna = _ready_cna() cna["credits"] = [] # make the CNA incomplete record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") # Workflow state went to DRAFT, but the CVE 5.x schema state # stays PUBLISHED — these are different fields. assert record["cveMetadata"]["state"] == "PUBLISHED" def test_ready_record_emits_review_workflow_state(self): # Legacy / non-gated default: a ready CNA advances to REVIEW # without any signal. This is what non-ASF adopters get when # they don't set [workflow].release_vote_gating in their config. cna = _ready_cna() record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "REVIEW" def test_ready_record_gated_without_vote_stays_draft(self): # Gated path (ASF adopters with release_vote_gating = true): when # the caller signals that no vote is happening, a ready CNA # stays at DRAFT. REVIEW is reserved for the actual vote window. cna = _ready_cna() record = wrap_cve_record( cna, cve_id="CVE-2026-00001", org_id="org", release_vote_in_progress=False, ) assert record["CNA_private"]["state"] == "DRAFT" def test_ready_record_gated_with_vote_emits_review(self): cna = _ready_cna() record = wrap_cve_record( cna, cve_id="CVE-2026-00001", org_id="org", release_vote_in_progress=True, ) assert record["CNA_private"]["state"] == "REVIEW" def test_incomplete_record_emits_draft_workflow_state(self): cna = _ready_cna() cna["credits"] = [] record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "DRAFT" def test_vendor_advisory_reference_emits_public_workflow_state(self): cna = _ready_cna() cna["references"] = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", extra_urls=[ "https://lists.apache.org/thread/abc123xyz789", ], ) record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "PUBLIC" def test_vendor_advisory_with_incomplete_fields_still_draft(self): # Even when the advisory URL is captured, an incomplete CNA # stays at DRAFT — PUBLIC requires the full review-ready set. cna = _ready_cna() cna["credits"] = [] cna["references"] = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", extra_urls=[ "https://lists.apache.org/thread/abc123xyz789", ], ) record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "DRAFT" def test_envelope_carries_cna_private_block(self): record = wrap_cve_record(_ready_cna(), cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["owner"] == "example" assert record["CNA_private"]["userslist"] == "users@example.apache.org" def test_emailed_is_none_in_draft_state(self): cna = _ready_cna() cna["credits"] = [] record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "DRAFT" assert record["CNA_private"]["emailed"] is None def test_emailed_is_none_in_review_state(self): record = wrap_cve_record(_ready_cna(), cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "REVIEW" assert record["CNA_private"]["emailed"] is None def test_emailed_is_yes_in_public_state(self): cna = _ready_cna() cna["references"] = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", extra_urls=[ "https://lists.apache.org/thread/abc123xyz789", ], ) record = wrap_cve_record(cna, cve_id="CVE-2026-00001", org_id="org") assert record["CNA_private"]["state"] == "PUBLIC" assert record["CNA_private"]["emailed"] == "yes" # --------------------------------------------------------------------------- # compute_cna_private_state # --------------------------------------------------------------------------- class TestComputeCnaPrivateState: def test_ready_without_advisory_is_review(self): assert compute_cna_private_state(_ready_cna(), "CVE-2026-00001") == "REVIEW" def test_ready_with_advisory_is_public(self): cna = _ready_cna() cna["references"] = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", extra_urls=["https://lists.apache.org/thread/abc123xyz789"], ) assert compute_cna_private_state(cna, "CVE-2026-00001") == "PUBLIC" def test_incomplete_is_draft(self): cna = _ready_cna() cna["credits"] = [] assert compute_cna_private_state(cna, "CVE-2026-00001") == "DRAFT" def test_gated_without_vote_is_draft(self): # Tri-state: explicit False ⇒ DRAFT even when ready. assert ( compute_cna_private_state( _ready_cna(), "CVE-2026-00001", release_vote_in_progress=False, ) == "DRAFT" ) def test_gated_with_vote_is_review(self): assert ( compute_cna_private_state( _ready_cna(), "CVE-2026-00001", release_vote_in_progress=True, ) == "REVIEW" ) def test_gated_with_advisory_overrides_vote_flag(self): # Even when release_vote_in_progress=False, a vendor-advisory # reference promotes the state to PUBLIC. The PUBLIC transition # is not gated by the vote signal — once the advisory shipped, # the record IS public. cna = _ready_cna() cna["references"] = build_references( mailing_list_field="", pr_field="https://github.com/apache/airflow/pull/123", extra_urls=["https://lists.apache.org/thread/abc123xyz789"], ) assert compute_cna_private_state(cna, "CVE-2026-00001", release_vote_in_progress=False) == "PUBLIC" # --------------------------------------------------------------------------- # compute_package_url # --------------------------------------------------------------------------- class TestComputePackageUrl: def test_pypi_python_org_returns_canonical_pypi_project_url(self): assert ( compute_package_url("https://pypi.python.org", "apache-example") == "https://pypi.org/project/apache-example/" ) def test_pypi_org_alias_also_supported(self): assert ( compute_package_url("https://pypi.org", "apache-example") == "https://pypi.org/project/apache-example/" ) def test_trailing_slash_is_tolerated(self): assert ( compute_package_url("https://pypi.python.org/", "apache-example-project-foo") == "https://pypi.org/project/apache-example-project-foo/" ) def test_unknown_collection_url_returns_none(self): assert compute_package_url("https://airflow.apache.org/", "apache-example-project-helm-chart") is None def test_empty_inputs_return_none(self): assert compute_package_url("", "apache-example") is None assert compute_package_url("https://pypi.python.org", "") is None # --------------------------------------------------------------------------- # format_version_range # --------------------------------------------------------------------------- class TestFormatVersionRange: def test_range_with_low_and_high(self): versions = [{"version": "2.0.0", "lessThan": "3.2.2", "status": "affected", "versionType": "semver"}] assert format_version_range(versions) == ">= 2.0.0, < 3.2.2" def test_open_lower_bound_uses_less_than_only(self): versions = [{"version": "0", "lessThan": "3.2.2", "status": "affected", "versionType": "semver"}] assert format_version_range(versions) == "< 3.2.2" def test_less_than_or_equal(self): versions = [ {"version": "0", "lessThanOrEqual": "3.2.1", "status": "affected", "versionType": "semver"} ] assert format_version_range(versions) == "<= 3.2.1" def test_bare_single_version(self): versions = [{"version": "3.1.5", "status": "affected", "versionType": "semver"}] assert format_version_range(versions) == "3.1.5" def test_round_trip_through_parser(self): # parse_affected_versions(...) → format_version_range(...) should # reconstruct the original human-readable shape. for raw, expected in [ (">= 2.0.0, < 3.2.2", ">= 2.0.0, < 3.2.2"), ("< 3.2.2", "< 3.2.2"), ("<= 3.2.1", "<= 3.2.1"), ("3.1.5", "3.1.5"), ]: parsed = parse_affected_versions(raw, version_start_override=None) assert format_version_range(parsed) == expected, raw def test_empty_input_returns_empty_string(self): assert format_version_range([]) == "" # --------------------------------------------------------------------------- # _build_attachment_body — the issue-body table above the embedded JSON # --------------------------------------------------------------------------- def _ready_cna_for_attachment() -> dict: """Same shape as _ready_cna(): affected[] entries as build_affected() actually emits them, i.e. carrying a packageURL rather than the legacy collectionURL / packageName pair.""" return _ready_cna() class TestBuildAttachmentBody: def test_metric_table_includes_title_state_and_size(self): cna = _ready_cna_for_attachment() body = _build_attachment_body( cve_id="CVE-2026-00001", json_text='{"x": 1}', cna=cna, cna_private_state="REVIEW", ) assert "| CVE ID | `CVE-2026-00001` |" in body assert "| Title | Example vulnerability |" in body assert "| Vulnogram state | `REVIEW` |" in body assert "| Affected packages | `apache-example` |" in body assert "| Size | 8 bytes |" in body def test_no_envelope_renders_state_placeholder(self): body = _build_attachment_body( cve_id="CVE-2026-00001", json_text='{"x": 1}', cna=_ready_cna_for_attachment(), cna_private_state=None, ) assert "| Vulnogram state | — (`--no-envelope`) |" in body def test_per_package_table_includes_project_url_and_versions(self): body = _build_attachment_body( cve_id="CVE-2026-00001", json_text="{}", cna=_ready_cna_for_attachment(), cna_private_state="REVIEW", ) assert "**Packages this JSON covers:**" in body assert "| # | Package | Product | Versions | Project URL |" in body assert ( "| 1 | `apache-example` | Apache Example | `>= 3.0.0, < 3.2.0` | <https://pypi.org/project/apache-example/> |" in body ) def test_per_package_table_renders_one_row_per_affected_entry(self): cna = _ready_cna_for_attachment() cna["affected"] = build_affected( "apache-example-project-foo <=6.5.0\napache-example-project-kerfluffle <=1.9.0", vendor="Apache Software Foundation", product="Apache Example", package_name="apache-example", collection_url="https://pypi.python.org", version_start=None, ) body = _build_attachment_body( cve_id="CVE-2026-00001", json_text="{}", cna=cna, cna_private_state="REVIEW", ) assert ( "| 1 | `apache-example-project-foo` | Apache Example Project Foo " "| `<= 6.5.0` | <https://pypi.org/project/apache-example-project-foo/> |" ) in body assert ( "| 2 | `apache-example-project-kerfluffle` | Apache Example Project Kerfluffle " "| `<= 1.9.0` | <https://pypi.org/project/apache-example-project-kerfluffle/> |" ) in body assert ( "| Affected packages | `apache-example-project-foo`, `apache-example-project-kerfluffle` |" ) in body def test_unknown_collection_url_renders_dash_in_url_column(self): cna = _ready_cna_for_attachment() cna["affected"] = [ { "vendor": "Apache Software Foundation", "product": "Apache Example Helm Chart", "packageName": "apache-example-project-helm-chart", "collectionURL": "https://airflow.apache.org/", "versions": [ {"version": "0", "lessThan": "1.18.0", "status": "affected", "versionType": "semver"}, ], "defaultStatus": "unaffected", } ] body = _build_attachment_body( cve_id="CVE-2026-00001", json_text="{}", cna=cna, cna_private_state="REVIEW", ) assert ( "| 1 | `apache-example-project-helm-chart` | Apache Example Helm Chart | `< 1.18.0` | — |" in body ) def test_pipe_in_title_is_escaped(self): cna = _ready_cna_for_attachment() cna["title"] = "Pipe | in | title" body = _build_attachment_body( cve_id="CVE-2026-00001", json_text="{}", cna=cna, cna_private_state="REVIEW", ) assert "| Title | Pipe \\| in \\| title |" in body def test_output_is_deterministic(self): first = _build_attachment_body( cve_id="CVE-2026-00001", json_text='{"x": 1}', cna=_ready_cna_for_attachment(), cna_private_state="REVIEW", ) second = _build_attachment_body( cve_id="CVE-2026-00001", json_text='{"x": 1}', cna=_ready_cna_for_attachment(), cna_private_state="REVIEW", ) assert first == second # --------------------------------------------------------------------------- # combine_remediation_developers # --------------------------------------------------------------------------- class TestCombineRemediationDevelopers: def test_body_field_only(self): assert combine_remediation_developers("Alice Smith\nBob Jones", []) == [ "Alice Smith", "Bob Jones", ] def test_cli_only(self): assert combine_remediation_developers("", ["Alice Smith"]) == ["Alice Smith"] def test_body_first_then_cli(self): assert combine_remediation_developers("Alice", ["Bob"]) == ["Alice", "Bob"] def test_duplicates_dropped_silently(self): # Body says Alice, CLI also says Alice — single credit, body order wins. assert combine_remediation_developers("Alice Smith\nBob Jones", ["Alice Smith", "Carol"]) == [ "Alice Smith", "Bob Jones", "Carol", ] def test_no_response_field_treated_as_empty(self): # extract_field already collapses _No response_ to "" upstream; this # asserts the helper doesn't choke if a caller passes "" directly. assert combine_remediation_developers("", []) == [] def test_full_name_affiliation_pattern_preserved(self): # Same parsing rule as Reporter credited as: "Name, Affiliation" is one credit. assert combine_remediation_developers("Jed Cunningham, Astronomer", []) == [ "Jed Cunningham, Astronomer", ] # --------------------------------------------------------------------------- # normalise_severity # --------------------------------------------------------------------------- class TestNormaliseSeverity: def test_known_values_are_lowercased(self): for raw in ("None", "Low", "Moderate", "Medium", "High", "Important", "Critical"): assert normalise_severity(raw) == raw.lower() def test_already_lowercase_known_value_passes_through(self): assert normalise_severity("high") == "high" def test_unknown_value_is_stripped_but_not_changed(self): assert normalise_severity(" Informational ") == "Informational" def test_mixed_case_known_value_normalised(self): assert normalise_severity("HIGH") == "high" assert normalise_severity("CRITICAL") == "critical" assert normalise_severity("MODERATE") == "moderate" assert normalise_severity("Important") == "important" # --------------------------------------------------------------------------- # to_html # --------------------------------------------------------------------------- class TestToHtml: def test_plain_text_is_returned_unchanged(self): assert to_html("Hello world") == "Hello world" def test_html_angle_brackets_are_escaped(self): assert to_html("<script>alert(1)</script>") == "<script>alert(1)</script>" def test_ampersand_is_escaped(self): assert to_html("A & B") == "A & B" def test_double_newlines_become_br_br(self): assert to_html("Para one.\n\nPara two.") == "Para one.<br><br>Para two." def test_single_newlines_become_br(self): assert to_html("Line one.\nLine two.") == "Line one.<br>Line two." def test_windows_line_endings_normalised_before_conversion(self): assert to_html("Line one.\r\nLine two.") == "Line one.<br>Line two." def test_mixed_newlines_in_multiline_text(self): result = to_html("Intro.\n\nBullet one.\nBullet two.") assert result == "Intro.<br><br>Bullet one.<br>Bullet two." # ------------------------------------------------------- -
__init__.py 0 B
-
-
.gitignore 81 B · in bundle
-
pyproject.toml 3 KB
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. [build-system] requires = ["hatchling"] build-backend = "hatchling.build" [project] name = "generate-cve-json" version = "0.1.0" description = "Generate a CVE 5.x JSON record from a tracking issue in the active project's security tracker (Vulnogram adapter)." readme = "README.md" requires-python = ">=3.11" license = { text = "Apache-2.0" } # Runtime deps are deliberately empty — the script is stdlib-only and shells # out to `gh` for GitHub access. Keeping the runtime closed means `uv run` # can resolve the environment in milliseconds. dependencies = [] [project.scripts] generate-cve-json = "generate_cve_json:main" [tool.hatch.build.targets.wheel] packages = ["src/generate_cve_json"] [tool.ruff] line-length = 110 target-version = "py311" src = ["src", "tests"] [tool.ruff.lint] select = [ "E", # pycodestyle errors "W", # pycodestyle warnings "F", # pyflakes "I", # isort "B", # flake8-bugbear "UP", # pyupgrade "SIM", # flake8-simplify "C4", # flake8-comprehensions "RUF", # ruff-specific ] ignore = [ "E501", # line-too-long — the 110-char limit above is already generous ] [tool.ruff.lint.per-file-ignores] "tests/**" = ["B", "SIM"] # test clarity beats these [tool.mypy] python_version = "3.11" files = ["src", "tests"] # The script manipulates untyped JSON-dict shapes extensively; bare # `dict` / `list` annotations are pragmatic. Keep the useful checks # (unreachable code, implicit Optional, return types) and turn off the # ones that would require a heavy TypedDict refactor. warn_unused_ignores = true warn_redundant_casts = true warn_unreachable = true check_untyped_defs = true no_implicit_optional = true disallow_untyped_defs = true disallow_incomplete_defs = true [[tool.mypy.overrides]] module = "tests.*" # Tests aren't expected to type-annotate everything and often mock freely. disallow_untyped_defs = false disallow_incomplete_defs = false [tool.pytest.ini_options] minversion = "8.0" addopts = "-ra -q" testpaths = ["tests"] [dependency-groups] # The shared toolchain (mypy, pytest, ruff) comes from `magpie-dev` # (tools/dev), declared once for the whole workspace. The checks run each tool # via `uv run --directory <member> --project . python -m <tool>` — see # tools/dev/run-workspace-check.sh. dev = ["magpie-dev"] -
README.md 3.3 KB
<!-- SPDX-License-Identifier: Apache-2.0 https://www.apache.org/licenses/LICENSE-2.0 --> <!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> **Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - [generate-cve-json](#generate-cve-json) - [Run](#run) - [Test](#test) - [Lint / type-check](#lint--type-check) <!-- END doctoc generated TOC please keep comment here to allow auto update --> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> # generate-cve-json Small Python project that generates a CVE 5.x JSON record from an `<tracker>` tracking issue, ready to paste into the Vulnogram `#source` tab of the [ASF CVE tool](https://cveprocess.apache.org/). The behavioural contract and the security-process context live in [`SKILL.md`](SKILL.md). This README covers the local-setup and test workflow for the project itself. ## Run From the framework's root (this repository when running standalone; the `.apache-magpie/` snapshot path inside an adopting tracker repo): ```bash uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <ISSUE-NUMBER> [options] ``` Skill files reference the same invocation via the `<framework>` placeholder so the path resolves in either context: ```bash uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <ISSUE-NUMBER> ``` `<framework>` substitutes to `.apache-magpie/apache-magpie` in adopting projects and to `.` (the repository root) in framework standalone — see the placeholder convention in [`AGENTS.md`](../../../AGENTS.md#placeholder-convention-used-in-skill-files). Equivalent forms: ```bash # as a module uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json python -m generate_cve_json <ISSUE-NUMBER> # from inside the project dir cd <framework>/tools/cve-tool-vulnogram/generate-cve-json uv run generate-cve-json <ISSUE-NUMBER> ``` Flags are documented in `generate-cve-json --help` and in [`SKILL.md`](SKILL.md). ## Test ```bash cd tools/cve-tool-vulnogram/generate-cve-json uv run --group dev pytest ``` ## Lint / type-check ```bash cd tools/cve-tool-vulnogram/generate-cve-json uv run --group dev ruff check src tests uv run --group dev ruff format --check src tests uv run --group dev mypy ``` The `prek` hooks configured in `.pre-commit-config.yaml` at the repository root run `ruff check`, `ruff format --check`, and `mypy` on the project files automatically on every commit that touches them. -
SKILL.md 32.6 KB
--- # SPDX-License-Identifier: Apache-2.0 # https://www.apache.org/licenses/LICENSE-2.0 name: generate-cve-json description: | Generate a CVE 5.x JSON document from an <tracker> tracking issue, ready to paste into the Vulnogram `#source` tab of the ASF CVE tool at https://cveprocess.apache.org/cve5/<CVE-ID>#source. The conversion is deterministic: same issue in, same JSON bytes out. Handles multiple credits (one per line) and multiple references (URLs extracted from the issue's "Public advisory URL" and "PR with the fix" fields; the "Security mailing list thread" field is treated as internal-only and never exported). when_to_use: | Invoke when a security team member says "generate CVE JSON for NNN", "update the CVE tool entry for NNN", "paste-ready CVE for NNN", or is about to publish the advisory for a tracking issue and wants the CVE record filled in from the issue body in one paste-and-save step. Not appropriate before the CVE has been allocated (the script needs a CVE ID either from the issue body's `CVE tool link` field or from a `--cve-id` override). --- # generate-cve-json This skill produces a CVE 5.x JSON document from a tracking issue in [`<tracker>`](https://github.com/<tracker>), ready to paste into the Vulnogram **"#source"** tab of the ASF CVE tool. The goal is to eliminate the manual "copy each field from the issue into the right Vulnogram form input" step when you are preparing to publish an advisory. > **Project-agnostic by design.** All project-specific values > (vendor, top-level product / package name, project display map, > CNA org id, generator tag, …) are loaded from a TOML config the > adopting project ships at `<project-config>/tools/cve-tool-vulnogram/cve-json-config.toml`. > Concrete `apache-foo-project-*` strings appearing in this > document are **illustrative examples** of how a project with a > project-style package layout would configure things; replace > them mentally with the adopter's own package taxonomy. The > schema is documented in the package [README](README.md). **Golden rule:** the script generates a *proposal* JSON document. It parses a handful of structured fields from the issue body, but it cannot read the security team member's mind. Always review the generated JSON before pasting, and always do the final review inside Vulnogram before moving the CVE from DRAFT → REVIEW → READY → PUBLIC. **Release-vote gating (opt-in, recommended for ASF projects).** The emitted `CNA_private.state` follows a tri-state state machine: - `DRAFT` — the CNA is incomplete *or* the project has opted into release-vote gating and no vote is in progress yet. - `REVIEW` — the CNA is review-ready (CVE ID + title + description + affected versions + CWE + severity + ≥ 1 credit + ≥ 1 reference) *and* either the project hasn't opted into gating (legacy: ready ⇒ REVIEW) or an RC vote is in progress (signalled by the configured tracker label or a `--review` CLI flag). - `PUBLIC` — the CNA is review-ready *and* the public advisory has shipped (a `vendor-advisory` reference is present). Projects opt into gating by setting `[workflow].release_vote_gating = true` in their `cve-json-config.toml` and choosing the label name via `[workflow].rc_voting_label` (default `"rc voting"`). The sync skill is responsible for detecting [VOTE] threads on the project's dev list (e.g. `dev@<project>.apache.org`) and proposing the label add/remove; the generator only reads the label on the tracker. Non- ASF adopters who publish advisories without a separate release-vote step typically leave gating off — the legacy "ready ⇒ REVIEW" behaviour is the right default for that workflow. **Determinism:** the same input issue body produces exactly the same JSON bytes on every run. The script uses only the Python standard library, has no timestamps or machine-dependent values in its output, sorts JSON keys, and sorts references alphabetically. This lets you paste the result into Vulnogram, tweak fields in the tool, re-run the script later, and cleanly diff the two to see what the tool has added / what you changed by hand. --- ## Inputs - **Issue number** (required) — e.g. `232`. - Optional CLI overrides: - `--cve-id CVE-YYYY-NNNN+` — override the CVE ID if the issue body's `CVE tool link` field has not yet been filled in, or retarget the JSON to a different CVE ID. - `--title "<vendor>: <product>: …"` — override the CVE title. Default is the GitHub issue title with the project's `<vendor>: <product>:` prefix (sourced from the TOML config) when it does not already start with that phrase. - `--version-start X.Y.Z` — override the start of the affected version range (the `affected[].versions[].version` field). Default is the lower bound parsed from the Affected versions field when it uses `>= X, < Y` syntax, otherwise `"0"`. - `--remediation-developer "Name"` — append a `type: "remediation developer"` credit on top of whatever the body's *Remediation developer* field already lists (auto-populated by the `security-issue-sync` skill from the linked PR's author). Repeat the flag to add multiple developers; duplicates between the body field and CLI flags are dropped silently. The reporter credit(s) from the *Reporter credited as* field are always emitted with `type: "finder"`. - `--vendor` / `--product` / `--package-name` / `--collection-url` — override the product identity fields. The defaults come from the project's TOML config (`product.vendor`, `product.default_product`, `product.default_package_name`, `product.default_collection_url`). They are used as the identity for *Affected versions* lines that don't start with a recognisable per-package directory name (see the multi-product note below). `--collection-url` reaches the record only through the purl it is used to derive (see `product.purl_type` below) — except for a product that opts out of purls, the one case a record still carries a `collectionURL`. - **Helm charts — `purl_type = "helm"`, and you must know where the chart is published.** A chart is identified by its repository, not by its name: two projects may both ship a chart called `superset`. So the generator emits ```text pkg:helm/<chart>?repository_url=<base URL> ``` taking the base URL from `product.default_collection_url` (or the package's `collection_url` override), and **refuses to emit a purl at all without one** rather than producing a chart identifier that matches somebody else's chart. **If the base URL is not recorded anywhere, ask whoever publishes the chart.** It is whatever a user would put in `helm repo add` — the `https://…` site serving `index.yaml`, or the `oci://…` registry path for a chart published as an OCI artifact. Either is accepted verbatim; a trailing slash is dropped so the same repository does not produce two identifiers. Do not infer it from the project's homepage: charts are routinely served from a different host than the project site. Note that `helm` is **not** one of the types the purl specification registers. It is used because the scanners that would match this advisory to a deployed chart emit it, and because the two spec-correct encodings — `pkg:oci/` for an OCI registry and `pkg:generic/` for a classic repository — would give the same chart two different identities depending on how it happened to be published. A strict validator may object to the type; being invisible to every scanner is the worse outcome. - `product.purl_type` *(config only)* — the Package URL type for the project's packages (`pypi`, `npm`, `cargo`, …). Every `affected[]` entry carries a `packageURL` (`pkg:<type>/<packageName>`). Per the CVE Record Format the purl never includes a version — the entry's `versions[]` carries the range. **The purl is the only package identifier the record carries.** The ASF CVE tool treats the Package URL as the recommended identifier and derives the legacy `collectionURL` / `packageName` pair from it when the record is serialised for publication, so the generator emits the purl alone. Writing the pair as well would hand the tool two identifiers that can disagree — which it reports rather than silently reconciles. **A purl is required, not optional.** A record needs one to be promoted, so the generator refuses to emit a record it cannot build a purl for rather than producing one the CNA will reject later. Leaving the key unset is fine when the type can be **derived from `product.default_collection_url`** — PyPI, npm, crates.io, RubyGems and NuGet hosts are recognised — and only hosts whose type is unambiguous are mapped, because a guessed type yields a valid-looking identifier pointing at the wrong ecosystem. Set `purl_type = "none"` to state deliberately that a product has no package host — a source-only release published to `dist.apache.org` and nowhere else, for instance. That is an explicit decision rather than an omission, which is the distinction the previous optional behaviour lost. Those entries — and only those — carry `collectionURL` / `packageName` instead, because there is no purl for the CVE tool to derive them from. Name normalisation follows the package-url spec per type, and is implemented for the types whose rules have been read from it: `pypi` (lowercased, `_` becomes `-`) and `npm` (scope becomes the namespace, so `@angular/animation` renders as `pkg:npm/%40angular/animation`; names keep their case, since mixed-case npm packages were grandfathered in). Any other type has its name passed through unchanged, and a name needing namespace semantics is covered by `product.purl_namespace` below, and without one such a type is an error rather than a guessed purl. - `product.purl_namespace` *(optional, config only)* — the namespace for purl types that require one: a Maven `groupId`, a Composer vendor, a Go module prefix. Types the spec gives no namespace (`pypi`, `cargo`, `gem`, `nuget`) ignore it, and an npm scope carried in the package name itself wins over it. A type that requires a namespace and has none is an error naming this key — a guessed Maven groupId would point at another organisation's artifact, and silently omitting the purl produces an unpromotable record. - `[packages.overrides."<packageName>"]` *(config only)* — per-package distribution identity, for a project that ships to more than one ecosystem. Accepts `product`, `collection_url` and `purl_type`; anything omitted falls through to the `product.*` default, so a package that differs only in purl type need not restate its collection URL. The case this exists for: a project whose packages are on PyPI **and** which also ships, say, a Helm chart from its own site. `product.*` describes only the majority ecosystem, so without an override the odd-one-out inherits it and the record claims `pkg:pypi/<chart>` — a package that host does not carry. Since purls are what scanners match on, a wrong one is acted upon, unlike the wrong free-text product name it replaced. ```toml [packages.overrides."apache-example-helm-chart"] product = "Apache Example Helm Chart" collection_url = "https://example.apache.org/" purl_type = "none" ``` The key is the resolved `packageName`, so the package must be one the configured `package_pattern` matches — extending that pattern is a prerequisite, not an extra. `--product-for` still wins over the `product` set here. - `--product-for PACKAGE=PRODUCT` — override the CVE product display name for a specific `packageName`. Repeat to override multiple packages. Useful when a package is not in the project's `project_display_map` config, or when an acronym needs different casing from the title-cased fallback. Example: `--product-for apache-foo-project-baz='Apache Foo Project Baz'`. - `--org-id <uuid>` — override the CNA assigner org id (defaults to the ASF org id). - `--discovery <word>` — override `source.discovery` (default `"UNKNOWN"`; valid CVE 5.x values include `UNKNOWN`, `INTERNAL`, `EXTERNAL`, `USER`). - `--no-envelope` — emit only the inner `cna` container instead of the full CVE 5.x record (envelope is the default). - `--review` / `--draft` (mutually exclusive) — force the emitted `CNA_private.state` to `REVIEW` or `DRAFT` regardless of the tracker's labels. Useful in two cases: - `--review` lets a release manager nudge a record forward by hand when the `rc voting` label is not yet set on the tracker. - `--draft` walks a record back when an RC vote was cancelled or failed and the label is still around. Both flags only matter when release-vote gating is enabled in the project's TOML config (see below); otherwise the state is derived from the CNA's readiness alone and these flags have no effect beyond what the legacy logic produces. - `--attach` — after generating the JSON, embed it at the end of the tracking issue's **body** (after the *CVE tool link* field), wrapped in a collapsible `<details>` block. The block is bracketed by HTML-comment markers (``<!-- generate-cve-json: cve=CVE-YYYY-NNNN+ version=v1 -->`` … ``<!-- generate-cve-json:end cve=CVE-YYYY-NNNN+ version=v1 -->``) that the script uses on later runs to find the existing block and **replace it in place**, so re-runs update the embedded attachment instead of duplicating it or breaking other body fields. The attachment lives in the body — not as a comment — so it stays above every status-change comment in the timeline (effectively "pinned" without needing any pin mechanism). Requires the positional issue argument; incompatible with `--stdin`. --- ## Prerequisites on the tracking issue For the generated JSON to be useful, the issue body should already be filled in through a prior `security-issue-sync` run. In particular: - **Short public summary for publish** — becomes the CVE description. - **Affected versions** — becomes the CVE `affected[]` list. The script understands the common version-expression shapes (`< 3.2.2`, `>= 2.0.0, < 3.2.2`, `<= 3.2.1`, a bare version like `3.1.5`, and a bare lower bound like `>= 2.0.0`). **Multi-product CVEs are supported** — put one package per line, prefixing each with the package directory name as it appears in the adopter's repo, and the script emits one `affected[]` entry per line with the right `product` and package identity. Example (illustrative — using a hypothetical `apache-foo` project's sub-project layout): apache-foo-project-alpha <=6.5.0 apache-foo-project-beta <=1.9.0 Known package directory names are resolved to the vendor-preferred display casing via the project's `packages.project_display_map` config table; unknown packages fall back to title-cased dash-split and can be overridden with `--product-for`. A line without a package prefix (or a single-line field) falls back to the `--product` / `--package-name` defaults, which preserves the single-product behaviour. **`< NEXT VERSION` placeholder** — multi-package trackers don't know which package version will ship the fix until the wave's release manager picks it during a release cut. Until then, the *Affected versions* lines use the literal token `NEXT VERSION` as the upper bound, e.g.: apache-foo-project-alpha < NEXT VERSION apache-foo-project-beta < NEXT VERSION The generator strips `< NEXT VERSION` before parsing each line and emits a `versions[]` entry without `lessThan` (open-ended upper bound — *"affected from \<low\> onwards, no fix released yet"*). When the wave ships and the version is known, the `security-issue-sync` skill replaces each `NEXT VERSION` with the actual `< X.Y.Z` and the next regen produces a fully-bounded entry. Case-insensitive; combines with a lower bound (e.g. `>= 2.0.0, < NEXT VERSION` becomes `{version: "2.0.0", status: "affected"}`). - **Security mailing list thread** — internal navigation reference only; the script **does not** export URLs from this field into `references[]`. Keep whatever the reporter or triager put there. - **Public advisory URL** — each URL in this field is extracted and added to `references[]` with `tags: ["vendor-advisory"]`. Populated by the release manager (or the `security-issue-sync` skill) once the advisory is archived on `<users-list>`. The `--advisory-url` CLI flag still exists for ad-hoc overrides. - **PR with the fix** — each URL in this field becomes a reference URL. **Multiple URLs are supported**: paste them on separate lines, as a bullet list, or comma-separated — the script extracts every `https?://…` token it finds. - **Reporter credited as** — each line becomes one CVE credit entry with `type: "finder"`. **Multiple credits are supported**: put each person on their own line. `Full Name, Affiliation` on a single line is treated as **one** credit, not two, so the common `Jed Cunningham, Astronomer` pattern works as expected. Bullets (`- `, `* `, `1. `) are stripped. Blank lines are ignored. If you need to credit many people:: Jed Cunningham Saurabh Banawar selen (Huntr bounty 3e88d364-5047-4768-a52c-6568f21ef35b) - **Remediation developer** — each line becomes one CVE credit entry with `type: "remediation developer"`. Same parsing rules as *Reporter credited as* (newline-separated, `Full Name, Affiliation` is one credit, bullets stripped). Auto-populated by the `security-issue-sync` skill from the linked PR's author the first time *PR with the fix* is set; manual edits survive subsequent syncs (the skill only proposes appending names that aren't already there). The `--remediation-developer` CLI flag adds further names on top of whatever the body already lists. > **Bot / AI credit policy.** This generator is intentionally > neutral on credit content: whatever a tracker's *Reporter credited > as* or *Remediation developer* field carries is what lands in > `credits[]`. The filtering of obvious bot / AI accounts (e.g. > `dependabot[bot]`, `*-scanner`, `automated-*`) happens **upstream > in the skills** at extraction time — see > [`bot-credits-policy.md`](../bot-credits-policy.md) for the > detection rule and the per-skill enforcement sites. Keeping the > filter upstream means an intentional human override (typed > directly into the field) survives every JSON regeneration without > needing a special bypass flag here. - **CWE** — `CWE-285: Improper Authorization` style works; so does a bare `CWE-285` or a plain sentence. The script extracts the `CWE-\d+` token for the `cweId` field and uses the rest as the human-readable description. - **Severity** — `None`, `Low`, `Medium`, `High`, `Critical` (case-insensitive) are emitted as the text content of a `metrics[].other` block. Vulnogram lets you replace this with a CVSS vector in its form if you want a numeric score. - **CVE tool link** — the ASF CVE tool URL, e.g. `https://cveprocess.apache.org/cve5/CVE-2026-40913`. The script extracts the `CVE-YYYY-NNNN+` token from this field. If the field is still `_No response_`, pass the CVE ID with `--cve-id`. If one of these fields is missing, the JSON still generates, but the reviewer will need to fill the gap in Vulnogram. The skill surfaces any empty field in the proposal so nothing is silently skipped. --- ## Prerequisites - **`gh` CLI authenticated** with collaborator access to `<tracker>` — the script reads the tracker via `gh`. - **`uv` installed** — the script is a small `uv`-managed Python project and is invoked as `uv run --project tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <N>`. See [Prerequisites for running the agent skills](../../../docs/quick-start/prerequisites.md#prerequisites-for-running-the-agent-skills) in `README.md`. --- ## Step 0 — Pre-flight check Before reading the tracker: 1. `gh api repos/<tracker> --jq .name` returns the adopter's tracker repo name (per `<project-config>/project.md`), **and** 2. `uv --version` returns. If either fails, stop and tell the user what to install or log in to. --- ## Step 1 — Verify the issue has the required fields Fetch the issue body and check every template field the script reads. If a field is missing or still `_No response_`, either run [`security-issue-sync`](../../../skills/security-issue-sync/SKILL.md) first to fill it in, or override it on the command line. ```bash gh issue view <N> --repo <tracker> --json body --jq .body \ | grep -E '^###|^_No response_' ``` Ask the user whether to proceed if any critical field is empty (description, affected versions, CVE tool link, credits). Do not silently generate a JSON with placeholder values. --- ## Step 2 — Run the generator Run the project's console script through `uv run --project`, which prepares the (cached) virtualenv on first use and reuses it on later runs: ```bash uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json <N> \ --output /tmp/<CVE-ID>.json \ --version-start <earliest-affected-version> ``` `--version-start` is the one flag the tracking issue body almost never contains and that Vulnogram expects filled in (the body field usually encodes only the upper bound). The remediation developer credit comes from the body's *Remediation developer* field, populated by the `security-issue-sync` skill from the linked PR's author — no CLI flag needed in the normal flow. For a fix that landed in `3.2.2` and was first introduced in `3.0.0`, for example: ```bash uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \ --output /tmp/CVE-2026-40913.json \ --version-start 3.0.0 ``` Pass `--remediation-developer "Name"` only when you need to add a developer credit on top of (or in place of) what the body already contains — for example a co-author who didn't end up as the PR's GitHub author. Additional flags, all optional: - `--cve-id CVE-YYYY-NNNN+` — override the CVE ID if the *CVE tool link* field is empty. - `--title "<vendor>: <product>: …"` — override the title. - `--vendor` / `--product` / `--package-name` / `--collection-url` — override product identity (defaults sourced from the project's TOML config under `[product]`). - `--org-id <uuid>` — override the CNA assigner org id (defaults to the ASF org id). - `--discovery UNKNOWN|INTERNAL|EXTERNAL|USER` — override `source.discovery`. - `--no-envelope` — emit only the `cna` container (no `cveMetadata`, no `dataType`/`dataVersion` wrapper). Use this if Vulnogram's `#source` tab is in "inner block only" mode. - `--stdin` — read the issue body from stdin instead of calling `gh`. Useful for offline iteration and for drafting by hand. The script is deterministic — re-running it with the same flags and the same tracking-issue body produces the same JSON bytes. ### Output shape (in brief) The generated record matches what Vulnogram exports after a save, minus editor cruft. Notable fields: - `containers.cna.affected[]` — `vendor`, `product`, `packageURL` (the version-less purl) and a `versions[]` entry with `version`, `lessThan`, `status: "affected"`, `versionType: "semver"`. An entry for a product configured with `purl_type = "none"` carries `collectionURL` / `packageName` in place of the purl. - `containers.cna.descriptions[]` — both a plain `value` and an HTML `supportingMedia` alternative (Vulnogram's WYSIWYG mode needs both). - `containers.cna.problemTypes[].descriptions[]` — `cweId`, human-readable `description`, `type: "CWE"`. - `containers.cna.metrics[].other` — `type: "Textual description of severity"` and `content.text` = the severity word. - `containers.cna.credits[]` — one entry per *Reporter credited as* line (type `"finder"`), plus one entry per *Remediation developer* body line and per `--remediation-developer` CLI override (type `"remediation developer"`); duplicates between the body field and CLI flags are dropped silently. - `containers.cna.references[]` — URLs with auto-tagged `tags`: - GitHub `pull/` or `commit/` URLs → `["patch"]`; - `lists.apache.org` / `security.apache.org` → `["vendor-advisory"]`; - everything else → no tag. - `containers.cna.source.discovery` — `"UNKNOWN"` by default. - `containers.cna.providerMetadata.orgId` — ASF assigner org id. - `cveMetadata` — `assignerOrgId`, `cveId`, `serial`, `state: "PUBLISHED"`. --- ## Step 3 — Surface the output to the user After the script finishes, print these three things in order: 1. **The output file path**, with a one-line `cat` suggestion so the user can review the JSON in the terminal: ``` Wrote /tmp/cve-CVE-2026-40913.json cat /tmp/cve-CVE-2026-40913.json ``` 2. **A clipboard-copy command** appropriate to the user's platform. On Linux with `xclip` installed: ``` xclip -selection clipboard < /tmp/cve-CVE-2026-40913.json ``` On Wayland: `wl-copy < /tmp/cve-...json`. On macOS: `pbcopy < …`. If `xclip` / `wl-copy` / `pbcopy` is not on PATH, skip the clipboard command and tell the user to copy manually. 3. **The Vulnogram `#source` paste URL**, as a clickable link rendered per the "Linking CVEs" rule in [`AGENTS.md`](../../../AGENTS.md): ``` Paste the JSON into the Vulnogram #source tab: [CVE-2026-40913](https://cveprocess.apache.org/cve5/CVE-2026-40913#source) ``` The #source tab on the ASF CVE tool is the direct "paste raw JSON" view of the Vulnogram form. The page loads the current record, you paste the script output over the top, click Save, and the form view reflects the new values. ### Optional: `--attach` to embed (or refresh) the JSON in the issue body If the user also wants the JSON *attached* to the tracking issue itself (so it is discoverable from the issue without needing the local file), add `--attach` to the invocation: ```bash uv run --project <framework>/tools/cve-tool-vulnogram/generate-cve-json generate-cve-json 232 \ --output /tmp/CVE-2026-40913.json \ --version-start 3.0.0 \ --attach ``` What `--attach` does: - After generating the JSON (exactly the same bytes as without `--attach`), edits the tracking **issue's body** to embed the full JSON inside a four-backtick fenced code block, collapsed behind a `<details>` disclosure so long records don't bloat the issue view. The block is appended *after* the existing template fields, right after the *CVE tool link* field, so it lives at the end of the body. - Brackets the attachment with a pair of hidden HTML-comment markers (``<!-- generate-cve-json: cve=CVE-YYYY-NNNN+ version=v1 -->`` … ``<!-- generate-cve-json:end cve=CVE-YYYY-NNNN+ version=v1 -->``) so subsequent runs can find the existing embedded block and **replace it in place**, without spawning duplicates and without touching any other text in the body. - Re-running with `--attach` is safe and idempotent: same issue body → same JSON → the script patches the body, leaving you with one and only one embedded attachment per CVE id. If the current body already matches what the script would write, the PATCH is skipped entirely (no no-op timestamp on the issue). - The script prints `Embedded CVE JSON in issue body on <tracker>#NNN` on first run and `Replaced CVE JSON in issue body on <tracker>#NNN` on subsequent runs, followed by a URL that deep-links to the `## CVE JSON — paste-ready for …` heading anchor inside the body. **Why embedded in the body and not as a comment?** Two reasons: 1. **Natural "pinning" without an API for it.** GitHub has no pin-comment API. A separate comment ends up buried below every status-change comment — so a newcomer looking at the issue sees a long comment timeline with no obvious way to find the CVE JSON. The issue body always renders *above* the entire comment timeline, so anything embedded in the body is effectively pinned. 2. **One place to read the tracker.** The reporter-template fields, the CVE metadata, and the paste-ready JSON are all in one place — no hunting through the timeline to reconstruct the current state. (GitHub also does not expose its `user-attachments` file-upload pipeline to the REST API — only the web UI drag-and-drop uses it — so a real file attachment isn't available to automation anyway. Embedding as body text is the closest automatable equivalent and is directly visible without a download round-trip.) **Confidentiality.** The embedded block lives inside the private repo, so it inherits the repo-wide confidentiality rules. Linking CVE references inside the block follows the "Linking CVEs" rule in [`AGENTS.md`](../../../AGENTS.md): before publication the block links the ASF CVE tool; after publication, re-running the script includes a `cve.org` link as well. --- ## Step 4 — Propose an issue comment recording the update Per the "Keeping the reporter informed" rule in [`README.md`](../../../README.md), any status change on an issue must be recorded in an issue comment. Pasting a new version of the CVE record is a status change. Propose (and, on confirmation, post) a short comment like: > **CVE entry regenerated from the tracking issue** — generated paste-ready > JSON for [`CVE-2026-40913`](https://cveprocess.apache.org/cve5/CVE-2026-40913) > from the current body fields (description, affected versions `< 3.2.2`, > CWE-285, Low severity, N credits, M references). Pasted into the > Vulnogram `#source` tab; the record is now in sync with the tracking > issue. Include a count of credits and references so a later reviewer can sanity-check that nothing was dropped. --- ## Step 5 — Never edit the JSON in place after pasting Once the JSON has been pasted into Vulnogram and saved, **do not** edit the local JSON file to match tool-side changes. Re-run the script instead (it is deterministic — you will get a clean baseline), diff the new output against the current Vulnogram state, and paste the merged JSON back. This keeps the tracking issue as the single source of truth: if Vulnogram shows something different from the generated JSON, either the issue body is out of date and needs a `security-issue-sync` run, or the tool-side difference is intentional and the reviewer will keep it. --- ## Guardrails - **Confidentiality.** The script deliberately **drops** any URL that points at `cveprocess.apache.org` or the project's `<tracker>` repo from the references list before serialising. Those URLs are private ASF-internal links and should not appear in a published CVE record. See the "Confidentiality of `<tracker>`" section of [`AGENTS.md`](../../../AGENTS.md). - **CVE IDs are always linked** per the "Linking CVEs" rule in [`AGENTS.md`](../../../AGENTS.md). When the skill mentions the CVE in proposals, recaps or comments on the `<tracker>` issue, it must render the ID as a markdown link — before publication to the ASF CVE tool, and additionally to `cve.org` after publication. - **`<tracker>` references are always linked** per the "Linking `<tracker>` issues and PRs" rule in [`AGENTS.md`](../../../AGENTS.md). When the skill mentions the tracking issue in its own comments, render it as a markdown link. - **Deterministic output is a feature.** Do not introduce timestamps, random UUIDs, ordering dependencies on dict iteration, or other sources of non-determinism into the script. If you need to add a new field, make sure the output still hashes the same across runs on the same input. - **Multi-entry fields.** Credits are split on **newlines only** to preserve the `Full Name, Affiliation` pattern. References are extracted from URL tokens in the field value. Do not reintroduce comma-splitting on credits. - **No envelope means no metadata.** `--no-envelope` drops the `cveMetadata` block which includes the CVE ID; the JSON is pure CNA content. Make sure the user knows they will have to set the CVE ID by hand in Vulnogram in that mode. --- ## References - [`AGENTS.md`](../../../AGENTS.md) — repo-wide conventions (confidentiality, Linking CVEs, Linking `<tracker>` issues and PRs, release-branch defaults). - [`README.md`](../../../README.md) — handling process, in particular step 13 (fill in CVE tool fields and send advisory from the tool) and step 15 (paste the attached JSON into Vulnogram's #source tab, move the CVE to PUBLIC, close the issue). - [`security-issue-sync`](../../../skills/security-issue-sync/SKILL.md) — the sibling skill that populates the tracking issue fields this skill consumes. - [`security-issue-fix`](../../../skills/security-issue-fix/SKILL.md) — the other sibling skill that opens a public PR and updates the tracking issue with the fix URL.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.