interface:
  display_name: "WinDiff Version Analysis"
  short_description: "Interpret security-relevant Windows build diffs"
  default_prompt: "Use $windiff-version-diff-analysis to compare two Windows builds and explain the security-relevant changes."

{
  "skill_name": "windiff-version-diff-analysis",
  "notes": "Evals point at the prebuilt databases in windiff_frontend/public/ (21H2 vs 22H2, amd64) so runs are deterministic and don't need network. Iteration 2 broadens the analysis lens beyond mitigations to any new security-relevant feature/component framed for three audiences: anti-malware/EDR, anti-cheat, and vulnerability researchers. This delta really contains non-mitigation signal (EtwTimLogRedirectionTrustPolicy = an EtwTi threat-intel telemetry event; Se*CodeIntegrityOriginClaimForFileObject = CI provenance), so a good report should surface and audience-tag those, not just the Redirection Guard mitigation.",
  "evals": [
    {
      "id": 0,
      "name": "ntoskrnl-security-writeup",
      "prompt": "I've already got WinDiff databases generated under windiff_frontend/public/ for Windows 21H2 and 22H2 (amd64, update BASE). Diff ntoskrnl.exe between those two builds and give me a security-research writeup of what changed. I care about more than just mitigation flags: call out any new security-relevant feature or component and tell me who it matters to — anti-malware/EDR, anti-cheat, or vulnerability researchers. Explain what the changes are likely for, not just a list of names.",
      "expected_output": "A structured report that uses windiff_diff.py, filters _unnamed_ noise, and interprets findings with Windows internals knowledge. Beyond the Redirection Guard mitigation, it should surface non-mitigation security signal in this delta — notably the new EtwTi threat-intelligence telemetry routine (EtwTimLogRedirectionTrustPolicy) as EDR-relevant, and the Se*CodeIntegrityOriginClaimForFileObject CI-provenance routines — and tag findings by audience. Should be honest that there are 0 new syscalls.",
      "files": []
    },
    {
      "id": 1,
      "name": "ntdll-attack-surface",
      "prompt": "Compare ntdll.dll between the 21H2 and 22H2 amd64 builds in windiff_frontend/public/ and tell me what new native API / attack surface or security-relevant feature Microsoft added or removed, and who each change matters to (EDR, anti-cheat, or vuln research). I care about the security angle.",
      "expected_output": "A report that diffs ntdll.dll, groups new symbols by prefix (Nt/Zw/Ldr/Rtl/Etw/Csr), distinguishes user-reachable stubs from internal helpers, interprets security relevance, tags audiences, and notes confidence. Should recognize the 13 new symbols are WIL feature-flag staging, not real API surface.",
      "files": []
    },
    {
      "id": 2,
      "name": "cross-binary-thorough",
      "prompt": "Using the prebuilt WinDiff databases under windiff_frontend/public/ (21H2 vs 22H2, amd64), do a thorough cross-binary diff of ntoskrnl.exe and ntdll.dll and write up the security-relevant deltas for anti-malware/EDR, anti-cheat, and vulnerability-research audiences. Pair up related changes across the two binaries, and don't limit yourself to mitigation flags — include new telemetry, callbacks, components, etc.",
      "expected_output": "A combined report covering both binaries with a cross-binary pairing section, a dedicated section for security-relevant features/components beyond mitigations (telemetry/EtwTi, CI provenance, etc.), audience tagging, highest-signal-first ordering, interpretations with confidence, and noise discounted.",
      "files": []
    }
  ]
}

# Report template

Use this structure. Lead with the highest-signal security findings, group by
component and feature, and give every nontrivial item an interpretation — not just
a name. Drop sections that have no findings rather than padding them.

```markdown
# WinDiff analysis: <binary(ies)> — <old version> → <new version> (<arch>)

## Scope
- Old: <version / update / build> · New: <version / update / build> · Arch: <arch>
- Binaries compared: <list>
- Focus: <syscalls / mitigations / attack surface / component / general>

## Executive summary
3–6 bullets: the most security-relevant changes and the overall theme of the
update (e.g. "hardening pass on win32k", "new process mitigation for X", "new
EtwTi event for EDR", "expanded ETW telemetry"). State confidence where it matters.
Tag findings with the audience that should care: **[EDR]**, **[anti-cheat]**,
**[vuln-research]** (a finding may carry more than one tag).

## New syscalls
For each (ntoskrnl `Nt*` and win32k `NtUser*`/`NtGdi*`):
- **`NtXxx`** [id N] — <subsystem from prefix>. <hypothesis on purpose>.
  Paired ntdll stub: yes/no. Impact: <new attack surface / capability>. **[tags]**.
  Confirm by: <reverse the handler / check public symbols>.
(Note renumbering only if relevant to the audience; don't list it as a feature.)

## New / changed security mitigations
For each new bitfield, enum value, or policy field:
- **<struct>::<field>** (in <binary>) — <which mitigation it implements>. **[tags]**.
  Width/offset if shown. Audit-only vs enforcing if inferable. Companion change
  in <other binary/struct> if any. Impact + how to confirm.

## New security-relevant features & components (beyond mitigations)
This section exists so EDR and anti-cheat findings aren't buried under mitigations.
See `references/windows-internals.md` §7. Cover, where present:
- **Detection & telemetry** — new ETW providers/events, especially `EtwTi*`
  threat-intelligence channels. **[EDR]** (often **[anti-cheat]**). What new
  event/visibility appeared, or what blind spot was closed.
- **Notification callbacks** — new/extended `Ps*NotifyRoutine`, `ObRegisterCallbacks`,
  `CmRegisterCallback*` surface. **[EDR][anti-cheat]**.
- **Code integrity / signing / boot trust** — `Ci*`/WDAC/ELAM/PPL-signer changes.
  **[EDR][vuln-research]**.
- **Process/object/handle & anti-tamper hardening** — **[anti-cheat][vuln-research]**.
- **VBS / secure kernel** — **[all]**.
- **New drivers/modules/components** — a new binary or a cluster under a new prefix;
  hypothesize its role and flag it for follow-up reversing.
For each: <component → subsystem → likely function → who cares and why → confirm-by>.

## New attack surface (routines, callbacks, object types)
Grouped by component (Ps/Ke/Mm/Se/Ci/...). For each notable routine or type:
- **`Name`** — <component → subsystem → likely function → security relevance>. **[tags]**.

## Notable structure changes
New named fields in core structs (`_EPROCESS`, `_TOKEN`, CI policy, ...) that
reveal an existing object gaining capability. Discount `_unnamed_0x...`-only diffs.

## Removed / deprecated
Anything retired, with a note on why it might have been removed.

## Research leads
Concrete next steps for a researcher: which routines to reverse, which findings
are hypotheses needing confirmation, suggested tooling (IDA/Ghidra/BinDiff,
public symbol servers, Microsoft docs).

## Appendix: raw counts
Per binary: +added / -removed / ~modified for exports, symbols, syscalls, types.
```

## Interpretation quality bar

- Never present a bare list of symbol names as the analysis. Names without
  interpretation are what the raw diff already provides; the skill's job is the *why*.
- Tie each finding to a subsystem via its prefix and to a plausible feature.
- Separate what the data *shows* (a name/field exists) from what you *infer*
  (its purpose). Use hedged language for inferences.
- Prefer a few well-reasoned, high-signal findings over an exhaustive dump. If
  there are many similar additions, characterize the group and call out exemplars.

# Roles of the tracked Windows binaries

Knowing what each binary is responsible for lets you predict where a given kind of
change should appear and judge whether a finding is significant. These are the
binaries WinDiff commonly tracks (see `ci/db_configuration.json` for the live set).

| Binary | Role | What new changes here usually mean |
|--------|------|-------------------------------------|
| **ntoskrnl.exe** | The NT kernel: process/thread/memory/object/security managers, the `Nt*` syscall table, most mitigation logic | The center of gravity. New syscalls, new `Ps/Ke/Mm/Ob/Se` routines, and new mitigation bitfields land here. Always diff it. |
| **ntdll.dll** | User-mode native API layer: syscall stubs (`Nt*`/`Zw*`), loader (`Ldr*`), heap, RTL helpers, CSR/`Csr*`, ETW user stubs | New `Nt*` stubs mirror new kernel syscalls. Loader/heap changes can indicate new user-mode mitigations (e.g. CFG/XFG metadata, heap hardening). |
| **win32k.sys** | Kernel GUI/window-manager syscall surface (`NtUser*`), the win32k shadow syscall table | A major LPE attack surface. New `NtUser*` syscalls and win32k lockdown/filter changes matter for sandbox-escape research. |
| **win32kbase.sys** | Base win32k services split out of win32k.sys | New shared GUI primitives; pairs with win32kfull. |
| **win32kfull.sys** | Full win32k window/messaging/GDI implementation | The bulk of GUI logic; new internal routines here often back new `NtUser*` surface. |
| **ci.dll** | Code Integrity: kernel-mode driver signature enforcement (DSE), WDAC/Device Guard policy, HVCI integration | New `Ci*` routines and policy structs = tightened code-signing / driver-blocking / WDAC enforcement. Key for DSE-bypass and supply-chain research. |
| **cng.sys** | Kernel Cryptography Next Generation provider | New algorithms/providers, FIPS, key isolation. Relevant to crypto-downgrade and key-protection research. |
| **hal.dll** | Hardware Abstraction Layer | Low-level platform/CPU changes; new speculation/CPU-feature handling occasionally appears. |
| **secure kernel / `securekernel.exe`, `skci.dll`** (if tracked) | VBS secure kernel and its code-integrity module | VBS/HVCI/Credential Guard surface — high security value when present. |

## Pairing changes across binaries

The strongest findings show a coherent story across binaries. Examples:

- A new `Nt*` **syscall in ntoskrnl** + a matching **stub in ntdll** = a fully new
  user-reachable kernel service. Confirm both before claiming "new syscall".
- A new **mitigation field** in `_PS_MITIGATION_OPTIONS*` (ntoskrnl) + a new
  enum value in `_PROCESSINFOCLASS`/`_PROCESS_MITIGATION_POLICY` = a new opt-in
  process mitigation surfaced through `NtSetInformationProcess`.
- New **`Ci*` policy fields** (ci.dll) + new mitigation toggles (ntoskrnl) can
  indicate tighter driver/usermode signing tied to a process policy.
- New **`NtUser*`** (win32k) + win32k filter/lockdown changes = adjusted GUI
  attack surface for sandboxed processes.

When you spot one half of such a pair, look for the other half in the companion
binary's diff and report them together.

# Windows Internals: inferring intent from a version diff

This is the reasoning toolkit for turning a raw symbol/type/syscall delta into a
security interpretation. The goal for each finding is to answer: *what subsystem
is this, what does it likely do, and why does it matter for security?*

## Table of contents
1. API name prefixes (routine namespaces)
2. Routine name suffixes and decorations
3. Syscalls: what additions/removals/renumbering mean
4. Security mitigation flags — where they live and how to spot new ones
5. Key structures to watch
6. How to phrase an interpretation (and admit uncertainty)
7. Security-relevant features & components beyond mitigations (EDR / anti-cheat / vuln research)

---

## 1. API name prefixes

Windows kernel/native routines are namespaced by a 2–3 letter prefix naming the
owning component. The prefix tells you the subsystem; the rest hints at the action.

| Prefix | Subsystem | Security relevance of new entries |
|--------|-----------|-----------------------------------|
| `Nt` / `Zw` | System call interface (user↔kernel). `Nt` = the syscall entry; `Zw` = same with kernel-mode previous-mode semantics | New `Nt*` = **new attack surface reachable from user mode**. Highest priority. |
| `Ps` | Process/thread management (Process Structure) | Process/thread creation, tokens, mitigations, protection (PPL). Watch for new mitigation/protection logic. |
| `Ke` | Kernel core (scheduling, sync, interrupts, CPU) | Low-level primitives; new CET/shadow-stack, speculation, or APC changes appear here. |
| `Mm` | Memory Manager | VAD, paging, sections, pool. New memory-safety/isolation features (e.g. kernel CFG, pool hardening). |
| `Ob` | Object Manager | Handle/object security, callbacks. New object types or handle-hardening. |
| `Se` | Security Reference Monitor | Tokens, privileges, access checks, ACLs, AppContainer. New `Se*` often = **authz/sandbox changes**. |
| `Cm` | Configuration Manager (registry) | Registry security, virtualization, callbacks. |
| `Cc` / `Fs` / `Io` / `Iop` | Cache / filesystem / I/O manager | Driver-facing surface, IRP handling, filter callbacks. |
| `Alpc` / `Lpc` | Advanced Local Procedure Call (IPC) | New ALPC surface is a classic LPE target. |
| `Etw` / `Wmi` | Event Tracing for Windows | New providers/events usually = **new telemetry/defensive instrumentation** (EDR-relevant). |
| `Ci` | Code Integrity (in `ci.dll`) | Driver/DSE signing, WDAC/HVCI policy. New `Ci*` = code-integrity policy/enforcement changes. |
| `Bcrypt`/`Ncrypt`/`Crypt` | CNG cryptography (`cng.sys`, `bcrypt.dll`) | New algorithms/providers, FIPS, key isolation. |
| `Vsl` / `Vbs` / `Skci` / `Securekernel` | Virtualization-Based Security / secure kernel | VBS, HVCI, Credential Guard, secure-kernel surface. High security value. |
| `Rtl` | Runtime Library (shared helpers) | Often supporting code for a feature whose policy lives elsewhere; trace callers. |
| `Exp`/`Psp`/`Mi`/`Obp`/`Sep`/`Cmp`/`Iop` | The `p`/`i` variants are the **internal** (private) implementations | Visible only as debug symbols, not exports; reveal the real logic behind a public stub. |
| `Etw`-style `Win32k`, `gre`, `Nt User`/`NtGdi` | win32k GUI/GDI syscalls | win32k is a huge LPE surface; new `NtUser*`/`NtGdi*` syscalls matter and tie into win32k lockdown. |

A name with no familiar prefix, or a brand-new prefix, can itself be the
signal — a new feature area. Note it and hypothesize from the rest of the name.

## 2. Suffixes and decorations

- `Ex` — extended version of an existing routine (new params/flags). Diff against
  the base routine to see what capability was added.
- `Worker`, `Callback`, `Notify`, `Routine` — registration/callback surface;
  new ones may be EDR/driver notification hooks.
- `Internal`, `Stub`, `Thunk` — wrappers; the interesting logic is elsewhere.
- Trailing digits / `2` — a v2 of an interface, usually because the struct or
  semantics changed; compare the associated types.
- `Mitigation`, `Cet`, `Cfg`, `Xfg`, `Shadow`, `Guard`, `Acg`, `Cig` in a name
  almost always indicate an exploit-mitigation feature (see §4).

## 3. Syscalls

`scripts/windiff_diff.py` reports syscalls as added / removed / renumbered.

- **Added syscalls** are the single highest-value finding: brand-new
  kernel-reachable surface. Interpret each from its `Nt`/`NtUser`/`NtGdi` name.
  Cross-reference whether a matching `Nt*` export and internal `Nt*`/`*p*`
  implementation also appeared.
- **Renumbered** (same name, different id) is normal across builds — the syscall
  table is regenerated. It matters only if you're hardcoding SSNs (e.g. for direct
  syscalls / EDR evasion research); call it out for that audience but don't treat
  it as a feature change.
- **Removed** syscalls are rare and noteworthy — a capability retired or merged.
- ntoskrnl carries the `Nt*` table; win32k binaries carry the `win32k` shadow
  table (`NtUser*`/`NtGdi*`). Diff both when GUI surface matters.

## 4. Security mitigation flags — the priority target

Mitigations are usually represented as **bitfields in a structure** or **enum
values**, not as standalone exports. Critically, the mitigation bitfields are
*anonymous* structs (`_EPROCESS::MitigationFlagsValues` is typed `_unnamed_0xNNNN`),
so a new bit hides inside an anonymous type whose id churns between builds. The
diff script handles this: check **`types.resolved_member_changes`** first — it
follows each anonymous member back to its named parent and reports new/removed bits
as `<parent>::<member>` (e.g. `_EPROCESS::MitigationFlags2Values` gaining
`RedirectionTrustPolicyEnabled : 1`). Then also look at these structures directly:

- **`_PS_MITIGATION_OPTIONS` / `_PS_MITIGATION_OPTIONS2` / `_PS_MITIGATION_AUDIT_OPTIONS`**
  — per-process mitigation policy bitmaps. New 4-bit nibble fields here = a new
  process mitigation (e.g. ACG, CIG, blocking non-MS binaries, redirection-guard,
  user-shadow-stack, pointer auth). This is the first place to check.
- **`_EPROCESS` / `_KPROCESS` flag bitfields** (e.g. `MitigationFlags`,
  `MitigationFlags2`, `MitigationFlags3`, `Flags`) — runtime mitigation state.
  New single-bit fields named `*Enabled`/`*Audit` are new mitigations or telemetry.
- **`_PS_PROTECTION`** — Protected Process Light (PPL) signer/type. New signer
  enum values = new protected-process classes.
- **CET / shadow stacks** — fields/types containing `Cet`, `ShadowStack`,
  `Ssp`, `UserCet`, `KernelCet`. Kernel CET (`kCET`) hardens ROP.
- **CFG / XFG** — Control Flow Guard / eXtended Flow Guard: `Guard`, `Cfg`, `Xfg`,
  `GuardFlags`. New bits tighten indirect-call protection.
- **Code Integrity (`ci.dll`)** — types/enums with `Ci`, `Policy`, `Wdac`, `Hvci`,
  `SiPolicy`, `Signing`. New policy fields = tightened driver/usermode signing.
- **Token/authz (`Se*`, `_SEP_TOKEN_*`, `_TOKEN`)** — new privilege bits,
  AppContainer/capability fields, trust labels.
- **VBS/secure kernel** — `Vsm`, `Vsl`, `Ium`, `Secure`, enclave fields.

When you see a new bitfield, state the structure, the field name, its width/offset
if shown, and the mitigation it most plausibly implements. If a new field merely
*reserves* bits (`SpareBits`, padding), say so — not every new bit is a feature.

## 5. Key structures to watch

`_EPROCESS`, `_KPROCESS`, `_ETHREAD`, `_KTHREAD` (process/thread state and flags);
`_PS_MITIGATION_OPTIONS*`, `_PS_PROTECTION`; `_TOKEN`, `_SEP_TOKEN_PRIVILEGES`;
`_OBJECT_HEADER`, object-type structs; `_HANDLE_TABLE*`; `_MMVAD*`, `_MMPTE`;
`_KPRCB`, `_KPCR` (per-CPU, where CET/speculation state hides); CI policy structs
in `ci.dll`; `_ALPC_*`. A new *field* in one of these is often more telling than a
whole new type, because it shows an existing object gaining a new capability.

## 6. Phrasing an interpretation

For each finding, aim for: **component (from prefix) → subsystem → hypothesis about
the feature/mitigation → security impact**. Example shape:

> `NtSetInformationProcess` gains a new info class `ProcessFooMitigationPolicy`
> (new enum value in `_PROCESSINFOCLASS`) alongside a new nibble in
> `_PS_MITIGATION_OPTIONS2`. This is consistent with a new opt-in process
> mitigation; the `Ps` ownership and the `Options2` overflow suggest the existing
> bitmap was full. Impact: defenders gain a new hardening toggle; researchers
> should reverse `PspSetMitigationPolicy` to learn the enforcement and whether it
> is audit-only first.

Be explicit about confidence. Use "likely / consistent with / appears to" for
inferences and reserve definite statements for what the data shows directly (a
name exists, a field was added). When unsure, name the concrete next step a
researcher would take (reverse the routine, check Microsoft docs / public PDB
symbols, diff the disassembly in IDA/Ghidra/BinDiff). Honest uncertainty beats a
confident wrong guess.

---

## 7. Security-relevant features & components beyond mitigations

Mitigation flags are the most obvious finding, but a version diff often reveals
other security-relevant work that matters just as much to three audiences:
**anti-malware / EDR developers**, **anti-cheat developers**, and **vulnerability
researchers**. Actively look for the categories below and, for each finding, say
which audience(s) should care and why. The same change can serve more than one.

### Detection & telemetry surface (primarily EDR, also anti-cheat)
EDR and kernel anti-cheat both live and die by the visibility the OS gives them.
New entries here are often the highest-value finding even though they aren't
"mitigations":

- **ETW providers / events** — symbols/types containing `Etw`, `Provider`,
  `EtwWrite`, GUID blobs, or `*EtwEvent*`. The crown jewel is the **Threat
  Intelligence** channel: `EtwTi*` routines (e.g. `EtwTiLogReadWriteVm`,
  `EtwTiLogAllocExecVm`, `EtwTiLogProtectExecVm`, `EtwTiLogSetContextThread`,
  `EtwTiLogDriverObjectLoad`, `EtwTiLogRedirectionTrustPolicy`). A new `EtwTiLog*`
  routine = a new kernel event EDR/anti-cheat can subscribe to (or that defenders
  must now account for). New non-Ti providers expand audit/forensic coverage.
- **Notification callbacks** — registration surface that drivers (EDR/AC) hook:
  `PsSetCreateProcessNotifyRoutine[Ex2]`, `PsSetCreateThreadNotifyRoutine[Ex]`,
  `PsSetLoadImageNotifyRoutine[Ex]`, `ObRegisterCallbacks` (handle-operation
  filtering), `CmRegisterCallback[Ex]` (registry). New `*NotifyRoutine*`,
  `*Callback*`, or callout-table entries change what products can observe or
  what attackers can tamper with. New `Ex`/`Ex2` variants usually add flags or
  context — diff the associated struct.
- **AMSI / script & content scanning** — `Amsi*`, scan interfaces, content
  inspection hooks: new places malware content gets surfaced to scanners.

### Code integrity, signing & boot trust (EDR + vuln research)
- **ELAM** (Early-Launch Anti-Malware) — `Elam*`, early-boot driver vetting.
- **Code Integrity / WDAC** (`ci.dll`, `Ci*`, `SiPolicy`, `Hvci`, `Wdac`) — driver
  blocklist, signing-level, and policy changes. New `Ci*` = tightened DSE / driver
  loading, directly relevant to BYOVD research and to EDR self-protection.
- **Protected Process Light** (`_PS_PROTECTION`, signer enums) — which signers can
  run protected. Central to anti-cheat (protect the game) and to EDR
  self-defense (protect the agent). New signer classes are notable.

### Process / object / handle hardening (anti-cheat + vuln research)
- New `Ob` object types, handle-table changes, `ObRegisterCallbacks` altitude or
  pre/post-op changes — anti-cheat uses these to block handle theft of game
  processes; researchers probe them for bypasses.
- Anti-tamper, integrity-check, or self-protection routines (`*Integrity*`,
  `*Tamper*`, `*SelfProtect*`).

### Virtualization-based security (all three)
`Vsm`/`Vsl`/`Ium`/`Secure*`/`Skci`/enclave surface — VBS/HVCI/Credential Guard.
New secure-kernel calls or trustlet surface matter for both hardening analysis and
secure-kernel vuln research.

### Brand-new drivers, modules, or components
A binary appearing in the diff that wasn't tracked before, or a large cluster of
new routines under a previously-absent prefix, can signal a **new feature/component**
(e.g. a new security driver, a new subsystem). Call it out as a unit, hypothesize
its role from the names, and flag it as something to pull the PE/PDB for and reverse.

### How to tag audiences (quick guide)
- New **telemetry / callbacks / ETW(Ti)** → **EDR** first (new visibility or a
  closed blind spot), often **anti-cheat** too.
- New **process/handle/object protection, PPL, anti-tamper, VBS** → **anti-cheat**
  and **EDR self-protection**.
- New **syscalls, IOCTLs, parsing surface, drivers, widened structs, low-priv
  callback registration** → **vulnerability researchers** (attack surface), and
  note if a mitigation simultaneously *removes* a known primitive.
- New **CI/WDAC/ELAM/signing** → **EDR** (self-protection, BYOVD defense) and
  **vuln research** (bypass surface).

Don't force a tag where it doesn't fit, and don't invent relevance — if a change is
purely functional with no security angle, say so briefly and move on.

#!/usr/bin/env python3
"""Build a minimal WinDiff CLI config for diffing two Windows versions.

WinDiff's config (see windiff_cli/src/configuration.rs) is:
    { "oses": [ {version, update, architecture}, ... ],
      "binaries": { "<name>": { "extracted_information": [FLAGS...] }, ... } }

This emits such a config restricted to the OS versions and binaries you want to
compare, so the CLI run downloads only what the diff needs.

Usage:
    make_config.py --os "VERSION:UPDATE:ARCH" --os "VERSION:UPDATE:ARCH" \
                   --binary ntoskrnl.exe [--binary ntdll.dll ...] \
                   [--info EXPORTS DEBUG_SYMBOLS MODULES TYPES SYSCALLS]

Example:
    make_config.py --os "21H2:BASE:amd64" --os "11-24H2:KB5074105:amd64" \
                   --binary ntoskrnl.exe --binary ntdll.dll --binary ci.dll
"""
import argparse
import json
import sys

ALL_INFO = ["EXPORTS", "DEBUG_SYMBOLS", "MODULES", "TYPES", "SYSCALLS"]
VALID_ARCH = {"i386", "wow64", "amd64", "arm", "arm64"}


def parse_os(spec):
    parts = spec.split(":")
    if len(parts) != 3:
        sys.exit(f"error: --os must be VERSION:UPDATE:ARCH, got {spec!r}")
    version, update, arch = parts
    if arch not in VALID_ARCH:
        sys.exit(f"error: arch must be one of {sorted(VALID_ARCH)}, got {arch!r}")
    return {"version": version, "update": update, "architecture": arch}


def main():
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--os", action="append", required=True, metavar="VERSION:UPDATE:ARCH")
    parser.add_argument("--binary", action="append", required=True, metavar="NAME")
    parser.add_argument(
        "--info",
        nargs="+",
        choices=ALL_INFO,
        default=ALL_INFO,
        help="Which data kinds to extract per binary (default: all)",
    )
    args = parser.parse_args()

    if len(args.os) < 2:
        sys.exit("error: pass --os at least twice (the two versions to diff)")

    config = {
        "oses": [parse_os(s) for s in args.os],
        "binaries": {name: {"extracted_information": args.info} for name in args.binary},
    }
    json.dump(config, sys.stdout, indent=4)
    print()


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""Diff two WinDiff databases for the same binary across two Windows versions.

WinDiff emits one gzip-compressed JSON file per (binary, OS) pair, named
    {binary}_{version}_{update}_{architecture}.json.gz
and an index.json.gz describing which OS versions / binaries are present.

This script does the deterministic part of a version diff so the analysis can
focus on interpretation rather than set arithmetic:

  - exports / debug symbols / modules / syscalls -> added & removed (set diff)
  - syscalls                                     -> also reports renumbering
  - reconstructed types                          -> added, removed, and for
                                                    types present in both, a
                                                    line-level diff of the
                                                    struct/enum definition so
                                                    new/removed fields and flags
                                                    are visible

Output is JSON on stdout (machine-readable, feed it back into the analysis) and
a human-readable summary on stderr.

Usage:
    windiff_diff.py <db_dir> <binary> <old_os_suffix> <new_os_suffix> [--kinds ...]

    db_dir        directory containing the *.json.gz databases (and index.json.gz)
    binary        e.g. ntoskrnl.exe, ntdll.dll, win32k.sys, ci.dll
    old/new       OS path suffix "version_update_architecture",
                  e.g. 21H2_BASE_amd64  (run with --list to see what's present)

Examples:
    windiff_diff.py ./out ntoskrnl.exe 21H2_BASE_amd64 22H2_BASE_amd64
    windiff_diff.py ./out --list
    windiff_diff.py ./out ntdll.dll 21H2_BASE_amd64 22H2_BASE_amd64 --kinds syscalls types
"""
import argparse
import difflib
import gzip
import json
import os
import re
import sys

KINDS = ["exports", "symbols", "modules", "syscalls", "types"]


def load_gz_json(path):
    with gzip.open(path, "rt", encoding="utf-8") as f:
        return json.load(f)


def db_path(db_dir, binary, suffix):
    return os.path.join(db_dir, f"{binary}_{suffix}.json.gz")


def list_databases(db_dir):
    """Print the OS versions and binaries available in db_dir."""
    index_path = os.path.join(db_dir, "index.json.gz")
    if os.path.exists(index_path):
        index = load_gz_json(index_path)
        print("OS versions (suffix = version_update_architecture):", file=sys.stderr)
        for os_entry in index.get("oses", []):
            suffix = f"{os_entry['version']}_{os_entry['update']}_{os_entry['architecture']}"
            print(f"  {suffix}", file=sys.stderr)
        print("\nBinaries:", file=sys.stderr)
        for b in index.get("binaries", []):
            print(f"  {b}", file=sys.stderr)
        return
    # Fall back to scanning the directory if there's no index.
    print("No index.json.gz; database files found:", file=sys.stderr)
    for name in sorted(os.listdir(db_dir)):
        if name.endswith(".json.gz") and name != "index.json.gz":
            print(f"  {name}", file=sys.stderr)


def diff_string_set(old_list, new_list):
    old, new = set(old_list), set(new_list)
    return {
        "added": sorted(new - old),
        "removed": sorted(old - new),
        "old_count": len(old),
        "new_count": len(new),
    }


def diff_syscalls(old_map, new_map):
    """Syscalls are {id: name}. Track added/removed names and renumbering."""
    old_names = set(old_map.values())
    new_names = set(new_map.values())
    # Map name -> id for renumbering detection.
    old_by_name = {v: k for k, v in old_map.items()}
    new_by_name = {v: k for k, v in new_map.items()}
    renumbered = []
    for name in sorted(old_names & new_names):
        if old_by_name[name] != new_by_name[name]:
            renumbered.append(
                {"name": name, "old_id": old_by_name[name], "new_id": new_by_name[name]}
            )
    return {
        "added": sorted(
            [{"id": new_by_name[n], "name": n} for n in (new_names - old_names)],
            key=lambda e: int(e["id"]) if str(e["id"]).isdigit() else e["id"],
        ),
        "removed": sorted(old_names - new_names),
        "renumbered": renumbered,
        "old_count": len(old_names),
        "new_count": len(new_names),
    }


ANON_PREFIX = "_unnamed_"
# A member declaration referencing an anonymous struct/union, e.g.
#   /* 0x09d4 */ _unnamed_0x19d4 MitigationFlags2Values;
# We capture the anon type id and the member name so we can follow the
# reference across builds (the synthetic id usually changes between builds).
ANON_MEMBER_RE = re.compile(r"\b(_unnamed_0x[0-9a-fA-F]+)\s+(\w+)")
# Any reference to an anonymous type id, used to detect lines that differ
# ONLY because the synthetic id churned (pure rebuild noise).
ANON_ID_RE = re.compile(r"_unnamed_0x[0-9a-fA-F]+")


def is_anon(name):
    """resym names anonymous structs/unions _unnamed_0xNNNN; these synthetic
    ids change between builds and are diff noise, not real additions."""
    return name.startswith(ANON_PREFIX)


def _strip_anon_ids(line):
    """Normalize a member line by erasing anonymous type ids, so two lines that
    differ only by `_unnamed_0x2b2c` vs `_unnamed_0x2b3d` compare equal."""
    return ANON_ID_RE.sub("_unnamed_", line)


def _clean_member(raw):
    """Strip resym's `/* ... */` offset/size/BitPos comments and collapse spaces.

    Removing these comments is deliberate: it means a bit inserted mid-bitfield
    (which shifts every following BitPos/offset) doesn't masquerade as dozens of
    changed members — only the genuinely added/removed declaration shows up."""
    c = re.sub(r"/\*.*?\*/", "", raw)
    return re.sub(r"\s+", " ", c).strip()


def _member_lines(body):
    """Field/enumerator declaration lines of a type body (skip braces/headers)."""
    out = []
    for raw in body.splitlines():
        c = _clean_member(raw)
        if not c.strip("{}; ") or c.startswith(("struct", "union", "enum")):
            continue
        out.append(c)
    return out


def _body_member_delta(old_body, new_body):
    """Added/removed member declaration lines between two type bodies."""
    diff = difflib.unified_diff(_member_lines(old_body), _member_lines(new_body), lineterm="", n=0)
    added, removed = [], []
    for l in diff:
        if l.startswith("+") and not l.startswith("+++"):
            added.append(l[1:].strip())
        elif l.startswith("-") and not l.startswith("---"):
            removed.append(l[1:].strip())
    return [a for a in added if a], [r for r in removed if r]


def _anon_members(definition):
    """Map member_name -> anonymous type id for a struct/union definition."""
    return {name: anon_id for anon_id, name in ANON_MEMBER_RE.findall(definition)}


def resolve_anon_member_changes(old_types, new_types):
    """Follow anonymous struct/union members back to their named parent and diff
    their contents across builds.

    This is what surfaces new bitfield flags — e.g. a new mitigation bit added to
    `_EPROCESS::MitigationFlags2Values` lives inside an anonymous `_unnamed_0xNNNN`
    struct whose synthetic id changes between builds. Diffing by member name
    (not by id) recovers the real per-bit delta and attributes it to
    `<parent>::<member>` so the change is readable, not noise.

    Returns a list of {path, parent, member, old_type, new_type, added, removed}.
    """
    results = []
    common = (set(old_types) & set(new_types))
    # Walk every named (non-anonymous) parent; recurse through nested anon members.
    roots = sorted(n for n in common if not is_anon(n))

    def walk(old_def, new_def, path, seen):
        old_anon = _anon_members(old_def)
        new_anon = _anon_members(new_def)
        for member in sorted(set(old_anon) & set(new_anon)):
            oid, nid = old_anon[member], new_anon[member]
            ob, nb = old_types.get(oid), new_types.get(nid)
            if ob is None or nb is None:
                continue
            mpath = f"{path}::{member}"
            key = (oid, nid, mpath)
            if key in seen:
                continue
            seen.add(key)
            if _strip_anon_ids(ob) != _strip_anon_ids(nb):
                added, removed = _body_member_delta(ob, nb)
                if added or removed:
                    results.append(
                        {
                            "path": mpath,
                            "parent": path,
                            "member": member,
                            "old_type": oid,
                            "new_type": nid,
                            "added": added,
                            "removed": removed,
                        }
                    )
            walk(ob, nb, mpath, seen)  # nested anonymous struct/union

    for parent in roots:
        walk(old_types[parent], new_types[parent], parent, set())
    results.sort(key=lambda r: r["path"])
    return results


def diff_types(old_types, new_types, hide_anon=True):
    """Types are {name: definition_text}. Diff definitions line by line."""
    old_keys, new_keys = set(old_types), set(new_types)
    if hide_anon:
        old_keys = {k for k in old_keys if not is_anon(k)}
        new_keys = {k for k in new_keys if not is_anon(k)}
    added = sorted(new_keys - old_keys)
    removed = sorted(old_keys - new_keys)
    modified = []
    for name in sorted(old_keys & new_keys):
        old_def = old_types[name]
        new_def = new_types[name]
        if old_def == new_def:
            continue
        old_lines = old_def.splitlines()
        new_lines = new_def.splitlines()
        diff = list(difflib.unified_diff(old_lines, new_lines, lineterm="", n=1))
        added_lines = [l[1:].strip() for l in diff if l.startswith("+") and not l.startswith("+++")]
        removed_lines = [l[1:].strip() for l in diff if l.startswith("-") and not l.startswith("---")]
        added_lines = [l for l in added_lines if l]
        removed_lines = [l for l in removed_lines if l]
        # Drop pairs that differ only by an anonymous-type id (pure rebuild churn);
        # the real change, if any, is recovered by resolve_anon_member_changes().
        anon_norm_removed = {_strip_anon_ids(l) for l in removed_lines}
        added_lines = [l for l in added_lines if _strip_anon_ids(l) not in anon_norm_removed]
        anon_norm_added = {_strip_anon_ids(l) for l in
                           [l[1:].strip() for l in diff if l.startswith("+") and not l.startswith("+++")]}
        removed_lines = [l for l in removed_lines if _strip_anon_ids(l) not in anon_norm_added]
        if not added_lines and not removed_lines:
            continue
        modified.append(
            {
                "name": name,
                "added_lines": added_lines,
                "removed_lines": removed_lines,
            }
        )
    return {
        "added": added,
        "removed": removed,
        "modified": modified,
        # Anonymous bitfield/struct member changes resolved back to <parent>::<member>.
        # Always computed (independent of hide_anon) since this is the signal that
        # hiding anonymous *names* would otherwise lose.
        "resolved_member_changes": resolve_anon_member_changes(old_types, new_types),
        "old_count": len(old_keys),
        "new_count": len(new_keys),
    }


def summarize(binary, old_suffix, new_suffix, result):
    out = sys.stderr
    print(f"\n=== {binary}: {old_suffix} -> {new_suffix} ===", file=out)
    for kind in KINDS:
        if kind not in result:
            continue
        r = result[kind]
        if kind == "syscalls":
            print(
                f"  syscalls : +{len(r['added'])} new, "
                f"{len(r.get('renumbered', []))} renumbered "
                f"({r['old_count']} -> {r['new_count']})",
                file=out,
            )
            for s in r["added"]:
                print(f"             + [{s['id']}] {s['name']}", file=out)
        elif kind == "types":
            resolved = r.get("resolved_member_changes", [])
            print(
                f"  types    : +{len(r['added'])} new, -{len(r['removed'])} removed, "
                f"~{len(r['modified'])} modified, "
                f"{len(resolved)} anon-member change(s) "
                f"({r['old_count']} -> {r['new_count']})",
                file=out,
            )
            for c in resolved:
                print(f"             ~ {c['path']}", file=out)
                for a in c["added"]:
                    print(f"                 + {a}", file=out)
                for d in c["removed"]:
                    print(f"                 - {d}", file=out)
        else:
            print(
                f"  {kind:<9}: +{len(r['added'])} added, -{len(r['removed'])} removed "
                f"({r['old_count']} -> {r['new_count']})",
                file=out,
            )


def main():
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("db_dir")
    parser.add_argument("binary", nargs="?")
    parser.add_argument("old_os_suffix", nargs="?")
    parser.add_argument("new_os_suffix", nargs="?")
    parser.add_argument("--kinds", nargs="+", choices=KINDS, default=KINDS)
    parser.add_argument(
        "--include-anon",
        action="store_true",
        help="Include _unnamed_0xNNNN anonymous types (hidden by default as diff noise)",
    )
    parser.add_argument("--list", action="store_true", help="List available OS versions and binaries")
    args = parser.parse_args()

    if args.list:
        list_databases(args.db_dir)
        return 0

    if not (args.binary and args.old_os_suffix and args.new_os_suffix):
        parser.error("binary, old_os_suffix and new_os_suffix are required (or use --list)")

    old_path = db_path(args.db_dir, args.binary, args.old_os_suffix)
    new_path = db_path(args.db_dir, args.binary, args.new_os_suffix)
    for p in (old_path, new_path):
        if not os.path.exists(p):
            print(f"error: missing database file: {p}", file=sys.stderr)
            print("Run with --list to see what's available.", file=sys.stderr)
            return 1

    old_db = load_gz_json(old_path)
    new_db = load_gz_json(new_path)

    result = {
        "binary": args.binary,
        "old_os": args.old_os_suffix,
        "new_os": args.new_os_suffix,
        "old_version": old_db.get("metadata", {}).get("version"),
        "new_version": new_db.get("metadata", {}).get("version"),
    }
    for kind in args.kinds:
        if kind == "syscalls":
            result["syscalls"] = diff_syscalls(old_db.get("syscalls", {}), new_db.get("syscalls", {}))
        elif kind == "types":
            result["types"] = diff_types(
                old_db.get("types", {}), new_db.get("types", {}), hide_anon=not args.include_anon
            )
        else:
            result[kind] = diff_string_set(old_db.get(kind, []), new_db.get(kind, []))

    summarize(args.binary, args.old_os_suffix, args.new_os_suffix, result)
    json.dump(result, sys.stdout, indent=2)
    print()
    return 0


if __name__ == "__main__":
    sys.exit(main())