{
  "name": "okf-wiki",
  "version": "0.8.3",
  "description": "Scaffold an Open Knowledge Format (OKF) knowledge base from docs, plans, notes, or a repo: one-concept-per-file Markdown with YAML frontmatter, navigation, and validation. Built for newsroom memory, research atlases, decision logs, and infrastructure maps.",
  "author": {
    "name": "Joe Amditis",
    "email": "jamditis@gmail.com"
  }
}

interface:
  display_name: "OKF wiki"
  short_description: "Builds an Open Knowledge Format (OKF) knowledge base from existing docs…"

# claude-skills-journalism wiki

An Open Knowledge Format (OKF) knowledge base: small markdown files, one concept each,
with provenance in YAML frontmatter. See `SPEC.md` for the full contract.

## Requirements

The validator parses YAML frontmatter with PyYAML:

```bash
pip install -r requirements.txt    # or: pip install pyyaml
```

## Validate

```bash
python3 scripts/validate.py --bundle bundle
```

It must exit 0. Run it before every commit.

## Add a concept

1. Create `bundle/<section>/<concept>.md` with the required frontmatter
   (`type, title, description, source, verified, timestamp, tags`).
2. Add a bullet for it in that section's `index.md`.
3. Validate.

## Session hooks

`.claude/` ships two hooks that orient Claude on this knowledge base before it works:

- `okf-anchor.py` (SessionStart) loads the bundle index into the session context.
- `okf-orient.py` (PreToolUse) blocks the first action once per session until Claude
  confirms it has read the index, then unblocks for the rest of the session.

They are one cross-platform python3 script each. No single interpreter name works on
every OS, so `settings.json` names the one for the OS this bundle was scaffolded on
(`python3` on macOS/Linux, `python` on Windows). If you move the bundle to a different
OS, change that one token in `settings.json` (or re-run the scaffolder there). Claude
Code treats a checked-in `.claude/settings.json` as untrusted, so the first time you
open this project it asks you to approve the hooks. They run automatically after that.
Delete `.claude/` (or set `disableAllHooks`) to turn them off.

## Security

Never put secret values in a concept. A credential concept documents the key name and
where it is retrieved, not the value. The validator scans for leaked secrets and fails on a hit.

# The validator (scripts/validate.py) parses YAML frontmatter with PyYAML.
PyYAML>=5.1

# OKF spec v1

Open Knowledge Format (OKF) is a convention for storing knowledge as small markdown
files that both people and agents can read. One file describes one concept and carries
its own provenance. Directory `index.md` files provide navigation. A validator enforces
the contract so the knowledge base stays consistent as it grows.

This is the generic spec. A project may layer its own conventions on top (extra tags,
naming patterns, a fixed section list), but must not weaken the rules below.

## A note on version numbers

Three separate numbers show up in this project, and none of them track each other:

1. **Upstream Google OKF's own spec version**, `0.1` (June 2026), then `0.2` (July 2026,
   adding the trust/provenance/attestation vocabulary this document adopts below). This is
   Google's number, not this fork's.
2. **This skill's package version** (the `version:` field in `SKILL.md`'s frontmatter),
   its own release-numbering axis for the skill/plugin itself (bug fixes, secret-scanner
   hardening, Codex compatibility, and so on). It has no relationship to either spec version.
3. **This fork's own bundle-format version** (the `okf_version` marker every bundle-root
   `index.md` declares, and what `SUPPORTED_VERSIONS` in `validate.py` checks), `0.1`, then
   `0.2` (new allowed types), then `0.3` (datetime-form `timestamp`), and now `0.4` (this
   patch: the v0.2 trust/provenance fields below, and the required-key rename they force).

So "OKF v0.2" (upstream Google's spec) and "`okf_version: 0.2`" (a bundle declared under
*this fork's* second format revision, from months before Google's v0.2 existed) are two
unrelated things that happen to share a digit. Where this document says "upstream v0.2" it
means Google's; a bare "`okf_version: 0.4`" always means this fork's own marker.

## Relationship to upstream OKF

This spec is a strict fork of Google's Open Knowledge Format
([GoogleCloudPlatform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf)).
It keeps the core idea (knowledge as small markdown files with YAML frontmatter and
`index.md` navigation) and tightens the contract so a validator can enforce it. If you
already know upstream OKF, these are the intentional differences to author against;
upstream conventions will otherwise produce files this validator rejects.

- Required keys. Upstream requires only `type` and treats everything else as recommended,
  with any extra key allowed. Here, for a bundle declaring `okf_version` `0.1` through `0.3`,
  all seven of `type`, `title`, `description`, `source`, `verified`, `timestamp`, and `tags`
  are required and non-empty. For a bundle declaring `0.4`, `verified` is renamed to
  `verified_on` in that required list (see "Trust and provenance (upstream v0.2 vocabulary)"
  for why), so a `0.4` bundle requires `type`, `title`, `description`, `source`,
  `verified_on`, `timestamp`, and `tags` instead. Either way, a file that conforms upstream
  (say, `type` alone) fails validation here.
- `resource` becomes `source`. Upstream's optional `resource` is one canonical URI for the
  underlying asset. This spec drops `resource` and requires `source`, a non-empty list of
  provenance pointers (paths, commands, URLs, events). Upstream v0.2 separately introduces its
  own richer `sources` (plural), this fork adopts that too, as an optional companion to the
  required `source`; see below.
- Citations fold into `source`. Upstream lists sources under a `# Citations` heading and
  allows a `references/` subdirectory. Here there is neither; all provenance lives in the
  `source` frontmatter list (and, optionally, the richer `sources` list below).
- `verified`/`verified_on` is added. Upstream v0.1 has no such key. Through `okf_version`
  `0.3`, this spec requires `verified`, the ISO date the fact was last confirmed true, kept
  distinct from `timestamp` (when the file was authored or edited). At `0.4`, this fork's own
  field is renamed `verified_on` to free the name `verified` for upstream v0.2's own
  `verified`, a different thing (see below).
- `timestamp`'s datetime form. Upstream's `timestamp` is an ISO 8601 datetime, and
  `check_dates` accepts that form for `timestamp` in a `0.3`-or-later bundle (with or without
  an offset, including a trailing `Z`), so an upstream bundle validates here unchanged. A
  `0.1`/`0.2` bundle's `timestamp` stays date-only. `verified`/`verified_on` is always
  date-only, in every version: it records the day a fact was last confirmed true, and a time
  of day there is false precision about the confirmation.
- The type vocab is closed. Upstream types are freeform and unregistered, and consumers
  must tolerate unknown ones. Here the set is fixed (see Type vocab) and an unlisted type
  fails the build, which catches typos; extending it is a deliberate spec edit.
- Links are strict, and must be relative. Upstream treats a broken link as tolerable
  ("consumers MUST tolerate broken links") and permits bundle-root-relative targets like
  `/tables/customers.md`. Here every intra-bundle link must resolve or validation fails, the
  `[[slug]]` wikilink form is rejected outright, and any target beginning with `/` is
  rejected as a root-relative link, so an upstream `/tables/customers.md` must be rewritten
  relative to the file that links it.
- A root `index.md` declaring `okf_version` is mandatory. Upstream treats `index.md` and
  `okf_version` as optional. Here the bundle root must contain an `index.md` whose
  frontmatter declares `okf_version` (and carries nothing else), so an upstream bundle with
  no root index, or one that omits the version marker, fails validation.
- Secret values fail the build. Upstream OKF has no secret-scanning rule. Here `validate.py`
  scans every markdown file for private-key blocks, cloud-token shapes, and `secret=<value>`
  assignments and fails the bundle on a hit, so an upstream bundle that inlines a credential
  value passes upstream but is rejected here (see Security).
- Trust/provenance/attestation fields are adopted, not required. Upstream v0.2 adds
  `generated`, `sources`, `status`, `stale_after`, `verified` (the new, upstream shape), and
  an `Attested Computation` type, all of it optional there, and all of it optional here too.
  Adopting none of it leaves a bundle exactly as valid as before; see the dedicated section
  below.

This fork's own format version differs from upstream's on a separate axis (see "A note on
version numbers" above): upstream is now `0.1`/`0.2`, this fork's current bundle-format
version is `0.4` (the validator still accepts `0.1`, `0.2`, and `0.3`). Every difference
above pulls the same direction: upstream stays minimal and needs no tooling to stay
portable, while this fork adds a validator that fails the build when a bundle drifts.

## Bundle model

A bundle is a directory tree. The simplest bundle is one directory of concept files
with an `index.md`. Larger bundles group concepts into subdirectories by subject.

```
<bundle-root>/
  index.md                 carries okf_version, here and nowhere else
  <section>/
    index.md               navigation for the section
    <concept>.md           one concept per file
```

## Files

- The bundle-root `index.md` carries `okf_version` (one of `"0.1"`–`"0.4"`; `scaffold.py`
  still writes `"0.3"` by default, `"0.4"` only with `--trust-signals`) in frontmatter,
  only there.
- Per-directory `index.md`: a heading, an optional one-line preamble, then bullet
  navigation. No frontmatter. (Keep the preamble to one line, it orients, it doesn't narrate.)
- `log.md` (optional, per directory): dated entries, newest first, no frontmatter.
- Concept files: one concept each, frontmatter required (below).
- Reserved filenames: `index.md`, `log.md`.
- All markdown files use a lowercase `.md` extension. A non-lowercase extension (`.MD`,
  `.Md`) is non-conforming and rejected, the validator discovers files case-insensitively
  so such a file cannot escape validation, and link checks match `.md` case-insensitively too.

The validator enforces the frontmatter rules here, reserved files carry no frontmatter, and
only the bundle-root `index.md` carries `okf_version` (and nothing else). The index/log body
shapes above are recommendations for human readers, not validator-checked structure.

The current format version is `0.4`, but `scaffold.py` still emits `0.3` by default,
unchanged from before this patch, and only writes `0.4` when explicitly asked
(`scaffold.py --trust-signals`; see Tooling). The validator accepts `0.1` through `0.4`
either way, so a newer validator still reads an older bundle. Version `0.2` added allowed
types. Version `0.3` admitted a full datetime in `timestamp`. Version `0.4` renames
`verified` to `verified_on` (freeing `verified` for the new upstream-v0.2 shape) and adds the
optional trust/provenance fields below, a bundle that adopts none of them still only needs to
declare `0.4` because of the rename; one that keeps declaring `0.1`–`0.3` keeps the old
`verified` field exactly as before and simply cannot use the new optional fields (see next
section). The marker makes these grammar changes explicit, so an older validator reports a
clear unsupported-version error instead of a misleading field error.

## Concept frontmatter (required keys, all non-empty)

For a bundle declaring `okf_version` `0.1` through `0.3`:

| key | value |
| --- | --- |
| `type` | one of the type vocab below |
| `title` | the concept name |
| `description` | one line |
| `source` | YAML list of provenance pointers (paths, commands, URLs, events) |
| `verified` | ISO date `YYYY-MM-DD` the fact was last confirmed true (see note below) |
| `timestamp` | ISO date authored/updated, or in a `0.3` bundle a full ISO 8601 datetime |
| `tags` | YAML list |

For a bundle declaring `okf_version` `0.4`, the table is identical except `verified` is
named `verified_on` (same required-ness, same meaning, same date-only rule, only the key
name changes, to make room for the new `verified` described below).

`verified`/`verified_on` note: it records when the fact was last confirmed true, which is not
always today. A fact you re-checked against reality now is confirmed today, as is one the user
is the authority for, a decision, preference, or intent they state directly. But a fact the
user is recalling about external or system state is a source claim, not a re-check: date it to
when that state was last checked or to the recollection's own date, not today. A claim copied
from a dated source without re-checking carries that source's date. A fact taken from an
undated record you cannot re-confirm (a memory file, an old conversation) carries the oldest
date you can evidence, file timestamp, introducing commit, or the date it was said, never
today; if no date can be evidenced, the fact is not yet verifiable, so find a datable source or
leave it out. When the date is uncertain, round it down: an older `verified`/`verified_on`
reads as "may be stale," today reads as "just confirmed." The date is the contract; a caveat in
the concept body does not undo it, because the validator and tools read only the date.

`timestamp` may be an ISO date (`YYYY-MM-DD`) in every supported format version. A `0.3` or
`0.4` bundle may instead preserve a full datetime in the exact form
`YYYY-MM-DDTHH:MM:SS[.fraction][Z|+HH:MM|-HH:MM]`; a space may replace `T`. Versions `0.1`
and `0.2` remain date-only. `verified`/`verified_on` is always date-only.

`source` quoting rule (hard): QUOTE every element of the `source` list. Source pointers
routinely carry YAML-significant characters, a `#` (e.g. `"issue #445"`) starts a comment
and corrupts the flow sequence, a colon-space `: ` splits a mapping, so a strict parser
rejects an unquoted source. Always quote them:

```yaml
source: ["README.md", "issue #445", "git log 9c2e510"]
```

The validator enforces this in both list styles. In flow style (`["a", b]`) an unquoted
element carrying a significant character fails to parse and is reported as an error. In block
style (`- a`) YAML would silently drop an inline `#` comment and pass, so the validator also
scans the raw source text and rejects an unquoted element with a `#`. An element that is
already quote-safe (a bare filename) is accepted either way, quote everything anyway so you
never have to judge which is which.

`tags` and `description` follow a lighter rule: quote an element only when it contains a
YAML-significant character (a colon-space `: `, a leading `[ { # * & ! | > % @` or quote,
or a trailing `:`); plain kebab tokens like `canonical` may stay unquoted. Quoting when
unsure is always safe. Hard quoting is `source` only.

Provenance lives in `source`, there is no separate citations section or references directory.

## Trust and provenance (upstream v0.2 vocabulary)

Upstream Google OKF v0.2 (July 2026) added a second kind of frontmatter field: not one that
describes a concept, but one a consumer uses to decide whether to trust it before reading the
body. This fork adopts that vocabulary as optional additions, available on any concept
regardless of type, in a bundle declaring `okf_version` `0.4`. None of it is required. A
concept that uses none of these fields is exactly as valid as one authored against `0.1`.

As with upstream, this fork records the raw signals and leaves scoring to the consumer, there
is no computed trust score anywhere in a concept file or in the validator's output. A tool that
wants "only surface human-reviewed metrics" derives that filter itself from the fields below.

- **`generated`**, an optional mapping `{by, at}`: who or what produced the current content,
  and when it last meaningfully changed. `by` is a non-empty string identifying the producer
  (a model/agent name, or `human:<id>`); `at` is an ISO date or full ISO 8601 datetime. This
  sits alongside the required `timestamp`, not in place of it, `timestamp` is this fork's own
  authored/updated marker and stays required; `generated` is the richer, optional upstream
  form for describing production, and the two may describe the same event.
- **`verified`** (only meaningful in a `0.4` bundle, where it is not the required key, see
  above), an optional YAML list of independent confirmations, each a mapping
  `{by, at}`: `by` a non-empty string (a `human:<id>` actor or a machine/agent identifier), `at`
  an ISO date. A consumer derives a trust tier from this list: no `verified` key is
  *unverified*; every entry from a machine/agent actor only is *machine-confirmed*; any entry
  from a `human:<id>` actor is *human-reviewed*. The validator checks only that the list is
  well-formed (non-empty entries, valid dates), deriving and filtering on a tier is a
  consumer's job, not this format's.
- **`sources`** (plural, distinct from the required singular `source`), an optional YAML list
  of structured provenance objects, each with a required `id` (non-empty string, unique within
  the list, used to key an in-body footnote like `[^warehouse-schema]`) and `resource`
  (non-empty string: a URL or bundle-relative path), plus optional `title`, `author`
  (non-empty strings), `usage_count` (a non-negative integer), and `last_modified` (an ISO
  date). Where the required `source` is a flat list of quoted pointers, `sources` lets each
  pointer carry its own credibility signals and be cited per-claim in the body via an ordinary
  markdown footnote. The validator checks shape only, it does not cross-check that every
  `[^id]` footnote in the body has a matching `sources[].id`, or vice versa; treat that
  cross-reference as a human/reviewer responsibility for now.
- **`status`**, an optional string, one of `draft`, `stable`, `deprecated`. Absent means
  `stable`. A `deprecated` concept is kept for history/reproducibility but should not be
  surfaced to new work.
- **`stale_after`**, an optional ISO date `YYYY-MM-DD`. An absolute date, deliberately, not a
  relative TTL: staleness is then a plain date comparison with no reference to when the
  concept happened to be read.

None of the above changes what the validator requires; it only makes the *absence* of these
fields distinguishable from their presence where they matter to a consumer deciding whether to
act on a concept. Absence means the signal was not supplied. A trust field that is explicitly
present with a YAML null value is invalid; present fields must have the shape documented above.

## Type vocab

Infrastructure and ops (fleet maps, system docs): `Machine`, `Network`, `Service`,
`Session`, `Project`, `Repo`, `Credential`, `Path`, `Process`.

Domain-neutral (newsrooms, research atlases, decision logs): `Concept`, `Decision`,
`Event`, `Person`, `Org`, `Source`.

`Reference` is the catch-all for a concept that is not one of the others. Index files carry
no frontmatter, so there is no `Index` type. The set is closed: an unlisted type fails the
build, which catches typos. To extend it, add the type here and in `scripts/validate.py`.
`Attested Computation` joins this closed vocabulary only in a bundle declaring
`okf_version` `0.4`; versions `0.1` through `0.3` reject it.

### `Attested Computation` (upstream v0.2)

A concept of this type carries a sanctioned way to compute a value, and the means to check
that the sanctioned computation actually ran, the answer to "was this number produced the way
we said it must be," distinct from `verified` above (which confirms a *definition* still
matches policy, not that any one run produced a correct value). It carries these keys in
addition to the seven (or, at `0.4`, six-plus-`verified_on`) base required keys:

| key | value |
| --- | --- |
| `runtime` | non-empty string naming the execution environment (e.g. `bigquery`) |
| `parameters` | YAML list (possibly empty) of mappings, each `{name, type, required}`, the declared inputs a caller may fill; nothing else |
| `executor` | mapping `{resource, receipt}`, `resource` points at the skill/tool that runs the computation, `receipt` a list naming what it returns (e.g. `[job_id, executed_sql, result]`) |
| `attester` | mapping `{resource}`, points at the deterministic, non-LLM checker that compares a receipt against this concept's sanctioned computation |

OKF records the computation and how to check it; this fork's validator checks only that these
four keys are present and correctly shaped when `type: Attested Computation`. It never runs
the computation, the executor, or the attester itself, that is a consumer's runtime
responsibility, entirely outside this format and this validator.

## Links

Relative markdown links. Every link to a file inside the bundle must resolve to a file that
exists; a link that escapes the bundle root or dangles fails validation. The bundle is
validated as one self-contained tree (see Federation for combining several).

A link's case must match the file on disk. Case-insensitive filesystems on macOS
and Windows can resolve `Concepts/Foo.md` to `concepts/foo.md`, while the same
link dangles on Linux. The validator checks the spelling of each link component
against directory listings before resolving can replace it with canonical casing.
Its error gives the corrected relative link. Symlink and `..` components stay in
filesystem traversal order, and the resolved destination must remain inside the
bundle root.

The `[[slug]]` wikilink form is not an OKF link, and the validator rejects it. It is the
auto-memory cross-reference idiom and easy to reach for by habit, but a `[[slug]]` is never
resolved or checked, so a dead reference would pass silently. Always link with
`[text](relative/path.md)`.

## Federation (optional)

Several bundles can be combined into one tree. Add a new root `index.md` that carries
`okf_version` and links to each member, then place each bundle under a uniquely named
subdirectory of that root. A member's own `index.md` is now a nested section index, so remove
its `okf_version` frontmatter block entirely, a nested `index.md` carries no frontmatter at
all. Write cross-bundle links as relative paths into the sibling directories. Validate by
pointing the validator at the new root, so every link resolves and the single `okf_version`
gate runs once at the top.

A member's marker is stripped when it is nested, so it can no longer be validated on its own
from inside the combined tree, validate the assembled root instead. (Per-node validation that
keeps a marker in each member is planned but not yet built.) Most single-repo knowledge bases
never need any of this.

## Security (hard)

- No secret VALUES anywhere. A credential concept documents the key name, where it lives, and
  how it is retrieved, never the value itself.
- The validator scans for private-key blocks, cloud-token shapes, and `secret=<value>`
  assignments and fails the build on a hit. If a pattern false-positives on legitimate text,
  narrow the pattern; do not delete the rule.
- OKF makes no claim about whether your bundle is public or private. That is your decision,
  but a bundle that documents real infrastructure is usually internal. Decide deliberately
  before publishing.

## Tooling

- `validate.py`, frontmatter conformance, date/list checks, link resolution, secret scan.
  Run it before every commit; it must exit 0.
- `scaffold.py`, generate a conforming starter bundle.

#!/usr/bin/env python3
"""gh-wiki-bootstrap.py, create the FIRST page of an empty GitHub wiki.

Why this exists: a repo with the wiki feature enabled but zero pages has no
`<repo>.wiki.git` repo yet. `git clone`/`git push` both fail with "Repository
not found," and GitHub exposes no REST API for wiki content. The only way to
create the first page is the web UI. After that one page exists, the wiki is a
normal git repo: clone it, add pages, push.

So this script does the minimum web-UI step, create one page (default "Home")
via Playwright using the saved GitHub web session, and prints the wiki URL.
Everything after (real content, multiple pages) should go through git.

Auth: reuses the saved Playwright storageState at ~/.cache/gh_state.json, the
same session kept alive by gh-session-keepwarm.py. The github PAT does NOT work
for this, wiki pages are a web-UI-only surface.

Usage:
  gh-wiki-bootstrap.py owner/repo
  gh-wiki-bootstrap.py owner/repo --title Home --body "Initializing wiki."
  gh-wiki-bootstrap.py owner/repo --headed   # watch it run, for debugging

Exit codes: 0 ok, 2 not authenticated, 3 body editor not found,
4 save button not found, 5 bad arguments, 6 save not confirmed.

Then:  git clone https://github.com/owner/repo.wiki.git
"""
import argparse
import os
import re
import sys
from urllib.parse import urlsplit

DEFAULT_STATE = os.path.expanduser("~/.cache/gh_state.json")


def _still_on_editor(url):
    """True while the browser is still sitting on the new-page editor.

    The new-wiki-page editor lives at the path `/<owner>/<repo>/wiki/_new` and,
    on a successful save, GitHub redirects to `/<owner>/<repo>/wiki/<slug>`. The
    save-success signal is leaving that editor path. Testing the whole URL for
    the substring "_new" misfires whenever "_new" appears anywhere else in it,
    the owner, the repo name (e.g. owner/service_new), or a saved page's slug:
    the redirected URL still contains the substring, so a real save reads as a
    failure and the tool wrongly reports it never saved. Match the editor by its
    path suffix instead, ignoring a trailing slash and any query or fragment.
    """
    return urlsplit(url).path.rstrip("/").endswith("/wiki/_new")


def parse_args():
    ap = argparse.ArgumentParser(description="Create the first page of an empty GitHub wiki.")
    ap.add_argument("repo", help="target repo as owner/name, e.g. jamditis/ccm")
    ap.add_argument("--title", default="Home", help="first page title (default: Home)")
    ap.add_argument("--body", default="Initializing wiki.", help="first page body text")
    ap.add_argument("--state", default=DEFAULT_STATE, help="Playwright storageState json (default: ~/.cache/gh_state.json)")
    ap.add_argument("--headed", action="store_true", help="run with a visible browser (debugging)")
    ap.add_argument("--screenshot-dir", default=None, help="if set, save before/after screenshots here")
    args = ap.parse_args()
    if not re.fullmatch(r"[^/\s]+/[^/\s]+", args.repo):
        ap.error("repo must look like owner/name")
    if not os.path.exists(args.state):
        ap.error(f"session state not found: {args.state} (run gh-session-keepwarm.py or re-auth)")
    return args


def shot(page, screenshot_dir, name):
    if screenshot_dir:
        os.makedirs(screenshot_dir, exist_ok=True)
        page.screenshot(path=os.path.join(screenshot_dir, name))


def main():
    args = parse_args()
    # Imported here so --help works without playwright installed.
    from playwright.sync_api import sync_playwright, TimeoutError as PWTimeout

    new_url = f"https://github.com/{args.repo}/wiki/_new"
    wiki_url = f"https://github.com/{args.repo}/wiki"

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=not args.headed)
        ctx = browser.new_context(storage_state=args.state)
        page = ctx.new_page()
        page.goto(new_url, wait_until="domcontentloaded", timeout=45000)
        url = page.url
        print("landed on:", url)
        if "/login" in url:
            print("error: not authenticated (redirected to login); refresh the session", file=sys.stderr)
            browser.close()
            sys.exit(2)

        # Wait for the editor to actually render rather than sleeping a fixed
        # interval, so a slow page load is not mistaken for a missing control.
        title_sel = "input#wiki_page_title, input[name='wiki[name]'], input[name='wiki[title]']"
        try:
            page.wait_for_selector(title_sel, state="visible", timeout=20000)
        except PWTimeout:
            pass  # editor never appeared; the body-editor check below reports it
        shot(page, args.screenshot_dir, "wiki_01_new.png")

        # Title field (often prefilled with "Home" on the first page).
        for sel in ["input#wiki_page_title", "input[name='wiki[name]']", "input[name='wiki[title]']"]:
            if page.locator(sel).count():
                page.fill(sel, args.title)
                print("title set via", sel)
                break

        # Body: plain textarea (classic gollum editor) first, then CodeMirror.
        body_filled = False
        if page.locator("textarea#gollum-editor-body").count():
            page.fill("textarea#gollum-editor-body", args.body)
            body_filled = True
            print("body filled via textarea#gollum-editor-body")
        else:
            for sel in [".CodeMirror textarea", ".cm-content", "div[contenteditable='true']", "textarea[name='wiki[content]']"]:
                if page.locator(sel).count():
                    page.click(sel)
                    page.keyboard.type(args.body)
                    body_filled = True
                    print("body typed via", sel)
                    break

        shot(page, args.screenshot_dir, "wiki_02_filled.png")
        if not body_filled:
            print("error: could not locate the wiki body editor", file=sys.stderr)
            browser.close()
            sys.exit(3)

        # Save.
        clicked = False
        for sel in ["button:has-text('Save Page')", "button[name='commit']", "input[type='submit'][value*='Save']", "button:has-text('Save')"]:
            if page.locator(sel).count():
                page.locator(sel).first.click()
                clicked = True
                print("clicked save via", sel)
                break
        if not clicked:
            print("error: could not find the Save button", file=sys.stderr)
            browser.close()
            sys.exit(4)

        # On success GitHub redirects away from the editor (.../_new) to the
        # created page. Wait for that navigation rather than a fixed sleep, so a
        # slow-but-successful save is not mistaken for a failure.
        saved = True
        try:
            page.wait_for_url(lambda u: not _still_on_editor(u), timeout=20000)
        except PWTimeout:
            saved = False
        final_url = page.url
        shot(page, args.screenshot_dir, "wiki_03_saved.png")
        print("final url:", final_url)
        browser.close()

    # If we never left the editor, the save was rejected (no wiki write access,
    # bad title, inline validation error), do not report success.
    if not saved or _still_on_editor(final_url):
        print(f"error: wiki editor never redirected to a created page ({final_url}), "
              "save not confirmed. Check the saved session has wiki write access and "
              "the title is valid.", file=sys.stderr)
        sys.exit(6)

    print("wiki bootstrapped. next:")
    print(f"  git clone https://github.com/{args.repo}.wiki.git")
    print(f"  open: {wiki_url}")


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""scaffold.py: generate a conforming OKF (Open Knowledge Format) starter bundle.

Creates a project that passes its own validator by construction:

    <target>/
      SPEC.md                 the format contract (copied from the skill)
      README.md               how to use and validate this bundle
      scripts/validate.py     the validator (copied from the skill)
      .claude/                session hooks that orient Claude on the bundle
        settings.json         registers the hooks (Claude Code asks you to approve once)
        hooks/okf-anchor.py   SessionStart: load the index into context
        hooks/okf-orient.py   PreToolUse: gate the first action on orientation
      bundle/                 the OKF bundle (this is what gets validated)
        index.md              carries okf_version
        <section>/
          index.md            section navigation
          example-concept.md  a starter concept with full frontmatter

SPEC.md and README.md live at the project root, NOT inside bundle/, because the
validator treats every non-reserved .md inside the bundle as a concept that needs
frontmatter. Keeping docs out of bundle/ means a fresh scaffold validates clean.

The .claude/ hooks are one cross-platform python3 script each; only the launch
command in settings.json differs per OS (python3 on macOS/Linux, python on Windows).
Claude Code treats a checked-in .claude/settings.json as untrusted, so the user
approves the hooks once on first session open. See SKILL.md for the hook contract.

Usage:
  scaffold.py ./my-knowledge-base
  scaffold.py ./kb --title "Team knowledge base" --sections concepts,services,decisions
  scaffold.py ./kb --no-validate        # skip the post-scaffold validation run
  scaffold.py ./kb --no-hooks           # do not write the .claude/ session hooks
  scaffold.py ./kb --hooks-os windows   # force the Windows launch command (default: this OS)
"""
from __future__ import annotations

import argparse
import datetime as dt
import importlib.util
import json
import platform
import re
import shlex
import shutil
import stat
import subprocess
import sys
from pathlib import Path

SKILL_ROOT = Path(__file__).resolve().parent.parent
SRC_SPEC = SKILL_ROOT / "spec" / "SPEC.md"
SRC_VALIDATOR = SKILL_ROOT / "scripts" / "validate.py"
SRC_HOOKS = SKILL_ROOT / "templates" / "hooks"
HOOK_SCRIPTS = ("okf-anchor.py", "okf-orient.py")
# The exact hook-command paths this scaffold writes (see cmd() in claude_settings).
# We identify our own hook entries by exact match on these strings, so a re-run
# replaces only what we generated and never a user's hook that merely lives at a
# similarly named path elsewhere (e.g. /opt/shared/.claude/hooks/okf-anchor.py).
OKF_HOOK_PATHS = frozenset(f"${{CLAUDE_PROJECT_DIR}}/.claude/hooks/{s}" for s in HOOK_SCRIPTS)

# validate.py imports PyYAML to parse frontmatter; a scaffolded project declares it
# so the dependency is explicit, not discovered via a traceback. Mirrors the skill's
# own requirements.txt.
REQUIREMENTS_TXT = (
    "# The validator (scripts/validate.py) parses YAML frontmatter with PyYAML.\n"
    "PyYAML>=5.1\n"
)


def slugify(name: str) -> str:
    return "-".join("".join(c if c.isalnum() else " " for c in name.lower()).split())


def write_if_absent(path: Path, content: str, preserved: list[Path]) -> None:
    """Write content unless the file already exists. On --force into a populated
    directory an existing file is the user's own, so preserve it instead of
    clobbering it with a generic template. On a fresh scaffold nothing exists, so
    every file is written as before."""
    if path.exists():
        preserved.append(path)
        return
    path.write_text(content, encoding="utf-8")


def copy_if_absent(src: Path, dst: Path, preserved: list[Path]) -> None:
    """copy2 src to dst unless dst exists (see write_if_absent for the rationale)."""
    if dst.exists():
        preserved.append(dst)
        return
    shutil.copy2(src, dst)


def root_index(title: str, sections: list[str], version: str = "0.3") -> str:
    nav = "\n".join(f"- [{s}]({s}/index.md)" for s in sections)
    return (
        f'---\nokf_version: "{version}"\n---\n'
        f"# {title}\n\n"
        "An Open Knowledge Format bundle. One concept per file; provenance in each file's frontmatter.\n\n"
        "## Sections\n\n"
        f"{nav}\n"
    )


def section_index(section: str) -> str:
    return (
        f"# {section}\n\n"
        f"Concepts in the {section} section.\n\n"
        "- [example concept](example-concept.md)\n"
    )


def example_concept(today: str, trust_signals: bool = False) -> str:
    # Pre-0.4, this fork's own field is named 'verified'. At 0.4 it is renamed
    # 'verified_on' to free 'verified' for upstream v0.2's own (differently
    # shaped) field of the same name -- see SPEC.md's "Trust and provenance"
    # section and validate.py's REQUIRED_KEYS_V04.
    verified_key = "verified_on" if trust_signals else "verified"
    return (
        "---\n"
        "type: Reference\n"
        "title: Example concept\n"
        "description: A starter concept showing the OKF frontmatter contract.\n"
        'source: ["SPEC.md", "scaffold.py"]\n'
        f"{verified_key}: {today}\n"
        f"timestamp: {today}\n"
        "tags: [example, starter]\n"
        "---\n"
        "# Example concept\n\n"
        "Replace this file with a real concept. Keep it to one concept per file.\n\n"
        "- Point every `source` element at real provenance (a path, command, URL, or event).\n"
        f"- Update `{verified_key}` when you re-check the fact against reality.\n"
        "- Link related concepts with relative markdown links, like this one to [the section index](index.md).\n"
    )


def readme(title: str, hooks: bool, interp: str = "python3", trust_signals: bool = False) -> str:
    verified_key = "verified_on" if trust_signals else "verified"
    hooks_section = (
        "## Session hooks\n\n"
        "`.claude/` ships two hooks that orient Claude on this knowledge base before it works:\n\n"
        "- `okf-anchor.py` (SessionStart) loads the bundle index into the session context.\n"
        "- `okf-orient.py` (PreToolUse) blocks the first action once per session until Claude\n"
        "  confirms it has read the index, then unblocks for the rest of the session.\n\n"
        "They are one cross-platform python3 script each. No single interpreter name works on\n"
        "every OS, so `settings.json` names the one for the OS this bundle was scaffolded on\n"
        "(`python3` on macOS/Linux, `python` on Windows). If you move the bundle to a different\n"
        "OS, change that one token in `settings.json` (or re-run the scaffolder there). Claude\n"
        "Code treats a checked-in `.claude/settings.json` as untrusted, so the first time you\n"
        "open this project it asks you to approve the hooks. They run automatically after that.\n"
        "Delete `.claude/` (or set `disableAllHooks`) to turn them off.\n\n"
    ) if hooks else ""
    return (
        f"# {title}\n\n"
        "An Open Knowledge Format (OKF) knowledge base: small markdown files, one concept each,\n"
        "with provenance in YAML frontmatter. See `SPEC.md` for the full contract.\n\n"
        "## Requirements\n\n"
        "The validator parses YAML frontmatter with PyYAML:\n\n"
        "```bash\n"
        "pip install -r requirements.txt    # or: pip install pyyaml\n"
        "```\n\n"
        "## Validate\n\n"
        "```bash\n"
        f"{interp} scripts/validate.py --bundle bundle\n"
        "```\n\n"
        "It must exit 0. Run it before every commit.\n\n"
        "## Add a concept\n\n"
        "1. Create `bundle/<section>/<concept>.md` with the required frontmatter\n"
        f"   (`type, title, description, source, {verified_key}, timestamp, tags`).\n"
        "2. Add a bullet for it in that section's `index.md`.\n"
        "3. Validate.\n\n"
        f"{hooks_section}"
        "## Security\n\n"
        "Never put secret values in a concept. A credential concept documents the key name and\n"
        "where it is retrieved, not the value. The validator scans for leaked secrets and fails on a hit.\n"
    )


def resolve_hooks_os(choice: str) -> str:
    """Map --hooks-os (auto|posix|windows) to the concrete launch target."""
    if choice == "auto":
        return "windows" if platform.system() == "Windows" else "posix"
    return choice


def interpreter_for(hooks_os: str) -> str:
    """The python command for an OS. No single name works everywhere: macOS/Linux
    have python3, stock Windows has python (python3 is usually absent there)."""
    return "python" if hooks_os == "windows" else "python3"


def claude_settings(hooks_os: str) -> dict:
    """Build the .claude/settings.json that registers the orientation hooks.

    The hook scripts are cross-platform python3; only the interpreter changes per OS
    (python3 on macOS/Linux, python on Windows). Each hook uses exec form -- the
    interpreter as `command` and the script path as a single `args` element -- which
    Claude Code spawns directly with no shell, so a project path with spaces or other
    special characters needs no quoting (the hooks docs recommend exec form for any
    hook that references a path placeholder). The path uses the ${CLAUDE_PROJECT_DIR}
    placeholder so it resolves even when the hook cwd has drifted from the project
    root, and stays correct when the bundle is cloned to another path -- unlike a
    baked-in absolute path. PreToolUse omits a matcher so it sees the first tool call
    of any kind.
    """
    interp = interpreter_for(hooks_os)

    def cmd(script: str) -> dict:
        path = f"${{CLAUDE_PROJECT_DIR}}/.claude/hooks/{script}"
        # exec form: the script path is one args element, spawned with no shell, so a
        # project path with spaces or special characters needs no quoting.
        return {"type": "command", "command": interp, "args": [path]}

    return {
        "hooks": {
            "SessionStart": [{"hooks": [cmd("okf-anchor.py")]}],
            "PreToolUse": [{"hooks": [cmd("okf-orient.py")]}],
        }
    }


def _is_okf_hook(h: dict) -> bool:
    """True if a hook entry launches one of the OKF orientation scripts we generate.

    We match the exact project-local path we write (`${CLAUDE_PROJECT_DIR}/.claude/
    hooks/<script>`), found either as an exec-form `args` element or, after shlex-
    splitting, as a token of a shell-form `command` string. Exact matching against
    OKF_HOOK_PATHS -- not a suffix or substring test -- means a re-run replaces only
    our own entries and never a user's hook that merely ends in the same filename
    (`okf-anchor.py.bak`) or sits at a different absolute path. The check is total: any
    hook shape returns a bool and never raises, so a malformed entry (e.g. a non-list
    `args`) is simply judged "not ours" and left in place rather than crashing --force.
    """
    args = h.get("args")
    tokens = list(args) if isinstance(args, (list, tuple)) else []
    cmd = h.get("command")
    if isinstance(cmd, str):
        try:
            tokens.extend(shlex.split(cmd))
        except ValueError:
            pass  # unbalanced quotes: nothing we generate parses to this
    return any(str(t) in OKF_HOOK_PATHS for t in tokens)


def merge_hook_settings(existing: dict, new: dict) -> dict:
    """Merge our hook groups into existing settings without losing anything recoverable.

    Every top-level key (e.g. permissions) and every unrelated hook event is preserved.
    Within an event we register, only our own hook entries are stripped: a group that
    also holds the user's hooks keeps them, and a group is dropped only when it empties.
    Then our fresh group is appended, so a re-run is idempotent and never deletes a
    user's hook that shared a group with ours. The function is total -- a non-dict
    `hooks`, a non-list event value, or a malformed group/entry holds no mergeable hook
    config and is treated as empty, so no input shape raises. write_claude_hooks backs
    up the original first when it had to reset such a malformed subtree.
    """
    merged = dict(existing)
    eh = merged.get("hooks")
    hooks = dict(eh) if isinstance(eh, dict) else {}
    for event, groups in new["hooks"].items():
        prior = hooks.get(event)
        prior = prior if isinstance(prior, list) else []
        kept = []
        for g in prior:
            inner = g.get("hooks") if isinstance(g, dict) else None
            if not isinstance(inner, list):
                kept.append(g)  # a shape we don't manage -- leave it untouched
                continue
            remaining = [h for h in inner if not (isinstance(h, dict) and _is_okf_hook(h))]
            if len(remaining) == len(inner):
                kept.append(g)                          # no OKF hooks here
            elif remaining:
                kept.append({**g, "hooks": remaining})  # keep the user's hooks
            # else: the group held only our hooks -- drop it
        hooks[event] = kept + list(groups)
    merged["hooks"] = hooks
    return merged


def write_claude_hooks(target: Path, hooks_os: str) -> None:
    """Copy the hook scripts into target/.claude/hooks and write settings.json.

    If settings.json already exists (e.g. --force into a project that already uses
    Claude Code), merge the OKF hooks into it rather than overwriting: the user's other
    settings and hook events survive, and re-running replaces only our own entries. A
    file that is not a JSON object at all is backed up to settings.json.bak and replaced.
    A JSON object whose hook subtree is malformed is repaired in place -- unrelated keys
    stay in the live file and the original is copied to settings.json.bak first, so the
    malformed copy is still recoverable.
    """
    hooks_dir = target / ".claude" / "hooks"
    hooks_dir.mkdir(parents=True, exist_ok=True)
    for script in HOOK_SCRIPTS:
        dest = hooks_dir / script
        shutil.copy2(SRC_HOOKS / script, dest)
        # make executable for the shebang case; exec form also names the interpreter
        # explicitly, so this is belt-and-suspenders.
        dest.chmod(dest.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)

    settings_path = target / ".claude" / "settings.json"
    new_settings = claude_settings(hooks_os)
    if settings_path.exists():
        try:
            existing = json.loads(settings_path.read_text(encoding="utf-8"))
        except (json.JSONDecodeError, OSError, UnicodeDecodeError):
            existing = None
        if isinstance(existing, dict):
            # The merge preserves every top-level key and unrelated event. If the hook
            # subtree itself is malformed (hooks not an object, or an event we write into
            # not a list), the merge resets just that subtree -- so copy the original to
            # .bak first to keep the malformed version recoverable, then merge in place so
            # unrelated settings (e.g. permissions) stay in the live file.
            existing_hooks = existing.get("hooks")
            clean = (existing_hooks is None or isinstance(existing_hooks, dict)) and (
                existing_hooks is None or all(
                    isinstance(existing_hooks.get(ev), (list, type(None)))
                    for ev in new_settings["hooks"]
                )
            )
            if not clean:
                backup = settings_path.with_name(settings_path.name + ".bak")
                shutil.copy2(settings_path, backup)
                print(f"warning: {settings_path.name} had malformed hook settings; "
                      f"repaired in place, original backed up to {backup.name}", file=sys.stderr)
            new_settings = merge_hook_settings(existing, new_settings)
        else:
            backup = settings_path.with_name(settings_path.name + ".bak")
            settings_path.replace(backup)
            print(f"warning: existing {settings_path.name} could not be merged "
                  f"(not a JSON object); backed up to {backup.name}", file=sys.stderr)
    settings_path.write_text(json.dumps(new_settings, indent=2) + "\n", encoding="utf-8")


def _canonical_req_name(line: str) -> str | None:
    """The PEP 503-normalized distribution name a requirements.txt line pins, or None
    when the line names no installable package (blank, comment, option, URL, or path).
    Used only to spot whether PyYAML is declared, so it is a name sniffer, not a full
    requirements parser."""
    s = line.split("#", 1)[0].strip()  # drop full-line and inline comments
    if not s or s.startswith("-") or s.startswith("."):
        return None  # option/include (-r, -e, ...) or a local path, not a bare name
    # The project name is the leading token, ending at the first version specifier,
    # marker, extras bracket, whitespace, or "@" direct-reference separator.
    name = re.split(r"[<>=!~;,@\[ (]", s, maxsplit=1)[0].strip()
    if not name or "://" in name or "/" in name:
        return None  # a bare URL or path names no distribution we can normalize
    return re.sub(r"[-_.]+", "-", name).lower()


def _strip_pip_options(text: str) -> str:
    """Drop pip's per-requirement options (`--hash`, `--global-option`, ...) from a
    requirements line so packaging.requirements.Requirement, which accepts only PEP 508 and
    rejects these options, can parse the requirement. Mirrors pip's own break_args_options:
    split on spaces and keep tokens up to the first that starts with '-'. Rejoining the
    original space-split tokens preserves the requirement's quotes (unlike shlex, which would
    strip a marker string's quotes and make a valid marker unparseable); a hash digest and a
    marker value never begin a bare '-' token, so the requirement is kept whole while the
    trailing options are dropped."""
    kept = []
    for token in text.split(" "):
        if token.startswith("-"):
            break
        kept.append(token)
    return " ".join(kept)


def _requirement_marker_selects_env(line: str) -> bool:
    """True if a requirements line has no PEP 508 environment marker, or one that selects the
    current interpreter, so pip would install it here. A marker that excludes this
    environment returns False: pip installs nothing from that line, so a PyYAML declaration
    gated to another platform (`PyYAML; sys_platform == "win32"` read on Linux) does not give
    validate.py the yaml module it imports on every platform. The full PEP 508 grammar (name,
    extras, direct-reference URL, marker) is parsed by `packaging`, so a '#' inside a quoted
    marker or a ';' inside a URL is not mistaken for a comment or the marker separator; pip's
    per-requirement options (`--hash` from a --generate-hashes lock file, etc.) are stripped
    first, since packaging rejects them. A trailing pip inline comment (which PEP 508 does not
    define) is only removed if the whole line fails to parse, so a '#' that is really inside a
    quoted marker value survives. If packaging is absent or the line does not parse as a
    requirement, returns True, so an unjudgeable line never becomes a false 'missing PyYAML'
    note (the behavior from before markers were read)."""
    # Drop pip per-requirement options (--hash, ...) first so packaging can parse the marker:
    # each is a space-separated token starting with '-', never part of a name or marker value.
    text = _strip_pip_options(line.strip())
    if not text:
        return True
    try:
        from packaging.requirements import Requirement, InvalidRequirement
    except ImportError:
        return True  # cannot parse without packaging: assume it installs, do not nag
    # A '#' inside a quoted marker value is valid PEP 508, so parse the whole line first; only
    # if that fails treat a trailing whitespace-'#' as a pip inline comment, strip it, and retry
    # -- removing a real comment without corrupting a marker value that legitimately holds '#'.
    try:
        req = Requirement(text)
    except InvalidRequirement:
        stripped = re.sub(r"(^|\s)#.*$", "", text).strip()
        if not stripped or stripped == text:
            return True  # unparseable even without a trailing comment: do not nag
        try:
            req = Requirement(stripped)
        except InvalidRequirement:
            return True  # not a parseable requirement line: do not nag
    if req.marker is None:
        return True  # no marker: the line always applies
    try:
        return bool(req.marker.evaluate())
    except Exception:
        # Any evaluation failure means the marker cannot be judged in this context: an undefined
        # variable or comparison, or a lock-file-only variable (dependency_groups) that raises a
        # raw KeyError in the default metadata context. Fall back to treating the line as
        # installable instead of crashing the --force path. The function is fail-open, so an
        # unjudgeable line never becomes a false missing-PyYAML note.
        return True


def _join_continuations(text: str):
    """Yield logical requirement lines, joining pip's backslash line-continuations the way
    pip's requirements parser does: a physical line ending in '\\' continues onto the next, so
    a hash-locked entry (its marker on the first physical line, indented `--hash` options on
    the following lines) is reassembled into one logical line before its name, directives, and
    marker are read. Without it, the first physical line keeps a trailing '\\' that breaks
    marker parsing, and a marker split across the continuation would be missed entirely. A
    full-line comment never continues, even when it ends in '\\': pip's parser treats it as a
    standalone comment, so it is emitted on its own rather than swallowing the next line (which
    would hide a PyYAML declaration or an -r/-e include directive)."""
    buf = []
    for raw in text.splitlines():
        if raw.lstrip().startswith("#"):
            if buf:  # flush a pending continuation before the standalone comment
                yield "".join(buf)
                buf = []
            yield raw
            continue
        if raw.endswith("\\"):
            buf.append(raw[:-1])
            continue
        buf.append(raw)
        yield "".join(buf)
        buf = []
    if buf:  # a final physical line ending in '\' with nothing after it
        yield "".join(buf)


def _preserved_requirements_lacks_pyyaml(path: Path) -> bool:
    """True only when we can say with confidence that a preserved requirements.txt does
    not declare an installable PyYAML: it is readable, no line names PyYAML that would
    install here, and it carries no include or editable directive (-r, -e) that could pull
    PyYAML from a file we do not read. An unreadable file or such a directive returns False,
    so the targeted warning fires only when the gap is certain and never nags a user who did
    declare it. A constraint file (-c/--constraint) only pins versions of packages installed
    elsewhere and cannot supply a missing one, so it does not suppress the note. A PyYAML
    line gated by a PEP 508 environment marker counts only when the marker selects this
    interpreter: one gated to another platform installs nothing here, so it does not (see
    _requirement_marker_selects_env). Physical lines are first joined on pip's backslash
    continuations, so a hash-locked entry split across lines is read as one requirement."""
    try:
        text = path.read_text(encoding="utf-8")
    except (OSError, UnicodeDecodeError):
        return False
    for raw in _join_continuations(text):
        s = raw.strip()
        if s.startswith(("-r", "--requirement", "-e", "--editable")):
            return False  # PyYAML may live in the included (-r) or editable (-e) target
        if _canonical_req_name(raw) == "pyyaml" and _requirement_marker_selects_env(raw):
            return False  # PyYAML declared and its marker (if any) installs here
    return True


def main() -> int:
    ap = argparse.ArgumentParser(description="Generate a conforming OKF starter bundle.")
    ap.add_argument("target", help="directory to create the project in")
    ap.add_argument("--title", default=None, help="bundle title (default: derived from target dir name)")
    ap.add_argument("--sections", default="concepts",
                    help="comma-separated section names (default: concepts)")
    ap.add_argument("--date", default=None, help="ISO date for sample frontmatter (default: today)")
    ap.add_argument("--force", action="store_true", help="write into a non-empty target directory")
    ap.add_argument("--no-validate", action="store_true", help="skip the post-scaffold validation run")
    ap.add_argument("--no-hooks", action="store_true", help="do not write the .claude/ session hooks")
    ap.add_argument("--hooks-os", choices=("auto", "posix", "windows"), default="auto",
                    help="launch command for the hooks (default: auto-detect this OS)")
    ap.add_argument(
        "--trust-signals", action="store_true",
        help="scaffold under okf_version 0.4: renames 'verified' to 'verified_on' and "
             "enables the optional upstream-v0.2 trust/provenance fields (generated, "
             "the new 'verified', sources, status, stale_after, Attested Computation). "
             "Default is okf_version 0.3, unchanged from before this flag existed.")
    args = ap.parse_args()

    if not SRC_SPEC.exists() or not SRC_VALIDATOR.exists():
        print(f"FAIL: skill assets missing ({SRC_SPEC} / {SRC_VALIDATOR})", file=sys.stderr)
        return 1
    if not args.no_hooks and not all((SRC_HOOKS / s).exists() for s in HOOK_SCRIPTS):
        print(f"FAIL: hook templates missing in {SRC_HOOKS} (or pass --no-hooks)", file=sys.stderr)
        return 1

    target = Path(args.target).resolve()
    if target.exists() and any(target.iterdir()) and not args.force:
        print(f"FAIL: {target} exists and is not empty (use --force to write anyway)", file=sys.stderr)
        return 1

    title = args.title or Path(args.target).name.replace("-", " ").replace("_", " ").strip() or "knowledge base"
    # A section name that is all punctuation slugifies to "" and would write its
    # index to bundle/index.md, clobbering the root index. Reject those, and
    # dedupe so a repeated name does not overwrite a directory mid-loop.
    sections: list[str] = []
    for raw in args.sections.split(","):
        if not raw.strip():
            continue
        slug = slugify(raw)
        if not slug:
            print(f"FAIL: section name {raw!r} has no alphanumeric characters", file=sys.stderr)
            return 1
        if slug not in sections:
            sections.append(slug)
    if not sections:
        print("FAIL: at least one section is required", file=sys.stderr)
        return 1
    today = args.date or dt.date.today().isoformat()
    try:
        dt.datetime.strptime(today, "%Y-%m-%d")
    except ValueError:
        print(f"FAIL: --date must be ISO YYYY-MM-DD, got {today!r}", file=sys.stderr)
        return 1

    bundle = target / "bundle"
    (target / "scripts").mkdir(parents=True, exist_ok=True)
    bundle.mkdir(parents=True, exist_ok=True)

    write_hooks = not args.no_hooks
    hooks_os = resolve_hooks_os(args.hooks_os)
    version = "0.4" if args.trust_signals else "0.3"

    # Skip-if-exists so --force into a populated directory never overwrites a user's
    # own SPEC.md/README.md/validator/bundle content with a generic template. (A
    # fresh scaffold has none of these, so all are written.) The .claude/ hooks are
    # stateless machinery, refreshed in place; settings.json is merged with backup.
    preserved: list[Path] = []
    copy_if_absent(SRC_SPEC, target / "SPEC.md", preserved)
    copy_if_absent(SRC_VALIDATOR, target / "scripts" / "validate.py", preserved)
    write_if_absent(target / "requirements.txt", REQUIREMENTS_TXT, preserved)
    write_if_absent(target / "README.md",
                    readme(title, write_hooks, interpreter_for(hooks_os), args.trust_signals), preserved)
    write_if_absent(bundle / "index.md", root_index(title, sections, version), preserved)
    for s in sections:
        sdir = bundle / s
        sdir.mkdir(parents=True, exist_ok=True)
        write_if_absent(sdir / "index.md", section_index(s), preserved)
        write_if_absent(sdir / "example-concept.md", example_concept(today, args.trust_signals), preserved)
    if write_hooks:
        write_claude_hooks(target, hooks_os)

    print(f"Scaffolded OKF project at {target}")
    print(f"  title: {title}")
    print(f"  sections: {', '.join(sections)}")
    if write_hooks:
        print(f"  hooks: .claude/ written for {hooks_os} (Claude Code asks you to approve them on first open)")
    else:
        print("  hooks: skipped (--no-hooks)")
    if preserved:
        rels = ", ".join(str(p.relative_to(target)) for p in sorted(preserved))
        print(f"  preserved {len(preserved)} existing file(s), not overwritten: {rels}")
        print("  (delete a file and re-run to regenerate it from the template)")

    if args.no_validate:
        return 0

    # The "validates by construction" guarantee only holds when the scaffold wrote
    # everything fresh. Once --force preserves any existing file, the result is not
    # guaranteed valid, and scripts/validate.py and bundle/index.md may themselves be
    # the user's own preserved files -- so running validation here could execute an
    # unrelated preserved validator or blame the scaffold for preserved user content.
    # Skip it and tell the user to validate when they have reconciled their files.
    if preserved:
        print("\nScaffold written, but skipping validation: existing files were "
              "preserved, so the bundle is not guaranteed valid by construction and "
              "scripts/validate.py may be your own.", file=sys.stderr)
        # A preserved requirements.txt is the user's own (#142), so we never edit it. But
        # if it omits PyYAML, the validate step below fails with ModuleNotFoundError, so
        # name the exact dependency here rather than leaving them to decode the traceback.
        req = target / "requirements.txt"
        if req in preserved and _preserved_requirements_lacks_pyyaml(req):
            print("  note: the preserved requirements.txt does not list PyYAML, which "
                  "scripts/validate.py imports. Add PyYAML>=5.1 to it (or run "
                  "`pip install pyyaml`) before validating. Without it, validation "
                  "fails with ModuleNotFoundError.", file=sys.stderr)
        print("  reconcile the preserved files, then validate when ready:", file=sys.stderr)
        print(f"    cd {target} && {interpreter_for(hooks_os)} scripts/validate.py --bundle bundle", file=sys.stderr)
        return 0

    # validate.py runs under this same interpreter, so its `import yaml` succeeds
    # only if PyYAML is findable here. Check first: an absent dependency is a setup
    # gap, not a scaffold bug, and the scaffold itself succeeded. Report it plainly
    # and exit 0 rather than letting the subprocess traceback be mislabeled below.
    if importlib.util.find_spec("yaml") is None:
        print("\nScaffold written, but skipping validation: PyYAML is not installed "
              "for this interpreter.", file=sys.stderr)
        print("  install it, then validate:", file=sys.stderr)
        print("    pip install -r requirements.txt    # or: pip install pyyaml", file=sys.stderr)
        print(f"    cd {target} && {interpreter_for(hooks_os)} scripts/validate.py --bundle bundle", file=sys.stderr)
        return 0

    print("\nValidating the new bundle...")
    res = subprocess.run(
        [sys.executable, str(target / "scripts" / "validate.py"), "--bundle", str(bundle)],
        capture_output=True, text=True)
    sys.stdout.write(res.stdout)
    if res.stderr:
        sys.stderr.write(res.stderr)
    if res.returncode != 0:
        print("FAIL: scaffold did not validate (this is a bug in scaffold.py)", file=sys.stderr)
        return res.returncode
    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""Validate an OKF (Open Knowledge Format) bundle against OKF spec v1 (see SPEC.md).

An OKF bundle is a tree of small markdown files: one concept per file, each with
YAML frontmatter carrying its provenance. Directory `index.md` files provide
navigation. This validator enforces the contract so a bundle stays machine- and
agent-readable.

Checks:
  1. Every non-reserved .md file has a parseable YAML frontmatter block. A YAML
     parse error is reported (commonly an unquoted colon-space or '#' in a
     string field, quote the value).
  2. Frontmatter carries every required key, non-empty. Through okf_version 0.3:
     type, title, description, source, verified, timestamp, tags. At 0.4:
     the same list with 'verified' renamed to 'verified_on' (see #6).
       - type     is one of the spec type vocab.
       - source   is a non-empty list of non-empty strings (provenance pointers).
                  An unquoted '#' in a block-style element (which YAML would silently
                  drop as a comment, losing the rest) is rejected, quote it.
       - tags     is a list.
       - verified/verified_on parses as an ISO date (YYYY-MM-DD).
       - timestamp parses as an ISO date, or under okf_version 0.3+ as a full
                    ISO 8601 datetime (upstream OKF writes a datetime).
  3. Reserved filenames (index.md, log.md) name no concept and carry no
     frontmatter, except the bundle-root index.md may carry okf_version only.
  4. Internal markdown links resolve. Links must be relative, a root-relative
     ('/'-prefixed) link is rejected. Every link to a .md file inside the bundle
     must point at a file that exists, with the case it has on disk (a
     case-insensitive filesystem would otherwise let a wrong-case link pass on
     macOS or Windows and dangle on Linux); a link that escapes the bundle root
     or dangles is a hard failure. Optional link titles and <>-wrapped destinations
     are handled. The bundle is validated as one self-contained tree (to validate
     federated content, assemble the bundles into one tree and point --bundle at
     that root).
  5. No file leaks a secret VALUE (private-key blocks, cloud API tokens,
     secret=<blob> assignments). Credential concepts document key NAMES and
     paths, never the values. Heuristic; narrow a pattern if it false-positives,
     do not delete the rule.
  6. At okf_version 0.4, optional upstream-v0.2 trust/provenance fields are
     checked for shape when present (never required): 'generated' ({by, at});
     the new 'verified' (a list of {by, at} confirmations, distinct from the
     required 'verified_on' at that version); 'sources' (plural; structured
     provenance objects, distinct from the required singular 'source'); 'status'
     (draft/stable/deprecated); 'stale_after' (an ISO date). A concept typed
     'Attested Computation' additionally requires 'runtime', 'parameters',
     'executor', and 'attester' to be present and correctly shaped.

Exits non-zero on any hard failure.
Usage: python3 validate.py --bundle DIR
"""
from __future__ import annotations

import argparse
import datetime as dt
import math
import os
import re
import sys
from collections import Counter
from pathlib import Path, PureWindowsPath

import yaml

REQUIRED_KEYS_LEGACY = ("type", "title", "description", "source", "verified", "timestamp", "tags")
# At TRUST_SIGNALS_VERSION, 'verified' is renamed to 'verified_on' in the required
# set -- it frees the bare name 'verified' for upstream v0.2's own optional field (a
# list of {by, at} confirmations; see check_verified_trust), which is a different
# shape and would otherwise collide with this fork's older single-date field.
REQUIRED_KEYS_V04 = ("type", "title", "description", "source", "verified_on", "timestamp", "tags")
LIST_KEYS = ("source", "tags")
# Keys that may also carry a full ISO 8601 datetime (see check_dates). The
# bundle must explicitly opt into this grammar through okf_version 0.3 so an
# older validator rejects the format at its version gate instead of later on a
# timestamp it does not understand.
DATETIME_KEYS = ("timestamp",)
DATETIME_TIMESTAMP_VERSION = "0.3"
# The version at which 'verified' is renamed to 'verified_on' and the optional
# upstream-v0.2 trust/provenance fields (generated, verified, sources, status,
# stale_after, Attested Computation) become available. See REQUIRED_KEYS_V04 above
# and SPEC.md's "Trust and provenance (upstream v0.2 vocabulary)" section.
TRUST_SIGNALS_VERSION = "0.4"
LEGACY_ALLOWED_TYPES = {
    # Infrastructure / ops (fleet maps, system docs)
    "Machine", "Network", "Service", "Session", "Project",
    "Repo", "Credential", "Path", "Process",
    # Domain-neutral (newsrooms, research atlases, decision logs)
    "Concept", "Decision", "Event", "Person", "Org", "Source",
    # Catch-all
    "Reference",
}
TRUST_SIGNAL_TYPES = {
    # Upstream v0.2: a sanctioned computation plus how to check a run of it
    # (see check_attested_computation). Kept as the literal upstream spelling
    # (with a space), not renamed to fit a no-space convention.
    "Attested Computation",
}
ALLOWED_TYPES = LEGACY_ALLOWED_TYPES | TRUST_SIGNAL_TYPES
ALLOWED_STATUSES = {"draft", "stable", "deprecated"}
RESERVED = {"index.md", "log.md"}
# okf_version values this validator accepts. The last entry is the current format
# version, but it is opt-in, not the scaffold default -- scaffold.py still writes
# "0.3" unless --trust-signals asks for "0.4" (see scaffold.py's own default).
# Older entries stay supported so a newer validator still reads an older bundle.
# Adding allowed types is backward compatible and bumps the format version
# (0.1 -> 0.2). Accepting a datetime in timestamp changes the field grammar and
# bumps it again (0.2 -> 0.3). Renaming 'verified' to 'verified_on' and adopting
# the optional upstream-v0.2 trust fields bumps it once more (0.3 -> 0.4).
SUPPORTED_VERSIONS = ("0.1", "0.2", DATETIME_TIMESTAMP_VERSION, TRUST_SIGNALS_VERSION)
SPEC_VERSION = SUPPORTED_VERSIONS[-1]  # current (opt-in) format version -- see note above


def required_keys_for(bundle_version):
    """The required-key tuple for a declared okf_version.

    Only TRUST_SIGNALS_VERSION ("0.4") renames 'verified' to 'verified_on'; every
    other declared (or missing/unsupported) version keeps the legacy name, so an
    unsupported-version bundle is still checked against a sensible required set
    instead of crashing before the version-gate error is reported.
    """
    return REQUIRED_KEYS_V04 if bundle_version == TRUST_SIGNALS_VERSION else REQUIRED_KEYS_LEGACY


def allowed_types_for(bundle_version):
    """The closed type vocabulary for a declared okf_version."""
    return ALLOWED_TYPES if bundle_version == TRUST_SIGNALS_VERSION else LEGACY_ALLOWED_TYPES


def date_keys_for(bundle_version):
    """The date-checked key names for a declared okf_version (see required_keys_for)."""
    return ("verified_on", "timestamp") if bundle_version == TRUST_SIGNALS_VERSION else ("verified", "timestamp")


def supports_datetime_timestamp(bundle_version):
    """Whether a supported bundle version carries full timestamp precision."""
    try:
        return (
            SUPPORTED_VERSIONS.index(bundle_version)
            >= SUPPORTED_VERSIONS.index(DATETIME_TIMESTAMP_VERSION)
        )
    except ValueError:
        return False


# Inline markdown link. The destination group allows one level of balanced
# parens so a filename like `missing(v2).md` is still captured (a plain [^)]+
# would stop at the first ')' and skip the link entirely).
LINK_RE = re.compile(r"\[[^\]]*\]\(((?:[^()]|\([^()]*\))*)\)")
# OKF links are relative markdown links only. The [[slug]] wikilink idiom (from the
# auto-memory system) is not OKF, so a typo'd or deleted [[ref]] would otherwise pass
# the link check unseen. It is reported as an error. Checked against strip_code output,
# so a [[x]] shown inside a code fence is illustrative, not flagged.
WIKILINK_RE = re.compile(r"\[\[([^\[\]]+)\]\]")

# Secret-value detectors. These match credential VALUES, not the key names/paths
# a credential concept is allowed to document. The generic assignment pattern
# requires a separator (`:`/`=`) directly before a high-entropy blob, so a
# documented key name like `service/api/...-secret` does not trip it.
#
# The key labels that mark a value as a credential, shared by the generic base64
# assignment pattern and the opt-in entropy scan so the two agree on what counts
# as a labeled secret.
SECRET_LABEL = (
    r"(?:password|passwd|secret|api[_-]?key|apikey|client[_-]?secret"
    r"|access[_-]?token|auth[_-]?token)")

SECRET_PATTERNS = [
    ("Tailscale key", re.compile(r"tskey-(?:api|auth|client)-[A-Za-z0-9]+-[A-Za-z0-9]{10,}")),
    ("private-key block", re.compile(r"-----BEGIN (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY-----")),
    ("AWS access key id", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
    ("Google API key", re.compile(r"\bAIza[0-9A-Za-z_\-]{35}\b")),
    ("Slack token", re.compile(r"\bxox[baprs]-[0-9A-Za-z-]{10,}")),
    ("GitHub token", re.compile(r"\bgh[pousr]_[0-9A-Za-z]{36,}\b")),
    ("GitHub fine-grained PAT", re.compile(r"\bgithub_pat_[0-9A-Za-z_]{22,}\b")),
    # More provider tokens with fixed literal prefixes and exact or narrow shapes.
    # Providers whose token bodies allow path-like hyphens and underscores use the
    # entropy-gated patterns below instead.
    ("Stripe secret key", re.compile(r"\b[sr]k_(?:live|test)_[0-9A-Za-z]{24,}\b")),
    ("Stripe organization key", re.compile(r"\bsk_org_[0-9A-Za-z]{24,}\b")),
    ("Stripe webhook secret", re.compile(r"\bwhsec_[0-9A-Za-z]{32,}\b")),
    # GitLab documents this exact cookie label as a token prefix. Keep the value
    # shape narrow enough that its `_gitlab_session=...` documentation placeholder
    # stays clean while an actual serialized cookie is caught.
    ("GitLab session cookie", re.compile(
        r"(?i)\b_gitlab_session\s*=\s*['\"]?"
        r"[0-9A-Za-z%+/_=-]{20,}(?![0-9A-Za-z%+/_=.\-])")),
    ("npm token", re.compile(r"\bnpm_[0-9A-Za-z]{36}\b")),
    ("SendGrid API key", re.compile(r"\bSG\.[0-9A-Za-z_-]{22}\.[0-9A-Za-z_-]{43}")),
    # Legacy personal OpenAI keys carry no project/service segment. `sk-` alone is a
    # weak prefix, so the 40-char solid-base62 run does the signal work; it stays
    # disjoint from the entropy-gated project-key detector below, whose `-` breaks
    # the run.
    ("OpenAI legacy key", re.compile(r"\bsk-[0-9A-Za-z]{40,}\b")),
    ("secret assignment", re.compile(
        r"(?i)" + SECRET_LABEL +
        # Base64-standard value charset only -- deliberately excludes - and _. A
        # credential concept documents key paths like `secret: svc/api/prod-key-path`,
        # and a hyphen/underscore-rich path must not read as a high-entropy value.
        # Structured tokens that use -/_ (fine-grained PATs, Slack, etc.) have their
        # own specific patterns above; the opt-in entropy scan below covers the rest.
        r"\s*[:=]\s*['\"]?[A-Za-z0-9+/]{24,}['\"]?")),
]

# Some provider token bodies allow the same hyphens and underscores used in
# human-readable vault paths. A prefix alone would therefore flag documentation
# such as `openai/sk-proj-production-primary-key-path`. These patterns capture a
# complete base64url-like body. They still run by default because the provider
# prefix is strong, but the entropy gate keeps path documentation clean. OpenAI
# and Anthropic use the generic 4.0 bits/character floor. GitLab accepts 20-char
# bodies, whose repeated characters lower their observable entropy; its narrower
# floor is 88% of the maximum entropy observable at the captured body length,
# capped at 4.0. That is 3.80 bits/character at 20 chars and rises to 4.0 at 24,
# so short tokens are not judged against a longer sample's ceiling while the
# longer path regressions stay clean.
#
# GitLab prefixes are from its token overview (checked 2026-07-23):
# https://docs.gitlab.com/security/tokens/#token-prefixes
PREFIXED_SECRET_PATTERNS = [
    ("GitLab token", re.compile(
        r"\b(?:glpat|gloas|gldt|glrtr?|glcbt|glptt|glft|glimt|glagent|glwt"
        r"|glsoat|glffct)-([0-9A-Za-z_-]{20,})(?![0-9A-Za-z_/-])"), 0.88),
    ("Anthropic API key", re.compile(
        r"\bsk-ant-([0-9A-Za-z_-]{20,})(?![0-9A-Za-z_/-])"), 1.0),
    ("OpenAI project key", re.compile(
        r"\bsk-(?:proj|svcacct)-([0-9A-Za-z_-]{20,})(?![0-9A-Za-z_/-])"), 1.0),
]

# Opt-in entropy scan (--secret-entropy-scan). The generic assignment pattern
# above uses a base64-standard value charset that excludes - and _, so a labeled
# secret whose value is URL-safe (base64url: - _ =) slips past it. Widening that
# charset would re-flag OKF key paths like `secret: svc/api/prod-key-path`, the
# precision an earlier review round asked us to keep. This optional pass instead
# matches only the base64url charset -- base64-standard values with `/` stay the
# generic pattern's job -- and keeps precision two ways. Structurally (the primary
# guard), the captured run must be a complete token: the trailing lookahead rejects
# a run that is followed by another value char (we truncated a longer token) or a
# `/` (it is a path segment, not a standalone value), so a documented key path like
# `secret: prd-usw2-...-key-path/service` cannot leak its first segment as a value.
# Excluding `/` from the class alone did not do this -- it stopped the match at the
# separator but still captured a >=24-char leading segment. Statistically, a
# Shannon-entropy floor backstops the slashless case: a random token scores above a
# short dictionary-and-separator name (measured: 24-char base64url secrets land
# ~4.05-4.4 bits/char, short human-readable names stay under 4.0). The floor is
# imperfect -- a long, varied slashless name can clear it, the acknowledged
# precision-for-recall tradeoff -- which is why the structural check, not this
# threshold, is the primary guard. It is off by default so a normal run keeps the
# narrow, zero-false-positive base64 behavior; the flag trades some precision for
# recall.
SECRET_ENTROPY_RE = re.compile(
    r"(?i)" + SECRET_LABEL + r"\s*[:=]\s*['\"]?([A-Za-z0-9_=-]{24,})"
    r"(?![A-Za-z0-9_=/-])['\"]?")
SECRET_ENTROPY_MIN_BITS = 4.0


def shannon_entropy(s: str) -> float:
    """Shannon entropy of s in bits per character (0.0 for the empty string)."""
    if not s:
        return 0.0
    n = len(s)
    return -sum((c / n) * math.log2(c / n) for c in Counter(s).values())


def prefixed_secret_labels(text: str):
    """Yield each provider label with a complete, random-looking token body."""
    for label, pattern, max_entropy_fraction in PREFIXED_SECRET_PATTERNS:
        for match in pattern.finditer(text):
            body = match.group(1)
            # A sample of n characters cannot exhibit more than log2(n) bits of
            # entropy per character, even when every character is unique.
            min_entropy = min(
                SECRET_ENTROPY_MIN_BITS,
                max_entropy_fraction * math.log2(len(body)),
            )
            if shannon_entropy(body) >= min_entropy:
                yield label
                break


def entropy_secret_values(text: str):
    """Yield the labeled, high-entropy base64url values in text that the generic
    base64 pattern misses. A value must carry a base64url-only character (- _ =) --
    otherwise the generic pattern already covers it -- and clear the entropy floor,
    so a low-entropy hyphenated name is left alone."""
    for m in SECRET_ENTROPY_RE.finditer(text):
        value = m.group(1)
        if not any(ch in value for ch in "-_="):
            continue  # plain base64/alnum -- already covered by SECRET_PATTERNS
        if shannon_entropy(value) >= SECRET_ENTROPY_MIN_BITS:
            yield value


FRONTMATTER_RE = re.compile(r"^---\n(.*?)\n---\n?(.*)$", re.DOTALL)


def parse_frontmatter(text: str):
    """Return (frontmatter_dict_or_None, body). Raises yaml.YAMLError on bad YAML."""
    if not text.startswith("---"):
        return None, text
    m = FRONTMATTER_RE.match(text)
    if not m:
        return None, text
    return yaml.safe_load(m.group(1)) or {}, m.group(2)


def frontmatter_block(text: str) -> str | None:
    """Return the raw YAML frontmatter text (between the --- fences), or None. Used to
    enforce rules on the source text before YAML strips comments (see
    check_source_quoting)."""
    if not text.startswith("---"):
        return None
    m = FRONTMATTER_RE.match(text)
    return m.group(1) if m else None


def link_destination(raw: str) -> tuple[str, str]:
    """Pull the path and fragment out of a markdown link's (...) contents: strip
    <...> wrapping and an optional "title"/'title'. Markdown allows
    [text](dest "title") and [text](<dest with spaces>), treating the whole
    contents as the path would falsely flag those as dangling."""
    s = raw.strip()
    if s.startswith("<"):
        end = s.find(">")
        if end != -1:
            destination = s[1:end].strip()
            path, separator, fragment = destination.partition("#")
            return path, separator + fragment
    s = s.split(None, 1)[0] if s else s  # dest ends at first space; rest is a title
    path, separator, fragment = s.partition("#")
    return path, separator + fragment


def resolve_link(target: str, md_file: Path) -> Path:
    """Resolve a relative link destination against the file's directory.
    .resolve() collapses ../ so the bundle-boundary check is not fooled by a path
    like ../../outside.md."""
    return (md_file.parent / target).resolve()


def is_rooted_link(target: str) -> bool:
    """True for POSIX roots, Windows roots, drive paths, and UNC paths."""
    windows_path = PureWindowsPath(target)
    return target.startswith(("/", "\\")) or bool(windows_path.drive)


def real_case_path(
    dest: Path, bundle: Path, *, allow_nonconforming_md: bool = False
) -> Path | None:
    """The path on disk that `dest` names, ignoring case, or None if nothing matches.

    Returns `dest` unchanged when every component already matches the real name.

    Path.exists() asks the filesystem, and macOS answers yes for `Concepts/Foo.md`
    when the file is really `concepts/foo.md`. A bundle written there passes
    validation on the author's machine and dangles the first time Linux CI or a
    Linux reader opens it, which is the class of break this validator exists to
    catch before it ships. So walk the components against the real directory
    listings instead of asking whether the path exists.

    The walk doubles as the existence check: a component that matches nothing,
    case or no case, means the link dangles.

    Walk the link's spelling before Path.resolve() can canonicalize its case on
    Windows. Keep each symlink and '..' component in order: collapsing '..'
    lexically can change the destination after a symlinked directory.

    `dest` is the uncollapsed link path starting at `bundle`. The caller keeps
    a separately resolved path for the bundle-boundary check.
    """
    current = bundle
    for part in dest.relative_to(bundle).parts:
        if part == "..":
            current = current / part
            continue
        try:
            names = set(os.listdir(current))
        except OSError:
            return None  # not a directory, or unreadable: nothing below it resolves
        if part in names:
            current = current / part
            continue
        # Exactly one case-variant is a mismatch worth naming. Several means a
        # case-sensitive filesystem holding both, and no way to say which was meant.
        # Do not recommend a file with an uppercase .md extension as a link fix: that
        # filename is rejected elsewhere in this validator. The caller can opt into a
        # second lookup solely to give that non-conforming file a rename diagnostic.
        variants = [n for n in names if n.lower() == part.lower()]
        if not allow_nonconforming_md and Path(part).suffix.lower() == ".md":
            variants = [n for n in variants if Path(n).suffix == ".md"]
        if len(variants) != 1:
            return None
        current = current / variants[0]
    return current


def strip_code(text: str) -> str:
    """Blank out fenced code blocks and inline code spans so a link shown as an
    example (e.g. a ```md fence containing [x](sample.md)) is not mistaken for a
    real bundle link. The secret scan still runs on the raw text, a secret in a
    code block is still a leak.

    Heuristic, not a full CommonMark parser: it handles ```/~~~ fences (matching
    the closing fence's char and length, so a longer fence can wrap a shorter one)
    and backtick-run inline spans (``code with a ` inside``). Rare forms, 4-space
    indented code blocks, code spans spanning lines, are out of scope; an OKF
    concept that needs those can wrap the example in a fence."""
    out = []
    fence = None  # (char, length) of the open fence, or None
    for line in text.splitlines():
        stripped = line.lstrip()
        if fence is None:
            m = re.match(r"(`{3,}|~{3,})", stripped)
            if m:
                fence = (stripped[0], len(m.group(1)))
                out.append("")
            else:
                out.append(re.sub(r"(`+)(.+?)\1", "", line))  # drop inline code spans
        else:
            ch, length = fence
            m = re.match(r"(`{3,}|~{3,})\s*$", stripped)
            if m and stripped[0] == ch and len(m.group(1)) >= length:
                fence = None
            out.append("")
    return "\n".join(out)


def _plain_scalar_dropped_comment(node, raw_fm):
    """True if a YAML comment directly truncated this scalar's provenance.

    `node` is a scalar node from the parsed frontmatter and `raw_fm` the raw text it was
    parsed from. Only a plain (unquoted) scalar can lose data: a quoted or block scalar
    (node.style set to ', ", |, or >) keeps a '#' as string content. For a plain scalar,
    YAML stops the value at the space before an inline '#', so the comment shows up in the
    raw text immediately after the scalar's end mark, on the same line. That is exactly the
    silent-truncation case (`issue #445` -> "issue") the SPEC quoting rule guards against;
    `issue#445` (no space) is one scalar and a '#' on a later line is a standalone comment,
    and neither trips this."""
    if node.style is not None:
        return False
    j = node.end_mark.index
    while j < len(raw_fm) and raw_fm[j] in (" ", "\t"):
        j += 1
    return j < len(raw_fm) and raw_fm[j] == "#"


# PyYAML resolves both `<<` and an explicit `!!merge` tag to this tag; a key carrying it is a
# merge directive regardless of how it is spelled, so it is never a real mapping key.
_MERGE_TAG = "tag:yaml.org,2002:merge"


def _child_ref_counts(root):
    """Map id(node) -> how many times it is referenced as a child across the tree.

    A YAML alias makes the composer reuse the anchor's node object, so an alias TARGET
    is the one node referenced 2+ times. Each unique node is traversed once (guarded by
    a seen set) so a recursive anchor cannot loop; references are still counted with
    multiplicity."""
    counts: dict[int, int] = {}
    seen: set[int] = set()
    stack = [root]
    while stack:
        node = stack.pop()
        if id(node) in seen:
            continue
        seen.add(id(node))
        children: list = []
        if isinstance(node, yaml.MappingNode):
            for k, v in node.value:
                children += (k, v)
        elif isinstance(node, yaml.SequenceNode):
            children += list(node.value)
        for c in children:
            counts[id(c)] = counts.get(id(c), 0) + 1
            stack.append(c)
    return counts


def _value_uses_alias(val_node, counts):
    """True if any node in this value's subtree is reached via a YAML alias.

    An alias target is the same node object referenced 2+ times in the whole frontmatter
    (counts), or reached twice while walking this one subtree (a cycle). An unused anchor
    is referenced once, so an anchored literal is not flagged, consistent with allowing
    `source: [&p "x"]`. Rejecting alias USES closes #169: an aliased scalar shares its
    anchor's position marks, so the end-mark quoting check cannot see a comment dropped
    after the alias."""
    seen: set[int] = set()
    stack = [val_node]
    while stack:
        node = stack.pop()
        if id(node) in seen:
            return True  # reached twice within source -> an alias points back into it
        seen.add(id(node))
        if counts.get(id(node), 0) >= 2:
            return True  # shared with another reference -> an alias target
        if isinstance(node, yaml.MappingNode):
            for k, v in node.value:
                stack += (k, v)
        elif isinstance(node, yaml.SequenceNode):
            stack += list(node.value)
    return False


def _mapping_yields_source(node, seen):
    """True if this mapping node's effective keys include 'source', counting keys reached
    through its own (possibly nested) merge keys.

    A merged mapping can itself merge in another mapping ('<<: *d' where d is '<<: {source:
    ...}'), so checking only the immediate keys misses a source that safe_load still
    materializes. Recurse through each merge key's mapping(s). The seen set guards a
    recursive anchor (&d {<<: *d}) from looping."""
    if not isinstance(node, yaml.MappingNode) or id(node) in seen:
        return False
    seen.add(id(node))
    for key, val in node.value:
        if not isinstance(key, yaml.ScalarNode):
            continue
        if key.tag == _MERGE_TAG:
            for merged in (val.value if isinstance(val, yaml.SequenceNode) else [val]):
                if _mapping_yields_source(merged, seen):
                    return True
        elif key.value == "source":
            return True
    return False


def _merge_supplies_source(root):
    """True if a top-level YAML merge key (<<) merges in a mapping that yields its own
    'source' key, directly or through a further nested merge.

    YAML lets a literal 'source:' override a merged one, so a source smuggled in through
    '<<: {source: ...}' (or a chain of merges that resolves to source) beside a literal is
    silently dropped and the top-level source-key scan (which keys on literal 'source'
    nodes) never sees it. A merge value is a mapping, or a sequence of mappings ('<<: [*a,
    *b]'); compose resolves an alias to the shared node, so an aliased merge map is a
    MappingNode here. Detecting it lets a merge-supplied source be rejected whether or not
    a literal source sits beside it."""
    for key, val in root.value:
        if not (isinstance(key, yaml.ScalarNode) and key.tag == _MERGE_TAG):
            continue
        for node in (val.value if isinstance(val, yaml.SequenceNode) else [val]):
            if _mapping_yields_source(node, set()):
                return True
    return False


def check_source_quoting(rel, fm, raw_fm, errors):
    """Enforce the SPEC 'source' quoting rule, where YAML's comment stripping would
    otherwise silently drop part of a provenance pointer.

    Quoting source elements is a hard SPEC rule, but YAML drops an unquoted inline '#'
    comment with no error: `- issue #445` parses to "issue", losing "#445". The parsed
    value alone cannot reveal the loss, so this re-parses the frontmatter into its node
    tree (which carries source position marks) and, for every top-level `source` element,
    checks whether a comment directly truncated a plain scalar (see
    _plain_scalar_dropped_comment). Delegating the lexing to YAML covers every shape,
    block items, single- and multi-line flow lists, wrapped scalars, quoted strings with
    escapes, anchors/tags, and block scalars, without re-implementing the parser. It is
    scoped to the top-level `source` key only (a nested `source:` under other metadata is
    not the OKF provenance list). A real parse error is reported by the schema check.

    `fm` is the safe_load result the caller already parsed. A `source` can enter it through
    a YAML merge key (`<<: {source: *r}`), a whole-node alias, an alias in key position, or a
    duplicate `source` key, each materializes the field, but YAML keeps the LAST of duplicate
    keys, so the value safe_load returns is not necessarily the first clean `source:` the scan
    finds. OKF source must be one literal top-level list, so this requires the effective source
    to come from exactly one literal, unshared `source:` key and rejects every indirection
    (merge, alias, duplicate) up front, which also keeps the node-tree quoting scan total."""
    if not raw_fm:
        return
    try:
        root = yaml.compose(raw_fm, Loader=yaml.SafeLoader)
    except yaml.YAMLError:
        return
    if not isinstance(root, yaml.MappingNode):
        return
    counts = _child_ref_counts(root)
    # YAML keeps the LAST of duplicate keys, so the value safe_load returns for `source` is
    # decided by the last top-level key that resolves to "source", not the first clean one.
    # Collect every such key. An alias in key position (`*k` resolving to "source") makes
    # compose reuse the anchor's node, so the key reads as a "source" scalar while the written
    # key is an alias, count >= 2 marks that sharing, so an aliased key is not unshared. A key
    # SPELLED "source" but carrying the explicit merge tag (`!!merge source:`) is a merge
    # directive, not a source key, PyYAML classifies merge by the tag, not the spelling, so
    # exclude it here: counting it would both falsely reject a file whose only real source is a
    # separate literal, and let a source merged in through it bypass the scan below.
    source_keys = [
        (k, v) for k, v in root.value
        if isinstance(k, yaml.ScalarNode) and k.value == "source"
        and k.tag != _MERGE_TAG
    ]
    unshared = [(k, v) for k, v in source_keys if counts.get(id(k), 0) < 2]
    if fm.get("source") is not None and (
        len(source_keys) != 1 or not unshared or _merge_supplies_source(root)
    ):
        errors.append(
            f"{rel}: 'source' is not a single literal top-level 'source:' key, it enters "
            f"through a YAML merge key (<<), an alias, or a duplicate 'source' key; OKF "
            f"'source' must be one literal top-level list of provenance pointers, declare "
            f"it directly instead of merging, aliasing, or duplicating it")
        return
    for key_node, val_node in source_keys:
        # A YAML anchor/alias that entangles source with the rest of the frontmatter
        # shares node identity and position marks, so the dropped-comment scan below
        # reads the anchor's definition line, not the use site, and can miss a truncated
        # pointer (#169). OKF source is a flat list of literal, self-contained pointers
        # anyway, so any anchor/alias SHARING is invalid here, reject it, which keeps the
        # quoting scan total. An unused anchor definition (`[&p "x"]`) shares nothing and
        # stays allowed.
        if _value_uses_alias(val_node, counts):
            errors.append(
                f"{rel}: a top-level 'source' value shares a YAML anchor/alias (& or *) "
                f"with the rest of the frontmatter; OKF 'source' must be a flat list of "
                f"literal provenance pointers, write each pointer out literally instead "
                f"of anchoring or aliasing it")
            return
        if isinstance(val_node, yaml.SequenceNode):
            scalars = [n for n in val_node.value if isinstance(n, yaml.ScalarNode)]
        elif isinstance(val_node, yaml.ScalarNode):
            scalars = [val_node]
        else:
            scalars = []
        if any(_plain_scalar_dropped_comment(n, raw_fm) for n in scalars):
            errors.append(
                f"{rel}: a top-level 'source' element has an unquoted '#' that YAML "
                f"reads as a comment, dropping the rest of the pointer, quote each "
                f"source element that contains a '#'")
            return  # one report per concept is enough


ISO_DATE_RE = re.compile(r"[0-9]{4}-[0-9]{2}-[0-9]{2}\Z")
ISO_DATETIME_RE = re.compile(
    r"[0-9]{4}-[0-9]{2}-[0-9]{2}[T ][0-9]{2}:[0-9]{2}:[0-9]{2}"
    r"(?:[.,][0-9]+)?(?:Z|[+-][0-9]{2}:[0-9]{2})?\Z"
)


def _is_iso_date(s):
    """True only for the literal YYYY-MM-DD form required by the SPEC."""
    if not ISO_DATE_RE.fullmatch(s):
        return False
    try:
        dt.datetime.strptime(s, "%Y-%m-%d")
    except ValueError:
        return False
    return True


def _is_iso_datetime(s):
    """True for the SPEC's full ISO 8601 datetime spelling.

    ``fromisoformat`` checks calendar and clock ranges, but it is deliberately
    not the lexical contract: both it and PyYAML accept wider timestamp forms.
    The regular expression first requires the exact zero-padded source shape.
    A trailing ``Z`` is normalised for Python versions that do not parse it.
    """
    if not ISO_DATETIME_RE.fullmatch(s):
        return False
    try:
        dt.datetime.fromisoformat(s[:-1] + "+00:00" if s.endswith("Z") else s)
    except ValueError:
        return False
    return True


def _literal_scalar_node(raw_fm, *path):
    """Return an unshared literal scalar node along a mapping/sequence path.

    String path components select the effective (last) literal mapping key;
    integer components index a sequence. Aliases are rejected because their
    source marks describe the anchor rather than the use site.
    """
    if not raw_fm or not path:
        return None
    try:
        root = yaml.compose(raw_fm, Loader=yaml.SafeLoader)
    except yaml.YAMLError:
        return None
    if not isinstance(root, yaml.MappingNode):
        return None

    counts = _child_ref_counts(root)
    node = root
    for component in path:
        if isinstance(component, str):
            if not isinstance(node, yaml.MappingNode):
                return None
            candidates = [
                (key_node, val_node)
                for key_node, val_node in node.value
                if isinstance(key_node, yaml.ScalarNode)
                and key_node.value == component
                and key_node.tag != _MERGE_TAG
                and counts.get(id(key_node), 0) < 2
            ]
            if not candidates:
                return None
            _, node = candidates[-1]  # safe_load keeps the last duplicate key
        elif isinstance(component, int):
            if (not isinstance(node, yaml.SequenceNode)
                    or component < 0
                    or component >= len(node.value)):
                return None
            node = node.value[component]
        else:
            return None
        if counts.get(id(node), 0) >= 2:
            return None

    return node if isinstance(node, yaml.ScalarNode) else None


def _raw_mapping_scalar(raw_fm, *path):
    """Return a literal scalar's unnormalised text along a node path, or None.

    ``safe_load`` constructs a broad family of YAML timestamp spellings as
    ``date``/``datetime`` objects. Calling ``isoformat`` on those objects would
    silently turn a malformed source spelling into a conforming value. Compose
    the same frontmatter into a node tree and inspect each effective (last)
    literal key along the path instead. Simple quoted strings remain supported;
    YAML aliases, tags, block scalars, and escape-based spellings are not literal
    date fields.
    """
    node = _literal_scalar_node(raw_fm, *path)
    if node is None:
        return None
    written = raw_fm[node.start_mark.index:node.end_mark.index]
    if node.style is None and written == node.value:
        return written
    if node.style in ("'", '"') and written == node.style + node.value + node.style:
        return node.value
    return None


def _mapping_scalar_dropped_comment(raw_fm, *path):
    """True when a literal scalar at path was truncated by a YAML comment."""
    node = _literal_scalar_node(raw_fm, *path)
    return node is not None and _plain_scalar_dropped_comment(node, raw_fm)


def _raw_top_level_scalar(raw_fm, key):
    """Return a literal top-level scalar's unnormalised text, or None."""
    return _raw_mapping_scalar(raw_fm, key)


def check_dates(rel, fm, raw_fm, bundle_version, errors):
    accepts_datetime = supports_datetime_timestamp(bundle_version)
    for key in date_keys_for(bundle_version):
        val = fm.get(key)
        if val is None:
            continue  # missing/empty already reported by required-key check
        raw = _raw_top_level_scalar(raw_fm, key)
        if raw is not None and _is_iso_date(raw):
            continue
        # `timestamp` is upstream OKF's key and upstream writes it as a full ISO
        # 8601 datetime. Version 0.3 and later accept and carry that precision;
        # older formats remain date-only. `verified` is this spec's own key and
        # always stays date-only because a time of day invites false precision.
        if (key in DATETIME_KEYS
                and accepts_datetime
                and raw is not None
                and _is_iso_datetime(raw)):
            continue
        if key in DATETIME_KEYS and not accepts_datetime:
            expected = (
                f"an ISO date YYYY-MM-DD under okf_version {bundle_version or '<missing>'}; "
                f"full ISO 8601 datetimes require okf_version {DATETIME_TIMESTAMP_VERSION}"
            )
        elif key in DATETIME_KEYS:
            expected = "an ISO date YYYY-MM-DD or a full ISO 8601 datetime"
        else:
            expected = "an ISO date YYYY-MM-DD"
        shown = raw if raw is not None else val
        errors.append(f"{rel}: '{key}' must be {expected}, got {shown!r}")


def check_lists(rel, fm, errors):
    for key in LIST_KEYS:
        val = fm.get(key)
        if val is None:
            continue  # required-key check handles absence
        if not isinstance(val, list):
            errors.append(f"{rel}: '{key}' must be a YAML list, got {type(val).__name__}")
            continue
        if key == "source" and not val:
            errors.append(f"{rel}: 'source' must be a non-empty list of provenance pointers")
        for el in val:
            if not isinstance(el, str) or not el.strip():
                errors.append(f"{rel}: '{key}' has a non-string/empty element {el!r}")


def check_generated(rel, fm, raw_fm, errors):
    """Optional upstream-v0.2 'generated' field: {by, at} -- who/what produced the
    current content and when."""
    if "generated" not in fm:
        return
    val = fm["generated"]
    if not isinstance(val, dict):
        errors.append(f"{rel}: 'generated' must be a mapping {{by, at}}, got {type(val).__name__}")
        return
    by = val.get("by")
    if not isinstance(by, str) or not by.strip():
        errors.append(f"{rel}: 'generated.by' must be a non-empty string")
    at = val.get("at")
    if at is None:
        errors.append(f"{rel}: 'generated.at' is required when 'generated' is present")
        return
    raw = _raw_mapping_scalar(raw_fm, "generated", "at")
    if raw is None or not (_is_iso_date(raw) or _is_iso_datetime(raw)):
        shown = raw if raw is not None else at
        errors.append(f"{rel}: 'generated.at' must be an ISO date or a full ISO 8601 datetime, got {shown!r}")


def check_verified_trust(rel, fm, raw_fm, bundle_version, errors):
    """Optional upstream-v0.2 'verified' field: a list of independent {by, at}
    confirmations, from which a consumer derives a trust tier (unverified /
    machine-confirmed / human-reviewed). Only meaningful at TRUST_SIGNALS_VERSION,
    where 'verified' is not the required key (see REQUIRED_KEYS_V04's
    'verified_on') -- at every earlier version 'verified' IS the required legacy
    single-date field, already checked by check_dates, so this must not also
    re-validate it there under the new shape."""
    if bundle_version != TRUST_SIGNALS_VERSION:
        return
    if "verified" not in fm:
        return  # optional; absence reads as "unverified" to a consumer, not an error
    val = fm["verified"]
    if not isinstance(val, list) or not val:
        errors.append(
            f"{rel}: 'verified' (trust confirmations) must be a non-empty YAML list "
            f"of {{by, at}} mappings when present, got {type(val).__name__}")
        return
    for i, entry in enumerate(val):
        if not isinstance(entry, dict):
            errors.append(f"{rel}: 'verified[{i}]' must be a mapping with 'by' and 'at', got {type(entry).__name__}")
            continue
        by = entry.get("by")
        if not isinstance(by, str) or not by.strip():
            errors.append(f"{rel}: 'verified[{i}].by' must be a non-empty string")
        at = entry.get("at")
        if at is None:
            errors.append(f"{rel}: 'verified[{i}].at' is required")
        else:
            raw = _raw_mapping_scalar(raw_fm, "verified", i, "at")
            if raw is None or not _is_iso_date(raw):
                shown = raw if raw is not None else at
                errors.append(
                    f"{rel}: 'verified[{i}].at' must be an ISO date YYYY-MM-DD, "
                    f"got {shown!r}")


def check_sources_plural(rel, fm, raw_fm, errors):
    """Optional upstream-v0.2 'sources' field (plural) -- structured provenance
    objects, distinct from the required singular 'source' (a flat list of quoted
    pointers). Each entry needs a unique 'id' and a 'resource'; title/author/
    usage_count/last_modified are optional credibility signals. Shape-checked
    only -- this does not cross-check in-body [^id] footnotes against these ids
    (see SPEC.md)."""
    if "sources" not in fm:
        return
    val = fm["sources"]
    if not isinstance(val, list) or not val:
        errors.append(
            f"{rel}: 'sources' must be a non-empty YAML list of provenance objects "
            f"when present, got {type(val).__name__}")
        return
    seen_ids = set()
    for i, entry in enumerate(val):
        if not isinstance(entry, dict):
            errors.append(f"{rel}: 'sources[{i}]' must be a mapping, got {type(entry).__name__}")
            continue
        sid = entry.get("id")
        if not isinstance(sid, str) or not sid.strip():
            errors.append(f"{rel}: 'sources[{i}].id' must be a non-empty string")
        elif sid in seen_ids:
            errors.append(
                f"{rel}: 'sources[{i}].id' {sid!r} duplicates an earlier entry -- "
                f"ids must be unique within 'sources'")
        else:
            seen_ids.add(sid)
        resource = entry.get("resource")
        if not isinstance(resource, str) or not resource.strip():
            errors.append(f"{rel}: 'sources[{i}].resource' must be a non-empty string")
        elif _mapping_scalar_dropped_comment(raw_fm, "sources", i, "resource"):
            errors.append(
                f"{rel}: 'sources[{i}].resource' has an unquoted '#' that YAML "
                f"reads as a comment, dropping the rest of the provenance pointer")
        for opt_key in ("title", "author"):
            v = entry.get(opt_key)
            if v is not None and (not isinstance(v, str) or not v.strip()):
                errors.append(f"{rel}: 'sources[{i}].{opt_key}' must be a non-empty string when present")
        uc = entry.get("usage_count")
        if uc is not None and (not isinstance(uc, int) or isinstance(uc, bool) or uc < 0):
            errors.append(f"{rel}: 'sources[{i}].usage_count' must be a non-negative integer when present, got {uc!r}")
        lm = entry.get("last_modified")
        if lm is not None:
            raw = _raw_mapping_scalar(raw_fm, "sources", i, "last_modified")
            if raw is None or not _is_iso_date(raw):
                shown = raw if raw is not None else lm
                errors.append(
                    f"{rel}: 'sources[{i}].last_modified' must be an ISO date "
                    f"YYYY-MM-DD when present, got {shown!r}")


def check_status(rel, fm, errors):
    """Optional upstream-v0.2 'status' field: draft/stable/deprecated. Absent means
    stable (nothing to check)."""
    if "status" not in fm:
        return
    val = fm["status"]
    if not isinstance(val, str) or val not in ALLOWED_STATUSES:
        errors.append(f"{rel}: 'status' must be one of {sorted(ALLOWED_STATUSES)} when present, got {val!r}")


def check_stale_after(rel, fm, raw_fm, errors):
    """Optional upstream-v0.2 'stale_after' field: an absolute ISO date, deliberately
    not a relative TTL (see SPEC.md)."""
    if "stale_after" not in fm:
        return
    val = fm["stale_after"]
    raw = _raw_mapping_scalar(raw_fm, "stale_after")
    if raw is None or not _is_iso_date(raw):
        shown = raw if raw is not None else val
        errors.append(
            f"{rel}: 'stale_after' must be an ISO date YYYY-MM-DD, got {shown!r}")


def check_attested_computation(rel, fm, errors):
    """Upstream-v0.2 'Attested Computation' type: a sanctioned computation plus the
    means to check that a run of it actually matches. Checks shape only -- this
    validator never executes the computation, the executor, or the attester; that
    is a consumer's runtime job (see SPEC.md)."""
    if fm.get("type") != "Attested Computation":
        return

    runtime = fm.get("runtime")
    if not isinstance(runtime, str) or not runtime.strip():
        errors.append(f"{rel}: 'Attested Computation' requires a non-empty 'runtime' string")

    params = fm.get("parameters")
    if not isinstance(params, list):
        errors.append(f"{rel}: 'Attested Computation' requires a 'parameters' list")
    else:
        for i, p in enumerate(params):
            if not isinstance(p, dict):
                errors.append(f"{rel}: 'parameters[{i}]' must be a mapping {{name, type, required}}, got {type(p).__name__}")
                continue
            extra = sorted(set(p) - {"name", "type", "required"})
            if extra:
                errors.append(f"{rel}: 'parameters[{i}]' has undeclared keys {extra}")
            for key in ("name", "type"):
                v = p.get(key)
                if not isinstance(v, str) or not v.strip():
                    errors.append(f"{rel}: 'parameters[{i}].{key}' must be a non-empty string")
            if not isinstance(p.get("required"), bool):
                errors.append(f"{rel}: 'parameters[{i}].required' must be true or false")

    executor = fm.get("executor")
    if not isinstance(executor, dict):
        errors.append(f"{rel}: 'Attested Computation' requires an 'executor' mapping {{resource, receipt}}")
    else:
        resource = executor.get("resource")
        if not isinstance(resource, str) or not resource.strip():
            errors.append(f"{rel}: 'executor.resource' must be a non-empty string")
        receipt = executor.get("receipt")
        if not isinstance(receipt, list) or not receipt or not all(
                isinstance(r, str) and r.strip() for r in receipt):
            errors.append(f"{rel}: 'executor.receipt' must be a non-empty list of non-empty strings")

    attester = fm.get("attester")
    if not isinstance(attester, dict):
        errors.append(f"{rel}: 'Attested Computation' requires an 'attester' mapping {{resource}}")
    else:
        resource = attester.get("resource")
        if not isinstance(resource, str) or not resource.strip():
            errors.append(f"{rel}: 'attester.resource' must be a non-empty string")


def declared_bundle_version(bundle):
    """Read the root marker for version-dependent field checks.

    This is a non-reporting pre-pass; the main file loop remains responsible for
    all root-index diagnostics. Reading it up front means a root-level concept
    named ``a.md`` receives the right grammar even though it sorts before
    ``index.md``.
    """
    root_index = bundle / "index.md"
    if not root_index.is_file():
        return None
    try:
        fm, _ = parse_frontmatter(root_index.read_text(encoding="utf-8-sig"))
    except (OSError, yaml.YAMLError, ValueError):
        return None
    if not isinstance(fm, dict) or fm.get("okf_version") is None:
        return None
    return str(fm["okf_version"]).strip()


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--bundle", default="bundle", help="path to the OKF bundle directory (default: bundle)")
    ap.add_argument(
        "--secret-entropy-scan", action="store_true",
        help="also flag a labeled URL-safe/base64url value whose Shannon entropy "
             "clears the secret floor (opt-in; trades some precision for recall on "
             "hyphenated secret values the base64 pattern misses)")
    args = ap.parse_args()
    bundle = Path(args.bundle).resolve()

    if not bundle.exists():
        print(f"FAIL: bundle not found at {bundle}")
        return 1
    if not bundle.is_dir():
        print(f"FAIL: bundle path is not a directory: {bundle}")
        return 1

    # Discover markdown files case-insensitively (suffix .lower() == ".md") so a
    # non-conforming Foo.MD cannot hide from validation behind a case-sensitive glob;
    # it is found here and rejected below. rglob("*") also yields a directory named
    # like "archive.md"; reading one raises IsADirectoryError, so keep only real files
    # and report any .md-suffixed path that is a directory rather than crashing.
    md_entries = sorted(p for p in bundle.rglob("*") if p.suffix.lower() == ".md")
    md_files = [p for p in md_entries if p.is_file()]
    errors: list[str] = [
        f"{p.relative_to(bundle)}: a '*.md' path must be a file, not a directory"
        for p in md_entries if not p.is_file()
    ]
    # A bundle must have a root index.md. The okf_version gate below only runs when
    # that file exists, so without this check a bundle that simply omits the root
    # index (or an empty directory) would validate clean and bypass version gating.
    if not (bundle / "index.md").is_file():
        errors.append("index.md: bundle-root index is required and must declare okf_version")
    bundle_version = declared_bundle_version(bundle)
    type_counts: Counter = Counter()
    concepts = 0

    for f in md_files:
        rel = f.relative_to(bundle)
        # utf-8-sig strips a leading byte-order mark if present. A BOM (common from
        # Windows editors) would otherwise defeat the startswith("---") frontmatter
        # check, reporting valid frontmatter as missing.
        text = f.read_text(encoding="utf-8-sig")

        # secret scan on every file, including index.md and a non-conforming Foo.MD
        # (a leak is a leak regardless of extension, scan before rejecting below).
        for label, pat in SECRET_PATTERNS:
            if pat.search(text):
                errors.append(
                    f"{rel}: possible secret leak ({label}), remove the value, "
                    f"document the key name/path instead")
        for label in prefixed_secret_labels(text):
            errors.append(
                f"{rel}: possible secret leak ({label}), remove the value, "
                f"document the key name/path instead")
        if args.secret_entropy_scan and next(entropy_secret_values(text), None):
            errors.append(
                f"{rel}: possible secret leak (high-entropy assignment flagged by "
                f"--secret-entropy-scan), remove the value, document the key "
                f"name/path instead")

        # OKF concept and index files use a lowercase .md extension. A non-lowercase
        # extension (Foo.MD) is non-conforming: it was discovered case-insensitively
        # above so its content is still secret-scanned, then rejected here instead of
        # validated as a concept. With the case-insensitive link check below, this
        # closes the bypass where an uppercase-extension file and links to it both
        # escaped validation.
        if f.suffix != ".md":
            errors.append(
                f"{rel}: non-conforming filename, OKF concept files use a lowercase "
                f"'.md' extension, found {f.suffix!r}; rename it to .md")
            continue

        try:
            fm, body = parse_frontmatter(text)
        except (yaml.YAMLError, ValueError) as e:
            # ValueError covers a date-shaped scalar PyYAML auto-constructs and
            # rejects (e.g. an invalid month), which is not a YAMLError subclass.
            errors.append(
                f"{rel}: YAML frontmatter parse error ({e.__class__.__name__}: {e}), "
                f"quote any string field that holds a YAML-significant character. "
                f"Common triggers: a colon-space (': ') anywhere in description, title, "
                f"or a source element; a bare '#' in a source element; an invalid date "
                f"in verified/timestamp.")
            continue

        # Past this point fm is either None or a mapping. Syntactically valid YAML
        # that is a list/scalar (e.g. a stray top-level list) would otherwise crash
        # the later fm.get(...) calls; report it as a clean failure instead.
        if fm is not None and not isinstance(fm, dict):
            errors.append(f"{rel}: frontmatter must be a YAML mapping, got {type(fm).__name__}")
            continue

        if f.name in RESERVED:
            if str(rel) == "index.md":
                # the bundle-root index.md may carry frontmatter, but only the
                # okf_version marker, never concept metadata or stray keys.
                keys = set(fm) if fm else set()
                if "okf_version" not in keys:
                    errors.append("index.md: bundle-root index must declare okf_version in frontmatter")
                else:
                    version = str(fm.get("okf_version")).strip()
                    if version not in SUPPORTED_VERSIONS:
                        errors.append(
                            f"index.md: okf_version {fm.get('okf_version')!r} is not supported "
                            f"(this validator supports {', '.join(SUPPORTED_VERSIONS)})")
                extra = sorted(keys - {"okf_version"})
                if extra:
                    errors.append(f"index.md: bundle-root index may carry only okf_version, found {extra}")
            elif fm is not None:
                # any other reserved file (subdir index.md, log.md) carries no frontmatter.
                errors.append(f"{rel}: reserved file should not carry frontmatter")
            continue

        if fm is None:
            errors.append(f"{rel}: missing YAML frontmatter")
            continue

        concepts += 1
        ctype = fm.get("type", "<none>")
        # type must be a scalar string. A list/dict (a plausible YAML typo like
        # `type: [Reference]`) is unhashable and would crash both the Counter
        # increment and the closed-vocabulary membership test, so report it and
        # fall back to "<none>" to keep the rest of this concept's checks running.
        if not isinstance(ctype, str):
            errors.append(f"{rel}: 'type' must be a string, got {type(ctype).__name__}")
            ctype = "<none>"
        type_counts[ctype] += 1

        for key in required_keys_for(bundle_version):
            val = fm.get(key)
            if val is None or (isinstance(val, str) and not val.strip()):
                errors.append(f"{rel}: missing/empty required frontmatter key '{key}'")

        allowed_types = allowed_types_for(bundle_version)
        if ctype not in allowed_types and ctype != "<none>":
            errors.append(f"{rel}: type '{ctype}' not in the spec vocab {sorted(allowed_types)}")

        check_lists(rel, fm, errors)
        raw_fm = frontmatter_block(text)
        check_dates(rel, fm, raw_fm, bundle_version, errors)
        check_source_quoting(rel, fm, raw_fm, errors)
        if bundle_version == TRUST_SIGNALS_VERSION:
            check_generated(rel, fm, raw_fm, errors)
            check_verified_trust(rel, fm, raw_fm, bundle_version, errors)
            check_sources_plural(rel, fm, raw_fm, errors)
            check_status(rel, fm, errors)
            check_stale_after(rel, fm, raw_fm, errors)
            check_attested_computation(rel, fm, errors)

    # Link resolution: every internal link to a .md file must resolve to a file
    # that exists inside the bundle. A link escaping the bundle root or pointing at
    # a missing file is a hard failure, the bundle is validated as one
    # self-contained tree. To validate federated content, assemble the bundles into
    # a single tree and point --bundle at that root.
    for f in md_files:
        if f.suffix != ".md":
            continue  # non-conforming file already reported; don't pile on link errors
        text = strip_code(f.read_text(encoding="utf-8-sig"))
        for m in WIKILINK_RE.finditer(text):
            slug = m.group(1).strip()
            errors.append(
                f"{f.relative_to(bundle)}: '[[{slug}]]' is not an OKF link, use a "
                f"relative markdown link like [text]({slug}.md). The [[slug]] form is "
                f"the auto-memory convention, not OKF.")
        for raw in LINK_RE.findall(text):
            target, fragment = link_destination(raw)
            if not target:
                continue
            low = target.lower()
            # External/anchor links are out of scope. Lower-case the scheme test so an
            # uppercase scheme (HTTPS://...) is still recognized and skipped, not
            # resolved as a local path (which would falsely fail as escaping/dangling).
            if low.startswith(("http://", "https://", "mailto:", "#", "tel:")):
                continue
            # Match .md case-insensitively, the same way discovery now does, so a link
            # to an uppercase-extension file (ghost.MD) is checked for dangling/escape
            # instead of silently skipped. If such a target file exists it is separately
            # rejected as non-conforming above, so the two checks stay in agreement.
            if ".md" not in low:
                continue
            if is_rooted_link(target):
                errors.append(f"{f.relative_to(bundle)}: root-relative link not allowed "
                              f"(use a relative path) -> {target}")
                continue
            spelled = f.parent / target
            real = real_case_path(spelled, bundle)
            if real is not None:
                try:
                    resolved_real = real.resolve()
                except (OSError, RuntimeError):
                    real = None
                else:
                    if not resolved_real.is_relative_to(bundle):
                        errors.append(f"{f.relative_to(bundle)}: link escapes bundle root -> {target}")
                        continue
            try:
                dest = resolve_link(target, f)
            except (OSError, RuntimeError):
                errors.append(f"{f.relative_to(bundle)}: dangling link -> {target}")
                continue
            inside = dest == bundle or bundle in dest.parents
            if not inside and real is None:
                errors.append(f"{f.relative_to(bundle)}: link escapes bundle root -> {target}")
            else:
                if real is not None and not real.exists():
                    real = None
                if real is None:
                    nonconforming = real_case_path(
                        spelled, bundle, allow_nonconforming_md=True
                    )
                    if nonconforming is not None:
                        try:
                            resolved_nonconforming = nonconforming.resolve()
                        except (OSError, RuntimeError):
                            nonconforming = None
                        else:
                            if not resolved_nonconforming.is_relative_to(

# OKF spec v1

Open Knowledge Format (OKF) is a convention for storing knowledge as small markdown
files that both people and agents can read. One file describes one concept and carries
its own provenance. Directory `index.md` files provide navigation. A validator enforces
the contract so the knowledge base stays consistent as it grows.

This is the generic spec. A project may layer its own conventions on top (extra tags,
naming patterns, a fixed section list), but must not weaken the rules below.

## A note on version numbers

Three separate numbers show up in this project, and none of them track each other:

1. **Upstream Google OKF's own spec version**, `0.1` (June 2026), then `0.2` (July 2026,
   adding the trust/provenance/attestation vocabulary this document adopts below). This is
   Google's number, not this fork's.
2. **This skill's package version** (the `version:` field in `SKILL.md`'s frontmatter),
   its own release-numbering axis for the skill/plugin itself (bug fixes, secret-scanner
   hardening, Codex compatibility, and so on). It has no relationship to either spec version.
3. **This fork's own bundle-format version** (the `okf_version` marker every bundle-root
   `index.md` declares, and what `SUPPORTED_VERSIONS` in `validate.py` checks), `0.1`, then
   `0.2` (new allowed types), then `0.3` (datetime-form `timestamp`), and now `0.4` (this
   patch: the v0.2 trust/provenance fields below, and the required-key rename they force).

So "OKF v0.2" (upstream Google's spec) and "`okf_version: 0.2`" (a bundle declared under
*this fork's* second format revision, from months before Google's v0.2 existed) are two
unrelated things that happen to share a digit. Where this document says "upstream v0.2" it
means Google's; a bare "`okf_version: 0.4`" always means this fork's own marker.

## Relationship to upstream OKF

This spec is a strict fork of Google's Open Knowledge Format
([GoogleCloudPlatform/knowledge-catalog](https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf)).
It keeps the core idea (knowledge as small markdown files with YAML frontmatter and
`index.md` navigation) and tightens the contract so a validator can enforce it. If you
already know upstream OKF, these are the intentional differences to author against;
upstream conventions will otherwise produce files this validator rejects.

- Required keys. Upstream requires only `type` and treats everything else as recommended,
  with any extra key allowed. Here, for a bundle declaring `okf_version` `0.1` through `0.3`,
  all seven of `type`, `title`, `description`, `source`, `verified`, `timestamp`, and `tags`
  are required and non-empty. For a bundle declaring `0.4`, `verified` is renamed to
  `verified_on` in that required list (see "Trust and provenance (upstream v0.2 vocabulary)"
  for why), so a `0.4` bundle requires `type`, `title`, `description`, `source`,
  `verified_on`, `timestamp`, and `tags` instead. Either way, a file that conforms upstream
  (say, `type` alone) fails validation here.
- `resource` becomes `source`. Upstream's optional `resource` is one canonical URI for the
  underlying asset. This spec drops `resource` and requires `source`, a non-empty list of
  provenance pointers (paths, commands, URLs, events). Upstream v0.2 separately introduces its
  own richer `sources` (plural), this fork adopts that too, as an optional companion to the
  required `source`; see below.
- Citations fold into `source`. Upstream lists sources under a `# Citations` heading and
  allows a `references/` subdirectory. Here there is neither; all provenance lives in the
  `source` frontmatter list (and, optionally, the richer `sources` list below).
- `verified`/`verified_on` is added. Upstream v0.1 has no such key. Through `okf_version`
  `0.3`, this spec requires `verified`, the ISO date the fact was last confirmed true, kept
  distinct from `timestamp` (when the file was authored or edited). At `0.4`, this fork's own
  field is renamed `verified_on` to free the name `verified` for upstream v0.2's own
  `verified`, a different thing (see below).
- `timestamp`'s datetime form. Upstream's `timestamp` is an ISO 8601 datetime, and
  `check_dates` accepts that form for `timestamp` in a `0.3`-or-later bundle (with or without
  an offset, including a trailing `Z`), so an upstream bundle validates here unchanged. A
  `0.1`/`0.2` bundle's `timestamp` stays date-only. `verified`/`verified_on` is always
  date-only, in every version: it records the day a fact was last confirmed true, and a time
  of day there is false precision about the confirmation.
- The type vocab is closed. Upstream types are freeform and unregistered, and consumers
  must tolerate unknown ones. Here the set is fixed (see Type vocab) and an unlisted type
  fails the build, which catches typos; extending it is a deliberate spec edit.
- Links are strict, and must be relative. Upstream treats a broken link as tolerable
  ("consumers MUST tolerate broken links") and permits bundle-root-relative targets like
  `/tables/customers.md`. Here every intra-bundle link must resolve or validation fails, the
  `[[slug]]` wikilink form is rejected outright, and any target beginning with `/` is
  rejected as a root-relative link, so an upstream `/tables/customers.md` must be rewritten
  relative to the file that links it.
- A root `index.md` declaring `okf_version` is mandatory. Upstream treats `index.md` and
  `okf_version` as optional. Here the bundle root must contain an `index.md` whose
  frontmatter declares `okf_version` (and carries nothing else), so an upstream bundle with
  no root index, or one that omits the version marker, fails validation.
- Secret values fail the build. Upstream OKF has no secret-scanning rule. Here `validate.py`
  scans every markdown file for private-key blocks, cloud-token shapes, and `secret=<value>`
  assignments and fails the bundle on a hit, so an upstream bundle that inlines a credential
  value passes upstream but is rejected here (see Security).
- Trust/provenance/attestation fields are adopted, not required. Upstream v0.2 adds
  `generated`, `sources`, `status`, `stale_after`, `verified` (the new, upstream shape), and
  an `Attested Computation` type, all of it optional there, and all of it optional here too.
  Adopting none of it leaves a bundle exactly as valid as before; see the dedicated section
  below.

This fork's own format version differs from upstream's on a separate axis (see "A note on
version numbers" above): upstream is now `0.1`/`0.2`, this fork's current bundle-format
version is `0.4` (the validator still accepts `0.1`, `0.2`, and `0.3`). Every difference
above pulls the same direction: upstream stays minimal and needs no tooling to stay
portable, while this fork adds a validator that fails the build when a bundle drifts.

## Bundle model

A bundle is a directory tree. The simplest bundle is one directory of concept files
with an `index.md`. Larger bundles group concepts into subdirectories by subject.

```
<bundle-root>/
  index.md                 carries okf_version, here and nowhere else
  <section>/
    index.md               navigation for the section
    <concept>.md           one concept per file
```

## Files

- The bundle-root `index.md` carries `okf_version` (one of `"0.1"`–`"0.4"`; `scaffold.py`
  still writes `"0.3"` by default, `"0.4"` only with `--trust-signals`) in frontmatter,
  only there.
- Per-directory `index.md`: a heading, an optional one-line preamble, then bullet
  navigation. No frontmatter. (Keep the preamble to one line, it orients, it doesn't narrate.)
- `log.md` (optional, per directory): dated entries, newest first, no frontmatter.
- Concept files: one concept each, frontmatter required (below).
- Reserved filenames: `index.md`, `log.md`.
- All markdown files use a lowercase `.md` extension. A non-lowercase extension (`.MD`,
  `.Md`) is non-conforming and rejected, the validator discovers files case-insensitively
  so such a file cannot escape validation, and link checks match `.md` case-insensitively too.

The validator enforces the frontmatter rules here, reserved files carry no frontmatter, and
only the bundle-root `index.md` carries `okf_version` (and nothing else). The index/log body
shapes above are recommendations for human readers, not validator-checked structure.

The current format version is `0.4`, but `scaffold.py` still emits `0.3` by default,
unchanged from before this patch, and only writes `0.4` when explicitly asked
(`scaffold.py --trust-signals`; see Tooling). The validator accepts `0.1` through `0.4`
either way, so a newer validator still reads an older bundle. Version `0.2` added allowed
types. Version `0.3` admitted a full datetime in `timestamp`. Version `0.4` renames
`verified` to `verified_on` (freeing `verified` for the new upstream-v0.2 shape) and adds the
optional trust/provenance fields below, a bundle that adopts none of them still only needs to
declare `0.4` because of the rename; one that keeps declaring `0.1`–`0.3` keeps the old
`verified` field exactly as before and simply cannot use the new optional fields (see next
section). The marker makes these grammar changes explicit, so an older validator reports a
clear unsupported-version error instead of a misleading field error.

## Concept frontmatter (required keys, all non-empty)

For a bundle declaring `okf_version` `0.1` through `0.3`:

| key | value |
| --- | --- |
| `type` | one of the type vocab below |
| `title` | the concept name |
| `description` | one line |
| `source` | YAML list of provenance pointers (paths, commands, URLs, events) |
| `verified` | ISO date `YYYY-MM-DD` the fact was last confirmed true (see note below) |
| `timestamp` | ISO date authored/updated, or in a `0.3` bundle a full ISO 8601 datetime |
| `tags` | YAML list |

For a bundle declaring `okf_version` `0.4`, the table is identical except `verified` is
named `verified_on` (same required-ness, same meaning, same date-only rule, only the key
name changes, to make room for the new `verified` described below).

`verified`/`verified_on` note: it records when the fact was last confirmed true, which is not
always today. A fact you re-checked against reality now is confirmed today, as is one the user
is the authority for, a decision, preference, or intent they state directly. But a fact the
user is recalling about external or system state is a source claim, not a re-check: date it to
when that state was last checked or to the recollection's own date, not today. A claim copied
from a dated source without re-checking carries that source's date. A fact taken from an
undated record you cannot re-confirm (a memory file, an old conversation) carries the oldest
date you can evidence, file timestamp, introducing commit, or the date it was said, never
today; if no date can be evidenced, the fact is not yet verifiable, so find a datable source or
leave it out. When the date is uncertain, round it down: an older `verified`/`verified_on`
reads as "may be stale," today reads as "just confirmed." The date is the contract; a caveat in
the concept body does not undo it, because the validator and tools read only the date.

`timestamp` may be an ISO date (`YYYY-MM-DD`) in every supported format version. A `0.3` or
`0.4` bundle may instead preserve a full datetime in the exact form
`YYYY-MM-DDTHH:MM:SS[.fraction][Z|+HH:MM|-HH:MM]`; a space may replace `T`. Versions `0.1`
and `0.2` remain date-only. `verified`/`verified_on` is always date-only.

`source` quoting rule (hard): QUOTE every element of the `source` list. Source pointers
routinely carry YAML-significant characters, a `#` (e.g. `"issue #445"`) starts a comment
and corrupts the flow sequence, a colon-space `: ` splits a mapping, so a strict parser
rejects an unquoted source. Always quote them:

```yaml
source: ["README.md", "issue #445", "git log 9c2e510"]
```

The validator enforces this in both list styles. In flow style (`["a", b]`) an unquoted
element carrying a significant character fails to parse and is reported as an error. In block
style (`- a`) YAML would silently drop an inline `#` comment and pass, so the validator also
scans the raw source text and rejects an unquoted element with a `#`. An element that is
already quote-safe (a bare filename) is accepted either way, quote everything anyway so you
never have to judge which is which.

`tags` and `description` follow a lighter rule: quote an element only when it contains a
YAML-significant character (a colon-space `: `, a leading `[ { # * & ! | > % @` or quote,
or a trailing `:`); plain kebab tokens like `canonical` may stay unquoted. Quoting when
unsure is always safe. Hard quoting is `source` only.

Provenance lives in `source`, there is no separate citations section or references directory.

## Trust and provenance (upstream v0.2 vocabulary)

Upstream Google OKF v0.2 (July 2026) added a second kind of frontmatter field: not one that
describes a concept, but one a consumer uses to decide whether to trust it before reading the
body. This fork adopts that vocabulary as optional additions, available on any concept
regardless of type, in a bundle declaring `okf_version` `0.4`. None of it is required. A
concept that uses none of these fields is exactly as valid as one authored against `0.1`.

As with upstream, this fork records the raw signals and leaves scoring to the consumer, there
is no computed trust score anywhere in a concept file or in the validator's output. A tool that
wants "only surface human-reviewed metrics" derives that filter itself from the fields below.

- **`generated`**, an optional mapping `{by, at}`: who or what produced the current content,
  and when it last meaningfully changed. `by` is a non-empty string identifying the producer
  (a model/agent name, or `human:<id>`); `at` is an ISO date or full ISO 8601 datetime. This
  sits alongside the required `timestamp`, not in place of it, `timestamp` is this fork's own
  authored/updated marker and stays required; `generated` is the richer, optional upstream
  form for describing production, and the two may describe the same event.
- **`verified`** (only meaningful in a `0.4` bundle, where it is not the required key, see
  above), an optional YAML list of independent confirmations, each a mapping
  `{by, at}`: `by` a non-empty string (a `human:<id>` actor or a machine/agent identifier), `at`
  an ISO date. A consumer derives a trust tier from this list: no `verified` key is
  *unverified*; every entry from a machine/agent actor only is *machine-confirmed*; any entry
  from a `human:<id>` actor is *human-reviewed*. The validator checks only that the list is
  well-formed (non-empty entries, valid dates), deriving and filtering on a tier is a
  consumer's job, not this format's.
- **`sources`** (plural, distinct from the required singular `source`), an optional YAML list
  of structured provenance objects, each with a required `id` (non-empty string, unique within
  the list, used to key an in-body footnote like `[^warehouse-schema]`) and `resource`
  (non-empty string: a URL or bundle-relative path), plus optional `title`, `author`
  (non-empty strings), `usage_count` (a non-negative integer), and `last_modified` (an ISO
  date). Where the required `source` is a flat list of quoted pointers, `sources` lets each
  pointer carry its own credibility signals and be cited per-claim in the body via an ordinary
  markdown footnote. The validator checks shape only, it does not cross-check that every
  `[^id]` footnote in the body has a matching `sources[].id`, or vice versa; treat that
  cross-reference as a human/reviewer responsibility for now.
- **`status`**, an optional string, one of `draft`, `stable`, `deprecated`. Absent means
  `stable`. A `deprecated` concept is kept for history/reproducibility but should not be
  surfaced to new work.
- **`stale_after`**, an optional ISO date `YYYY-MM-DD`. An absolute date, deliberately, not a
  relative TTL: staleness is then a plain date comparison with no reference to when the
  concept happened to be read.

None of the above changes what the validator requires; it only makes the *absence* of these
fields distinguishable from their presence where they matter to a consumer deciding whether to
act on a concept. Absence means the signal was not supplied. A trust field that is explicitly
present with a YAML null value is invalid; present fields must have the shape documented above.

## Type vocab

Infrastructure and ops (fleet maps, system docs): `Machine`, `Network`, `Service`,
`Session`, `Project`, `Repo`, `Credential`, `Path`, `Process`.

Domain-neutral (newsrooms, research atlases, decision logs): `Concept`, `Decision`,
`Event`, `Person`, `Org`, `Source`.

`Reference` is the catch-all for a concept that is not one of the others. Index files carry
no frontmatter, so there is no `Index` type. The set is closed: an unlisted type fails the
build, which catches typos. To extend it, add the type here and in `scripts/validate.py`.
`Attested Computation` joins this closed vocabulary only in a bundle declaring
`okf_version` `0.4`; versions `0.1` through `0.3` reject it.

### `Attested Computation` (upstream v0.2)

A concept of this type carries a sanctioned way to compute a value, and the means to check
that the sanctioned computation actually ran, the answer to "was this number produced the way
we said it must be," distinct from `verified` above (which confirms a *definition* still
matches policy, not that any one run produced a correct value). It carries these keys in
addition to the seven (or, at `0.4`, six-plus-`verified_on`) base required keys:

| key | value |
| --- | --- |
| `runtime` | non-empty string naming the execution environment (e.g. `bigquery`) |
| `parameters` | YAML list (possibly empty) of mappings, each `{name, type, required}`, the declared inputs a caller may fill; nothing else |
| `executor` | mapping `{resource, receipt}`, `resource` points at the skill/tool that runs the computation, `receipt` a list naming what it returns (e.g. `[job_id, executed_sql, result]`) |
| `attester` | mapping `{resource}`, points at the deterministic, non-LLM checker that compares a receipt against this concept's sanctioned computation |

OKF records the computation and how to check it; this fork's validator checks only that these
four keys are present and correctly shaped when `type: Attested Computation`. It never runs
the computation, the executor, or the attester itself, that is a consumer's runtime
responsibility, entirely outside this format and this validator.

## Links

Relative markdown links. Every link to a file inside the bundle must resolve to a file that
exists; a link that escapes the bundle root or dangles fails validation. The bundle is
validated as one self-contained tree (see Federation for combining several).

A link's case must match the file on disk. Case-insensitive filesystems on macOS
and Windows can resolve `Concepts/Foo.md` to `concepts/foo.md`, while the same
link dangles on Linux. The validator checks the spelling of each link component
against directory listings before resolving can replace it with canonical casing.
Its error gives the corrected relative link. Symlink and `..` components stay in
filesystem traversal order, and the resolved destination must remain inside the
bundle root.

The `[[slug]]` wikilink form is not an OKF link, and the validator rejects it. It is the
auto-memory cross-reference idiom and easy to reach for by habit, but a `[[slug]]` is never
resolved or checked, so a dead reference would pass silently. Always link with
`[text](relative/path.md)`.

## Federation (optional)

Several bundles can be combined into one tree. Add a new root `index.md` that carries
`okf_version` and links to each member, then place each bundle under a uniquely named
subdirectory of that root. A member's own `index.md` is now a nested section index, so remove
its `okf_version` frontmatter block entirely, a nested `index.md` carries no frontmatter at
all. Write cross-bundle links as relative paths into the sibling directories. Validate by
pointing the validator at the new root, so every link resolves and the single `okf_version`
gate runs once at the top.

A member's marker is stripped when it is nested, so it can no longer be validated on its own
from inside the combined tree, validate the assembled root instead. (Per-node validation that
keeps a marker in each member is planned but not yet built.) Most single-repo knowledge bases
never need any of this.

## Security (hard)

- No secret VALUES anywhere. A credential concept documents the key name, where it lives, and
  how it is retrieved, never the value itself.
- The validator scans for private-key blocks, cloud-token shapes, and `secret=<value>`
  assignments and fails the build on a hit. If a pattern false-positives on legitimate text,
  narrow the pattern; do not delete the rule.
- OKF makes no claim about whether your bundle is public or private. That is your decision,
  but a bundle that documents real infrastructure is usually internal. Decide deliberately
  before publishing.

## Tooling

- `validate.py`, frontmatter conformance, date/list checks, link resolution, secret scan.
  Run it before every commit; it must exit 0.
- `scaffold.py`, generate a conforming starter bundle.

"""Tests for the okf-wiki skill: the scaffolder and the validator.

Each test runs the real CLI scripts in a temp directory, the same way a user
would. Run: python3 -m pytest okf-wiki/tests/ -q
"""
import json
import os
import subprocess
import sys
from pathlib import Path

import pytest

SKILL = Path(__file__).resolve().parent.parent
SCAFFOLD = SKILL / "scripts" / "scaffold.py"
VALIDATE = SKILL / "scripts" / "validate.py"
TEMPLATE_ANCHOR = SKILL / "templates" / "hooks" / "okf-anchor.py"
TEMPLATE_ORIENT = SKILL / "templates" / "hooks" / "okf-orient.py"

# Import the scaffolder module to unit-test its pure helpers directly. The CLI tests
# below drive scaffold.py as a subprocess; the requirements name-sniffing contract is
# fine-grained enough to pin here without a subprocess round-trip.
sys.path.insert(0, str(SKILL / "scripts"))
import scaffold as scaffold_mod  # noqa: E402  the module under test; the local scaffold() helper below shadows the bare name

# Load gh-wiki-bootstrap.py by path (its hyphenated name is not importable) so its
# pure save-detection helper can be unit-tested without a browser. Its playwright
# import lives inside main(), so importing the module top-level touches only stdlib.
import importlib.util  # noqa: E402
_boot_spec = importlib.util.spec_from_file_location(
    "gh_wiki_bootstrap", SKILL / "scripts" / "gh-wiki-bootstrap.py")
gh_wiki_bootstrap = importlib.util.module_from_spec(_boot_spec)
_boot_spec.loader.exec_module(gh_wiki_bootstrap)

# The scaffolder evaluates a PEP 508 environment marker on a preserved requirements.txt's
# PyYAML line only when `packaging` is importable; without it, it falls back to treating the
# line as installable. Tests that assert an EXCLUDING marker warns therefore need packaging
# present, so they guard on this. A matching or absent marker holds either way (the fallback
# also selects the env), so those need no guard.
HAS_PACKAGING = importlib.util.find_spec("packaging") is not None

GOOD = """---
type: Process
title: good
description: a good concept
source: ["README.md"]
verified: 2026-06-23
timestamp: 2026-06-23
tags: ["x"]
---
# good
"""


def scaffold(target, *args):
    r = subprocess.run([sys.executable, str(SCAFFOLD), str(target), *args],
                       capture_output=True, text=True)
    return r.returncode, r.stdout + r.stderr


def validate(bundle, *args):
    r = subprocess.run([sys.executable, str(VALIDATE), "--bundle", str(bundle), *args],
                       capture_output=True, text=True)
    return r.returncode, r.stdout + r.stderr


def write_concept(bundle, text, name="concepts/c.md"):
    p = bundle / name
    p.parent.mkdir(parents=True, exist_ok=True)
    p.write_text(text, encoding="utf-8")


def run_hook(script, project_dir, stdin=None, cwd=None, extra_env=None):
    env = dict(os.environ, CLAUDE_PROJECT_DIR=str(project_dir))
    if extra_env:
        env.update(extra_env)
    r = subprocess.run([sys.executable, str(script)], input=stdin,
                       capture_output=True, text=True, env=env, cwd=cwd)
    return r.returncode, r.stdout, r.stderr


def settings_of(target):
    return json.loads((target / ".claude" / "settings.json").read_text())


# --- scaffold ---------------------------------------------------------------

def test_default_scaffold_validates(tmp_path):
    rc, out = scaffold(tmp_path / "kb")
    assert rc == 0, out
    assert "PASS" in out
    assert (tmp_path / "kb" / "bundle" / "index.md").exists()
    assert (tmp_path / "kb" / "SPEC.md").exists()
    assert (tmp_path / "kb" / "scripts" / "validate.py").exists()


def test_multi_section_scaffold_validates(tmp_path):
    rc, out = scaffold(tmp_path / "kb", "--sections", "concepts,services,decisions")
    assert rc == 0, out
    for s in ("concepts", "services", "decisions"):
        assert (tmp_path / "kb" / "bundle" / s / "index.md").exists()


def test_root_index_carries_okf_version(tmp_path):
    scaffold(tmp_path / "kb")
    root = (tmp_path / "kb" / "bundle" / "index.md").read_text()
    assert 'okf_version: "0.3"' in root


def test_refuses_nonempty_dir_without_force(tmp_path):
    target = tmp_path / "kb"
    target.mkdir()
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target)
    assert rc == 1 and "not empty" in out


def test_force_writes_into_nonempty_dir(tmp_path):
    target = tmp_path / "kb"
    target.mkdir()
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out


def test_bad_date_arg_rejected(tmp_path):
    rc, out = scaffold(tmp_path / "kb", "--date", "2026-13-99")
    assert rc == 1 and "ISO" in out


def test_section_slugifying_to_empty_is_rejected(tmp_path):
    # a punctuation-only section name slugifies to "" and would otherwise write
    # its index over bundle/index.md, clobbering the root. Must be refused.
    rc, out = scaffold(tmp_path / "kb", "--sections", "..")
    assert rc == 1 and "no alphanumeric" in out
    assert not (tmp_path / "kb" / "bundle" / "index.md").exists()


def test_duplicate_sections_collapse(tmp_path):
    rc, out = scaffold(tmp_path / "kb", "--sections", "notes,notes")
    assert rc == 0, out
    assert (tmp_path / "kb" / "bundle" / "notes" / "index.md").exists()


def test_scaffold_writes_requirements(tmp_path):
    # the validator depends on PyYAML; the scaffolded project must declare it.
    rc, out = scaffold(tmp_path / "kb")
    assert rc == 0, out
    assert "PyYAML" in (tmp_path / "kb" / "requirements.txt").read_text()


def test_force_preserves_existing_content(tmp_path):
    # --force into a populated project must not clobber a user's own files with the
    # generic template. Scaffold once, edit content, re-scaffold --force: the edits
    # survive and the run reports what it preserved.
    target = tmp_path / "kb"
    scaffold(target)
    readme, root = target / "README.md", target / "bundle" / "index.md"
    readme.write_text("MY OWN README\n", encoding="utf-8")
    root.write_text("MY OWN INDEX\n", encoding="utf-8")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    assert readme.read_text() == "MY OWN README\n"
    assert root.read_text() == "MY OWN INDEX\n"
    assert "preserved" in out and "README.md" in out


def test_force_preserve_skips_validation(tmp_path):
    # when --force preserves a non-OKF bundle/index.md, the scaffold is no longer
    # valid by construction; it must skip validation, not fail as "a bug in
    # scaffold.py" over intentionally preserved user content.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "bundle" / "index.md").write_text("# my repo index\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "skipping validation" in out.lower()
    assert "bug in scaffold.py" not in out
    # the printed validate command must cd into the target, not the caller's cwd
    assert f"cd {target}" in out


def test_force_preserve_does_not_run_preserved_validator(tmp_path):
    # a preserved user scripts/validate.py must never be executed by the scaffolder.
    target = tmp_path / "kb"
    scaffold(target)
    sentinel = tmp_path / "ran"
    (target / "scripts" / "validate.py").write_text(
        f"import pathlib; pathlib.Path(r'{sentinel}').write_text('x')\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert not sentinel.exists(), "preserved validate.py must not be run"


def test_missing_pyyaml_skips_validation(tmp_path):
    # validate.py needs PyYAML; if it is absent the scaffold must still succeed and
    # say so plainly, not mislabel the missing dependency as a bug in scaffold.py.
    # Simulate absence with -S (no site-packages); skip if yaml is still findable.
    probe = subprocess.run(
        [sys.executable, "-S", "-c",
         "import importlib.util, sys; "
         "sys.exit(0 if importlib.util.find_spec('yaml') is None else 3)"])
    if probe.returncode != 0:
        pytest.skip("PyYAML is importable even under -S; cannot simulate its absence")
    target = tmp_path / "kb"
    r = subprocess.run([sys.executable, "-S", str(SCAFFOLD), str(target)],
                       capture_output=True, text=True)
    out = r.stdout + r.stderr
    assert r.returncode == 0, out
    assert "PyYAML is not installed" in out
    assert "bug in scaffold.py" not in out
    assert (target / "bundle" / "index.md").exists()
    # the printed validate command must cd into the target, not the caller's cwd
    assert f"cd {target}" in out


def test_force_preserve_warns_when_requirements_lacks_pyyaml(tmp_path):
    # A preserved requirements.txt is the user's own and stays untouched (#142). When it
    # omits PyYAML, the skip-validation message must name the exact missing dependency so
    # the user is not left to decode a later ModuleNotFoundError from validate.py.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text("requests>=2\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert (target / "requirements.txt").read_text() == "requests>=2\n"  # never edited
    assert "does not list PyYAML" in out
    assert "PyYAML>=5.1" in out


def test_force_preserve_no_pyyaml_warning_when_declared(tmp_path):
    # If the preserved requirements.txt already declares PyYAML (any case or pin), the
    # targeted warning must not fire: validation is still skipped for the preserve, but we
    # do not nag a user who did the right thing. Lowercase + pin proves name normalization.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text("pyyaml==6.0.1\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "skipping validation" in out.lower()  # the preserve still skips validation
    assert "does not list PyYAML" not in out


def test_force_preserve_no_pyyaml_warning_with_include(tmp_path):
    # A "-r base.txt" include can declare PyYAML in a file the scaffolder does not read, so
    # an uncertain requirements.txt suppresses the targeted warning rather than making a
    # false "missing" claim.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text("-r base.txt\nrequests\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "skipping validation" in out.lower()
    assert "does not list PyYAML" not in out


def test_force_preserve_warns_with_constraint_file_and_no_pyyaml(tmp_path):
    # A "-c constraints.txt" only pins versions of packages installed elsewhere; unlike an
    # include it cannot supply a missing package, so a preserved file that carries only a
    # constraint and no PyYAML declaration must still warn. Guards against treating -c as an
    # include and silently suppressing the note (pip: -c "Constrain versions", not install).
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text("-c constraints.txt\nrequests\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" in out
    assert "PyYAML>=5.1" in out


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_force_preserve_warns_when_pyyaml_marker_excludes_env(tmp_path):
    # #185: a PyYAML line gated by a PEP 508 marker to another environment installs nothing
    # here, so the preserved file still lacks an importable PyYAML. The name sniffer alone
    # reads it as declared and stays silent; the marker must be evaluated so the note fires
    # when the declaration will not install for the interpreter that runs validate.py.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        'PyYAML; sys_platform == "no-such-platform"\n', encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" in out
    assert "PyYAML>=5.1" in out


def test_force_preserve_no_warning_when_pyyaml_marker_matches(tmp_path):
    # the other side: a PyYAML line whose marker selects this interpreter installs normally,
    # so the note must stay silent. Guards against over-nagging on a valid environment gate.
    # Holds with or without packaging: the marker matches, and the no-packaging fallback also
    # treats the line as installable.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        'PyYAML; python_version >= "3.0"\n', encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "skipping validation" in out.lower()
    assert "does not list PyYAML" not in out


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_force_preserve_warns_when_pyyaml_hash_line_marker_excludes_env(tmp_path):
    # #185 (review): pip-compile --generate-hashes writes a requirement as a backslash-
    # continued block: the marker on the first physical line, indented `--hash` options on
    # following lines. Read physically, the first line keeps a trailing '\' that breaks marker
    # parsing, so an environment-excluded PyYAML read as installable and the note went silent.
    # Joining continuations first reconstructs the logical line, drops the hash options, and
    # the excluding marker fires the note.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        'pyyaml==6.0.1 ; sys_platform == "no-such-platform" \\\n'
        '    --hash=sha256:aaaa \\\n'
        '    --hash=sha256:bbbb\n', encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" in out


def test_force_preserve_no_warning_when_pyyaml_hash_line_no_marker(tmp_path):
    # guard on the join: a hash-locked PyYAML with no marker reconstructs to an installable
    # line, so the note stays silent. Holds with or without packaging.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        'pyyaml==6.0.1 \\\n    --hash=sha256:aaaa \\\n    --hash=sha256:bbbb\n', encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" not in out


def test_force_preserve_no_warning_when_comment_ends_in_backslash(tmp_path):
    # #185 (review): pip does not continue a full-line comment even when it ends in '\'. If the
    # join treated the comment as continuing, it would swallow the following PyYAML line into
    # the comment, hiding the declaration and firing a false 'missing PyYAML' note. The comment
    # must be emitted on its own so the real PyYAML line is read. Holds with or without packaging.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        "# a trailing note that ends in a backslash \\\n"
        "PyYAML==6.0.1\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" not in out


def test_force_preserve_no_warning_when_comment_backslash_precedes_include(tmp_path):
    # #185 (review): same join bug, higher-impact variant. A comment ending in '\' just before
    # an `-r` include directive would swallow the directive, so the include (which may supply
    # PyYAML from a file we do not read) goes unseen and the note wrongly fires. The comment
    # must not continue, so the `-r` line is read and suppresses the note.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        "# via requirements.in \\\n"
        "-r base.txt\n", encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" not in out


@pytest.mark.parametrize("line,expected", [
    ("PyYAML>=5.1", True),                       # no marker: installs here
    ('PyYAML; python_version >= "3.0"', True),   # marker selects this interpreter
    ("PyYAML  # just a note", True),             # inline comment, no marker
    ("PyYAML; ", True),                          # empty marker after ';'
])
def test_requirement_marker_selects_env_true_cases(line, expected):
    # a missing, empty, or matching marker selects the current environment. These hold with
    # or without packaging, since the packaging-absent fallback also returns True, so no
    # skip guard is needed here (only the excluding case below depends on packaging).
    assert scaffold_mod._requirement_marker_selects_env(line) == expected


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_excludes_env():
    # an excluding marker returns False only when packaging can evaluate it; the fallback
    # (packaging absent) returns True, so this case is guarded on packaging being present.
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML; sys_platform == "no-such-platform"') is False


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_hash_inside_quoted_marker_excludes():
    # a '#' inside a quoted marker string is not a pip comment; stripping at the first '#'
    # would corrupt the marker. Parsing the full requirement keeps it intact, so the
    # excluding marker still evaluates (no implementation is named "cpython#not").
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML; implementation_name == "cpython#not"') is False


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_semicolon_inside_url_excludes():
    # a ';' inside a direct-reference URL is part of the URL, not the marker separator.
    # Splitting on the first ';' would misread the URL as the marker; parsing the full
    # requirement extracts the real trailing marker after ' ; ' and evaluates it.
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML @ https://example.com/pkg;param ; sys_platform == "no-such-platform"') is False


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_hashed_requirement_excluded():
    # a hash-pinned line (pip-compile --generate-hashes) carries pip's per-requirement
    # `--hash` option, which packaging.requirements.Requirement rejects. The pip option must
    # be stripped before parsing, or the excluding marker is never seen and the missing-PyYAML
    # warning is wrongly suppressed. With the option dropped the marker evaluates and excludes.
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML==6.0.1; sys_platform == "no-such-platform" '
        '--hash=sha256:aaaa --hash=sha256:bbbb') is False


def test_requirement_marker_hashed_requirement_matches():
    # a hash-pinned PyYAML with no marker still counts as installable: dropping the `--hash`
    # options leaves a bare, applicable requirement. Holds with or without packaging (the
    # packaging-absent fallback also returns True), so no skip guard is needed.
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML==6.0.1 --hash=sha256:aaaa') is True


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_whitespace_hash_inside_quoted_marker_excludes():
    # #185 (review): a '#' preceded by whitespace *inside* a quoted marker value is part of the
    # marker, not a pip inline comment. Stripping the comment before parsing (the earlier
    # regex-first approach) corrupts the marker and the line reads as installable, wrongly
    # suppressing the note. Parsing the whole line first keeps the '#' intact, so the excluding
    # marker still evaluates (no implementation is named "cpython #not").
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML; implementation_name == "cpython #not"') is False


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_trailing_comment_after_marker_excludes():
    # guard on the parse-first change: a real pip inline comment after the marker must still be
    # stripped so the marker evaluates. The whole-line parse fails on the trailing comment, then
    # the comment is removed on retry and the excluding marker evaluates to False.
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML; sys_platform == "no-such-platform"  # windows only') is False


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_requirement_marker_unevaluable_lockfile_marker_does_not_crash():
    # #185 (review): packaging 26 added lock-file-context marker variables. A line such as
    # `PyYAML; dependency_groups == "docs"` parses, but Marker.evaluate() raises KeyError in the
    # default metadata context because that variable is not defined there. Any evaluation failure
    # must be treated as unjudgeable and fall back to installable, not crash. Asserts the outcome
    # (True, no exception), which also holds on older packaging that rejects the marker at parse
    # time (InvalidRequirement, then True via the parse fallback).
    assert scaffold_mod._requirement_marker_selects_env(
        'PyYAML; dependency_groups == "docs"') is True


@pytest.mark.skipif(not HAS_PACKAGING, reason="marker evaluation needs the packaging library")
def test_force_preserve_no_crash_on_unevaluable_marker(tmp_path):
    # #185 (review): a preserved requirements.txt whose PyYAML line carries a marker that parses
    # but cannot be evaluated in the default context (dependency_groups, a lock-file-only
    # variable) must not crash the --force run. The marker is unjudgeable, so PyYAML counts as
    # possibly installable and the note stays silent; the run exits 0.
    target = tmp_path / "kb"
    scaffold(target)
    (target / "requirements.txt").write_text(
        'PyYAML; dependency_groups == "docs"\n', encoding="utf-8")
    rc, out = scaffold(target, "--force")
    assert rc == 0, out
    assert "does not list PyYAML" not in out


@pytest.mark.parametrize("line,expected", [
    ("PyYAML>=5.1", "pyyaml"),              # version specifier stripped, lowercased
    ("pyyaml==6.0.1  # pinned", "pyyaml"),  # pin and inline comment stripped
    ("PyYAML[extra]", "pyyaml"),            # extras bracket stripped
    ("pyyaml @ https://example.com/p.whl", "pyyaml"),  # direct reference: name kept
    ("ruamel.yaml", "ruamel-yaml"),         # PEP 503 separator collapse (not PyYAML)
    ("# a comment", None),                  # comment names no package
    ("-r base.txt", None),                  # include option names no package
    ("https://example.com/p.whl", None),    # bare URL names no distribution
    ("./local/pkg", None),                  # local path names no distribution
    ("", None),                             # blank line
])
def test_canonical_req_name(line, expected):
    # The pure name sniffer the preserved-requirements PyYAML check relies on: PEP 503
    # normalization of the leading distribution name, and None for any line that names no
    # bare package (comment, option, include, URL, or path). Pinned here so a future edit
    # to the normalization cannot silently break PyYAML detection.
    assert scaffold_mod._canonical_req_name(line) == expected


# --- validator (negative cases) ---------------------------------------------

def test_validator_passes_on_good_concept(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD)
    rc, out = validate(b)
    assert rc == 0, out


def test_missing_required_key_fails(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.replace('tags: ["x"]\n', ""))
    rc, out = validate(b)
    assert rc == 1 and "tags" in out


def test_bad_type_fails(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.replace("type: Process", "type: Wizard"))
    rc, out = validate(b)
    assert rc == 1 and "not in the spec vocab" in out


def test_domain_neutral_type_validates(tmp_path):
    # the vocab is a superset: domain-neutral types (newsroom/research/decision-log)
    # validate alongside the infrastructure types. Closed-set typo rejection is still
    # covered by test_bad_type_fails above.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    for t in ("Concept", "Decision", "Event", "Person", "Org", "Source"):
        write_concept(b, GOOD.replace("type: Process", f"type: {t}"), name=f"concepts/{t}.md")
    rc, out = validate(b)
    assert rc == 0, out


def test_invalid_date_shape_reports_cleanly(tmp_path):
    # date-shaped but invalid (month 13), PyYAML raises ValueError during parse,
    # which is not a YAMLError. Must report cleanly, not crash with a traceback.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.replace("verified: 2026-06-23", "verified: 2026-13-99"))
    rc, out = validate(b)
    assert rc == 1 and "parse error" in out
    assert "Traceback" not in out


def test_source_must_be_list(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.replace('source: ["README.md"]', 'source: "README.md"'))
    rc, out = validate(b)
    assert rc == 1 and "must be a YAML list" in out


def test_secret_value_fails(tmp_path):
    # build an AWS-key-shaped string from fragments so no literal secret-shaped
    # token lives in this test file.
    fake = "AKIA" + "IOSFODNN7" + "EXAMPLE"
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\nkey = {fake}\n")
    rc, out = validate(b)
    assert rc == 1 and "secret leak" in out


# A labeled base64url secret value: URL-safe (- and _), so the base64-standard
# generic pattern misses it. Built from fragments so no secret-shaped token lives
# in this file. Its Shannon entropy is ~4.84 bits/char, well above the 4.0 floor.
URLSAFE_SECRET = "Zk9" + "_qX2" + "-Lm7" + "vB4t" + "Nc1w" + "Rp8h" + "Ej6" + "-uYs"


def test_urlsafe_secret_passes_without_entropy_scan(tmp_path):
    # Default behavior is unchanged: the opt-in scan is off, and the generic
    # base64 pattern deliberately does not match a hyphen/underscore value.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\napi_key: {URLSAFE_SECRET}\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_entropy_scan_flags_urlsafe_secret(tmp_path):
    # With the opt-in flag, the same URL-safe value is caught by entropy.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\napi_key: {URLSAFE_SECRET}\n")
    rc, out = validate(b, "--secret-entropy-scan")
    assert rc == 1 and "high-entropy assignment" in out


def test_entropy_scan_keeps_okf_key_path(tmp_path):
    # The precision the earlier review round asked us to keep: a slash-delimited
    # OKF key path is not a secret even under the strict scan. Its own entropy
    # (~4.07) clears the floor, so this proves the structural `/` exclusion, not
    # just the threshold, is what protects documented key paths.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nsecret: services/api/production-primary-key-path\n")
    rc, out = validate(b, "--secret-entropy-scan")
    assert rc == 0, out


def test_entropy_scan_keeps_key_path_with_long_first_segment(tmp_path):
    # Regression for the #150 review: the `/` exclusion stops the match at the
    # separator, but a first path segment of >=24 url-safe chars was still captured
    # and entropy-checked on its own. Here `prd-usw2-mysql-rw-20260722-key-path`
    # (35 chars, entropy 4.01, above the floor) precedes the `/service` suffix, so
    # the old pattern flagged the segment as a value. The trailing lookahead now
    # requires a complete token, so a documented key path stays clean regardless of
    # how long its leading segment is.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nsecret: prd-usw2-mysql-rw-20260722-key-path/service\n")
    rc, out = validate(b, "--secret-entropy-scan")
    assert rc == 0, out


def test_entropy_scan_ignores_low_entropy_name(tmp_path):
    # A slashless but human-readable hyphenated value stays under the floor, so
    # the strict scan does not flag it.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\napi_key: prod-key-path-name-placeholder\n")
    rc, out = validate(b, "--secret-entropy-scan")
    assert rc == 0, out


def test_entropy_scan_flags_secret_just_above_floor(tmp_path):
    # Recall is the flag's whole reason to exist, so pin it at the knife-edge: a
    # 24-char base64url value whose entropy is 4.054, just over the 4.0 floor,
    # must still flag. Below this the scan silently misses, the acknowledged
    # precision-for-recall tradeoff, so this marks where that boundary sits.
    marginal = "Ab-Cd" + "_Ef-Gh" + "_Ij-Kl" + "_Mn-Op1"
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\napi_key: {marginal}\n")
    rc, out = validate(b, "--secret-entropy-scan")
    assert rc == 1 and "high-entropy assignment" in out


def test_dangling_link_fails(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nSee [missing](nope.md).\n")
    rc, out = validate(b)
    assert rc == 1 and "dangling" in out


def test_bundle_path_not_a_directory_fails_cleanly(tmp_path):
    # pointing --bundle at a file (not a dir) must report a clean failure, not
    # crash with a NotADirectoryError traceback from rglob/iterdir.
    f = tmp_path / "notabundle.md"
    f.write_text(GOOD, encoding="utf-8")
    rc, out = validate(f)
    assert rc == 1 and "not a directory" in out
    assert "Traceback" not in out


def test_root_index_with_concept_frontmatter_fails(tmp_path):
    # the bundle-root index.md may carry only okf_version; arbitrary concept
    # metadata there must not slip through unchecked.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    (b / "index.md").write_text(
        '---\ntype: Credential\nnonsense: yes\n---\n# bad root\n', encoding="utf-8")
    rc, out = validate(b)
    assert rc == 1 and "only okf_version" in out


def test_root_index_missing_okf_version_fails(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    (b / "index.md").write_text("# no frontmatter here\n", encoding="utf-8")
    rc, out = validate(b)
    assert rc == 1 and "okf_version" in out


def test_root_index_unsupported_okf_version_fails(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    (b / "index.md").write_text('---\nokf_version: "0.9"\n---\n# root\n', encoding="utf-8")
    rc, out = validate(b)
    assert rc == 1 and "not supported" in out


@pytest.mark.parametrize("version", ["0.1", "0.2"])
def test_root_index_legacy_version_validates(tmp_path, version):
    # Backward compatibility: older date-only bundles still validate under the
    # newer validator.
    scaffold(tmp_path / "kb", "--no-validate")
    root = tmp_path / "kb" / "bundle" / "index.md"
    root.write_text(root.read_text().replace(
        'okf_version: "0.3"', f'okf_version: "{version}"'), encoding="utf-8")
    rc, out = validate(tmp_path / "kb" / "bundle")
    assert rc == 0, out


def build_federated_tree(root, members=("nodeA", "nodeB"), strip_member_markers=True):
    """Assemble a combined tree the way SPEC.md "Federation" describes: a new root
    index.md carrying okf_version, each member under its own subdirectory."""
    nav = "\n".join(f"- [{m}]({m}/index.md)" for m in members)
    (root).mkdir(parents=True, exist_ok=True)
    (root / "index.md").write_text(
        f'---\nokf_version: "0.1"\n---\n# Atlas\n\n{nav}\n', encoding="utf-8")
    for m in members:
        (root / m).mkdir(parents=True, exist_ok=True)
        marker = '---\nokf_version: "0.1"\n---\n' if not strip_member_markers else ""
        (root / m / "index.md").write_text(
            f"{marker}# {m}\n\n- [concept](concept.md)\n", encoding="utf-8")
        (root / m / "concept.md").write_text(GOOD, encoding="utf-8")


def test_federated_tree_validates(tmp_path):
    # the documented strip-and-merge procedure must actually pass: one root marker,
    # members nested as marker-less section indexes.
    root = tmp_path / "atlas"
    build_federated_tree(root)
    rc, out = validate(root)
    assert rc == 0, out


def test_nested_member_marker_fails(tmp_path):
    # the failure the SPEC warns about: leaving okf_version on a nested member index.
    root = tmp_path / "atlas"
    build_federated_tree(root, strip_member_markers=False)
    rc, out = validate(root)
    assert rc == 1
    assert "reserved file should not carry frontmatter" in out


def test_link_escaping_bundle_fails(tmp_path):
    # a link resolving above the bundle root is a hard failure even if such a file
    # exists on disk.
    (tmp_path / "outside.md").write_text("x", encoding="utf-8")
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nSee [x](../../../outside.md).\n")
    rc, out = validate(b)
    assert rc == 1 and "escapes bundle root" in out


def test_link_with_title_resolves(tmp_path):
    # a CommonMark link with a title, [text](dest "title"), must not be flagged
    # dangling: the title is not part of the path.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + '\nSee [idx](index.md "the section index").\n')
    rc, out = validate(b)
    assert rc == 0, out


def test_root_relative_link_rejected(tmp_path):
    # a '/'-prefixed link is absolute, which the spec forbids; it must fail even
    # if bundle/index.md happens to exist.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nSee [r](/index.md).\n")
    rc, out = validate(b)
    assert rc == 1 and "root-relative link not allowed" in out


def test_code_fence_link_examples_ignored(tmp_path):
    # a markdown link shown inside a fenced code block is illustrative, not a real
    # bundle link, and must not be reported as dangling.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\n\n```md\n[x](does-not-exist.md)\n```\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_inline_code_link_example_ignored(tmp_path):
    # a link inside an inline code span is also an example, not a real link.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\n\nUse `[x](nope.md)` syntax.\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_dangling_link_with_parens_caught(tmp_path):
    # a real (non-fenced) dangling link whose filename has balanced parens must
    # still be caught, the regex must not stop at the first ')'.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nSee [x](missing(v2).md).\n")
    rc, out = validate(b)
    assert rc == 1 and "dangling" in out


def test_four_backtick_fence_wraps_triple(tmp_path):
    # a 4-backtick fence enclosing a 3-backtick example stays closed until a
    # >=4-backtick fence; the inner example link must not leak out as dangling.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\n\n````md\n```\n[x](nope.md)\n```\n````\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_multi_backtick_inline_span_ignored(tmp_path):
    # a multi-backtick inline span (used when the code itself contains a backtick)
    # is still code, not a real link.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\n\nUse ``[x](nope.md)`` here.\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_nonmapping_frontmatter_fails(tmp_path):
    # syntactically valid YAML that is a list (not a mapping) must fail cleanly,
    # not crash the validator with an AttributeError traceback.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, "---\n- a\n- b\n---\n# oops\n")
    rc, out = validate(b)
    assert rc == 1 and "must be a YAML mapping" in out
    assert "Traceback" not in out


def test_md_directory_fails_cleanly(tmp_path):
    # rglob("*.md") also matches a directory named like "archive.md"; reading it
    # would raise IsADirectoryError. The validator must report a clean failure, not
    # crash with a traceback (it is copied into every scaffolded project).
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    (b / "concepts" / "archive.md").mkdir(parents=True)
    rc, out = validate(b)
    assert rc == 1 and "must be a file, not a directory" in out
    assert "Traceback" not in out


def test_nonscalar_type_reports_cleanly(tmp_path):
    # type as a list/dict (a plausible YAML typo) is unhashable; counting it or
    # testing membership would crash. Must report cleanly, not traceback.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.replace("type: Process", "type: [Process]"))
    rc, out = validate(b)
    assert rc == 1 and "'type' must be a string" in out
    assert "Traceback" not in out


def test_missing_root_index_fails(tmp_path):
    # a bundle with concepts but no root index.md must fail: the okf_version gate
    # only runs when that file exists, so its absence would otherwise pass.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD)
    (b / "index.md").unlink()
    rc, out = validate(b)
    assert rc == 1 and "bundle-root index is required" in out


def test_bom_frontmatter_is_parsed(tmp_path):
    # a leading UTF-8 BOM (common from Windows editors) must not make valid
    # frontmatter read as missing; utf-8-sig strips it.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, "" + GOOD)
    rc, out = validate(b)
    assert rc == 0, out


def test_github_pat_secret_detected(tmp_path):
    # build a fine-grained PAT shape from fragments so no real-looking token lives
    # in this test file. The classic gh*_ pattern misses github_pat_.
    fake = "github_pat_" + "11ABCDE" + "FGHIJKLMNOPQRSTUVWXYZ"
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\ntoken = {fake}\n")
    rc, out = validate(b)
    assert rc == 1 and "secret leak" in out


@pytest.mark.parametrize("label, token", [
    ("Stripe secret key", "sk_" + "live_" + "A1b2C3d4E5f6G7h8I9j0K1L2"),
    ("Stripe organization key", "sk_" + "org_" + "A1b2C3d4E5f6G7h8I9j0K1L2"),
    ("Stripe webhook secret", "whsec_" + "A1b2C3d4E5f6G7h8I9j0K1L2M3n4O5p6"),
    ("GitLab token", "glpat-" + "A1b2C3d4E5f6G7h8I9j0"),
    # A valid 20-character GitLab token body can repeat characters and land just
    # below the generic 4.0-bit entropy floor (3.984 bits/character here).
    ("GitLab token", "glpat-" + "5lRDXNfPxOMFQmlFCcFZ"),
    ("GitLab token", "gloas-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "gldt-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glrt-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glrtr-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glcbt-" + "abc_" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glptt-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glft-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glimt-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glagent-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glwt-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glsoat-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("GitLab token", "glffct-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("npm token", "npm_" + "A1b2C3d4E5f6G7h8" + "I9j0K1L2M3n4O5p6Q7r8"),
    ("SendGrid API key", "SG." + "A" * 22 + "." + "B" * 43),
    ("Anthropic API key", "sk-" + "ant-" + "api03-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("OpenAI project key", "sk-" + "proj-" + "A1b2C3d4E5f6G7h8I9j0"),
    ("OpenAI legacy key", "sk-" + "A1b2C3d4E5f6G7h8I9j0" + "K1L2M3n4O5p6Q7r8S9t0"),
])
def test_provider_token_secret_detected(tmp_path, label, token):
    # Prefix-anchored provider detectors (issue #150 move A). Each token is built
    # from fragments so no real-looking secret lives in this file. They run on the
    # default validate (no flag); the negative test below pins the path-documentation
    # precision boundary.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\nkey = {token}\n")
    rc, out = validate(b)
    # Assert the specific detector, not just that some leak fired, so a token that
    # matched the wrong pattern would still be caught.
    assert rc == 1 and f"secret leak ({label})" in out


def test_gitlab_session_cookie_secret_detected(tmp_path):
    # GitLab lists the session-cookie assignment itself alongside its fixed token
    # prefixes. Build the fake cookie from fragments so no real-looking value
    # lives in this test file.
    fake = "_gitlab_" + "session=" + "A1b2C3d4E5f6G7h8I9j0K1L2M3n4O5p6"
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + f"\nCookie: {fake}\n")
    rc, out = validate(b)
    assert rc == 1 and "secret leak (GitLab session cookie)" in out


def test_provider_prefixes_do_not_flag_prose(tmp_path):
    # The provider prefixes must not fire on documentation: a placeholder ellipsis,
    # an env-var name, words that merely contain an rk_/sk_ substring (the \b anchor
    # guards these), and provider-specific OKF key paths all stay clean on the
    # default validate.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    prose = (
        "Set your sk_test_... key from the dashboard.\n"
        "The npm_config_registry env var points at the mirror.\n"
        "Use the work_live and mark_live feature flags.\n"
        "The pointer secret: svc/api/prod-key-path names a vault key, not a value.\n"
        "The pointer secret: openai/sk-proj-production-primary-key-path is a vault path.\n"
        "The pointer secret: anthropic/sk-ant-production-primary-key-path is a vault path.\n"
        "The pointer secret: gitlab/gldt-production-deploy-token-path is a vault path.\n"
        "The pointer secret: openai/sk-proj-A1b2C3d4E5f6G7h8I9j0/key-path is a vault path.\n"
        "GitLab documents the placeholder _gitlab_session=... for browser sessions.\n"
    )
    write_concept(b, GOOD.rstrip() + "\n" + prose)
    rc, out = validate(b)
    assert rc == 0, out


def test_link_to_existing_uppercase_md_fails(tmp_path):
    # the case the prior case-sensitive design worried about: a link to an existing
    # Foo.MD used to "resolve" as valid while discovery never scanned that file. Now
    # discovery finds Target.MD and rejects it as non-conforming, so a bundle that
    # contains it fails -- the file check and the link check stay in agreement.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD, name="concepts/Target.MD")
    write_concept(b, GOOD.rstrip() + "\n\nSee [target](target.md).\n", name="concepts/c.md")
    rc, out = validate(b)
    assert rc == 1, out
    assert "rename Target.MD to target.md" in out
    assert "write Target.MD" not in out


def test_nonconforming_extension_rename_uses_real_parent_case(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD, name="concepts/Target.MD")
    write_concept(
        b,
        GOOD.rstrip() + "\n\nSee [target](../Concepts/target.md).\n",
        name="concepts/c.md",
    )
    rc, out = validate(b)
    assert rc == 1, out
    assert "rename Target.MD to target.md" in out
    assert "rename Target.MD to ../Concepts/target.md" not in out


def test_nonconforming_extension_rename_normalizes_link_extension(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD, name="concepts/Target.MD")
    write_concept(
        b,
        GOOD.rstrip() + "\n\nSee [target](target.MD).\n",
        name="concepts/c.md",
    )
    rc, out = validate(b)
    assert rc == 1, out
    assert "rename Target.MD to target.md" in out
    assert "rename Target.MD to target.MD" not in out


def test_wrong_case_dangling_symlink_is_reported_as_dangling(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    try:
        (b / "concepts" / "Broken.md").symlink_to("missing.md")
    except OSError as exc:
        pytest.skip(f"symlink unavailable on this filesystem: {exc}")
    write_concept(
        b,
        GOOD.rstrip() + "\n\nSee [broken](broken.md).\n",
        name="concepts/c.md",
    )
    rc, out = validate(b)
    assert rc == 1, out
    assert "dangling link -> broken.md" in out
    assert "write Broken.md" not in out


def test_uppercase_scheme_link_not_flagged(tmp_path):
    # an external link with an uppercase scheme must be recognized as external and
    # skipped, not resolved as a local path (which falsely fails as escaping).
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nSee [s](HTTPS://example.com/readme.md).\n")
    rc, out = validate(b)
    assert rc == 0, out


def test_documented_credential_path_not_flagged(tmp_path):
    # OKF credential concepts document key NAMES/paths, not values; a hyphenated
    # path after a secret-ish label must not read as a high-entropy leaked value.
    scaffold(tmp_path / "kb", "--no-validate")
    b = tmp_path / "kb" / "bundle"
    write_concept(b, GOOD.rstrip() + "\nsecret: service/api/prod-client-secret-path\n")
    rc, out = validate(b)
    assert rc == 0, out


# --- session hooks: scaffold wiring -----------------------------------------

def test_default_scaffold_writes_hooks(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    c = tmp_path / "kb" / ".claude"
    assert (c / "settings.json").exists()
    assert (c / "hooks" / "okf-anchor.py").exists()
    assert (c / "hooks" / "okf-orient.py").exists()


def test_no_hooks_skips_claude_dir(tmp_path):
    rc, out = scaffold(tmp_path / "kb", "--no-validate", "--no-hooks")
    assert rc == 0, out
    assert not (tmp_path / "kb" / ".claude").exists()
    assert "skipped" in out


def test_scaffold_with_hooks_still_validates(tmp_path):
    # the .claude/ dir sits outside bundle/, so a default scaffold (hooks on)
    # must still validate clean.
    rc, out = scaffold(tmp_path / "kb")
    assert rc == 0 and "PASS" in out, out
    assert (tmp_path / "kb" / ".claude" / "settings.json").exists()


def test_settings_json_structure(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate")
    s = settings_of(tmp_path / "kb")
    assert "SessionStart" in s["hooks"] and "PreToolUse" in s["hooks"]
    start = s["hooks"]["SessionStart"][0]["hooks"][0]
    pre_group = s["hooks"]["PreToolUse"][0]
    assert "matcher" not in pre_group  # no matcher => fires on the first tool call of any kind
    pre = pre_group["hooks"][0]
    # exec form: interpreter in `command`, script path as one `args` element (no shell
    # tokenization). The path must use the ${CLAUDE_PROJECT_DIR} placeholder, not a
    # bare cwd-relative path: the hook cwd is not guaranteed to be the project root.
    for hook, script in ((start, "okf-anchor.py"), (pre, "okf-orient.py")):
        assert hook["command"] in ("python3", "python"), hook
        assert hook["args"] == [f"${{CLAUDE_PROJECT_DIR}}/.claude/hooks/{script}"], hook


def test_hooks_os_windows_uses_python(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate", "--hooks-os", "windows")
    hook = settings_of(tmp_path / "kb")["hooks"]["SessionStart"][0]["hooks"][0]
    assert hook["command"] == "python"  # the py launcher, not python3


def test_hooks_os_posix_uses_python3(tmp_path):
    scaffold(tmp_path / "kb", "--no-validate", "--hooks-os", "posix")
    hook = settings_of(tmp_path / "kb")["hooks"]["SessionStart"][0]["hooks"][0]
    assert hook["command"] == "python3"


def test_readme_validate_command_matches_os(tmp_path):
    # the generated README's validate command must use the same interpreter as the
    # hooks; stock Windows has no python3, so a windows scaffold must say python.
    scaffold(tmp_path / "win", "--no-validate", "--hooks-os", "windows")
    win = (tmp_path / "win" / "README.md").read_text()
    assert "python scripts/validate.py" in win and "python3 scripts/validate.py" not in win
    scaffold(tmp_path / "nix", "--no-validate", "--hooks-os", "posix")
    assert "python3 scripts/validate.py" in (tmp_path / "nix" / "README.md").read_text()


def test_force_merges_into_existing_settings(tmp_path):
    # scaffolding --force into a project that already has .claude/settings.json must
    # preserve the user's settings (permissions, unrelated events, their own
    # SessionStart hook) and add the OKF hooks, never overwrite the file wholesale.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    existing = {
        "permissions": {"allow": ["Bash(ls:*)"]},
        "hooks": {
            "SessionStart": [{"hooks": [{"type": "command", "command": "echo hi"}]}],
            "Stop": [{"hooks": [{"type": "command", "command": "echo bye"}]}],
        },
    }
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    assert s["permissions"] == {"allow": ["Bash(ls:*)"]}  # untouched
    assert s["hooks"]["Stop"] == [{"hooks": [{"type": "command", "command": "echo bye"}]}]
    start_cmds = [h.get("command") for g in s["hooks"]["SessionStart"] for h in g["hooks"]]
    assert "echo hi" in start_cmds  # user's own hook preserved alongside ours
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if h.get("args") == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]]
    assert len(anchors) == 1
    assert "PreToolUse" in s["hooks"]


def test_force_merge_is_idempotent(tmp_path):
    # running the scaffold twice must not duplicate the OKF hook entries.
    target = tmp_path / "kb"
    scaffold(target, "--no-validate")
    scaffold(target, "--force", "--no-validate")
    s = settings_of(target)
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if h.get("args") == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]]
    orients = [h for g in s["hooks"]["PreToolUse"] for h in g["hooks"]
               if h.get("args") == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-orient.py"]]
    assert len(anchors) == 1 and len(orients) == 1


def test_force_backs_up_unparseable_settings(tmp_path):
    # a settings.json that is not valid JSON must be backed up, not silently
    # discarded, before the OKF settings are written in its place.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    (target / ".claude" / "settings.json").write_text("not json{", encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    assert (target / ".claude" / "settings.json.bak").read_text() == "not json{"
    assert "backed up" in out
    assert "SessionStart" in settings_of(target)["hooks"]


def test_force_replaces_shellform_hook(tmp_path):
    # if an OKF hook is recorded in shell form (our exact path inside a `command`
    # string, e.g. hand-edited), --force must recognize it and replace it with the
    # exec-form entry, not leave both active. shlex-splitting the command exposes the
    # path token, which is then matched exactly against the paths we generate.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    legacy = {"hooks": {
        "SessionStart": [{"hooks": [{"type": "command",
            "command": 'python3 "${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"'}]}],
        "PreToolUse": [{"hooks": [{"type": "command",
            "command": 'python3 "${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-orient.py"'}]}],
    }}
    (target / ".claude" / "settings.json").write_text(json.dumps(legacy), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if any("okf-anchor.py" in str(t) for t in [h.get("command")] + (h.get("args") or []))]
    assert len(anchors) == 1, s["hooks"]["SessionStart"]  # legacy entry replaced, not duplicated
    assert anchors[0]["args"] == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]  # exec form


def test_force_preserves_user_hook_sharing_okf_group(tmp_path):
    # a user may add their own hook into the same group as the OKF hook; replacing our
    # entry must strip only ours and keep theirs -- no whole-group drop (data loss).
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    existing = {"hooks": {
        "SessionStart": [{"hooks": [
            {"type": "command", "command": "python3",
             "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]},
            {"type": "command", "command": "echo mine"},
        ]}],
    }}
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    cmds = [h.get("command") for g in s["hooks"]["SessionStart"] for h in g["hooks"]]
    assert "echo mine" in cmds  # the user's hook in the shared group survives
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if any("okf-anchor.py" in str(t) for t in [h.get("command")] + (h.get("args") or []))]
    assert len(anchors) == 1, s["hooks"]["SessionStart"]  # ours replaced once, not duplicated


def test_force_keeps_lookalike_user_hook(tmp_path):
    # a user hook whose path merely starts with our script name (a .bak/-wrapper
    # variant) is NOT ours; a whole-token match means --force must not strip it.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    lookalike = "${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py.bak"
    existing = {"hooks": {
        "SessionStart": [{"hooks": [
            {"type": "command", "command": "python3", "args": [lookalike]},
        ]}],
    }}
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    kept = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
            if h.get("args") == [lookalike]]
    assert len(kept) == 1, s["hooks"]["SessionStart"]  # lookalike untouched, not stripped as ours


def test_force_keeps_same_named_hook_at_other_path(tmp_path):
    # a user hook that runs okf-anchor.py from a DIFFERENT location (a shared/global
    # hooks dir, not ${CLAUDE_PROJECT_DIR}) is not ours; exact-path matching means
    # --force must leave it in place.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    other = "/opt/shared/.claude/hooks/okf-anchor.py"
    existing = {"hooks": {
        "SessionStart": [{"hooks": [
            {"type": "command", "command": "python3", "args": [other]},
        ]}],
    }}
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    kept = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
            if h.get("args") == [other]]
    assert len(kept) == 1, s["hooks"]["SessionStart"]  # foreign-path hook left untouched


def test_force_backs_up_malformed_event_value(tmp_path):
    # a parseable settings.json whose event value is the wrong shape (not a list) must
    # be backed up rather than crashing the merge, preserving the original on disk.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    malformed = '{"hooks": {"SessionStart": 5}}'
    (target / ".claude" / "settings.json").write_text(malformed, encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    assert (target / ".claude" / "settings.json.bak").read_text() == malformed  # original preserved
    assert "backed up" in out
    s = settings_of(target)
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if h.get("args") == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]]
    assert len(anchors) == 1, s["hooks"]["SessionStart"]  # fresh hooks written after backup


def test_force_tolerates_malformed_hook_entry(tmp_path):
    # a list-shaped event holding a malformed hook entry (args not a list) must not
    # crash the merge; the odd entry is preserved (we can't claim it) and ours is added.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    existing = {"hooks": {
        "SessionStart": [{"hooks": [{"type": "command", "args": 5}]}],
    }}
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    weird = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"] if h.get("args") == 5]
    assert len(weird) == 1, s["hooks"]["SessionStart"]  # malformed entry left in place
    anchors = [h for g in s["hooks"]["SessionStart"] for h in g["hooks"]
               if h.get("args") == ["${CLAUDE_PROJECT_DIR}/.claude/hooks/okf-anchor.py"]]
    assert len(anchors) == 1, s["hooks"]["SessionStart"]  # our hook still added


def test_force_preserves_unrelated_keys_when_hooks_malformed(tmp_path):
    # a parseable settings.json with unrelated live config (permissions) but a malformed
    # hooks subtree must keep the unrelated config in the LIVE file and just repair the
    # hooks; the original is copied to .bak. No whole-file reset, no lost permissions.
    target = tmp_path / "kb"
    (target / ".claude").mkdir(parents=True)
    existing = {"permissions": {"allow": ["Bash(ls)"]}, "hooks": {"SessionStart": 5}}
    (target / ".claude" / "settings.json").write_text(json.dumps(existing), encoding="utf-8")
    (target / "keep.txt").write_text("x")
    rc, out = scaffold(target, "--force", "--no-validate")
    assert rc == 0, out
    s = settings_of(target)
    assert s["permissions"] == {"allow":