{
  "schema_version": 1,
  "skill_name": "ai-governance",
  "evals": [
    {
      "id": "operating-model-design",
      "prompt": "A 120-engineer company is scaling its use of AI and has no standing governance. Design a governance operating model for them: the decision-rights map, the review council, named stewards and accountable owners, RACI for intake and launch decisions, and whether to run federated or centralized. State when the lightweight vs. high-assurance mode applies.",
      "expected_output": "A concrete governance operating model: a named council with charter, a decision-rights map assigning who decides/consults/informs per decision type with evidence standards, stewards and accountable owners, a federated-vs-centralized recommendation with rationale, and an explicit operating-mode choice (lightweight or high-assurance) with the trigger condition for switching.",
      "assertions": [
        "names a review council and a governance charter as the accountability home",
        "assigns decision rights (who decides, consults, informs) and evidence standards per decision type",
        "recommends a federated or centralized model and justifies it for the scale given",
        "selects lightweight or high-assurance mode with an explicit trigger condition",
        "assigns named stewards and an accountable owner for AI governance"
      ]
    },
    {
      "id": "use-case-risk-tiering",
      "prompt": "Tier these three AI use cases by governance risk and prescribe the required controls for each: (1) a customer-support email summarizer with no autonomous action, (2) a loan-approval model that makes a consequential decision about an individual, and (3) an internal agent that executes code changes automatically.",
      "expected_output": "Each use case is assigned a risk tier (low, medium, high, or critical) based on data sensitivity, autonomy, exposure, and decision impact, with the controls that tier requires (e.g., human-in-the-loop, model card, bias review, red-teaming, access restrictions) stated explicitly rather than as generic recommendations.",
      "assertions": [
        "assigns each of the three use cases to an explicit risk tier",
        "derives the tier from autonomy, exposure, decision impact, and data sensitivity",
        "prescribes tier-appropriate controls for every use case, including elevated controls for the autonomous agent",
        "does not reduce all three to the same tier and control set"
      ]
    },
    {
      "id": "llm-app-governance-review",
      "prompt": "Review an internal RAG copilot that retrieves company documents and answers employee questions. Identify the governance and security gaps across trust boundaries, prompt injection, data exposure, excessive agency, and system-prompt control, then recommend the controls and evidence gates needed before deployment.",
      "expected_output": "A governance and security gap review that walks the RAG copilot through trust boundaries and threat surfaces (prompt injection, data exposure, excessive agency, system-prompt integrity), maps each gap to a concrete control, and defines the lifecycle evidence gates the system must pass before it can be deployed.",
      "assertions": [
        "identifies prompt injection and data-exposure risks specific to a RAG retrieval flow",
        "addresses excessive agency and system-prompt control for the assistant",
        "maps each gap to a concrete control and an accountable owner",
        "defines the evidence gates that must pass before deployment"
      ]
    },
    {
      "id": "six-level-governance-loop",
      "prompt": "A company is moving an internal customer-support assistant from a SaaS experiment to an API-integrated pilot and may later self-host it. Apply the Six-Level Governance framework to the transition. Define the evidence, owners, metrics, maturity target, and reassessment triggers for Strategy & Policy, Risk & Impact Assessment, Implementation Review, Acceptance Testing, Operations & Monitoring, and Learning & Improvement. Explain how this overlay complements, rather than replaces, lifecycle stage gates.",
      "expected_output": "A cross-cutting 6L-G governance plan with all six levels named, each level's accountable owner, required artifact or evidence, useful outcome metric, common failure mode, proportional maturity target, and material-change trigger. It clearly distinguishes the 6L-G governance disciplines from ordinary lifecycle stages and does not present 6L-G as a legal or standards requirement.",
      "assertions": [
        "covers all six 6L-G levels with a decision question, owner, artifact or evidence, and metric",
        "distinguishes the 6L-G overlay from lifecycle timing rather than replacing lifecycle gates",
        "sets a proportional maturity target and avoids averaging away a critical weak level",
        "defines reassessment triggers for posture, model, data, tool, user, incident, or material behavior changes",
        "labels 6L-G as an author-developed framework rather than a mandatory standard"
      ]
    },
    {
      "id": "deployment-posture-boundary",
      "prompt": "Compare governance for the same confidential RAG assistant in three forms: a vendor-hosted SaaS application, an API integrator with an organization-owned UI and middleware, and a self-hosted model. For each posture, map who controls the interface, prompts, RAG data, logs, guardrails, model updates, security, deletion, incident response, and exit. Identify what must be proven internally versus obtained from the vendor, and do not rank a posture as safe by default.",
      "expected_output": "A posture comparison that assigns control ownership at each trust boundary, identifies the evidence and contract terms needed for vendor-controlled layers, distinguishes organization-controlled pre/post-processing from provider-controlled model behavior, and explains the distinct accountability and operational burden of self-hosting without declaring any posture inherently safe.",
      "assertions": [
        "compares SaaS, API-integrator, and model-hoster control boundaries explicitly",
        "assigns evidence or contract responsibilities for retention, training use, logs, model changes, deletion, and incident response",
        "identifies organization-side controls available in the API-integrator posture",
        "identifies supply-chain, guardrail, and full operational responsibilities introduced by self-hosting",
        "states that posture changes where risk and accountability sit rather than determining safety by itself"
      ]
    },
    {
      "id": "agentic-control-and-egress-review",
      "prompt": "Review an agent that can read customer records, query an external knowledge service, send email, update tickets, and retain memory across sessions. It discovers tools dynamically through an MCP-compatible connector. Design the governance controls before pilot: the six-factor exposure assessment, approved tool registry, external authorization policy, action-approval tiers, purpose-aware egress gate, memory tiers and erasure, sandbox and kill switch, runtime traces, and multi-step red-team tests.",
      "expected_output": "An agentic governance review that inventories environment/model/input/data/change/agency exposure; allowlists individual tools and operations with identity, scope, version, owner, and destination; makes authorization external to model reasoning and deny-by-default; calibrates approvals by consequence; checks purpose, necessity, and destination before egress; governs memory and deletion; and provides tested containment, observability, and multi-step adversarial evidence.",
      "assertions": [
        "assesses all six exposure factors and explains that the ladder is not a severity score",
        "requires an approved registry for individual tools or operations, scopes, versions, destinations, and owners",
        "places runtime authorization outside model reasoning with deny-by-default enforcement",
        "uses consequence-based approval tiers and addresses approval fatigue without allowing high-impact actions to bypass review",
        "checks purpose, necessity, and destination for every outbound transfer and records allow/redact/block/approval outcomes",
        "defines tiered memory retention, poisoning tests, erasure coverage, sandboxing, and a tested kill switch",
        "requires structured run traces and multi-step tests for tool chaining, memory poisoning, and goal manipulation"
      ]
    },
    {
      "id": "genai-privacy-surfaces",
      "prompt": "A support agent uses customer chat history for RAG, stores embeddings and long-term memory, calls an external translation API, and drafts responses that may affect a customer's claim. Review privacy using four GenAI pillars. Cover collection and purpose, storage and memorization, output integrity, user rights and governance, then specify purpose-aware egress, retention and deletion evidence, contestability, and what technical limitations must be disclosed. Keep it as governance guidance, not legal advice.",
      "expected_output": "A four-pillar GenAI privacy review that addresses collection and secondary use, memorization and embeddings, false or inferred personal claims in outputs, and access/correction/erasure rights. It includes a fail-closed purpose/necessity/destination egress decision, tiered memory retention, source-linked deletion across derived stores and downstream recipients, meaningful human contestability, and an explicit statement of technical uncertainty and legal re-verification.",
      "assertions": [
        "covers collection and purpose, storage and memorization, output integrity, and user rights as distinct privacy surfaces",
        "treats embeddings, memory, logs, caches, backups, and model-update inputs as deletion and retention surfaces",
        "addresses hallucinated or inferred personal claims and does not treat a rubber-stamp human review as sufficient",
        "specifies purpose, necessity, and destination checks with fail-closed egress outcomes",
        "provides memory inspection, correction, deletion, and future-use prevention paths",
        "states the output is guidance rather than legal advice and that current obligations require primary-source verification"
      ]
    },
    {
      "id": "impact-assessment-closure",
      "prompt": "A company proposes a high-impact employee-benefits eligibility assistant that uses confidential HR records, an external model API, and an approval workflow. Design the AI impact assessment as a coordination record rather than a one-time checklist. Identify which privacy, fairness, security, vendor, data-governance, validation, and human-oversight follow-ups are required; assign owners and due dates; define evidence and closure or bounded-exception rules; include process-health metrics that detect bypassed or overdue reviews; and end with a go, conditional-go, or no-go decision. Keep it as governance guidance, not legal advice.",
      "expected_output": "An operational AI impact-assessment record that states the affected people and residual risks, links each required specialist assessment to a named owner, due date, status, evidence, and closure decision, blocks approval when required work is incomplete unless a bounded exception is approved, and measures assessment coverage, bypasses, overdue actions, repeated findings, escalation, and time to decision. It distinguishes governance coordination from legal interpretation and preserves uncertainty.",
      "assertions": [
        "identifies the affected population, consequential impact, deployment posture, data sensitivity, and residual-risk decision",
        "links privacy, fairness, security, vendor, data-governance, validation, and human-oversight follow-ups to named owners and due dates",
        "requires evidence and an explicit closure status or bounded exception for each required assessment",
        "makes incomplete required assessments block approval unless the exception has an approver, compensating control, expiration, and re-review trigger",
        "includes process-health metrics for coverage, bypasses, overdue actions, repeated findings, escalation, and time to decision",
        "states the output is governance guidance rather than legal advice and preserves material uncertainty"
      ]
    },
    {
      "id": "fairness-accountability-review",
      "prompt": "Review a hiring model for fairness without reducing fairness to a single metric. Lay out the fairness metrics you would compute and their limits, the trade-offs between accuracy and parity, the bias sources to probe, and how accountability and auditability should be established in the model's documentation.",
      "expected_output": "A fairness and accountability review that computes multiple fairness metrics (e.g., demographic parity, equalized odds, calibration), explains the limits and trade-offs of each rather than picking one number, enumerates the bias sources to probe in data and labels, and ties fairness decisions to model-card documentation and a named accountable owner.",
      "assertions": [
        "proposes more than one fairness metric and explains the limits of each",
        "surfaces accuracy-vs-parity trade-offs rather than an absolute fairness override",
        "names specific bias sources in data, labels, and evaluation to probe",
        "connects the fairness decisions to auditable model-card documentation and an accountable owner"
      ]
    },
    {
      "id": "regulatory-compliance-mapping",
      "prompt": "Map the EU AI Act's requirements for high-risk AI systems to a concrete compliance and control plan for a company building an AI system, covering risk management, data governance, technical documentation, transparency, human oversight, and registration. Keep it as guidance, not legal advice.",
      "expected_output": "A compliance and control plan that maps each high-risk obligation of the EU AI Act to a specific control and evidence artifact (risk management system, data governance, technical documentation, transparency and human-oversight measures, registration), with an accountable owner per control, and a clear statement that it is guidance to be re-verified against the primary source.",
      "assertions": [
        "maps EU AI Act high-risk obligations to named controls and evidence artifacts",
        "covers data governance, technical documentation, and transparency obligations",
        "assigns an accountable owner to each control",
        "includes human-oversight measures and the registration obligation",
        "states the output is guidance, not legal advice, to be re-verified"
      ]
    },
    {
      "id": "board-governance-reporting",
      "prompt": "Design the board-level AI governance reporting for a company: the metrics to report, the cadence, the risk register and exceptions to surface, the escalation path, and how to structure a board AI-governance report so directors can exercise oversight.",
      "expected_output": "A board reporting design with a defined metric set (adoption, incidents, open high-risk use cases, material exceptions, compliance posture), a cadence, the risk-register and exception content to surface, a documented escalation path, and a report structure that lets directors exercise oversight rather than drown in detail.",
      "assertions": [
        "defines a concrete metric set for board reporting on AI",
        "specifies a cadence and which risks, exceptions, and incidents get surfaced",
        "documents the escalation path to the board or risk committee",
        "structures the report so directors can exercise oversight and ask pointed questions"
      ]
    },
    {
      "id": "gxp-ai-data-integrity-overlay",
      "prompt": "A pharmaceutical company wants to use an AI system to summarize laboratory results and flag potential out-of-specification investigations. Design the governance overlay before pilot and release. Cover GxP applicability, ALCOA+ data integrity, electronic records and audit trails, risk-based validation or assurance, AI-specific evidence, QMS interfaces, supplier controls, monitoring, change control, and retirement. Do not claim legal compliance; distinguish AI governance from QA, validation, and regulatory responsibilities.",
      "expected_output": "A bounded GxP AI governance plan that connects the AI lifecycle to the quality system, explicitly evaluates ALCOA+ attributes and electronic-record/audit-trail controls, defines a risk-based validation or assurance evidence plan, adds AI-specific context-of-use and performance evidence, assigns QA/QMS/validation/data/system owners, and defines monitoring, change, deviation/CAPA, supplier, and retirement controls without presenting legal advice as a compliance conclusion.",
      "assertions": [
        "screens GxP applicability and records the system boundary, domain, intended use, and accountable owners",
        "explicitly covers ALCOA+ rather than treating lineage and provenance as sufficient",
        "addresses electronic records, signatures, audit trails, access, time, retention, and retrieval as applicable",
        "defines risk-based validation or assurance evidence including context of use, acceptance criteria, traceability, failure cases, and change triggers",
        "connects AI governance to QA/QMS processes such as release, deviation, CAPA, change control, monitoring, periodic review, and retirement",
        "states that the plan is guidance and does not determine legal applicability or certify compliance",
        "includes supplier/model controls such as provider change notification, evidence access, continuity, and exit planning",
        "includes AI-specific performance evidence such as context of use, representativeness, generalizability, uncertainty, or drift"
      ]
    }
  ]
}

# AI Lifecycle Governance

This reference explains how to keep an AI system accountable across its full lifetime, from
the moment someone proposes an idea or decides to buy a model, through the data, build,
evaluation, deployment, and monitoring phases, and finally into orderly retirement. The
backbone is the **stage gate**: a fixed checkpoint with a named decision-maker and a
published set of criteria that a use case or model must satisfy before it is allowed into
the next phase. Around those gates sit the controls that make the whole sequence visible
and auditable — a **model inventory** that lists every system the organization runs,
**lineage** that shows where data and models originated and how they were transformed,
**drift detection** that flags degradation once a model is live, and **incident response**
for when something goes wrong despite the gates.

The content below is an original synthesis of the ideas in *Platform and Model Design for
Responsible AI*, *Designing Data Governance from the Ground Up*, and the *Data Governance
Handbook*, refreshed against current research on technical governance controls (see
research-technical-controls.md). Read it alongside `risk-management-and-frameworks.md`,
which supplies the risk tiers and registers the gates feed into, and
`governance-operating-model.md`, which names the owners and councils who operate the gates.
Nothing here is legal advice.

## Why Governance Must Cover the Entire Lifetime

A model is not a thing you construct once and forget. *Platform and Model Design for
Responsible AI* argues that responsible AI is a continuing practice that runs from an early
prototype all the way to a production system that may be decommissioned years afterward,
and it warns that each step in the development life cycle brings hazards of its own. Risk
does not begin at the training run: an ill-chosen use case, a biased or outdated dataset, a
build nobody validated, a launch with no monitoring, a model that drifts silently, or a
decommissioned system still making decisions can each do damage independently.

Because the failure modes differ from phase to phase, oversight has to be spread across the
whole sequence rather than concentrated at a single approval moment. Current research on
technical controls echoes this: the NIST AI RMF and ISO/IEC 42001 both treat governance as
an activity that extends throughout the AI life cycle, and the EU AI Act makes logging,
technical documentation, and post-market surveillance explicit duties for high-risk systems
precisely because obligations do not end at launch. The AI Omnibus deferral has pushed the
high-risk start dates out — most high-risk systems now face those duties from 2 December
2027, and systems embedded in regulated products from 2 August 2028 — so treat
`regulatory-landscape.md` as the authoritative timeline rather than any single effective
date.

Three commitments hold the sequence together:

- **Every phase has a named owner and a documented gate.** Someone is accountable for each
  transition, and the bar for passing is written down in advance.
- **Every phase is recorded.** The inventory, the lineage, and the logs let anyone
  reconstruct what happened, which is what makes the process auditable.
- **Every phase can feed back.** Monitoring and incidents push lessons back into earlier
  phases so the next model starts from a smarter place.

## Stage Gates as the Spine of Lifecycle Governance

A stage gate is a defined point between phases where a model or use case is checked against
fixed criteria and either cleared to continue, sent back for rework, or stopped. Gates earn
their keep three ways: they force a review before expensive work is committed; they leave an
auditable trail of who approved what and on what evidence; and they stop teams from quietly
skipping a required control.

*Platform and Model Design for Responsible AI* catalogues the recurring questions a model
practice should be able to answer at each gate — whether the applicable regulatory tools
have been identified, whether inventorying and classification happens during development,
how model naming and version control are handled, whether formal policies and audit
checklists exist for development, validation, use, monitoring, and retirement, whether
compliance checklists are maintained, and whether models are properly documented. These
questions are the substance of a gate.

A well-defined gate records four things:

| Gate element | What it captures |
|---|---|
| Entry criteria | What must already be true before work on the next phase begins |
| Review artifacts | The evidence the owner must present (cards, assessments, logs) |
| Decision rights | Who approves, who is consulted, who must be told (see the RACI material in `governance-operating-model.md`) |
| Exit criteria | The concrete, verifiable conditions that clear the gate |

How heavy a gate should be depends on the model's risk tier (see
`risk-management-and-frameworks.md`). A low-risk internal utility might clear a light gate
with a one-page intake; a high-impact system warrants the full assessment, validation, and
oversight chain. Calibrating gate depth to tier keeps the gate from being either a rubber
stamp or an unusable bottleneck.

The [use-case-intake-form.md](../templates/use-case-intake-form.md) template turns the earliest gate into a fillable form, and
[model-risk-assessment.md](../templates/model-risk-assessment.md) records the tiering decision that sets how demanding every later
gate must be.

## The Ideation and Procurement Stage

Oversight starts before any model exists, at the point where someone floats an AI idea or
decides to bring in a vendor-built model. The ideation gate exists to force one question
first: is this a sound AI use case at all, and is the risk worth it?

At this stage the organization should record:

- The business problem the AI is meant to solve and why AI — as opposed to a rule or a human
  process — is the right answer.
- The intended users and everyone the system could affect, including any group that might be
  harmed.
- The decision impact, meaning the consequential outcome the model will influence or
  automate.
- The data involved and how sensitive it is.
- The owner, the sponsor, and the decision rights for the use case.
- A first risk-tiering so the depth of later review is set up front.

For procured and third-party models, the ideation gate merges with procurement diligence:
the organization is choosing to host an external model, so it must establish who is
accountable for it, what documentation and assurances the vendor will provide, and how it
will be watched after adoption. That supply-chain thread is developed in
`procurement-third-party-and-board-oversight.md`. The governing rule, consistent with the
*Data Governance Handbook*'s point that a governed asset has clear ownership, is that no
model — whether built in-house or bought — enters the portfolio without a named owner and a
stated purpose.

A practical intake gate is a checklist rather than a blank approval:

- [ ] Problem statement and intended use recorded
- [ ] Affected population and decision impact assessed
- [ ] Data sensitivity and sourcing identified
- [ ] Named owner and sponsor assigned
- [ ] Initial risk tier proposed and recorded
- [ ] Applicable laws, policies, and standards identified

## The Data Stage

A model is only as sound as the data it learns from, which is why the data phase is where
many failures take root. *Designing Data Governance from the Ground Up* contends that data
oversight must persist through the life cycle and that a plan ignoring drift and end-of-life
phases will come up short. The *Data Governance Handbook* frames data ownership and data
quality as foundational: governed data has a clear owner, is fit for its purpose, and can be
traced.

The data gate should confirm:

- **Ownership.** A named steward is accountable for the dataset and its quality (see the
  steward roles in `governance-operating-model.md`).
- **Provenance and lineage.** Where the data came from and how it was transformed is
  recorded, so the model's inputs can be followed back (see `Lineage` below).
- **Quality.** The data is profiled and checked for completeness, accuracy, consistency, and
  timeliness against defined rules, with a plan for any material gaps.
- **Suitability.** The data reflects the intended use and population, and known gaps, biases,
  or proxies — for instance, a sensitive attribute correlated with an innocuous-looking one —
  are surfaced rather than buried.
- **Consent and rights.** The data was lawfully obtained and its use respects privacy and
  licensing constraints, in line with `privacy-and-data-governance.md`.

The *Data Governance Handbook* describes data quality as fitness for use, delivered through
defined rules, profiling resources, dashboards that report current quality openly, and plans
to resolve material issues. Those four steps map directly onto the data gate: set the rules,
measure against them, surface the results, and act before the model builds on shaky data.

## The Build Stage

The build phase turns data and a use case into a working model. Oversight here is less a
single approval point and more a set of habits woven into how the team works: reproducible
experiments, versioned code and data, documented choices, and acknowledged limits.

*Platform and Model Design for Responsible AI* treats governance as an engineering
discipline: reproducible pipelines, clear naming and versioning of models, documented
assumptions and limitations, and sharing so features and models are reused rather than
reinvented in isolation. The build gate confirms that a model's development is traceable —
that the code, data, and configuration behind any given version can be reproduced and
examined later. Current research on MLOps governance tooling backs this up: before a model
can advance from build to evaluation it must have its documentation finished, its risk tier
accepted, and a clear connection to its data lineage, because those artifacts are what make
the build auditable after the fact.

Build-phase practices worth enforcing:

- Reproducible training runs with recorded code, data versions, and configuration.
- A clear, documented naming and versioning scheme for models and their artifacts.
- Recorded assumptions, limitations, and design decisions.
- Registration in a model registry or inventory so the versioned artifact is tracked from
  the start.
- Bias and robustness checks at the model level, guided by
  `fairness-bias-accountability.md`.

## The Evaluation Stage

Evaluation is where the organization decides whether the model is actually good enough and
safe enough to ship. Good governance demands that evaluation be planned, not improvised:
the metrics, the datasets, and the acceptance thresholds are agreed before the run, and the
results are written down against a model card.

The evaluation gate should confirm:

- **Performance.** The model meets the agreed accuracy, reliability, and quality thresholds
  on appropriate test data, including performance broken out by relevant subgroups so
  the spread across groups is apparent instead of buried in one aggregate — the model-card
  discipline described in current research.
- **Fairness.** Bias is measured against relevant metrics, and trade-offs are explicit and
  defensible (see `fairness-bias-accountability.md`).
- **Explainability.** Where needed, the model's behavior can be explained and the disclosure
  a user will see is defined (see `transparency-and-explainability.md`).
- **Robustness and security.** The model holds up under expected variation and known
  adversarial cases (see `llm-and-agent-security.md`).
- **Limitations.** The situations where the model is likely to underperform are documented
  so it is not applied somewhere it is unsuited to.

The evaluation gate is also where the **model card** is finalized. As the mission's
technical-controls research records, model cards began as a proposal to standardize model
documentation — intended use, benchmarked results by subgroup, evaluation method, and
limitations — and have since become the standard artifact attached to a registry entry. A
finished model card is both the evidence that clears the evaluation gate and the reference
that the monitoring and incident-response phases consult later. The fillable [model-card.md](../templates/model-card.md)
template ships with this skill.

## The Deployment Stage

Deployment is the moment a model starts influencing real decisions, which makes this the
highest-stakes gate before launch. Its purpose is to confirm that a model enters production
only with the controls it needs to run safely and to be pulled back if it misbehaves.

*Platform and Model Design for Responsible AI* stresses that oversight continues after
launch and that live systems need monitoring and a fast way to respond. Current research
adds that monitoring must be set up in advance — with defined owners, thresholds, and
runbooks for responding — rather than thrown together after a problem appears, and that the
EU AI Act turns post-market monitoring and incident reporting into legal duties for
high-risk systems.

A deployment gate should confirm:

- A completed model card and approved risk tier on record.
- A monitoring configuration with defined metrics, alert thresholds, owners, and a runbook
  for drift and quality issues.
- A rollback plan, so the team can revert to the previous version or take the model offline
  quickly.
- Access controls and logging, so who can invoke or change the model is controlled and
  recorded (see `llm-and-agent-security.md`).
- Human oversight arrangements where the decision calls for them (see
  `foundations-and-principles.md`).
- A named operator or steward who owns the live system.

Current research describes the desired end state as one where deployment is gated
automatically: the pipeline blocks a model from reaching production until its documentation
is complete, its risk tier is accepted, and its monitoring is configured, enforcing those
checks mechanically instead of trusting someone to remember them.

## The Monitoring Stage

Monitoring is the runtime control that catches a deployed model when it stops performing as
expected. It is increasingly treated as a required part of responsible deployment rather
than an add-on, and it is what changes "we shipped it" into "we keep watching it."
*Designing Data Governance from the Ground Up* makes the parallel case for data: oversight
is not done at launch, and a plan that does not think about what happens to data in
production will fall short.

Monitoring generally tracks two things at once:

- **Operational health** — latency, throughput, error rates, and availability.
- **Model behavior** — prediction distributions, output quality, and whether the outputs
  still look like the ones the model was evaluated on.

The specific hazard monitoring exists to catch is **drift**, covered in depth in `Drift
Detection and Response` below. The key governance principle is that monitoring is
pre-configured and owned: someone is responsible for the metrics, the thresholds, and the
escalation path, and the setup is recorded before launch rather than bolted on after an
incident. Current research notes that monitoring is well developed for ordinary predictive models but
weaker for generative and agentic ones, whose outputs are not predetermined and are hard to
grade against a ground truth, so those systems call for proxy signals and staged human
review rather than a single accuracy figure.

## The Retirement Stage

Every model eventually reaches the end of its useful life. Retirement, or decommissioning,
is the governed end-of-life phase where the organization confirms a model is no longer
serving its purpose, shuts it down cleanly, records the fact, and disposes of the related
data responsibly.

*Platform and Model Design for Responsible AI* treats retirement as part of the inventory
life cycle: the inventory notes each model's development stage, including whether it is in
use, under development, or recently retired. *Designing Data Governance from the Ground Up*
applies the same discipline to data, holding that a governance plan must cover the
end-of-life phases — data usage, archiving, and destruction — and that governed teams decide
retention, storage, and deletion in advance rather than hoarding everything.

A retirement gate should confirm:

- **A reason and an owner.** Someone documents why the model is being retired and answers
  for the call.
- **A clean handoff.** Any models or systems that depended on this one are re-pointed or
  validated first, because retiring a model others rely on can cascade.
- **Access removal.** The model is taken offline and its entry points are disabled.
- **Data disposition.** Training and operational data is archived or destroyed according to
  retention policy, balancing compliance obligations against the cost and risk of holding
  data indefinitely.
- **Records.** The retirement is logged in the inventory and lineage so the system's history
  stays reconstructable for audit.

The *Data Governance Handbook*'s ownership principle applies at the end as much as the
start: even retirement needs a named accountable owner, or the model lingers unmanaged and
keeps making decisions nobody watches.

## Model Inventory

An inventory, or registry, is the foundation the other lifecycle controls rest on, because
an organization cannot govern systems it cannot list. As the mission's technical-controls
research emphasizes, a dependable inventory is usually the highest-value governance artifact
there is: it converts broad policy into a tangible, queryable register that audit,
monitoring, and incident-response teams can use, and it tends to be the first deficiency
surfaced when an organization measures its maturity, since most firms struggle to enumerate
every model running in their systems.

*Platform and Model Design for Responsible AI* describes the model inventory as holding all
the models running in production, along with the terms stakeholders need to see what each
model does and where its limits are. Crucially, the inventory makes it possible to build a
**dependency tree** showing how models interact, which reveals which models carry the most
inherent risk because a failure in one could ripple into others. A typical inventory entry
records:

- The model's identity, name, and version.
- Its development stage (in use, under development, or retired).
- Its owner and business sponsor.
- A high-level risk assessment and tier.
- Its intended use, goals, assumptions, and limitations.
- Its volume and context of use, plus its financial or customer impact.
- Its data lineage plus pointers to its documentation and monitoring setup.

The book also flags a practical trap: inventories tend to go stale. Its remedy is to hand
explicit responsibility for upkeep to a designated owner who works closely with model owners
to keep each entry current. The *Data Governance Handbook* makes the parallel point for
data — that governed assets have clear owners and that stewards keep them current — which
carries over directly to a model registry.

Current research treats the inventory as the scaffolding for all the other controls: a
governance-focused MLOps stack is built around a model registry that keeps model cards and
versioned artifacts, and ISO/IEC 42001 assumes the organization can name the AI systems
within its management system's scope. If the organization cannot produce a complete,
current list of its models on request, that is the first control to build.

## Lineage

Lineage is the record of where data and models came from and how they changed over time. It
is what makes the whole sequence reconstructable, and therefore auditable. *Platform and
Model Design for Responsible AI* insists that data and model lineage be captured across the
development phases so that, at any moment, the team can see how a model evolved. *Designing
Data Governance from the Ground Up* defines data lineage as confirming the origin of data,
recording the path it follows over time, and showing its flow from one system to another,
and it argues that lineage underpins transparency and root-cause analysis.

The book demonstrates the value with a hiring-algorithm example: when a bias concern is
raised about a deployed model, lineage lets the team trace back to the data and discover
that an indirect bias — a sensitive attribute tracking a non-sensitive one such as a zip
code — had been baked into the training set. With lineage in place, the fix is to roll back
to a version before the biased data and retrain, rather than guessing at the cause.

Lineage is not free; it depends on solid metadata management. The *Data Governance Handbook*
treats lineage as a capability built on metadata: to trace data, the organization must first
define the metadata that matters and assign stewards who keep it current. Two views are
commonly useful:

- **End-to-end lineage**, which shows the complete path of an asset from its source inputs
  to its final uses.
- **Vertical, or granular, lineage**, which focuses on a single asset or feature to reveal
  its details.

A **feature store** is one concrete lineage mechanism in practice: it catalogues features in
a single place, tracks how each feature was generated and used across projects, and gives
teams one location to answer questions about compliance, accuracy, and use. Lineage serves
several masters at once — finding errors, meeting privacy obligations, migrating systems,
and supporting bias investigations — which is why the mission's research calls it a core
technical control rather than a nice-to-have.

## Drift Detection and Response

Drift is the slow erosion of a model's validity after it goes live. It is the reason
monitoring exists, and it is the failure mode that most often catches teams off guard.
*Designing Data Governance from the Ground Up* is direct: drift is unavoidable, and a
governance plan that ignores it will fall short — but a model losing predictive power is not
fatal if the team detects it in time and has a plan.

The book defines data drift as unexpected, undocumented shifts in data structure, semantics,
or infrastructure that hurt quality, and it observes that environmental change alters how
the data a model is exposed to interacts with the training process. The mission's
technical-controls research narrows this to the two standard forms:

- **Data drift** — the input features no longer resemble the distribution they had during
  training.
- **Concept drift** — the connection between inputs and the predicted outcome shifts as
  circumstances change.

Detection usually works by running statistical tests and distribution-comparison measures
across sliding windows of live data, with warning levels and escalation points that prompt an
investigation, a retrain, or a revert. Because fresh labels for production output are
frequently missing or arrive late, *Designing Data Governance from the Ground Up* recommends
proxies: train a model on
data from six months ago and compare its outcomes with a model trained on current data, or
build pipelines whose features carry quality expectations and correct errors by updating part
of the pipeline. *Platform and Model Design for Responsible AI* describes the remediation
loop the same way — detect the drift, set a threshold for how much is tolerable, and trigger
model replacement or calibration once the threshold is crossed.

The governance discipline is to make the drift response pre-planned and owned:

- Define the drift metrics and the acceptable thresholds before deployment.
- Assign a named owner who answers for acting when an alert fires.
- Pre-write the response runbook: investigate, re-evaluate against fresh labeled data,
  retrain, or roll back.
- Log the decision so the response is auditable.

Current research adds that drift detection is routine for predictive models yet still
developing for generative and agentic systems, which rely on proxy signals and human
oversight; do not treat a plain accuracy metric as enough there.

## Incident Response

Even with good gates, incidents happen — a biased outcome surfaces, a model leaks data, a
security flaw is exploited, a system misfires in a way that harms someone. Incident response
is the controlled way the organization reacts: recognize the problem, stop the harm,
investigate the cause, fix it, and learn so it does not recur.

*Platform and Model Design for Responsible AI* includes real incident cases — data leaks and
breaches at major companies, with resulting fines and settlements — to show that incidents
are genuine, costly, and often the product of avoidable mistakes such as misconfigured
storage or weak access control. The mission's technical-controls research stresses that the
evidence base for any incident review is the **audit trail**: a log of what the system did
and when, the inputs it acted on, and who signed off on or launched each operation. The EU
AI Act treats such logging as a core obligation for high-risk systems so outcomes can be
reconstructed, and that same logging is what lets an assessor tell whether a decision
followed the intended procedure.

A workable incident-response posture for AI follows the operating model's escalation path
(see `governance-operating-model.md`):

1. **Detect.** Monitoring, lineage, user reports, or audits surface a problem.
2. **Triage.** Assess severity and impact, and decide whether to pause, roll back, or take
   the model offline.
3. **Contain.** Stop further harm — restrict access, disable the model, or roll back to a
   known-good version.
4. **Investigate.** Use the inventory, lineage, and audit logs to reconstruct what happened
   and find the root cause.
5. **Remediate.** Fix the model, the data, the process, or the control that failed, and
   validate the fix.
6. **Report and learn.** Escalate to the right body (including the board for material
   incidents), record the lessons, and feed them back into earlier stage gates so the next
   model starts smarter.

Two principles keep incident response trustworthy rather than reactive. First, **predefine
the escalation path**: know in advance whom to call and who is accountable, so the response
is not improvised when the incident strikes. Second, **keep the logs that make
reconstruction possible**: if the organization cannot reconstruct what a system did, it
cannot credibly investigate an incident, defend a decision, or show a regulator or auditor
that it governed the system correctly. Log design should balance completeness against
privacy and storage cost, with defined retention periods, access controls, and immutability
so the records cannot be quietly rewritten later.

## Running Lifecycle Governance with the Rest of the Skill

Lifecycle governance schedules every other control in this skill across time:

- **Principles** set the values the gates are protecting (`foundations-and-principles.md`).
- **The operating model** names the owners, councils, and escalation paths that run the gates
  (`governance-operating-model.md`).
- **Risk frameworks and tiering** decide how heavy each gate must be and feed the register
  (`risk-management-and-frameworks.md`).
- **Fairness, transparency, privacy, and security** each contribute specific checks to the
  data, build, evaluation, and deployment gates (`fairness-bias-accountability.md`,
  `transparency-and-explainability.md`, `privacy-and-data-governance.md`,
  `llm-and-agent-security.md`).
- **Procurement and board oversight** cover the third-party models entering at the ideation
  gate and the aggregate lifecycle picture reported upward
  (`procurement-third-party-and-board-oversight.md`).

Use [use-case-intake-form.md](../templates/use-case-intake-form.md) to open the lifecycle with a governed intake,
[model-risk-assessment.md](../templates/model-risk-assessment.md) to set the tier that calibrates every later gate, and
[model-card.md](../templates/model-card.md) to produce the documentation that carries the model through evaluation,
monitoring, and incident response.

## Where to Go Next

- **`governance-operating-model.md`** — who owns and runs each gate.
- **`risk-management-and-frameworks.md`** — the tiers and registers the gates route through.
- **`privacy-and-data-governance.md`** — the data ownership and quality underpinning the data
  and retirement stages.
- **`procurement-third-party-and-board-oversight.md`** — the supply-chain and board tiers of
  the lifecycle.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *Platform and Model Design for
Responsible AI*, *Designing Data Governance from the Ground Up*, and the *Data Governance
Handbook*. Current context on model and data cards, model inventories, monitoring and drift,
audit trails, and MLOps governance tooling is drawn from the mission research note
research-technical-controls.md. All prose is an original paraphrase and synthesis of the
ideas in these sources; idea-level attribution is consolidated in `source-index.md`. It is
not legal advice.

# Fairness, Bias, and Accountability

This reference teaches how to reason about and operationalize fairness, bias, and
accountability in AI systems. Its central message is that fairness cannot be collapsed into a
single number: the choice of any one metric embeds a contested moral judgment, and a defensible
program measures the right things, documents the reasoning behind the choice, and assigns someone
who can be held answerable for the outcome. It synthesizes (never reproduces) the ideas in *AI
Fairness*, *Introduction to Responsible AI*, and *Responsible AI: Best Practices*. Read it with
`foundations-and-principles.md` (which frames fairness and accountability as governance
principles), `transparency-and-explainability.md` (which covers the explanation and disclosure
half of accountability), and `ai-lifecycle-governance.md` (which places fairness review at stage
gates). Use the [model-card.md](../templates/model-card.md) and [model-risk-assessment.md](../templates/model-risk-assessment.md) templates to record the decisions
this reference describes. It is not legal advice.

## Why Fairness Defies a Single Metric

It is tempting to ask "is our model fair?" and expect a yes-or-no answer backed by one score. The
books that inform this reference converge on a different lesson: fairness is context-dependent,
partly subjective, and measured in several irreducibly different ways, so any attempt to reduce it
to one threshold is an oversimplification that can hide real harm. *Responsible AI: Best Practices*
stresses that fairness is so bound up with context and so many-sided that no small set of numbers
can do it justice, and warns that metric-driven toolkits risk a compliance checklist mentality in
which a check mark stands in for genuine scrutiny. *AI Fairness* makes a related point about so-called "ethics
washing": running a model through a fairness library, picking whichever measurements it happens to
satisfy, and then claiming the model was audited is a way of manufacturing the appearance of
diligence rather than exercising judgment.

The practical consequence is that an organization must be able to say *which* notion of fairness
it is pursuing and *why*. There is no neutral default. Choosing a metric is a value-laden decision
that deserves an explicit ethical justification, not a habit of reaching for the newest or most
popular measurement. *AI Fairness* argues that the choice of metrics should be grounded in an
independent theory of justice rather than in convention, and at minimum the reasoning behind the
choice must be stated and defensible. *Introduction to Responsible AI* reinforces the point that
defining a universally acceptable notion of fairness is unachievable because different contexts
demand different definitions, and the balance between equal treatment and correcting historical
disparities is inherently contested.

## Where Bias Comes From

Before measuring fairness it helps to name the sources of bias, because each has a different
remedy. Drawing on the Barocas-Selbst analysis that *AI Fairness* recounts, a model can absorb
bias in at least three distinct ways:

| Source of bias | What it looks like | How hard it is to address |
|---|---|---|
| Sample inequality | The training set under-represents a group (for example, far more resumes from men than women) | Easiest; collect a larger, balanced sample |
| Label inequality | The outcomes used to train the model are themselves skewed by biased human judgments (for example, managers rated women's performance lower) | Manageable; rework or reweigh the labels |
| Underlying social inequality | Historical oppression means a group genuinely has fewer qualifying features, so even "true" labels encode disadvantage | Hardest; this may be injustice rather than a fixable data flaw |

*Introduction to Responsible AI* frames bias as a human tendency that seeps into AI through the
data it learns from, and it documents well-known real-world failures to illustrate the stakes:
recruiting tools that downgraded resumes containing female-coded terms, recidivism risk scores
that overestimated reoffense for minority groups, photo auto-tagging that mislabeled people, and
facial recognition that misidentifies darker-skinned faces. The book notes that eliminating all
bias is unfeasible, so the realistic goal is to mitigate the biases that produce unjust or harmful
consequences. Bias can enter at collection, at labeling, through proxy features that stand in for
protected attributes, and through the choice of target variable itself.

## Fairness Metrics and Their Limits

Fairness metrics divide broadly into individual and group measures. *AI Fairness* catalogs the
families, and *Responsible AI: Best Practices* adds the practitioner vocabulary. The important
distinction is between *equal treatment* (whether the same rule is applied to everyone) and *equal
impact* (whether outcomes balance across groups); the two are in mathematical tension with one
another and with overall accuracy.

| Metric | Kind | What it requires | Caveat |
|---|---|---|---|
| Blindness | Individual | No protected attributes (or proxies) in the data | Being blind to a feature does not mean being blind to its proxies |
| Counterfactual tests | Individual | Changing an irrelevant feature should not change the outcome; changing a relevant one should | Needs a defensible notion of "relevant" and "similar" |
| Demographic parity | Group | Positive-selection rates are equal across groups | Can conflict with actual qualification rates; resembles quotas |
| Equalized odds | Group | True-positive and false-positive rates are equal across groups | Trades off against selection parity; needs a clear decision threshold |
| Equal opportunity | Group | True-positive rates (recall) are equal across groups | Widely endorsed, relatively minimal; respects actual qualifications |
| Calibration | Group | A given predicted score means the same likelihood of the outcome in every group | Not always satisfiable alongside other parity notions |

The most consequential limit is formal: *AI Fairness* describes the "impossibility" results of
2017, showing that in realistic conditions no single model can satisfy all of the reasonable
fairness requirements at once. Equalized odds, equal opportunity, calibration, and demographic
parity pull against one another. Practitioners must therefore decide priorities rather than chase
every parity at once. The book also flags the shallowness of a field that produces many metrics
with little underlying moral content, urging that metric selection be anchored in ethical argument.

### Choosing and Reporting the Metric Set

Because the metrics conflict, the defensible move is to state a priority order. *AI Fairness*
proposes a hierarchical approach: a model must meet a minimally acceptable accuracy bar; it must
expose its most significant causal features so that irrelevant (especially protected) features can
be ruled out; it should equalize recall for qualified candidates across protected groups; and it
should equalize selection rates unless doing so would push accuracy below the minimum standard or
below default human practice. Whatever ordering an organization adopts, the reasoning should be
recorded in the model's documentation, where it becomes auditable. This is the substance behind the
fairness section of the [model-card.md](../templates/model-card.md) template: not a single number but an explicit statement of
which metrics were computed, what they showed, and why those metrics were the right ones.

## Algorithmic Justice and the Trade-offs of Enforcing Fairness

Correcting a model to improve group parity is not free. *AI Fairness* documents that enforcing
selection parity almost always reduces overall accuracy, and often reduces accuracy and precision
most sharply for the very groups the mitigation is meant to help. There are also "losses in
fairness": a mitigated model can create reverse-discrimination cases where someone from a majority
group can truthfully claim that they would have been approved under the original model but were
rejected after the correction. These are genuine costs that must be weighed, not dismissed.

Because demographic parity, equalized odds, equal opportunity, and calibration make incompatible
demands, an organization that enforces one parity notion necessarily gives ground on the others
and on raw accuracy. Fairness also has to be reconciled with the other trustworthiness
properties. A purely accuracy-maximizing model can be blatantly unfair, but a heavily constrained
one can be so degraded that it fails its purpose. The discipline this reference recommends is to treat fairness
as one dimension of a risk assessment rather than an absolute override: surface the trade-offs,
quantify what is given up, and let the governance process decide whether the residual outcome is
acceptable. This is why fairness review belongs in the same workflow as the [model-risk-assessment.md](../templates/model-risk-assessment.md)
template and the tiering logic in `risk-management-and-frameworks.md`.

## Disparate Treatment Versus Disparate Impact

The legal vocabulary for discrimination usefully frames the fairness problem. As *AI Fairness*
explains, US anti-discrimination law recognizes disparate treatment (using a protected attribute in
a decision that harms an individual) and disparate impact (a practice that produces unequal outcomes
across protected groups even without intent). Algorithms complicate this picture because they have
no intent, so the traditional "intent to discriminate" test does not map cleanly onto them. Yet the
law has long recognized that discrimination need not be intentional, and regulators such as the
Consumer Financial Protection Bureau have applied an "effects test" to practices that are neutral on
their face but disproportionately harmful, unless a legitimate business need justifies them.

This matters for accountability because many unfair AI outcomes are unintentional and driven by
proxy features rather than explicit protected attributes. A model that never sees race can still
produce racially disparate results through correlated proxies such as income, geography, or
residence. *Responsible AI: Best Practices* makes the same point at the level of the system: a
company can assert that it never collects protected attributes, yet the model perpetuates historical
bias through the data it was trained on. Consequently, "we are blind to protected attributes" is not
an accountability claim; it is a reason to test whether the model is nonetheless reproducing
disparate outcomes. Notable policy responses, such as New York City's Local Law 144 requiring
public bias audits of automated hiring tools, show the direction of travel toward mandatory
disparate-impact reporting.

## Accountability: Assigning Answerability

Accountability turns the analysis of bias into an obligation with an owner. *Responsible AI: Best
Practices* frames it as a principle that the people responsible for each phase of the AI lifecycle
should be identifiable and answerable for outcomes, and it offers the *role-level accountability
contract* as the concrete mechanism: a document describing the specific roles and responsibilities
of every individual or organization involved in the system's development and use, sometimes
formalized as a signed contract. The intent is that when a system misbehaves, there is a clear,
accessible answer to the question of who is accountable at each stage, rather than a diffuse
organizational fog. The book acknowledges the tension between this clarity and the loss of
flexibility that rigid role boundaries can create, and it pairs role-level accountability with
related controls such as a code of ethics and oversight structures.

*Introduction to Responsible AI* links accountability to transparency: a system that is explainable
and documented is one whose developers can be held responsible for failures, and one that can be
audited for compliance. Accountability is therefore not a separate tool but a property that the rest
of the governance stack produces — clear ownership, recorded decisions, audit trails, and disclosure
make answerability possible. The `governance-operating-model.md` reference assigns these roles and
decision rights; this reference supplies the fairness-specific content those roles are accountable
for.

## Model Cards as the Accountability Record

Model cards are the canonical documentation artifact that makes fairness and accountability
verifiable after the fact. Originating in the 2019 model-cards paper (current research, see
research-technical-controls.md), a model card is a short document accompanying a released model
that records its intended use, the evaluation procedure, and benchmarked performance *disaggregated
by subgroup* — so that accuracy variation across demographic and other groups is visible rather than
hidden inside a single average. The disaggregation is the point for fairness: an overall accuracy
figure can mask a model that performs well for the majority and poorly for a minority. The card
also documents limitations, so the model is not applied in contexts for which it is unsuited.

*Responsible AI: Best Practices* treats model cards and data sheets as the checklist-and-template
layer of a responsible-AI program, with the caveat that such artifacts only work when high-quality,
standardized practices sit behind each check box; otherwise they become superficial compliance
theater. For governance purposes, a completed model card records the fairness metrics that were
chosen, why, what they showed across groups, and which decisions were made in response — turning the
discussions in this reference into a durable, audit-ready record. The [model-card.md](../templates/model-card.md) template in
this skill implements exactly that, and research-technical-controls.md documents the current
state of model and data cards as a widely adopted control.

## Running Fairness, Bias, and Accountability with the Rest of the Skill

Fairness work is not a standalone activity; it threads through every other governance surface:

- **Principles** define what the organization values about fairness and accountability
  (`foundations-and-principles.md`).
- **Risk frameworks** host fairness measures alongside accuracy, safety, and robustness metrics,
  and route fair models through the same tiering and register as any other risk
  (`risk-management-and-frameworks.md`).
- **Lifecycle gates** schedule when fairness is assessed — from ideation through post-deployment
  monitoring for drift that reintroduces bias (`ai-lifecycle-governance.md`).
- **Transparency and explainability** provide the disclosure and explanation that make
  accountability meaningful (`transparency-and-explainability.md`).
- **Board and third-party oversight** consume the fairness findings as part of the aggregate risk
  picture and hold the accountable owners to account
  (`procurement-third-party-and-board-oversight.md`).

Use the [model-card.md](../templates/model-card.md) template to record metric choice, subgroup results, and limitations; the
[model-risk-assessment.md](../templates/model-risk-assessment.md) template to weigh fairness against other risk dimensions; and the
[use-case-intake-form.md](../templates/use-case-intake-form.md) template to flag high-stakes, fairness-sensitive use cases early.

## Where to Go Next

- **`foundations-and-principles.md`** — fairness and accountability as governing principles.
- **`transparency-and-explainability.md`** — the explanation and disclosure that enable
  accountability.
- **`ai-lifecycle-governance.md`** — where fairness review and monitoring happen across the
  lifecycle.
- **research-technical-controls.md** — current state of model cards, data cards, and monitoring.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *AI Fairness*, *Introduction to
Responsible AI*, and *Responsible AI: Best Practices*. Current context on model cards and
monitoring is drawn from the mission research note research-technical-controls.md. All prose is
an original paraphrase and synthesis of the ideas in these sources; idea-level attribution is
consolidated in `source-index.md`. The legal discussion is educational context, not legal advice,
and specific laws and enforcement should be confirmed against current sources at use time.

# AI Governance: Foundations and Core Principles

This reference is the entry point to the `ai-governance` skill. It defines what AI
governance is, introduces the core principles that anchor every other reference in this
skill, and draws the boundary between governance, compliance, and risk management. Read
this first, then route to the deeper references (operating model, risk frameworks,
lifecycle, fairness, transparency, privacy, security, regulation, board oversight) as your
question requires. Everything here is an original synthesis of the ideas in the source
books and current research; it is not legal advice.

## What AI Governance Is

AI governance is the set of structures, processes, roles, and decisions an organization
uses to direct, evaluate, and control how it designs, builds, deploys, and uses AI systems.
Where software governance generally cares about quality and reliability, AI governance
extends the field to cover the novel ways a model can harm people and organizations: biased
outcomes, opacity in decision-making, data-privacy exposure, unsafe behavior at scale, and
the erosion of human control over consequential choices.

Three ideas recur across the sources:

- **It is continuous, not a one-time check.** Responsible AI in the Enterprise frames
  governance as an ongoing discipline that covers the whole life of a model — from a
  prototype in a notebook to a monitored production system that may be retired years later.
- **It is cross-functional.** AI systems sit at the intersection of data, engineering,
  product, legal, risk, privacy, security, and the business. Governance only holds when
  people from these functions share explicit decision rights and a common set of policies
  (see `governance-operating-model.md`).
- **It is about behavior and accountability, not just documents.** Introduction to
  Responsible AI stresses that a responsible program must translate values into how people
  actually work every day, and that named humans must answer for outcomes.

In short, governance operationalizes the values and principles below into concrete
structures (charters, councils, committees), artifacts (inventories, risk registers,
model cards, audit reports), and controls (stage gates, monitoring, incident response)
that let an organization steer AI responsibly rather than merely react after something goes
wrong.

### Governance as a discipline vs. ethics as an aspiration

Ethics supplies the values; governance supplies the machinery. An AI ethics statement that
"we will be fair" is a declaration of intent; governance is what turns that declaration into
a fairness assessment before deployment, a documented owner for the model, and a way to
correct drift when the model starts producing biased results. Beyond the Algorithm argues
that ethical principles are a necessary starting point but become meaningful only when they
are wired into design, review, and audit processes. Governance is therefore best understood
as the engineering arm of AI ethics.

## The Core Principles

Across industry frameworks and the source books there is striking convergence on a short
set of core principles. Responsible AI: Best Practices catalogs the common principles and
notes that, while wording varies, the substance overlaps heavily across jurisdictions and
organizations. The OECD principles (adopted 2019, updated 2024) and the NIST AI Risk
Management Framework reinforce the same themes from the intergovernmental and standards
sides — current research (see research-standards.md) confirms these remain the shared
vocabulary in 2026.

For this skill the six core principles are:

1. **Fairness** — systems should not produce or reinforce unjust discrimination.
2. **Accountability** — named people and teams answer for each stage of a system's life.
3. **Transparency** — people should be able to understand when and how AI affects them.
4. **Privacy** — collection, use, and storage of data respects individual rights and legal
   limits.
5. **Safety** — systems behave reliably and avoid harm, including over time and under
   unexpected conditions.
6. **Human oversight** — humans retain meaningful control, especially over consequential or
   autonomous decisions.

The table below summarizes each principle, what it looks like in practice, and the main
risks it guards against.

| Principle | What it demands in practice | Failure mode it prevents |
|---|---|---|
| Fairness | Bias assessment, equity-aware metrics, trade-off decisions | Discriminatory outcomes and systemic inequity |
| Accountability | Named owners, decision rights, audit trail, escalation paths | "Nobody owns it" and invisible blame |
| Transparency | Disclosure, documentation, explainability where required | Opaque decisions people cannot challenge |
| Privacy | Minimization, consent, retention limits, protection | Data misuse and regulatory exposure |
| Safety | Robustness, monitoring, drift detection, fail-safes | Harm from unreliable or degrading behavior |
| Human oversight | Human-in-the-loop control, override, meaningful review | Unchecked automation of consequential choices |

Each principle is developed in its own section below, then tied together in the
governance-vs-compliance-vs-risk discussion.

## Fairness

Fairness is the demand that an AI system not treat people differently in ways that are
unjust, and that it not quietly reproduce historical or structural bias. Introduction to
Responsible AI describes fairness as among the most urgent of the responsible-AI concerns
because AI is increasingly embedded in consequential decisions about credit, employment,
healthcare, and opportunity.

Two commitments are central:

- **Understand where bias comes from.** Bias enters a system through training data that
  mirrors human decisions and social patterns, through the choices of model developers, and
  through how a system is used in the world. Fairness work starts by locating these sources
  rather than assuming a "neutral" model.
- **Do not reduce fairness to a single number.** There is no universally correct fairness
  metric; different contexts demand different definitions, and the fair thing to do often
  involves a genuine trade-off between competing goods (for example, equalizing error rates
  versus equalizing acceptance rates). Responsible AI: Best Practices and AI Fairness both
  stress that teams must be explicit about which notion of fairness they are optimizing and
  who bears the cost of the trade-off.

The concrete work of fairness — metrics, bias measurement, mitigation, and the model card
that records fairness choices — is covered in `fairness-bias-accountability.md`.

## Accountability

Accountability is the principle that named humans — not an algorithm — are answerable for
an AI system and its outcomes. Responsible AI in the Enterprise makes the point that
models are built and owned by people, and governance only works when responsibility for
each stage of the life cycle is explicit.

Three practical requirements follow:

- **Named owners.** Someone is responsible for the data, the model, the deployment, and the
  ongoing monitoring, and those assignments are written down.
- **An audit trail.** Decisions about design, data, thresholds, and deployment are recorded
  so that an outcome can be traced back to a human decision.
- **Escalation and contestability.** When something goes wrong, there is a defined path to
  raise, review, and correct it — including a way for affected people to challenge a
  decision. Responsible AI: Best Practices identifies contestability as a companion to
  accountability: people significantly affected should have a timely process to push back.

## Transparency

Transparency is the principle that people should be able to find out when AI is being used
and how it is affecting them, and — where the stakes justify it — to understand enough about
why a decision was made to evaluate or challenge it. Beyond the Algorithm connects
transparency to the "black box" problem of complex models whose internal reasoning is hard
to follow, which makes both oversight and remediation harder.

Transparency is not one thing; it operates at several levels:

- **Disclosure.** Telling people they are interacting with AI or that AI informed a decision
  about them.
- **Documentation.** Recording intended use, data, performance, and limitations in artifacts
  such as model cards and data cards.
- **Explainability.** Providing an explanation of a specific prediction, typically through
  interpretable methods or post-hoc explanation techniques, when the decision is
  consequential enough to require it.

Introduction to Responsible AI emphasizes that transparency also strengthens
accountability and auditability, because a system that can be understood can be reviewed and
held to account. The techniques and the "when is explainability required" question live in
`transparency-and-explainability.md`.

## Privacy

Privacy in AI governance is broader than legal compliance: it is the principle that data
about people should be collected, used, and kept only for legitimate purposes, with
appropriate limits and protections. Privacy is tightly coupled to AI because models depend
on large datasets, can make inferences about individuals from apparently unrelated signals,
and can be attacked in ways that leak information about the people in the training data.

Governing privacy means deciding — and enforcing — several things:

- **Minimization and purpose.** Collect and use only what the task genuinely requires.
- **Consent and rights.** Respect people's expectations and legal rights over their data.
- **Retention and lifecycle.** Delete or de-identify data when it is no longer needed.
- **Protection and privacy-enhancing techniques.** Apply security controls and, where
  appropriate, techniques that reduce exposure even if data is compromised.

Beyond the Algorithm and Introduction to Responsible AI both stress that privacy is a
precondition for trust: people will not rely on systems that expose them. The deeper
treatment of training-data versus operational-data governance, ownership, lineage, and
quality is in `privacy-and-data-governance.md`.

## Safety

Safety is the principle that an AI system should behave reliably and avoid harm, not just at
launch but as it operates, learns, and interacts with the world. Responsible AI in the
Enterprise flags that models can drift over time, degrade, or be used in ways their
designers did not anticipate, so safety is an ongoing property rather than a static
certification.

Safety concerns for AI include:

- **Reliability.** The system performs its intended function under expected conditions.
- **Robustness.** It degrades gracefully under unusual inputs, adversarial attempts, and
  changed environments.
- **Monitoring and drift detection.** Systems are watched in production so that degrading
  performance is caught before it causes harm.
- **Fail-safes and containment.** There are ways to pause, limit, or roll back a system.

Introduction to Responsible AI groups robustness and reliability with safety, and current
research on NIST's AI RMF and its Generative AI Profile (see research-standards.md)
extends these concerns to generative models — hallucination, misuse, and information
integrity. The security-specific mechanics are in `llm-and-agent-security.md`.

## Human Oversight

Human oversight is the principle that people should retain meaningful control over AI,
especially when systems make consequential or autonomous decisions. Responsible AI: Best
Practices explicitly pairs accountability with human oversight, and Introduction to
Responsible AI warns that without it AI can operate as an opaque, unchecked force rather
than a tool under human direction.

Human oversight takes practical forms:

- **Human-in-the-loop.** A human reviews or approves consequential decisions before they
  take effect.
- **Human-on-the-loop.** A human monitors an autonomous system and can intervene when
  needed.
- **Override and escalation.** There is a way for people to correct or reverse a decision,
  and a defined path to escalate when confidence is low.

The right degree of oversight depends on the stakes and the autonomy of the system; a
low-risk content tool needs far less than a system that makes employment or credit
decisions. Deciding that degree is a core governance activity.

## Governance, Compliance, and Risk: The Distinction

These three terms are often used interchangeably, but the sources draw a clear and useful
separation, and the difference matters for how you design a program.

- **Risk** is the fact that using AI can produce adverse outcomes — harm to people, financial
  loss, reputational damage, or regulatory penalty. Risk management is the practice of
  identifying, assessing, prioritizing, and mitigating those possibilities. It is analytical
  and continuous, and it is the substrate everything else sits on.
- **Compliance** is the narrower act of satisfying externally imposed rules — laws,
  regulations, and contractual obligations. Compliance asks "are we meeting the letter of
  the requirement?" and is inherently reactive to whatever the rulebook currently says.
- **Governance** is the broader system that decides *what to do about risk and
  responsibility at all*, including setting the policies, roles, decision rights, and
  accountability that go beyond any current regulation. Governance sets the objectives and
  guardrails; compliance is one mechanism for meeting a subset of them; risk management is
  the ongoing practice that informs governance and feeds compliance.

A useful way to think about it, grounded in Responsible AI in the Enterprise and Beyond the
Algorithm:

> Governance decides who decides, what we will and will not do, and who is accountable.
> Risk management tells us what could go wrong and how bad it might be. Compliance checks
> that we are meeting the rules we are required to meet. You need governance even in the
> absence of any specific law, because principles like fairness and human oversight are not
> fully captured by any rulebook.

Three consequences follow for building a program:

1. **Do not let compliance substitute for governance.** Passing every current legal check
   does not make a system fair or safe. Governance should always be able to demand more
   than the law does (a point Beyond the Algorithm makes by observing that ethics and
   responsibility routinely exceed the legal minimum).
2. **Anchor governance in risk, not in the rulebook.** Because the regulatory landscape is
   still stabilizing (see research-regulatory.md and research-standards.md for the 2026
   state), a risk-based approach lets you stay ahead of specific laws instead of reacting
   to each one. Current frameworks such as NIST's AI RMF and ISO/IEC 23894 are explicitly
   risk-based for exactly this reason.
3. **Make risk management continuous and accountable.** Risk registers, heat maps, and
   scoring are only useful if someone owns them, reviews them, and acts on them — which is
   where governance provides the missing structure that pure compliance does not.

### Governance as the connective layer

You can think of the three as nested: risk management is the ongoing analysis, compliance is
a subset of obligations drawn from law and contract, and governance is the whole system that
keeps the other two aligned with the organization's values and responsibilities. Responsible
AI in the Enterprise treats AI risk governance as one component of an enterprise's broader
risk and governance framework, and current research on board fiduciary duty (see
research-org-board-governance.md) reinforces that AI governance is now a recognized
director-level responsibility, not merely an engineering or legal task.

## Putting Principles into Practice

A principles-first approach fails without operational machinery. The rest of this skill
provides that machinery, but the checklist below is a useful starting point for turning the
six principles into a working program.

- [ ] **Adopt a written principles statement.** Name the six principles (or an equivalent
      set) and commit the organization to them publicly.
- [ ] **Designate named owners.** Assign accountability for data, model, deployment, and
      monitoring, and record those assignments.
- [ ] **Stand up a governance structure.** A council or review body with clear decision
      rights (see `governance-operating-model.md`).
- [ ] **Maintain an AI inventory and risk register.** Know what systems exist and what risk
      each poses (see `risk-management-and-frameworks.md`).
- [ ] **Gate the life cycle.** Apply stage gates from ideation through retirement so
      principles are checked at each step (see `ai-lifecycle-governance.md`).
- [ ] **Document the consequential choices.** For fairness and explainability trade-offs,
      record what was decided and why, in artifacts such as model cards.
- [ ] **Monitor in production.** Detect drift, incidents, and misuse, and act on them.
- [ ] **Report to the board.** Give leadership visibility into risk and incidents (see
      `procurement-third-party-and-board-oversight.md`).

Use the fillable templates in this skill — the governance charter, use-case intake form,
model risk assessment, model card, third-party due-diligence, and board report — to instantiate
these items rather than starting from a blank page.

## Where to Go Next

- **`governance-operating-model.md`** — councils, stewards, decision rights, RACI, maturity.
- **`risk-management-and-frameworks.md`** — NIST AI RMF, ISO/IEC 42001 and 23894, tiering.
- **`ai-lifecycle-governance.md`** — stage gates and controls across the model life cycle.
- **`fairness-bias-accountability.md`** and **`transparency-and-explainability.md`** — the
  two principles most in need of metric-level guidance.
- **`privacy-and-data-governance.md`**, **`llm-and-agent-security.md`** — the data and
  security underpinnings of safety and privacy.
- **`regulatory-landscape.md`** and **`procurement-third-party-and-board-oversight.md`** —
  the compliance and oversight layers.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from the following sources: *Responsible
AI in the Enterprise*; *Introduction to Responsible AI*; *Beyond the Algorithm*;
*Responsible AI: Best Practices* (the book on trustworthy AI systems). Current context on
standards and organizational governance is drawn from the mission research notes
research-standards.md and research-org-board-governance.md. All prose is an original
paraphrase and synthesis of the ideas in these sources; idea-level attribution is
consolidated in `source-index.md`.

# AI Governance Operating Model

This reference teaches the machinery that turns governance principles into a working
system: the six-step operating model, the councils and stewards that carry it, how decision
rights are assigned and made explicit (including RACI), how to choose between federated and
centralized structures, how to gauge maturity, and the culture that makes the whole thing
hold together. It is an original synthesis of the ideas in *Designing Data Governance from
the Ground Up*, the *Data Governance Handbook*, and current research on organizational and
board governance (see research-org-board-governance.md). Read it alongside
`foundations-and-principles.md`, which explains *why* governance matters; this file explains
*how to run it*. It is not legal advice.

## What a Governance Operating Model Is

A governance operating model is the concrete arrangement of roles, bodies, and decision
rules an organization uses to run governance day to day, as distinct from the principles
themselves. Principles tell you *what you value*; the operating model tells you *who gets to
decide, who must be consulted, who does the work, and how disagreements get resolved*. Think
of it as the wiring diagram under the policy statements.

Three commitments are essential to any operating model, and they recur across the sources:

- **Documented accountability.** Every significant decision and every piece of the life
  cycle has a named owner. *Designing Data Governance from the Ground Up* stresses that
  governance fails when nobody is explicitly accountable for a decision right, and current
  research extends this to the boardroom: director fiduciary duty of oversight now reaches
  mission-critical AI, so accountability must trace all the way up.
- **A bounded escalation path.** The operating model defines who approves higher-risk
  actions and how disputes are raised. A council exists precisely to arbitrate across
  domains so that a single team cannot quietly decide something consequential alone.
- **A living, maturing system.** The model is not a static org chart; it is reassessed as
  the organization grows, as its risk profile changes, and as its maturity rises. The books
  and the current research both treat the operating model as something that evolves rather
  than something you install once.

Because AI systems sit across data, engineering, product, legal, risk, privacy, and
security, the operating model is necessarily cross-functional. The *Data Governance
Handbook* argues that the people dimension is the cornerstone: identifying who is
responsible and accountable for a domain is what makes data (and by extension AI)
governable at all.

## The Six-Step Model

*Designing Data Governance from the Ground Up* offers a pragmatic, sequenced model for
building a governance program that is easy to translate from data to AI. The six steps are:

1. **Find your framework.** Decide which governance framework fits your organization and
   connect every initiative back to a clear mission statement. Without a shared "why," the
   rest of the machinery has no anchor.
2. **Select data stewards.** Identify people who own the strategic and tactical decisions
   within each domain. Stewards are the trusted advisors for their area, whether that is
   sales, finance, a specific dataset, or a specific AI model.
3. **Build the governance council.** Stewards cannot coordinate from silos; a cross-functional
   council brings them together, sets shared standards, and arbitrates across domains.
4. **Write a roadmap.** Turn the strategy into a living, single source of truth that maps
   initiatives to milestones and success metrics, so the program has a measurable direction.
5. **Practice governance-driven development.** Governance is not something bolted on after
   building; it is woven into how teams choose tools, partners, and architecture as they
   build and deploy.
6. **Monitor in production.** Governance continues after launch: stewards watch quality,
   behavior, and drift in the running system and act when things degrade.

The power of this sequence is that it is deliberately culture-first and people-first: it
starts with alignment and ownership before it reaches for tools or controls. For AI, the
"steward" and "council" steps map naturally onto model owners and an AI risk council, and
the monitoring step maps onto production drift detection and incident response (see
`ai-lifecycle-governance.md`).

## Roles: Stewards, Owners, and the Chief AI Officer

The operating model resolves into a small set of recurring roles. The *Data Governance
Handbook* distinguishes clear layers that transfer well to AI governance:

- **Executive / accountable owner.** The most senior person accountable for a domain's
  success or failure. In data this is often a chief data officer; in AI, organizations are
  increasingly naming a Chief AI Officer (CAIO) or equivalent, a trend current research
  shows is accelerating — including a U.S. federal mandate in 2024 for agency-level CAIOs.
  This role sets vision, owns policy, and reports up to the executive team and board.
- **Domain steward / owner.** The day-to-day leader for a specific domain, asset, or model.
  Stewards answer for data quality, access, and management within their area and keep the
  accountable executive informed. In *Designing Data Governance from the Ground Up*,
  stewards are split between *business stewards* (who own the data for business processes
  and know what "good" looks like for the domain) and *technical stewards* (who own the
  systems, pipelines, and implementation details).
- **Functional professionals.** The analysts, data scientists, engineers, and architects who
  do the hands-on work under the stewards and are consulted on decisions affecting their
  specialty.

The key discipline is that these roles are **named and documented**, not implied. Current
research on NIST's AI RMF "govern" function reinforces this: organizations are expected to
document roles, responsibilities, and lines of communication, and to put executive
ownership explicitly on the record for AI development and deployment decisions.

## Councils and Committees

Councils are the coordination layer that stops stewards from operating in silos. Both books
describe a tiered structure that scales with the organization, and the current research
confirms the same three-tier pattern for AI:

- **Board / board committee.** The top oversight tier, responsible for fiduciary oversight of
  mission-critical AI, reviewing risk and mitigation as a recurring agenda item, and being
  briefed promptly on material incidents. Research describes this as "noses in, fingers
  out": boards stay informed enough to govern without stepping into operations.
- **Enterprise council / committee.** The cross-functional body that sets policy and
  standards and approves higher-risk use cases. The *Data Governance Handbook* distinguishes
  an *enterprise data committee* (executives, meeting quarterly, accountable to the CEO and
  board) from an *enterprise data council* (senior leaders one to two levels below the
  C-suite, doing the detailed policy and monitoring work and reporting up to the committee).
  For AI this is the AI ethics or AI risk council that convenes compliance, legal, privacy, risk,
  security, data, product, and engineering.
- **Stewards / operating owners.** The people who run day-to-day controls and drive
  implementation inside their domains.

Each body should be run under a **charter** — a written terms of reference covering purpose,
key representatives, responsibilities, reporting lines, meeting cadence, communication plan,
and how it evaluates its own effectiveness. The *Data Governance Handbook* includes sample
charters for the committee and council, and this skill ships a fillable
[governance-charter.md](../templates/governance-charter.md) template to instantiate one. A recurring practical note from
*Designing Data Governance from the Ground Up* is that councils need a regular cadence (for
example, roughly an hour every other week, plus focused work between meetings) and ground
rules for psychological safety so that honest discussion, including disagreement, is
possible.

## Decision Rights and RACI

The operating model only works if it is explicit about who decides what. *Designing Data
Governance from the Ground Up* frames this as a set of questions: Who owns data within each
domain? Who is responsible for it? Who decides which teams and people gain access? The same
questions apply to an AI model or use case: who approves its development, who approves its
deployment, who owns its monitoring, and who can escalate if it drifts or fails.

**RACI** is the standard tool for making these decision rights legible. A RACI matrix assigns
each task or decision one or more of four labels:

| RACI letter | Meaning | Practical role in AI governance |
|---|---|---|
| **R**esponsible | Does the work | The steward/engineer who builds or monitors the model |
| **A**ccountable | Owns the outcome | The executive or owner who answers for it and signs off |
| **C**onsulted | Must be asked before deciding | Legal, risk, privacy, security, affected business units |
| **I**nformed | Must be told after deciding | Broader stakeholders, sometimes the board |

A well-built RACI makes disputes rare because it answers the question in advance: for any
given decision, exactly one person is accountable, the right people are consulted, and
everyone who needs to know is informed. For a use case this might assign the model owner as
Responsible, the head of the domain as Accountable, legal and privacy as Consulted, and the
AI council and board as Informed. Documenting this explicitly in the governance charter and
the use-case intake flow is what turns an abstract commitment to accountability into an
enforceable decision right.

## Federated vs. Centralized Operating Models

Organizations must choose how much authority sits at the center versus in the business units.
The *Data Governance Handbook* describes a spectrum of operating models:

- **Centralized.** All responsibilities sit in a central office. This maximizes consistency
  and is useful short term to remediate large maturity gaps or quality problems, but the book
  cautions that long-term centralization separates stewards from their business units and
  erodes the subject-matter expertise that makes a steward effective. The guidance is to
  treat full centralization as a bounded, roughly 12-to-24-month phase and then move toward a
  more distributed model.
- **Federated.** The center defines policy and does the bare minimum, while business units do
  the implementation. This empowers each unit but makes it hard to assemble an
  enterprise-wide view, and often needs a rationalization effort later once programs
  formalize.
- **Semi-federated.** Units stand up their own accountable leaders who implement centrally set
  policy within their divisions, with a data domain model assigning a steward to each asset.
- **Hub-and-spoke.** A strong central office provides expectations, policies, standards, and
  shared capabilities (the hub), while business units and functions drive implementation
  (the spokes). The book regards this as the strongest model for lasting success, precisely
  because it standardizes capabilities across the organization while keeping expertise in
  the business — but only when the central office is well established and funded.

There is no single correct answer. The books and the current research converge on the
principle that the best model is the one that drives progress and maturity for your specific
organization, and that most organizations try several arrangements before settling on one.
Current research adds that there is no one prescribed AI operating model either: the balance
between a CAIO, the governance council, and a board committee is not uniform, but authoritative
guidance consistently requires documented roles, clear accountability that reaches the
board, and periodic reporting.

## Maturity and Culture

**Maturity.** Governance programs are rarely complete; they improve in stages. The *Data
Governance Handbook* treats maturity assessment as an annual exercise that measures the
organization across governance dimensions (using models such as DCAM), gives each dimension
a level, surfaces gaps, and reports results up to the executive team and board in aggregate.
Current research reinforces the idea for AI: NIST's AI RMF functions (govern, map, measure,
manage) function like a maturity ladder — a nascent organization merely identifies its
systems, while a mature one governs, maps, measures, and manages AI risk continuously and
feeds results back into improvement. ISO/IEC 42001 provides a certifiable management-system
structure built around a Plan-Do-Check-Act cycle, which is itself a maturity mechanism. The
practical takeaway is that the operating model should include an explicit way to score
itself and a roadmap to move up, not a one-time compliance checkbox. (This skill ships the
`governance-maturity.py` script to operationalize such a self-assessment.)

**Culture.** Both books stress that governance succeeds on culture, not just structure.
*Designing Data Governance from the Ground Up* emphasizes transparency and psychological
safety: people must be able to surface problems without fear of blame, or they will hide
exactly the issues governance exists to catch. The *Data Governance Handbook* makes a more
provocative point: resist treating "data culture" or "data literacy" as separate campaigns.
Instead, weave data-informed decision-making into the ordinary culture of the company and
build trust through delivery; the culture change follows from demonstrated value rather than
from a formal program. The current research echoes this for AI adoption, where knowledge
gaps, budget constraints, and regulatory uncertainty remain the leading implementation
barriers — problems that are cultural and educational as much as structural.

## Running the Operating Model Day to Day

An operating model has value only when it is exercised. Concretely, this means:

- **A standing, documented decision rights map** (RACI) that is consulted for every new AI
  use case, not an afterthought.
- **A regular cadence of council and committee meetings**, with a charter that says how often
  they meet, what they approve, and how they report upward.
- **Use-case intake and tiering** that routes new AI proposals to the right level of review
  based on risk (see `risk-management-and-frameworks.md`).
- **Incident and escalation paths** so that when a system misbehaves in production, the
  operating model tells people whom to call and who is accountable for the response (see
  `ai-lifecycle-governance.md`).
- **Periodic board reporting** so that leadership has visibility into risk, incidents, and
  third-party exposure, consistent with the oversight duty described in the research (see
  `procurement-third-party-and-board-oversight.md`).
- **An annual maturity review** that scores the program, identifies gaps, and feeds a revised
  roadmap.

## Adapting the Model to AI

The six-step model and the role/council structure transfer to AI with only modest
adjustments, and current research points to where AI-specific attention is needed:

- **Name an accountable executive and, where warranted, a Chief AI Officer** to reduce
  fragmentation, because uncoordinated AI projects otherwise proliferate without executive
  oversight.
- **Make third-party risk first-class.** Much of an organization's AI footprint is procured,
  embedded, or inherited through vendors and M&A, so supply-chain diligence and vendor
  accountability must be embedded in the operating model from the start (see
  `procurement-third-party-and-board-oversight.md`).
- **Extend stewardship to models.** Assign a named owner to every model and dataset, using
  the same business/technical steward split the books describe for data.
- **Treat the board as a real tier.** Because oversight of mission-critical AI is now a
  recognized director responsibility, the operating model should include a standing board or
  board-committee view and prompt escalation of material incidents.

Use the fillable [governance-charter.md](../templates/governance-charter.md) template to instantiate the council terms of
reference, and the use-case intake and model-risk-assessment templates to exercise the
decision rights the operating model defines.

## Where to Go Next

- **`foundations-and-principles.md`** — the values this operating model operationalizes.
- **`risk-management-and-frameworks.md`** — how the model routes work through risk tiering.
- **`ai-lifecycle-governance.md`** — the stage gates the model enforces across the life cycle.
- **`procurement-third-party-and-board-oversight.md`** — the board and vendor tiers of the model.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *Designing Data Governance from the
Ground Up* and the *Data Governance Handbook*. Current context on board fiduciary duty, the
Chief AI Officer role, governance councils, maturity, and third-party risk is drawn from the
mission research note research-org-board-governance.md. All prose is an original paraphrase
and synthesis of the ideas in these sources; idea-level attribution is consolidated in
`source-index.md`.

# GxP AI Governance and Data Integrity

Use this reference when an AI system creates, transforms, analyzes, or influences GxP-relevant data, records, processes, decisions, or regulatory submissions. It adds a GxP quality-system overlay to the general AI governance method.

This is governance guidance, not legal advice, a validation package, a QMS replacement, or a declaration that a system is compliant. Confirm applicability and current requirements with the organization's QA/quality unit, regulatory, validation/assurance, privacy, security, and legal owners.

## The governing boundary

General AI governance answers:

- Who owns the AI system and its risks?
- What is the intended use and risk tier?
- What evidence gates development, release, monitoring, change, and retirement?
- How are model, data, supplier, security, fairness, and performance risks managed?

The GxP quality system additionally answers:

- Does the system create, modify, transmit, retain, or rely on regulated records or data?
- Which GxP domain and jurisdiction apply: GLP, GCP, GMP, GDP, or pharmacovigilance?
- What data-integrity attributes and electronic-record controls are required?
- What validation or computer-software-assurance evidence is proportionate to risk?
- How do deviations, CAPA, change control, training, periodic review, and quality-unit decisions operate?

Do not create a parallel AI bureaucracy. Connect AI governance decisions to the authoritative QMS, controlled records, validation repository, supplier-quality process, and incident/deviation system.

## GxP applicability screen

At intake, record:

- GxP domain(s), product/process, jurisdiction, and regulated business owner.
- Whether the AI system is used in discovery, clinical research, laboratory work, manufacturing, quality control, distribution, safety/pharmacovigilance, or regulatory submission support.
- Whether it creates, modifies, interprets, approves, transmits, or stores GxP-relevant records or data.
- Whether an AI output can affect patient safety, product quality, batch disposition, study integrity, subject safety, or a regulatory decision.
- System boundary: model, prompts, retrieval sources, data pipelines, human review, downstream systems, hosted services, and retained records.
- Named process owner, system owner, data owner, QA/quality-unit reviewer, validation/assurance lead, and AI-governance reviewer.
- Applicable internal procedures and external requirements, with a primary-source verification owner and review date.

If the system does not touch GxP-relevant processes or records, document why the GxP overlay is out of scope instead of silently assuming it.

## ALCOA+ data-integrity review

Use ALCOA+ as a control review, not as a slogan. For every GxP-relevant record or dataset, identify the control and evidence for each attribute:

| Attribute | Governance question |
|---|---|
| Attributable | Can the person, system, or agent responsible for each action be identified? |
| Legible | Can authorized users read and interpret the record throughout its retention period? |
| Contemporaneous | Is the record captured when the activity occurs, with reliable time handling? |
| Original | Is the source record preserved, or is the relationship to the original demonstrable? |
| Accurate | Are the record, transformation, calculation, and output correct and checked? |
| Complete | Are relevant data, metadata, failed runs, exceptions, changes, and audit-trail events retained? |
| Consistent | Are sequence, timestamps, formats, units, and meanings coherent across systems? |
| Enduring | Will the record remain intact and usable for the required retention period? |
| Available | Can authorized users retrieve the record, metadata, and audit history when needed? |

For AI, extend the review across the full chain: source data, labeling or curation, prompt/context, retrieval results, model/version, parameters and configuration, output, human review, downstream action, and retained evidence. A polished answer is not an acceptable substitute for the underlying record.

## Electronic records, signatures, and audit trails

When the system creates or relies on regulated electronic records, the gate must explicitly assess the applicable electronic-record and signature controls. Do not infer applicability from the presence of a model alone.

Confirm, as applicable:

- Validation or assurance evidence supports the intended use and critical functions.
- Access is limited to authorized individuals and service identities; privileges are reviewed.
- Records are protected against unauthorized alteration and remain accurately retrievable.
- Audit trails capture time-sequenced creation, modification, deletion, and relevant configuration or model changes.
- Audit trails are enabled, protected, periodically reviewed, and linked to investigations or deviations when needed.
- Electronic signatures identify the signer, bind the signature to the record, and cannot be repudiated casually.
- Time sources, time zones, clock changes, and synchronization are controlled and documented.
- The retention, archival, backup, restoration, export, and readability plan covers records plus relevant metadata and audit history.
- AI-generated or AI-assisted content is distinguishable from human review and approval where the process requires that distinction.

These are control prompts, not a legal conclusion about 21 CFR Part 11, EU GMP Annex 11, or another regime. Have the responsible quality and regulatory functions determine which requirements apply.

## Risk-based validation and assurance

Use a documented, risk-based validation or computer-software-assurance decision. The question is not “is AI validated?” in the abstract. The question is whether the system is fit for its intended GxP use and whether the evidence is proportionate to the risk.

The evidence plan should state:

1. **Intended use and context of use.** What the system may and may not do, who uses it, and what decisions it can influence.
2. **Criticality and risk.** Impact on patient safety, product quality, subject rights, study integrity, data integrity, and regulatory submissions.
3. **Requirements and acceptance criteria.** Functional, data-integrity, performance, security, human-oversight, and record-retention requirements agreed before testing.
4. **Traceability.** Links among requirements, risk controls, tests, results, deviations, approvals, and the released system/model/data versions.
5. **Test evidence.** Representative data, boundary and failure cases, relevant subgroups, abnormal inputs, uncertainty, fallback behavior, and human-review effectiveness.
6. **Release decision.** Named approver, QA/quality-unit involvement where required, unresolved deviations, residual risk, and operating restrictions.
7. **Change boundaries.** What changes require impact assessment, regression testing, revalidation or re-assurance, retraining review, or a new approval.
8. **Retirement and continuity.** Record retention, reproducibility, migration, rollback, decommissioning, and access to historical outputs and evidence.

For probabilistic, adaptive, or generative systems, add controls for nondeterminism, model/provider changes, prompt and retrieval changes, data drift, version pinning, output review, and the risk that a vendor changes behavior without the organization's approval.

## GxP overlay on AI lifecycle gates

| Gate | Minimum GxP questions and evidence |
|---|---|
| Intake / ideation | GxP applicability, domain, intended use, system boundary, criticality, owners, initial risk, and QMS route |
| Data | ALCOA+ review, provenance, authorized use, source and transformation history, quality, representativeness, retention, and access controls |
| Build / configure | Controlled versions of code, model, prompts, retrieval, data, configuration, and infrastructure; documented deviations and decisions |
| Evaluate / validate | Approved requirements, risk-based assurance plan, traceability, representative and edge-case testing, failure handling, human oversight, and evidence review |
| Release | QA/quality-unit decision as required, approved residual risk, complete records and audit trails, training, SOP/work-instruction updates, monitoring, rollback, and incident/deviation routes |
| Operate / monitor | Performance and data-integrity monitoring, audit-trail review, drift and vendor-change monitoring, periodic review, access review, incidents, deviations, CAPA, and escalation |
| Change | Impact assessment, change control, version and provenance update, regression/revalidation decision, approvals, and retained comparison evidence |
| Retire | Controlled decommissioning, retention and retrieval, archival integrity, migration or destruction evidence, supplier exit, and closure of open risks or CAPA |

## QMS and accountability interfaces

The governance record should identify which system owns each decision:

| Decision or event | Primary accountable function | AI-governance contribution |
|---|---|---|
| GxP applicability and process impact | Process owner with QA/quality unit | Ensure AI use case, boundaries, and risk are recorded |
| Validation/assurance strategy | Validation/assurance lead and system owner | Set AI-specific evidence and change questions |
| Data integrity controls | Data owner, system owner, and QA | Apply ALCOA+ across data and model lineage |
| Release for regulated use | Authorized quality and business approvers | Confirm governance gates and residual-risk record |
| Deviation, incident, or suspected integrity failure | QMS/quality process owner | Escalate AI-specific evidence and preserve affected artifacts |
| CAPA and change control | QMS/quality process owner | Ensure model, data, prompt, vendor, and configuration changes are in scope |
| Supplier/model due diligence | Procurement and supplier quality | Require provenance, audit rights, change notification, service continuity, and evidence access |
| Periodic review | System/process owner with QA | Recheck intended use, performance, integrity, access, drift, and changes |

## Supplier and hosted-model controls

For third-party models, hosted APIs, retrieval services, labeling vendors, or cloud systems, document:

- Model, provider, service, and dependency identity, version, location, and change-notification mechanism.
- Provider evidence about development data, evaluation, limitations, security, availability, incident handling, and business continuity.
- Contractual rights and practical ability to obtain records, audit trails, logs, evidence, and timely incident information.
- How provider updates, model substitutions, prompt changes, safety-filter changes, outages, and data-location changes trigger impact assessment.
- Whether the service permits retention, deletion, access restriction, and export of GxP-relevant prompts, inputs, outputs, metadata, and audit history.
- Exit, rollback, migration, and record-retention plans that do not depend on indefinite vendor availability.

## Verification checklist

- [ ] GxP domain, jurisdiction, intended use, and system boundary are documented.
- [ ] Named process, system, data, quality, validation, and AI-governance owners exist.
- [ ] ALCOA+ review covers source data through retained AI output and human action.
- [ ] Electronic-record, signature, audit-trail, time, access, and retention controls are assessed where applicable.
- [ ] Risk-based validation/assurance strategy and acceptance criteria were approved before testing.
- [ ] Model, data, prompt, retrieval, configuration, and provider versions are traceable.
- [ ] Testing covers representative use, failure modes, uncertainty, generalizability, and human oversight.
- [ ] Release, monitoring, change, deviation/CAPA, periodic-review, rollback, and retirement routes are connected to the QMS.
- [ ] Supplier changes and evidence-access limitations are controlled.
- [ ] Current primary sources and internal procedures were re-verified by the responsible functions.

## Primary sources and implementation references

Accessed 2026-08-20 UTC. Re-verify current versions and applicability at use time.

- FDA, [Data Integrity and Compliance With Drug CGMP](https://www.fda.gov/media/119267/download)
- FDA, [Part 11, Electronic Records; Electronic Signatures — Scope and Application](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/part-11-electronic-records-electronic-signatures-scope-and-application)
- eCFR, [21 CFR Part 11](https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11)
- MHRA, [Guidance on GxP Data Integrity](https://www.gov.uk/government/publications/guidance-on-gxp-data-integrity)
- PIC/S, [PI 041-1 Guidance on Data Integrity](https://picscheme.org/docview/4234)
- WHO, [TRS 1033 Annex 4: Guideline on Data Integrity](https://www.who.int/docs/default-source/medicines/norms-and-standards/guidelines/inspections/trs1033-annex4-guideline-on-data-integrity.pdf)
- EMA, [Guideline on Computerised Systems and Electronic Data in Clinical Trials](https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-computerised-systems-and-electronic-data-clinical-trials_en.pdf)
- EMA, [Reflection Paper on AI in the Medicinal Product Lifecycle](https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf)
- FDA, [Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological)
- ICH, [Q9(R1) Quality Risk Management](https://database.ich.org/sites/default/files/ICH_Q9(R1)_Guideline_Step4_2022_1219.pdf)
- ISPE, [GAMP 5 Guide, 2nd Edition](https://ispe.org/publications/guidance-documents/gamp-5-guide-2nd-edition)
- European Commission, [EudraLex Volume 4, Annex 11: Computerised Systems](https://health.ec.europa.eu/system/files/2016-11/annex11_01-2011_en_0.pdf)

### Synthesized from

This reference synthesizes the official regulator, inspectorate, standards-body, and industry sources listed above. The durable research basis is recorded in `../research/gxp-ai-governance-brief.md` and `../research/gxp-ai-governance-log.md`. The material is an original governance synthesis, not legal advice, a validation protocol, or a certification of compliance.

# LLM and Agent Security

This reference teaches how to secure systems built around large language models (LLMs) and the
autonomous agents that act on their output. Its central message is that LLM security is not a
single bug to patch but a set of trust-boundary decisions layered across the whole system: what you
let a model read, what you let it do, what you trust its output to be, and what it can reach when an
input turns adversarial. The same discipline extends to agents, where autonomy multiplies every
risk because a model that merely talks can be re-pointed to call tools, move files, and change real
state. This reference synthesizes (never reproduces) ideas from *The Developer's Playbook for LLM
Security*, *Beyond the Algorithm*, and *AI Governance*, and it marks "current research" where the
fast-moving OWASP and agentic guidance de-stales the books. Read it with
`foundations-and-principles.md` (which names safety and human oversight as governing principles),
`privacy-and-data-governance.md` (the data-protection half that security protects), and
`ai-lifecycle-governance.md` (where security gates sit in the model lifecycle). Use the
[model-risk-assessment.md](../templates/model-risk-assessment.md) and [model-card.md](../templates/model-card.md) templates to record the threat model and controls
this reference describes. It is not legal advice, and specific controls should be re-verified
against current OWASP and government guidance at use time.

## Why LLM Security Is Different

A conventional application separates code from data: the attacker can try to inject into the code
path, but the two are distinguishable. An LLM blurs that line because its instructions and its data
are the same natural-language tokens, processed together in one pass. *The Developer's Playbook for
LLM Security* makes this the heart of the threat model: an LLM cannot always tell
developer-authored guidance from user-supplied or retrieved content, which is why so many of its
risks are really about confusion between "instructions" and "data." *Beyond the Algorithm* frames
the same idea as a trust-boundary problem — a line that separates components worth trusting from
those treated as untrusted — and stresses that sanitizing and validating inputs before they reach
the model is the protective habit that keeps a boundary meaningful.

The practical consequence is that most LLM defenses are not about making the model immune; they are
about constraining what a compromised model can reach and what its outputs can do. Security teams
should assume injection may succeed and engineer the surrounding system so that a failure is
contained. This is the pessimistic trust boundary: treat every output the model produces, and every
action it initiates, as potentially harmful until validated.

## Trust Boundaries in an LLM Application

A trust boundary is the point where data or control moves from one trust level to another — user
input entering the model, model output returning to the user, data leaving for an external API, or
an agent reaching a database. *The Developer's Playbook* walks through each boundary an LLM
application crosses and what to secure there:

| Boundary | What crosses it | What to secure |
|---|---|---|
| User interaction | Inputs and outputs between person and model | Validate and sanitize inputs; filter and monitor outputs for toxic, inaccurate, or sensitive content |
| In-the-wild training data | Public/internet-sourced training and retrieval content | Treat as untrusted; watch for bias, toxicity, poisoning, and indirect injection |
| Internal data | Proprietary fine-tuning data, corporate documents | Protect confidentiality; beware leakage of sensitive or personally identifiable content |
| Live external sources | Web pages, third-party APIs, tool outputs read at runtime | Treat as untrusted; validate and scope what the model may ingest |
| Internal services | Databases, internal APIs, backend systems the model or agent can reach | Apply least privilege; prevent unauthorized access and lateral movement |

Two hosting choices shift the risk profile. Calling a third-party model API is convenient and
cheap, but the data crosses a boundary into an external system, raising exposure risk. Hosting a
model privately keeps data inside your network and tightens control over the boundary, but shifts
the burden to maintenance, patching, and verifying that an open-source model is genuine and
uncompromised. Neither choice is safe by default; each just moves where the boundary is drawn.

## The Six-Factor Exposure Ladder

Use this ladder from *AI Governance* to identify which security exposures a system has before
choosing controls. It is a dependency ladder, not a severity ranking or a numeric risk score. A
failure at the environment layer, such as exposed credentials, can be more damaging than a
higher-level planning failure. Assess severity and residual risk separately in the risk register.

| Factor | The factor applies when the system... | Governance question |
|---|---|---|
| **Environment** | runs with weak logging, secrets, isolation, access control, or vendor assurance | Can an incident be reconstructed, contained, and attributed? |
| **Model** | depends on an unvetted, externally controlled, fine-tuned, or provenance-uncertain model | What evidence supports model origin, version, behavior, and safety after updates? |
| **Input** | accepts untrusted user, document, web, or tool content | Which content is data, which is instruction, and how is indirect injection handled? |
| **Data access** | retrieves personal, confidential, regulated, or otherwise sensitive data at runtime | Does the model receive only the minimum authorized data for this request? |
| **Ability to make changes** | can write records, send messages, trigger workflows, change code, or move money | What validates parameters and what approval or reversal exists before execution? |
| **Agency** | chooses goals, tools, order, retries, or sub-agents across multiple steps | Can the system's plan expand its authority or create a new trust path? |

Every LLM application has at least an environment and model exposure; most have input exposure.
Data access, change capability, and agency increase the blast radius and the required evidence, but
they do not make the lower layers optional. A simple chatbot on an insecure environment can be
more dangerous than a carefully isolated agent. Record which factors apply, the controls at each
layer, and the evidence that the controls operate at the actual boundary.

## Prompt Injection

Prompt injection steers an LLM or agent using adversarial input — overriding developer
guidance, exposing protected information, circumventing policy, or triggering actions the operator
did not authorize. It ranks at or near the top of OWASP's risk lists and is commonly called the
field's most frequent AI exploit. *The Developer's Playbook* distinguishes the two delivery classes:

- **Direct injection** ("jailbreaking"): the attacker controls the input channel, crafting a prompt
  to override the system prompt or reveal hidden instructions. *Beyond the Algorithm* notes that
  this can let an attacker target backend systems the LLM can reach.
- **Indirect injection**: malicious instructions are hidden inside content the model ingests as part of
  its task — a retrieved document, a web page, a tool's output. This is the more dangerous variant
  for agents, because the untrusted material often arrives through a channel the system trusts, and
  it can be invisible to humans when only the model processes the text.

Prompt injection is hard because there is no solid, universal patch the way there is for SQL
injection. The books characterize mitigation as defense-in-depth, more like anti-phishing than like
parameterized queries. Layered controls include:

- **Rate limiting** (by IP, user, or session) to slow automated probing and concentrated attacks.
- **Input filtering** as a first line, though simple regex blocklists are brittle and degrade
  capability (blocking words like "bomb" also blocks legitimate discussion).
- **Adding prompt structure** — tagging data separately from instructions so the model treats
  injected text as data rather than as a high-priority command.
- **A special-purpose filtering LLM** trained to flag injection attempts, treated as one layer, not
  a silver bullet.
- **Adversarial training**, seeding malicious examples into the training set so the model learns to
  recognize them, with continuous updating as attacks evolve.
- **Pessimistic trust boundary and least privilege**, treating every output as untrusted, filtering
  it, restricting backend access, and requiring human approval for destructive actions.

## Sensitive Data Exposure

LLM applications can leak data through several routes: the model regurgitating memorized training
material, drawing on confidential internal data it was given access to, or being steered by
injection into revealing information. *The Developer's Playbook* connects exposure to the "know too
much" problem — an overfitted or too-broadly-accessed model can emit private information even when
the underlying data is secured. Controls include treating externally visible prompts and outputs as
a disclosure surface, restricting which internal corpora a model may draw on, sanitizing outputs,
monitoring for data exfiltration, and keeping sensitive and non-sensitive functions on separate
boundaries. For agentic systems this extends to what a tool can return and what an agent may send
out, so that a compromised agent cannot quietly exfiltrate data through a channel the operator
doesn't watch.

## Hallucination, Overreliance, and Misinformation

Hallucination is the model confidently generating plausible-sounding content that is not true or
grounded in its inputs. Alone it is an accuracy problem, but it becomes a security and governance
problem through **overreliance** — when people or downstream systems trust erroneous output and act
on it. *Beyond the Algorithm* and the playbook both treat overreliance as a top-ten-class risk
because fluent, confident output that drives a decision turns a wrong answer into a wrong outcome.
Mitigations pair technical controls (grounding via retrieval, validation of outputs) with
organizational ones (designing the human–AI interface so people know when to double-check, and
building automatic checkpoints before output becomes action). "current research": the OWASP GenAI
LLM Top 10 2026 has moved misinformation materially up the rankings because incident evidence
showed how often bad output produced real-world harm in systems that acted without checking.

## Excessive Agency and Agentic Risk

Excessive agency is granting an LLM-based system more capability, permission, or autonomy than it
safely should have. *The Developer's Playbook* treats it as a structural design vulnerability rather
than an output bug, and breaks it into three forms:

- **Excessive permissions**: the model or agent holds broader access than needed (e.g., write and
  delete rights where read-only would do), which a confused-deputy attack can exploit to modify
  records it should only view.
- **Excessive autonomy**: the system takes consequential actions without review (e.g., auto-trading
  a portfolio), so a single injected instruction can drive real financial or operational damage.
- **Excessive functionality**: features that sound attractive but expand the attack surface beyond
  what the application requires.

The mitigations center on **least agency**, the agentic counterpart to least privilege: scope each
agent's tools and permissions narrowly, require human sign-off for high-impact actions, keep
agents isolated so a takeover cannot spread across the fleet, and log tool use and decisions so
abuse leaves a trail. "current research": the OWASP GenAI LLM Top 10 2026 ranks excessive agency
third, and in mid-2026 the Five Eyes nations' cyber agencies (CISA, NSA, and counterparts) published
their first joint guidance on agentic AI, urging measured rollout backed by identity, access, and
monitoring safeguards.

## Agentic Control Plane

Agent security needs controls outside the model's reasoning. The agent may propose a legitimate
plan while an injected document, poisoned memory, or compromised tool steers the actual call. The
following control plane makes authority explicit and independently enforceable.

### Registry and identity

Maintain an approved registry of agents, tools, individual operations, data scopes, destinations,
versions, owners, and review dates. Registering a connector is not enough when it exposes both
read and destructive operations. Give every agent a distinct identity tied to one business
purpose, use short-lived and scoped credentials, and record the human or service principal on
whose behalf each call occurs. For multi-agent systems, define which agents may communicate and
what one agent may ask another to do.

### External authorization and action tiers

Evaluate authorization immediately before each tool call, outside natural-language reasoning. A
policy decision should consider agent identity, user or calling principal, purpose, tool and
operation, resource or tenant, data classification, parameters, destination, and current
approval. Default to deny when no rule explicitly permits the call. The policy engine makes the
decision; a separate enforcement point must prevent the call when the result is deny or approval
is missing.

Classify operations by consequence rather than by the agent's description of its intent:

- **Low-impact, reversible reads:** allow within least-privilege scope with logging and rate limits.
- **Medium-impact or externally visible actions:** require a targeted confirmation or human review.
- **High-impact, sensitive, or irreversible actions:** require explicit approval every time and a
  tested reversal, escalation, or containment path.

Do not interrupt a reviewer for every low-risk retrieval. Excessive approvals create fatigue and
turn the control into rubber-stamping. Conversely, do not assume a read is harmless: a web request,
URL, tool parameter, or downstream log can exfiltrate data without changing a record.

### Isolation, memory, and containment

Separate untrusted external research from sensitive internal analysis where possible. Use isolated
sessions, sub-agents, credentials, network egress, and filesystem scopes so a component that reads
untrusted content cannot also reach sensitive systems. Run agents in a sandbox with explicit network
allowlists, limited filesystem access, no unnecessary process spawning, and credentials outside the
sandbox boundary.

Treat memory as an attack surface. Use short-lived working memory, bounded task or session logs,
and reviewed or purpose-limited durable memory. Require explicit approval for durable memory writes,
protect memory by tenant and user, and provide a way to inspect, correct, and purge it. Test whether
poisoned memory survives a session, reaches another user, or changes a later plan.

Every deployed agent needs a tested kill switch that suspends active work, revokes credentials, and
isolates connected systems. Pair manual intervention with automated triggers for anomalous tool
use, data volume, destination, cost, or goal drift. Prefer graceful degradation, such as fewer
tools or more approvals, when the system can remain useful while under investigation.

### Browser and tool-ecosystem boundaries

An agentic browser can combine authenticated sensitive data, untrusted web content, and external
communication in one session. Break that combination where possible: use an isolated profile,
logged-out mode, no password-manager access, no saved credentials or corporate session cookies, and
real-time visibility into navigation and actions. For tool ecosystems such as MCP, pin versions,
review tool descriptions and updates, disable or hold dynamic tool discovery for review, scan both
repositories and developer workstations, and put an enforcement gateway between agents and tools.
Treat tool descriptions and tool outputs as untrusted input, not as policy.

For a fillable record of these decisions, use
[agentic-governance-review.md](../templates/agentic-governance-review.md).

## Denial of Service and Unbounded Consumption

LLM systems are vulnerable to attacks that exhaust their resources. A denial-of-service (DoS)
attack floods the model with requests to degrade performance or make it unavailable. The playbook
highlights a financially distinct variant, denial-of-wallet (DoW): because cloud models bill
per-token or per-operation, an adversary can drive an unsustainable cost by issuing excessive or
pathologically expensive requests, harming the operator's economics even without a service outage.
A related class is model cloning or theft, where an attacker floods the model with queries, records
the answers, and uses them to train a competing model, draining value from the operator's
intellectual property. Many of the same defenses apply because all three depend on repeated query
volume: rate limiting, quotas, request cost caps, anomaly monitoring, and access controls that
distinguish trusted users from automated abuse. "current research": the OWASP GenAI LLM Top 10 2026
introduces unbounded consumption, noting that per-token costs mean request-per-second limits alone
do not protect against expensive reasoning chains or repeated tool calls.

## Supply Chain Security

The LLM supply chain reaches well beyond conventional software dependencies. It includes the
foundation model itself, the datasets used to train or fine-tune it, the retrieval stores and
external services it calls, and any plugins or tools an agent invokes. *The Developer's Playbook*
warns that open-source model distribution is immature: in 2023, leaked API tokens and account
takeovers on model registries raised the real possibility that a trusted model could be swapped for
a malicious one, and a tainted model file can act as a back door. Training-data poisoning can
inject false information or bias for very little cost, and even accidental exposure to unsafe
training data can cause a deployed model to generate harmful content. Insecure plugins broaden the
surface further, letting a third-party component become a vector for malicious code or data
collection.

The controls are inventory and provenance. Teams should track the origin and version of every
model, dataset, and plugin they rely on, mirroring the software-bill-of-materials discipline.
Artifacts include the SBOM (software bill of materials), the model card (documenting intended use,
data, performance, and limitations), and emerging ML-BOMs for machine-learning components. *Beyond
the Algorithm* likewise calls for secure model development and deployment practices that treat
models and datasets as first-class supply-chain components. "current research": the OWASP GenAI LLM
Top 10 2026 broadens supply chain risk to include name-squatting in public model registries and
configuration-file attacks that alter model behavior without touching the model itself.

## Red-Teaming LLM and Agentic Systems

A red team carries out a structured, adversarial evaluation that probes an AI system for failures —
prompt injection, jailbreaks, data leaks, unsafe outputs, bias, and misuse of tools — before a real
attacker exploits them. *Beyond the Algorithm* describes it as crafting prompts that trigger harmful
or revealing behavior and notes it is creative and resource-intensive. The playbook adds that a red
team simulates realistic attacks and drives improvements, and it is complementary to traditional
penetration testing: a pen test is a point-in-time assessment of exploitable weaknesses, while red
teaming is an ongoing, creative effort to probe AI-specific behaviors that automated scans miss. A
red team exercises areas like hallucination triggers, bias, excessive agency, and injection with an
external, adversarial perspective that internal teams focused on functionality often lack.

Practitioners warn that red-teaming alone is not a guarantee: it mostly surfaces known classes of
weakness, so passing a red-team round can breed overconfidence about novel attacks, especially as
models and agents evolve. Strong programs pair red-teaming with layered injection defenses,
least-privilege agency design, active monitoring, and continuous re-evaluation, rather than treating
one session as proof of safety. "current research": government bodies have institutionalized this
practice — the U.K. AI Security Institute and its U.S. counterpart, the AI Safety Institute (now
NIST's Center for AI Standards and Innovation, or CAISI) both run technical adversarial evaluations,
and OWASP ships a red-teaming taxonomy alongside its Top 10.

## The Current OWASP Top 10 (verified)

OWASP's Top 10 for LLM Applications is the default risk checklist for the field, but its ranking is
volatile between editions — the research note and the books both stress that no published
ranking should be treated as a frozen checklist. This section is "current research" verified against the OWASP
GenAI Security Project's release, which moved from expert-vote-only ranking to blending expert
judgment with thousands of documented incidents. The ordering below follows the OWASP GenAI LLM Top 10
2026 and should be re-checked before it is asserted as current; earlier editions differed, and later
ones will again.

1. Prompt Injection
2. Sensitive Information Disclosure
3. Excessive Agency
4. Supply Chain
5. Data and Model Poisoning
6. Unbounded Consumption
7. Misinformation
8. Hidden Context Exposure (formerly system prompt leakage)
9. Vector and Embedding Weaknesses
10. Improper Output Handling

The governance takeaway is not to memorize the order but to see the categories it normalizes and to
map them onto the controls in this reference: trust boundaries and least privilege answer the top
three; inventory and provenance answer supply chain and poisoning; monitoring, quotas, and cost caps
answer unbounded consumption; output validation and human-in-the-loop answer misinformation and
improper output handling.

## Where to Go Next

- **`foundations-and-principles.md`** — safety and human oversight as the principles security
  operationalizes.
- **`privacy-and-data-governance.md`** — the data-protection controls (exposure, retention,
  PETs, purpose-aware egress, and agent memory) that security defends.
- **`ai-lifecycle-governance.md`** — where security reviews and red-teaming sit in the stage gates.
- **`risk-management-and-frameworks.md`** — NIST AI RMF, including its Generative AI Profile, into which
  the OWASP categories map.
- **`regulatory-landscape.md`** — current law that turns some of these risks into legal duties.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *The Developer's Playbook for LLM Security*,
*Beyond the Algorithm*, and *AI Governance*, and it is de-staled against the current state
by the mission research note on LLM/agent security (research-llm-agent-security.md) and a
verification of the OWASP GenAI LLM Top 10 2026 against the OWASP GenAI Security Project's published
release. All prose is an original paraphrase and synthesis of the ideas in these sources;
idea-level attribution is consolidated in `source-index.md`. Because the LLM/agent security
landscape changes quickly, the OWASP ranking and agentic guidance should be re-verified against
primary sources at use time. The security recommendations here are educational context, not legal
advice.

# Privacy and Data Governance

This reference teaches how to govern the data that feeds and runs AI systems, and how to protect
the individuals whose information flows through those systems. Its central message is that privacy
and data governance are two halves of the same discipline: data governance supplies the ownership,
lineage, and quality that make data trustworthy and auditable, while privacy supplies the limits —
consent, minimization, retention, and protective techniques — that keep that data from causing
harm. It synthesizes (never reproduces) ideas from *Designing Data Governance from the Ground Up*,
*Data Governance Handbook*, *Platform and Model Design for Responsible AI*, and *AI Governance*, and
it marks
"current research" where the 2023–2025 books are de-staled by recent regulation. Read it with
`foundations-and-principles.md` (which names privacy as a governing principle),
`ai-lifecycle-governance.md` (where data and model lineage surface at stage gates), and
`llm-and-agent-security.md` (the security half of data protection). Use the [model-card.md](../templates/model-card.md) and
[model-risk-assessment.md](../templates/model-risk-assessment.md) templates to record the data practices this reference describes. It is
not legal advice; data-protection duties should be confirmed against current law at use time.

## Data Governance Is the Foundation

*Data Governance Handbook* describes data governance as the deliberate alignment of people,
processes, and technology so that an organization delivers the correct data, to the right place,
under appropriate safeguards, and thus enables effective and safe use. The book treats governance as
an ongoing activity carried through several cooperating capabilities rather than one standing rule:
managing metadata, tracing lineage, assuring data quality, shaping data architecture, mastering core
reference data, and running data operations. Each capability
answers a concrete question — where data came from, how trustworthy it is, how it is classified,
who may use it — and together they turn a collection of scattered files into a governed, queryable
asset.

A useful mental model from the same book is to treat data as both asset and liability. Data is an
asset when it creates measurable value — a curated, well-owned dataset that many teams can rely on,
or a model that drives revenue. Data becomes a liability when it introduces risk, for example
records that were never catalogued, were never classified and secured, or have leaked. The goal of a governance
program is to grow the asset side and shrink the liability side, which the book calls creating
"data equity." This framing matters for AI because model quality and privacy risk both trace back
to how well the underlying data is owned and controlled; a model is only as trustworthy as the data
that trained it, and only as safe as the controls around the data it uses.

For AI specifically, the discipline splits into two regimes that this reference keeps distinct:

| Regime | What it governs | Typical questions |
|---|---|---|
| Training-data governance | The datasets used to fit and validate a model | Who owns this corpus? Where did it come from? Is its use authorized? |
| Operational-data governance | The data a deployed model ingests and produces | What PII flows in and out? Who may see it? How long is it kept? |

The two share the same toolkit — ownership, lineage, quality, and retention — but pose different
risks, so a governance program should treat them separately rather than folding them into one
bucket.

## Ownership and Accountability

Data governance begins with answering "who owns this data?" Both *Data Governance Handbook* and
*Designing Data Governance from the Ground Up* make ownership the first building block, because
accountability is impossible without a named owner. *Designing Data Governance from the Ground Up*
argues that data stewards are the human heart of governance: they own the strategic and tactical
decisions about data within their domains, act as trusted advisors on its meaning and limits, and
provide the context that data engineers and scientists need to use it correctly. Stewards can come
from business or technical roles; a sales director who knows the customer dataset is often a better
owner than an engineer who merely processes it, because ownership is about understanding and being
answerable, not just about touching the data.

The book distinguishes two stewardship flavors that should both be populated:

- **Business stewards** are accountable for the data used across business processes and workflows —
  the classification scheme, the business definitions, the quality expectations — and write the
  documentation that makes data usable across departments.
- **Technical stewards** own data for systems and pipelines — the metadata parameters, the marts and
  warehouses, the models and algorithms, the access limits that decide who can see and change data.

The same split appears in *Data Governance Handbook*, which assigns data-domain executives and data
stewards per functional area and pairs them with technical stewards who carry out the mechanics.
Both books stress a common failure: stewards may be experts in their data yet lack the authority to
enforce decisions. A governance program must give owners real decision rights, visibility, and
recognition, or ownership becomes an empty title. In AI terms, every training dataset and every
production data feed should have a named accountable owner who can answer for its provenance, its
authorized use, and its quality.

## Data Lineage and Provenance

Ownership is only useful if you can trace what a piece of data actually is and where it has been.
*Data Governance Handbook* treats data lineage as one of the core capabilities, and *Designing Data
Governance from the Ground Up* defines lineage as confirming the origin of data and the path it took
to reach its current state. Lineage underpins transparency and root-cause analysis: when a model
produces a wrong or harmful output, lineage is what lets you walk the trail backward from the result
to the source dataset and the transformation that introduced the problem. *Platform and Model Design
for Responsible AI* extends the idea to the model itself, arguing that data and model lineage should
be captured across the pipeline so that any deployment can be reproduced from its inputs.

For AI governance, lineage should cover at least three layers:

- **Data lineage.** Where each training or operational dataset originated, how it was collected or
  acquired, what transformations it passed through, and which version is in use.
- **Feature lineage.** Which engineered features were derived from which source columns, and with
  what definitions — important because a seemingly innocent feature can encode sensitive or biased
  information.
- **Model lineage.** Which model version was trained on which data snapshot, with which
  hyperparameters, feeding which deployed artifact.

Lineage is what converts "we used some data" into "we can reconstruct exactly what the system
ingested and why," which is essential for audits, incident response, and compliance inquiries. In
practice it is delivered through a data catalog and metadata management layer that records business
and technical metadata for every critical data element (the *Data Governance Handbook* lists
descriptions, schemas, business definitions, and data classification among the required catalog
fields).

## Data Quality

Data that is not trustworthy cannot be governed safely, so quality is a prerequisite for both
privacy and model reliability. *Data Governance Handbook* defines data quality simply as ensuring
data is "fit for use": the data meets the standards of accuracy, completeness, and timeliness needed
for the analytical or operational purpose at hand. The book recommends establishing data quality
rules for each critical data element, enabling monitoring on those elements, publishing transparent
quality dashboards, and creating remediation plans for material issues. *Designing Data Governance
from the Ground Up* ties quality to the cost side of the ledger, noting that poor-quality data costs
organizations heavily and that stewards are responsible for defining and conducting quality
assessments against agreed business metrics.

Quality work in AI is doubly important because defects compound: a small error in a training
dataset can become a systematic bias in a model, and a stale production feed can silently degrade
predictions. Governance should therefore treat data quality not as a one-time clean-up but as an
ongoing, monitored property with named owners, explicit rules, and visible measurement — the same
discipline that `ai-lifecycle-governance.md` applies to the model itself. Where quality gaps cannot
be fixed, they should at least be disclosed so that users and reviewers know the limits of what the
data can support.

## Consent and Lawful Basis

Privacy law, above all the GDPR, governs when personal data may be collected and used at all.
Because the primary AI books predate much of the current enforcement picture, this section marks
"current research." Under the GDPR (Regulation (EU) 2016/679), every use of personal data must rest
on a lawful basis, and "consent" is only one of several; others include contractual necessity,
legal obligation, and legitimate interest. **Consent** is a specific, informed, freely-given, and
revocable indication of agreement — not a passive default and not buried in terms the individual
cannot meaningfully decline. The European Data Protection Board's guidance on AI and personal data
(such as Opinion 28/2024) clarifies that the GDPR continues to govern AI models and that the
data-protection duties attached to lawful basis, transparency, and accountability persist in AI
processing. The EU AI Act does not displace the GDPR: an AI system that handles personal data must
comply with data protection law in addition to AI-specific rules.

For governance practice, this means:

- **Document a lawful basis per use.** Do not assume that scraping or accumulating data implies
  permission to train on it. Record, per training and operational dataset, the legal ground relied
  on and the reasoning behind it.
- **Treat consent as a living consent.** Consent that is opt-in, specific, and withdrawable must be
  revocable; if individuals later withdraw, the system must honor that withdrawal, which has
  implications for retention and retraining.
- **Reconcile AI Act and GDPR duties.** They are complementary, not interchangeable; a compliance
  mapping must satisfy both where personal data is involved.

Consent is one piece of a broader transparency duty: individuals should be told what data is
processed, for what purpose, and how the system reaches decisions that affect them (see
`transparency-and-explainability.md`).

## Minimization and Retention

Two data-protection principles constrain how much data an AI system may hold and for how long.
**Minimization** is the principle that you collect and keep only the personal data necessary for the
declared purpose — no more. For AI this is in tension with the common instinct to hoard every
available record, but it is directly protective: the less sensitive data you hold, the less there is
to leak, misattribute, or misuse. **Retention** is the related limit on time: personal data should
not be kept longer than needed for the purpose, after which it should be deleted or de-identified.
The EU AI Act's "high-quality datasets" requirement (which pushes providers to design training data
to minimize discriminatory outcomes) reinforces the spirit of minimization by tying the *content* of
data to the system's actual purpose rather than to convenience.

Practical controls a governance program can adopt:

- **Purpose-bound acquisition.** Collect data for a stated purpose and refuse scope creep — do not
  quietly repurpose a dataset collected for one use into training a model for an unrelated one.
- **Retention schedules.** Define, per dataset, how long it is retained, who may extend that period
  and why, and when deletion or anonymization is triggered. Record the schedule in the catalog.
- **Deletion and de-identification paths.** Ensure that when a retention period ends, or an
  individual exercises a right to erasure, the data is actually removed from training corpora,
  feature stores, and backups, not merely hidden.
- **Reasonable, not minimal-to-zero.** Minimization does not mean stripping data until a model
  cannot function; it means the amount of data is justified by and proportionate to the purpose.

## Four GenAI Privacy Surfaces

Traditional privacy controls still apply, but GenAI adds failure surfaces that are easy to miss if
the review only asks where a database row is stored. Use these four pillars as a coverage check;
they are a conceptual model from *AI Governance*, not a legal classification.

| Pillar | GenAI-specific question | Evidence and control direction |
|---|---|---|
| **Collection and purpose** | What enters training, fine-tuning, retrieval, prompts, feedback, or telemetry, and is each use necessary and authorized? | Purpose and data map, lawful-basis record where applicable, minimization/redaction, vendor data-use and retention terms |
| **Storage and memorization** | Where can data persist or be inferred: logs, caches, embeddings, indexes, memory, model weights, or backups? | Source-linked inventory, retention and deletion tests, access controls, leakage and membership-inference testing where justified |
| **Output integrity** | Can the model invent, infer, or disclose a personal fact, and could a person or downstream system act on it? | Output validation, disclosure, contestation, human review that can change the result, and tests for hallucinated or sensitive personal claims |
| **User rights and governance** | Can people understand, access, correct, object to, or erase relevant data and derived records? | Rights workflow, data-subject request package, memory view and deletion path, vendor flow-down, and an explicit record of technical limits |

The output-integrity pillar matters because privacy harm can be created by what a system says, not
only by unauthorized access. A fabricated medical, financial, employment, or criminal fact about a
person is not made safe by a human clicking “review” if the reviewer cannot verify or reject it.
Treat consequential output as a decision surface: provide source or uncertainty cues, a meaningful
contest path, and an accountable reviewer with authority to change the outcome.

## Agentic Privacy: Purpose-Aware Egress and Memory

Agentic systems intensify privacy risk because planning, observation, tool use, and persistent
memory let the system expand what it reads and where it sends data during a task. The declared goal
is not a sufficient boundary: an agent may infer a new purpose, call an unvetted tool, or pass
context to another agent without the user seeing the intermediate step.

### Purpose-aware egress gate

Put an enforcement point before every outbound tool call, external API request, browser fetch, or
agent-to-agent transfer. The gate should evaluate three attributes together:

1. **Purpose:** Is the proposed use compatible with the purpose declared for this task and the
   purpose for which the data was collected?
2. **Necessity:** Are the exact fields proposed for transfer required, or can the payload be
   redacted, generalized, tokenized, or replaced with a reference?
3. **Destination:** Is this tool, recipient, region, tenant, and subprocessor permitted?

Make the default fail closed. If no policy explicitly permits the combination of agent, principal,
purpose, data class, operation, and destination, block the call or route it to review. Record the
decision and the minimized payload, not only the agent's proposed payload. A policy engine can
return the decision, but a separate enforcement point must prevent a blocked call from executing.

Use explicit outcomes:

- **Allow** when purpose, necessity, and destination are all satisfied.
- **Allow with redaction** when the purpose is valid but some fields are unnecessary.
- **Block** when the use or destination is prohibited.
- **Request human approval** when the change is ambiguous or materially risky.
- **Request just-in-time consent** when a new user-facing purpose needs a fresh choice.

Keep security controls in the loop. Treat incoming documents, web pages, tool descriptions, and
tool results as untrusted; bind model output to typed parameters; validate those parameters before
execution; and isolate external fetches. Otherwise prompt injection can turn a privacy policy into
an instruction the model merely describes rather than a boundary the system enforces.

### Tiered memory and erasure

Treat agent memory as a data store and as a persistence channel for malicious or incorrect
instructions. Define separate tiers instead of one indefinite conversation history:

| Memory tier | Default governance posture |
|---|---|
| **Ephemeral working memory** | Reset after the task or retain only for a short, justified TTL. |
| **Bounded session or task log** | Retain for a documented operational, security, or accountability purpose, with restricted access and expiry. |
| **Long-term user or organizational memory** | Require a stated purpose, appropriate authorization or consent where applicable, reviewable writes, tenant isolation, and an inspect/correct/delete path. |
| **Derived model, embedding, or index state** | Maintain source lineage and a tested process for exclusion, rebuild, deletion, or retraining when the source must no longer be used. |

Memory controls should cover both privacy and integrity. Test that poisoned or false memories do
not cross users or sessions, that durable writes cannot silently expand purpose, and that clearing a
source also removes its derived embeddings, caches, replicas, and downstream copies where required.
Do not claim that deleting a source row removes its influence from model weights unless that has
been demonstrated; record the residual technical limitation and prevent future reuse.

For rights requests, maintain an inventory of prompts, outputs, logs, caches, embeddings, indexes,
memory stores, model-update inputs, and downstream tools or agents that received the data. A
memory dashboard or administrative search-and-purge tool is more reliable than a promise that a
user can email support later. Retain only the audit evidence that is necessary and justified, and
avoid storing unrestricted hidden reasoning when structured decision, source, policy, approval,
and outcome records are sufficient.

For a fillable review of posture, egress, memory, rights, and evidence, use
[agentic-governance-review.md](../templates/agentic-governance-review.md).

## Privacy-Enhancing Techniques

Privacy-enhancing techniques (PETs) are the technical tools that let organizations extract value
from data while reducing the exposure of the underlying individuals. *Platform and Model Design for
Responsible AI* treats these as a set of "privacy by design" defaults built into the model pipeline
rather than bolt-on afterthoughts. The book's model-level view distinguishes privacy across several
surfaces: the training data (an adversary should not be able to reverse-engineer it from the model),
the model inputs (protected from view during training), the model outputs (visible only to the
intended recipient), and model storage and access (restricted to authorized staff). Protecting each
surface requires different controls.

The core PETs the book and current practice converge on include:

| Technique | What it does | Typical use |
|---|---|---|
| Differential privacy (DP) | Adds calibrated noise so that including or excluding any one record barely changes outputs, bounding what an adversary can infer | Training and query-time privacy; measured by epsilon (ε) |
| K-anonymity | Groups records so no row is alone: each is one of at least k−1 peers on identifying attributes | Publishing or sharing structured data |
| Anonymization / pseudonymization | Removes or replaces identifiers; pseudonymization is reversible with a key, anonymization is not | Pre-processing before sharing or analysis |
| Encryption / access control | Protects data and model weights in storage and transit; restricts who can reach them | Model storage and access privacy |
| Federated learning | Trains on distributed data without centralizing raw records, exchanging only model updates | Keeping data local while still learning |

A recurring caution in the book is that overly flexible models can *memorize* training data, so an
overfitted model can leak private information even when the source data is secured. This is why
governance pairs PETs with monitoring and with careful feature selection: the protection has to hold
at the model output, not just at the dataset boundary. The book also notes that DP's epsilon metric
quantifies how much privacy loss a query or training run incurs, so practitioners can set a budget
for how much noise a system is willing to pay in exchange for how much utility.

## Privacy by Design Across the Model Pipeline

The lesson across all three sources is that privacy cannot be a last-minute audit step; it has to be
built into the design and every stage of the pipeline. *Platform and Model Design for Responsible AI*
lists the characteristics of privacy-aware AI as: being proactive and preventive rather than
reactive; making privacy the default; designing privacy in rather than bolting it on after the fact;
refusing to trade away essential functionality; protecting the model end to end across its lifecycle;
maintaining visibility and transparency; and putting the individual's interests first. These map
cleanly onto the governance controls this reference describes:

- **At intake**, classify each dataset and decide lawful basis, purpose, and owner before any
  modeling begins.
- **At build**, apply PETs, minimize sensitive features, and record lineage and quality so the model
  is auditable.
- **At deploy**, restrict access, monitor for drift and for leakage of sensitive outputs, and bind
  retention schedules to the operational data.
- **At retirement**, decommission data per its retention schedule and record the deletion.

Wiring these into the lifecycle gates (see `ai-lifecycle-governance.md`) and recording them in the
model card is what turns "privacy is important to us" into an enforced, verifiable practice.

## Where to Go Next

- **`foundations-and-principles.md`** — privacy as a governing principle and how training vs.
  operational data governance fit the principles.
- **`ai-lifecycle-governance.md`** — where ownership, lineage, quality, and retention are enforced
  across stage gates.
- **`llm-and-agent-security.md`** — the security controls (access, exposure, trust boundaries) that
  protect data in deployed systems, including agent tools and containment.
- **`six-level-governance-framework.md`** — the cross-cutting evidence loop and maturity model for
  applying these controls through strategy, review, operations, and learning.
- **`transparency-and-explainability.md`** — how disclosure of data practices supports consent and
  individual control.
- **`regulatory-landscape.md`** — GDPR and AI Act obligations that set the legal floor for the
  practices above.
- **`../templates/agentic-governance-review.md`** — the posture, purpose-aware egress, memory, and
  rights worksheet.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *Designing Data Governance from the Ground
Up*, *Data Governance Handbook*, *Platform and Model Design for Responsible AI*, and *AI Governance*,
and it is
de-staled against the current state by the mission research notes on the regulatory landscape
(GDPR lawful basis, EDPB AI guidance) and technical controls (data cards, lineage, monitoring). All
prose is an original paraphrase and synthesis of the ideas in these sources; idea-level attribution
is consolidated in `source-index.md`. The discussion of consent, retention, and minimization is
educational context, not legal advice, and specific requirements should be confirmed against current
regulation at use time.

# Procurement, Third-Party, and Board Oversight of AI

This reference teaches the acquisition-side and governance-top-side of an AI program: how an
organization decides what to buy versus build, how it vets and manages the vendors, models, data, and
open-source assets it brings in from outside, how it keeps that third-party surface safe across the
supply chain, and how it reports the whole picture to the board with meaningful metrics and audit
backing. Its central message is that governance must reach beyond the models built in-house. A large
share of an organization's AI surface arrives through procurement — hosted models, pretrained weights,
data licenses, tooling, and consultants — so the discipline of due diligence and supply-chain
management is what turns board-level intent into working control, and board reporting with metrics is
what keeps leadership honestly informed. Read it alongside `risk-management-and-frameworks.md` (the
tiering and control machinery), `regulatory-landscape.md` (the legal duties that make third-party and
board exposure material), `governance-operating-model.md` (roles, councils, and the decision chain that
reaches the board), `ai-lifecycle-governance.md` (where vendor-built assets enter and leave the
lifecycle), and `llm-and-agent-security.md` (the supply-chain and plugin threats specific to LLM
systems). Use the [third-party-due-diligence.md](../templates/third-party-due-diligence.md) template to run a vendor or model intake and the
[board-ai-governance-report.md](../templates/board-ai-governance-report.md) template to structure the periodic reporting this reference describes.
This is educational context, not legal advice.

## Why Procurement and Board Oversight Belong Together

Two observations pull third-party risk and board oversight into one discipline. The first is that AI
is increasingly procured rather than invented. Hosted model APIs, pretrained open-source weights,
curated datasets, ML platform tooling, and specialized consultancies supply a large share of the
systems an enterprise actually runs. When an AI system fails — it is biased, it leaks data, it makes
a consequential error — the organization is accountable for that failure even if the flawed component
came from a vendor it did not build. The second is that the board, not the engineering team, carries
the ultimate duty of oversight. Fiduciary expectations and enforcement pressure mean leadership cannot
delegate AI risk away and call it handled; it must be informed, ask pointed questions, and see
reporting that reflects reality.

The two themes reinforce each other. Third-party diligence produces the evidence a board needs to make
acquisition and risk decisions, and board reporting produces the accountability that forces the
procurement function to keep its diligence current. A governance program that treats vendor risk as an
engineering-only concern, or board reporting as a cosmetic slide deck, fails on both sides of the
chain. The practical shape of a working program is a documented accountability chain that runs from a
named owner on a procurement or due-diligence intake, through a governance council or risk committee
that reviews higher-risk acquisitions, up to a board committee that receives a metrics-backed report on
a fixed cadence.

## Third-Party and Vendor Due Diligence

Due diligence is the disciplined practice of investigating a proposed third-party engagement — a
vendor, a model, a dataset, or a consultancy — before you commit to it, so that you accept risk with
your eyes open rather than discover it after signing. The goal is not to refuse every external
dependency; it is to characterize what you are bringing in, what it could harm, and what you need to
protect yourself with in the contract and the controls.

### A due-diligence intake should answer before engagement

- [ ] **What is the component and how is it used?** Classify the engagement: a hosted model API, an
      open-source weight, a licensed dataset, an ML platform, or a professional service. State the use
      case it supports and the tier it lands in under `risk-management-and-frameworks.md`.
- [ ] **Who owns and operates it?** Identify the provider, its location and jurisdiction, its size and
      financial stability, and whether it is subject to any regime relevant to the data you will share
      (see `regulatory-landscape.md`).
- [ ] **What data flows to the vendor?** Enumerate inputs and outputs. Whether data is personal,
      regulated, proprietary, or open determines lawful-basis, transfer, and confidentiality duties.
- [ ] **What does the vendor do with our data and the model?** Ask about training, retention, storage
      location, subprocessors, and whether inputs can be retained or reused in ways you cannot accept.
- [ ] **What controls does the vendor actually operate?** Request evidence of security, privacy, and
      governance controls — certifications, audits, incident-response capability, and model-governance
      practices — rather than accepting assertions at face value.
- [ ] **What happens when things go wrong?** Confirm the vendor's incident-notification commitments,
      uptime and performance commitments, liability and indemnity terms, and the exit path.
- [ ] **Can we verify it?** Build in rights to audit or at least to review evidence, so diligence is not
      a one-time event.

Diligence should be proportional to risk. A low-impact internal productivity tool and a system making
consequential employment or credit decisions are not the same due-diligence job. The proportionality
lens in `risk-management-and-frameworks.md` — inherent risk, impact, and likelihood — should drive how
deep the investigation goes and how many signatures the intake needs.

### Common pitfalls in diligence

- Diligence that happens *after* the contract is signed, when leverage is gone.
- Accepting vendor claims without independent verification or the right to inspect evidence.
- Ignoring the full chain — the vendor's own suppliers and subprocessors — rather than just the direct
  counterparty.
- Treating diligence as a static document instead of a living file that is revisited when the system,
  the vendor, or the risk profile changes.
- Failing to record the outcome so a board or audit can see how a decision was reached.

## Supply Chain Risk

Supply chain risk is the modern, expanded form of third-party risk. Traditional manufacturing tracked
every part and its source through a bill of materials; software and AI face the same problem with far
more moving pieces and far fewer guarantees. For AI systems the supply chain includes not only hosted
services and licensed data but pretrained model weights, model registries, open-source libraries,
training data of uncertain provenance, and LLM plugin extensions. A vulnerability in any of these can
infect the finished system, introduce bias, leak data, or make the model behave unpredictably in
production.

### Why AI supply chains are fragile

- **Pretrained and open-source models are reused at scale.** Model registries host thousands of models
  that teams fine-tune or embed without always knowing their origin, training data, or weaknesses.
  Their convenience and low cost hide genuine uncertainty about integrity and lineage.
- **Dependencies multiply silently.** The software libraries, plugins, and packages that surround a
  model often have weak version tracking and can carry their own vulnerabilities.
- **Data provenance is hard to guarantee.** Training or operational data gathered from external or
  crowd-sourced sources can be tampered with, biased, out of date, or drawn from material you have no
  right to use.
- **Models go stale and unsupported.** A model that served well can hit end-of-support, leaving no
  security updates and creating drift and exposure over time.

### BOMs for AI

Just as a software bill of materials (SBOM) lists every component in an application, an **AI bill of
materials (AI BOM)** records, component by component, what an AI system is assembled from: the model
and its architecture, the data
sources used for training, the libraries and dependencies, how the model is used or deployed, and the
assumptions and attestations behind it. Maintaining an AI BOM delivers several practical benefits:

- **Transparency and traceability** of what is really inside a system, so developers, auditors, and
  stakeholders can assess quality, reliability, and security.
- **Faster root-cause work** when a system fails, shows bias, or is breached — you can identify the
  problematic component quickly instead of hunting through an undocumented stack.
- **Due-diligence and compliance evidence** that the organization can show to auditors, regulators, and
  the board.
- **Contingency planning** for high-risk third-party AI systems: knowing what the system depends on lets
  you plan for failure or incident response.

SBOMs are increasingly expected in regulated environments and are being mandated for software sold to
governments, so AI governance programs should treat component-level transparency as a baseline
expectation rather than a nice-to-have.

### Supply-chain threats to monitor

- Outdated or vulnerable third-party packages and dependencies.
- Reliance on a vulnerable or untrusted pretrained model for fine-tuning.
- Training on tampered crowd-sourced or external data.
- Using end-of-support models or libraries with no security updates.
- Ambiguous terms and privacy policies that allow misuse of sensitive or copyrighted data.

Because LLM systems amplify these threats — model manipulation, data poisoning, and malicious plugin
behavior all travel through the supply chain — read this section together with `llm-and-agent-security.md`
for the specific attack surface.

## Managing the Vendor Lifecycle

Due diligence is the entry point, but third-party risk is a lifecycle, not a form. A working vendor
management program covers onboarding, continuous monitoring, and exit.

### Onboarding

Onboarding is where diligence becomes an enforceable relationship. The contract should encode the
commitments identified during diligence: data-handling and retention rules, subprocessor disclosure,
incident notification windows, performance and uptime targets, liability and indemnification, and the
right to verify compliance. Roles and responsibilities should be explicit — who inside the organization
owns the relationship, who monitors the vendor, and who gets escalated to. Without a written onboarding
record, the diligence that justified the relationship is lost and cannot be reviewed or audited later.

### Continuous monitoring

The relationship is not a one-time deal. Recurring review should track the vendor's actual performance
against commitments, watch for changes in the vendor's controls, ownership, or financial health, and
recheck the system as it or the surrounding risk profile evolves. How you observe the vendor can range
from automatically collected system telemetry to scheduled reviews, structured surveys, and interviews,
supplemented by the incident and violation reports the contract requires them to surface. When a
materially risky third-party AI system is involved, this monitoring should be explicit and documented
— a gap that surfaces only at incident time is a governance failure.

### Exit

Exiting a vendor should be planned before you need it. What happens to the data the vendor holds when
the contract ends? What is the migration path for the models or services that depended on it? Building
exit terms into the contract at onboarding — data return or deletion, transition assistance, and
notice periods — prevents a procurement decision from becoming a lock-in liability later. The
board-reporting cadence should surface material dependency concentrations so the organization is not
surprised by its own reliance on a single vendor.

## Board Oversight and Fiduciary Duty

Board oversight of AI is no longer a forward-looking nicety; it has become a recognized director
responsibility. The legal anchor, especially in the United States, is the fiduciary duty of oversight
rooted in the 1996 Delaware *Caremark* decision. Under that standard, directors act improperly if
they knowingly let the corporation run without a sound information and reporting apparatus, or if
they neglect to track management and thus stay in the dark about risks that called for their
attention. Recent cases have tested whether that duty reaches "mission critical" risks — the same
kind of exposure courts already attach to cybersecurity, and one that commentators expect to extend
to high-impact AI shaping consequential outcomes such as hiring, lending, health, and similar
decisions.

### Applying a fiduciary mindset to AI

- **Treat AI as strategy and risk, not just technology.** Boards should understand which AI systems are
  mission critical, how they affect employees, customers, stakeholders, and the environment, and how
  they create or destroy value.
- **Oversee compliance.** Directors should ensure management has assigned clear responsibility for
  AI-related legal and regulatory compliance and risk mitigation, and that there are AI-specific
  policies and guidance in place.
- **Stay informed without operating.** The governing discipline is often summarized as "noses in,
  fingers out": a board should observe enough to direct well but never slip into hands-on management.
  The failure mode is the reverse — ignoring mission-critical AI risk by failing to hand monitoring
  responsibility for a safety- or compliance-critical AI system to any committee.
- **Assign committee responsibility.** AI risk should be mapped to a specific board committee (audit,
  risk, or a dedicated technology or AI committee) so it has a defined owner, not an ambiguous
  "everyone and no one."
- **Demand director education.** When directors across the board lack AI expertise, that gap is itself
  a governance risk; boards should invest in their own learning so their questions are substantive
  rather than perfunctory.

The arrangement the leading guidance keeps converging on is a documented accountability chain: the
board or a named committee holds oversight, an internal AI ethics or governance council weighs policy
and clears the higher-risk use cases, and named owners or stewards at the operating level handle the
day-to-day controls. That chain is exactly the operating model described in
`governance-operating-model.md`.

### When oversight becomes exposure

Enforcement is making AI governance material to directors. Regulators are pursuing "AI washing" —
marketing claims that overstate AI capability or responsibility — and taking action on AI-related
misconduct, while Delaware-style oversight doctrine continues to be tested for mission-critical AI. The
practical consequence is that boards cannot treat AI as a technical or compliance matter that is fully
delegated away. Being able to show an informed, recurring review process with named responsibility is
itself a governance control that reduces both legal exposure and the chance of being blindsided by an
AI failure.

## Board Reporting with Metrics

Board reporting is how oversight becomes real rather than nominal. Reporting turns the noisy detail of
day-to-day governance into a compact, decision-useful picture that directors can actually evaluate, and
it forces management to define what good looks like. Without metrics, a board report is an opinion;
with the right metrics, it is a state of the system.

### What a reporting cadence should cover

- [ ] **Inventory and tiering:** how many AI systems exist, their risk tiers, and how the portfolio is
      shifting.
- [ ] **Material risks and incidents:** the highest-risk systems, their residual risk, and any material
      incidents or near-misses since the last report, including briefings on those that need prompt
      attention.
- [ ] **Compliance and regulatory exposure:** open obligations, audit findings, and any enforcement or
      investigation activity (see `regulatory-landscape.md`).
- [ ] **Third-party and supply-chain posture:** concentration on critical vendors, outstanding
      diligence, and material supply-chain or model events.
- [ ] **Governance operations:** status of policies, council decisions, approvals, and the state of the
      control environment.
- [ ] **Progress against goals:** how the program is advancing against the board-approved objectives and
      any identified gaps.

### Choosing metrics that actually inform

Metrics should be chosen to tell directors whether the system is safe, compliant, and being governed —
not to flatter the program. Good candidates include:

| Type | Example indicators |
|---|---|
| Exposure | Count of systems by risk tier; coverage of systems with a named owner; material-vendor concentration |
| Control | Percentage of high-risk systems with completed risk assessments, model cards, or monitoring; audit findings open versus closed |
| Performance | Drift alerts, accuracy or fairness metric regressions, system availability, mean time to resolve incidents |
| Compliance | Number of open compliance obligations, incidents reported to regulators, outstanding audit recommendations |
| Third-party | Vendors with current vs overdue diligence, data flows that lack a documented lawful basis, open supply-chain findings |
| Incidents | Number and severity of AI incidents in the period, time to detect and to remediate |

The metric set will differ by organization, but it should be **stable enough to trend over time**,
**comparable across periods**, and **honest** — a metric that never changes or never shows a problem is
a signal the wrong thing is being measured. This reporting structure is implemented concretely in the
[board-ai-governance-report.md](../templates/board-ai-governance-report.md) template.

### Cadence and escalation

Recurring review should be on a fixed cadence — commonly at least quarterly, with a standing
board-level agenda item — so AI risk is not buried until an incident forces attention. Beyond the
recurring report, there should be a defined escalation path for material AI incidents so the board or
its committee is briefed promptly when something rises to that threshold, rather than at the next
scheduled meeting. Management should prepare, own, and be accountable for the report's accuracy; the
board should ask questions and challenge the answers, which is the entire point of the exercise.

## Audit

Audit is the independent check that the controls and reporting described above actually exist and work.
It is what converts a self-reported governance program into one that can withstand scrutiny from
regulators, customers, and the board itself. Audit sits alongside monitoring: monitoring is continuous
and operational, while audit is a structured, independent examination against stated criteria.

### The role of audit in AI governance

- **Independent verification:** an auditor (internal audit, or an external third party) evaluates
  whether the program's controls — diligence records, model cards, monitoring, tiering — are in place
  and effective, rather than taking management's word for it.
- **Evidence for the board:** a clean audit gives the board confidence that reported metrics reflect
  reality; findings give it a concrete list of what to fix.
- **Finding what monitoring misses:** because monitoring is designed by the same people who run it,
  audit offers an outside view that can surface gaps, exceptions, and weaknesses that daily operations
  overlook.
- **Due-diligence credibility:** the right to audit, or to receive audit evidence from a vendor, is a
  powerful procurement tool — it forces suppliers to demonstrate the controls they claim.

### What a program should be able to show an auditor

- The inventory and tiering of all AI systems, including those procured from third parties.
- Completed due-diligence records and an AI BOM or equivalent component-level transparency for each
  material system.
- Risk assessments, model cards, and monitoring evidence tied to each system (see
  `risk-management-and-frameworks.md` and the [model-card.md](../templates/model-card.md) template).
- A documented accountability chain from named owners up through a council to a board committee.
- The metric set and reporting cadence used for the board, with a track record of actual reports.
- Incident logs and evidence that material incidents were escalated and remediated.
- Exception registers and the process for approving and tracking waivers.

An audit is most valuable when it is genuinely independent and when its findings are acted on. A
governance program should treat audit findings as a feedback loop that feeds back into diligence,
monitoring, and reporting, so the system improves rather than merely passing the next check.

## A Practitioner's Workflow

- [ ] **Classify and tier** every external AI engagement on intake, and route higher-risk ones for
      deeper diligence and sign-off (see `risk-management-and-frameworks.md`).
- [ ] **Run due diligence** before contract, proportional to risk, and record the outcome.
- [ ] **Maintain an AI BOM** for each material system so the supply chain is transparent and auditable.
- [ ] **Encode commitments in the contract** at onboarding, including the right to verify, and assign a
      named owner.
- [ ] **Monitor continuously** and revisit diligence when the system, vendor, or risk profile changes.
- [ ] **Plan the exit** before you need it.
- [ ] **Report on a fixed cadence** with stable, honest metrics and a defined incident-escalation path.
- [ ] **Audit independently** and feed findings back into the program.

## Where to Go Next

- **`risk-management-and-frameworks.md`** — the tiering and risk-assessment machinery that determines
  how deep diligence and reporting must go.
- **`governance-operating-model.md`** — the councils, stewards, and accountability chain that run from
  the board down to named owners.
- **`regulatory-landscape.md`** — the legal duties that make third-party and board exposure material.
- **`llm-and-agent-security.md`** — the supply-chain, plugin, and model-specific threats for LLM
  systems.
- **`ai-lifecycle-governance.md`** — where vendor-built assets enter and leave the model lifecycle.
- Use the **[third-party-due-diligence.md](../templates/third-party-due-diligence.md)** template to run a vendor or model intake and the
  **[board-ai-governance-report.md](../templates/board-ai-governance-report.md)** template to structure board reporting.

---

### Synthesized from

This reference synthesizes the acquisition, supply-chain, vendor-management, reporting, and audit
material from *The AI Product Manager's Handbook* and from *Developing Cybersecurity Programs and
Policies* (the fourth edition, written for an AI-driven world), together with the mission research note
on organizational and board governance (research-org-board-governance.md, current to August 2026). The
research note supplies the current
fiduciary, board-oversight, and third-party expectations; the two books supply the procurement,
vendor-lifecycle, supply-chain, BOM, reporting-metric, and audit mechanics. All prose is an original
paraphrase and synthesis of the ideas in these sources; no passage is reproduced verbatim, and
idea-level attribution is consolidated in `source-index.md`. This reference is educational context, not
legal advice, and the in-flux regulatory and case-law elements it touches should be verified against
primary sources at use time.

# AI Regulatory Landscape

This reference teaches the current obligations organizations face across the major AI
jurisdictions, with an eye to what enforcers actually do. Its central message is that AI
compliance is now a *place-based* problem: the same model can be lawful to sell in one country,
illegal to deploy in another, and only lightly regulated in a third. Rather than one global rule,
there are several converging but distinct regimes — the European Union's horizontal AI Act, the
General Data Protection Regulation (GDPR), a fragmented United States patchwork, the United
Kingdom's context-based approach, and China's binding sectoral measures — and a governance program
must map each system to each regime by its risk tier and its geographic footprint. This reference
is authoritative on the current state, built from the mission research note on the regulatory
landscape (research-regulatory.md); the regulatory chapters of the source books are treated here
as historical context only, because the law has moved materially past their 2023–2025 publishing
dates. Read it with `risk-management-and-frameworks.md` (the NIST/ISO machinery a compliance program
applies), `procurement-third-party-and-board-oversight.md` (the board-level obligations that flow
from these laws), and `llm-and-agent-security.md` (where security duties become legal duties). Use
the [model-risk-assessment.md](../templates/model-risk-assessment.md) and [board-ai-governance-report.md](../templates/board-ai-governance-report.md) templates to record the
regulatory mapping this reference describes. It is not legal advice, and the in-flux sections it
flags must be verified against primary sources at use time.

## Why Regulation Is the Fastest-Moving Input to AI Governance

The first practical fact for a governance practitioner is that the regulatory map changes faster
than almost any other input to the discipline. The laws themselves are recent — the EU AI Act was
only adopted in 2024 — and the deadlines, fines, and even the *existence* of a rule have shifted
during the writing window of the source books. *Responsible AI in the Enterprise*, for example,
described a compliance landscape in which the EU was still finalizing its AI regulation and the
United States had no unified federal framework; that general shape is still accurate, but the
specific obligations have since become far more concrete in Europe and remain genuinely unsettled
in the United States. The books are therefore a reliable guide to the *principles* a compliance
program should embody — risk-based tiering, documentation, accountability, human oversight — but
not to the *current* deadlines or penalty levels. Any serious governance program should track
primary regulator sources continuously rather than rely on a static reference like this one.

A second practical fact is that compliance is increasingly a risk-tier exercise. Most AI in use
today sits in a "minimal risk" bucket that no regime newly regulates, while a small set of
highest-risk uses — biometric identification, employment and credit decisions, critical
infrastructure, influence over public life — draws the heaviest duties. The governance skill
overall, and the risk-tiering in `risk-management-and-frameworks.md`, exists precisely to answer
"which tier is this system, and therefore what must we do." Regulation supplies the legal
underpinning for that tiering, but the analytical work is the same everywhere: characterize the
system's data sensitivity, autonomy, and potential for harm, then apply the rules that attach to
that characterization in each jurisdiction where the system is deployed.

## European Union: The AI Act as the Global Baseline

EU Regulation 2024/1689 — the AI Act — is the world's first comprehensive, horizontal AI statute,
and it has effectively become the reference point other jurisdictions either imitate or explicitly
reject. Its reach extends to *providers* (who put an AI system or general-purpose AI (GPAI) model
on the market) and to *deployers* (who use one), whenever they operate in or serve the EU market.
Its organizing idea is a **four-tier risk pyramid**:

| Tier | What it covers | Consequence |
|---|---|---|
| Unacceptable | Prohibited practices (harmful manipulation, social scoring, scraping untargeted facial imagery, emotion recognition in workplaces or schools, and live remote biometric ID in public places) | Banned outright |
| High-risk | AI in sensitive areas (biometrics, critical infrastructure, education or employment, migration and asylum, plus border control) and AI embedded in regulated products | Strict duties before and after market placement |
| Limited / transparency | Generative AI and chatbots that interact with people or produce content | Disclosure and labelling duties |
| Minimal | The large majority of ordinary AI | Largely unregulated by the Act |

The AI Act does not switch on all at once; its obligations arrive in phases. The bans on prohibited
practices and the AI-literacy duties have been live since February 2025. From 2 August 2025, makers
of general-purpose models took on duties covering transparency, copyright, and, for the most
capable models, systemic-risk management. Commission guidance published in 2025 sets out when a
model qualifies as general-purpose, adopts a pragmatic view that only providers making substantial
changes are caught, and describes the open-source carve-outs. Then, in August 2026, fresh
transparency duties arrived for those who build or run generative systems: telling users when they
are conversing with a machine, and labelling AI-made images, audio, deepfakes, and public-facing
text.

### EU high-risk timing: verify against current state

> **Currency flag.** The most consequential recent change is the deferral of the high-risk
> obligations. Under the original schedule, the central high-risk duties would have started on
> 2 August 2026. The AI Omnibus — floated by the Commission in November 2025, politically agreed in
> May 2026, and effective from 27 July 2026 — pushed those dates out. High-risk systems in areas
> like biometrics, critical infrastructure, education and employment, migration and asylum, and
> border control now face strict rules from 2 December 2027, while high-risk systems embedded in
> regulated products like lifts and toys follow from 2 August 2028. **Because this timeline is in active flux
> and subject to litigation and further amendments, verify the effective dates against the European
> Commission's current publications at the time you rely on them.** The deferral buys organizations
> a window, but it also leaves real uncertainty about the final scope once the supporting standards
> mature.

When the high-risk rules do bite, they are heavy and practical. Owners must carry out risk
assessment and mitigation, feed models with datasets curated to reduce discrimination, log activity
for traceability, keep documentation examinable by authorities, give deployers clear instructions,
build in human oversight, and hit exacting targets for robustness, security, and accuracy.
Deployers owe duties of their own for oversight, monitoring, and notifying serious incidents.
Compliance machinery should be built to these duties *now*, even where the hard deadline has
slipped, because the obligation set is not going away — only its start date.

## Enforcement and Penalties in the EU

Enforcement is split between the AI Office at the European Commission and the authorities of the member states. The
AI Office took charge of general-purpose models in August 2026, with authority to demand technical
docs, run evaluations, order corrective measures, and impose penalties; that same day the
Commission's broader enforcement and the new transparency rules also went live. Behind all this sit
the AI Board, plus a Scientific Panel and an Advisory Forum, along with practical fixtures such as a service
desk, a channel for whistleblowers, and a complaints mechanism.

The penalty ceiling is what makes the AI Act a board-level concern. Fines can reach **EUR 35
million or 7% of global annual turnover** for the gravest breaches, with violations tied to
general-purpose models capped at EUR 15 million or 3% of turnover. These figures mirror the scale
of GDPR penalties and make AI compliance an enterprise-risk matter, not an engineering checkbox.
The compliance angle is to treat the AI Act as carrying roughly the same organizational weight as
GDPR, with the same need for accountability, audit trail, and senior ownership.

### GDPR and the AI Act as complementary duties

The AI Act does not displace the GDPR; systems touching personal data stay squarely within
data-protection law. The two regimes overlap rather than collapse into one another: the AI Act
governs a system's design and risk profile, while the GDPR governs the data flowing through it —
lawful basis, transparency, minimization, data-subject rights, and accountability. The European
Data Protection Board's Opinion 28/2024 is the key EU data-protection gloss on AI: it explains when
a model counts as processing under the GDPR and how duties like lawful basis, transparency, and
accountability flow into AI workloads. Governance practitioners should treat the two as
complementary — a compliant AI system satisfies both its AI-Act risk-tier duties and its GDPR data
duties, and a compliance plan should document each separately rather than assume one implies the
other.

## United States: Federal and State Patchwork

> **Currency flag.** The U.S. position is the least settled of the major jurisdictions. There is no
> comprehensive federal AI statute as of the date of this reference; rule-making proceeds through
> executive action and existing agency authority (above all the Federal Trade Commission), and
> a rapidly growing and actively litigated body of state law. **Because federal policy can change
> with each administration and state laws are being enacted, amended, challenged, and enjoined
> month to month, verify the current federal and state position against primary sources at the time
> you rely on it.**

At the federal level, policy has swung with the executive. An executive order from January 2025,
EO 14179, scrapped the prior administration's AI directive and ordered an AI Action Plan, which
appeared that July. A White House order in December 2025 championed one national policy and
criticized what it called "onerous and excessive" state rules, and a 2026 national AI framework then
called on Congress to unify the field federally. What this means in practice is that federal
*guidance* can shift direction quickly even while the underlying agency powers (especially FTC
authority over unfair and deceptive practices) remain available to police harmful AI conduct.

At the state level the picture is a genuine patchwork. Colorado's SB 24-205 was America's first
broad consumer-protection AI statute, yet its rollout was delayed and a Colorado court in 2026
temporarily stopped the state attorney general from applying it, following legal challenges.
California, Texas, Illinois, Connecticut, and Utah are among the states that have passed or are
progressing their own measures on algorithmic bias, deepfakes, and AI in hiring; more than a
thousand AI bills surfaced in statehouses in a single recent year. For governance practitioners,
this means the practical compliance surface is both broad and shifting: a national deployer must
track a growing set of state-specific duties, monitor which are actually in force versus enjoined,
and treat the federal pre-emption question as genuinely open.

## United Kingdom: Context-Based and Sectoral

The UK has not enacted its own AI law; it governs AI by the context of use, leaning on current
regulators and legal regimes. That stance, set out in a March 2023 white paper that framed AI
regulation as a pro-innovation exercise, has been kept by the Labour government, which argues that
most AI should
be overseen where it is used and that sector specialists are the right overseers. Oversight is
spread among bodies like the ICO (for data protection), the FCA, the CMA, and Ofcom, which polices
online services including AI chatbots under the 2023 Online Safety Act. A promised statutory scheme
for the most advanced frontier models has still not been passed into law, so frontier-model
governance in the UK today rests more on the voluntary and safety-research side, including the UK
AI Security Institute, than on statute.

The compliance angle for the UK is that obligations are *sector- and context-specific* rather than
a single horizontal law: the same system may face data-protection duties from the ICO if it touches
personal data, consumer-protection and competition duties if it reaches consumers, and
platform-safety duties if it operates as an online service — but there is no one UK AI statute to
map against. Governance programs with UK exposure should therefore map each use case to the
relevant sectoral regulator and treat "is this AI subject to UK rules" as a question answered per
context.

## China: Binding Sectoral Measures

China lacks one overarching AI statute yet runs what many consider the world's densest set of
binding, sector-specific AI rules, largely administered by a single body, the Cyberspace
Administration of China (CAC). The cornerstone instruments are the recommendation-algorithm and deep-synthesis regulations
plus the August 2023 Interim Measures for generative-AI services, which load providers with
content, safety, and data duties. Draft rules on anthropomorphic AI chat services, covering
chatbots and virtual companions, were released by China in 2026. Beijing's regime pairs innovation
goals with tight curbs on content, data security, and national interest, and the "AI+" program
within the current five-year plan signals continued growth.

For multinationals, the compliance takeaway is that China adds a substantial, *distinct* compliance
surface on top of the EU, US, and UK frameworks — content controls and data-security duties that
have no direct EU or US analogue and that apply to providers operating within or serving the
Chinese market. A governance program cannot assume that a system compliant in the EU or US is
compliant in China; the obligations are qualitatively different and require separate mapping.

## Sectoral Rules: Where Horizontal and Vertical Law Meet

Beyond the horizontal regimes, AI is increasingly caught by sectoral law that was not written for
AI but applies to it. Financial services, health care, employment, and consumer protection each
carry existing duties that an AI system inherits when it operates in that sector. Two patterns
matter for governance:

- **AI Act as a coordinating layer:** in the EU, the AI Act explicitly clarifies its relationship
  with sectoral product-safety legislation (such as the Machinery Regulation) to avoid duplicating
  rules, so a system already governed by a sectoral regime must still satisfy its AI-Act risk-tier
  duties where applicable.
- **Sectoral duties as independent obligations:** in the US and UK especially, AI deployed in a
  regulated sector is bound by that sector's existing rules — fair-lending and employment law, for
  example, apply to an AI that makes or informs credit and hiring decisions regardless of whether
  a dedicated AI statute exists.

The governance implication is a **mapping discipline**: for each AI system, record the horizontal
regimes (AI Act, GDPR, and any applicable US/UK/China law) *and* the sectoral rules that the system
inherits from the domain it operates in. `risk-management-and-frameworks.md` and the
[model-risk-assessment.md](../templates/model-risk-assessment.md) template provide the structure for recording this mapping.

## A Practitioner's Compliance Workflow

- [ ] **Inventory:** maintain a model/system inventory (see `ai-lifecycle-governance.md`) with the
      jurisdictions where each system is offered or deployed.
- [ ] **Tier:** characterize each system's risk tier per the EU AI Act's categories and any local
      analogues (see `risk-management-and-frameworks.md`).
- [ ] **Map:** for each system, list the horizontal regimes (AI Act, GDPR, US federal/state, UK
      sectoral, China sectoral) that apply in each deployment location.
- [ ] **Assign duties:** translate each applicable rule into concrete obligations — documentation,
      logging, human oversight, transparency, data minimization — and record them.
- [ ] **Track currency:** subscribe to primary regulator sources (European Commission, EDPB, FTC,
      relevant state AGs, UK ICO, CAC) and re-verify the in-flux sections flagged in this reference
      on a fixed cadence.
- [ ] **Report:** surface material obligations and penalty exposure to the board (see
      [board-ai-governance-report.md](../templates/board-ai-governance-report.md)).

## Horizon Scanning: What to Watch

- **EU:** the final scope of the high-risk obligations once the deferred deadlines arrive and the
  supporting standards mature; how the AI Act, GDPR, and sectoral rules interact in enforcement
  practice.
- **US:** whether a federal pre-emption framework is enacted; the outcome of litigation over state
  laws like Colorado's; how the FTC exercises existing authority over AI.
- **UK:** whether the planned binding regulation of frontier models is enacted, and how the
  sectoral regulators coordinate on cross-cutting risks.
- **China:** the scope of the draft chatbot/virtual-companion rules and continued expansion of
  generative-AI content and data duties.

## Where to Go Next

- **`risk-management-and-frameworks.md`** — the NIST AI RMF and ISO/IEC 42001 machinery a
  compliance program applies to meet these legal duties.
- **`procurement-third-party-and-board-oversight.md`** — how regulatory obligations surface in
  third-party due diligence and board reporting.
- **`ai-lifecycle-governance.md`** — where regulatory checks and documentation sit in the model
  lifecycle.
- **`llm-and-agent-security.md`** — security duties that become legal duties under these regimes.

---

### Synthesized from

This reference is authoritative on the current state, built from the mission research note on the
regulatory landscape (research-regulatory.md, current to August 2026). The regulatory chapters of
the source books — primarily *Responsible AI in the Enterprise* and *Beyond the Algorithm* — are
treated as historical context only, because the obligations described there have moved materially
past their publication dates. All prose is an original paraphrase and synthesis of the ideas in
these sources; idea-level attribution is consolidated in `source-index.md`. Because the EU AI Act
high-risk timeline and the entire US federal/state landscape are in active flux, those sections are
explicitly flagged for verification against primary sources at use time. This reference is
educational context, not legal advice.

# AI Risk Management and Frameworks

This reference teaches the structured discipline of AI risk management: how an
organization identifies, assesses, prioritizes, and mitigates the risks its models can
produce. It explains the two most influential risk frameworks — the NIST AI Risk
Management Framework (RMF) and the ISO/IEC 42001 / 23894 pair — and covers the
concepts those frameworks rely on: inherent versus residual risk, model risk tiering,
and the risk register. It is an original synthesis of the ideas in *Responsible AI in
the Enterprise*, *Platform and Model Design for Responsible AI*, and current research on
standards and frameworks (see research-standards.md). Read it with
`foundations-and-principles.md` (which draws the governance-versus-risk-versus-compliance
distinction) and `governance-operating-model.md` (which assigns the roles that own these
risks). It is not legal advice, and framework details should always be confirmed against
the governing bodies' current releases.

## Why Structured AI Risk Management Exists

The starting point of AI risk management is the recognition that a model can harm the
business, its customers, and its reputation in ways that ordinary software rarely does.
*Responsible AI in the Enterprise* lists concrete failure examples: an automated resume
screener that filters candidates along biased lines, a manufacturing defect detector
that degrades as the model drifts, and a credit model that refuses loans on the basis of
an applicant's employment type. Each is a plausible, ordinary deployment that can go
quietly wrong.

Risk management is the ongoing, analytical practice of making those possibilities
visible and tractable: identifying what could go wrong, assessing how bad it could be
and how likely, prioritizing the risks worth acting on, and mitigating them. It differs
from compliance, which only asks whether you meet the current letter of the law. As the
mission's standards research notes, the leading frameworks are explicitly risk-based
rather than rule-based for exactly this reason: the regulatory landscape is still
stabilizing, so an organization that anchors on risk stays ahead of whatever specific
laws arrive rather than reacting to each one.

Structured AI risk management typically hangs on four mechanisms, all of which this
reference explains below:

- **A vocabulary and a workflow** for naming, scoring, and escalating risk (the
  frameworks and the register).
- **A distinction between inherent and residual risk**, which drives how much mitigation
  a given model deserves.
- **Model risk tiering**, which routes each model in the inventory to the right depth of
  review.
- **A risk register** that is owned, reviewed, and kept current rather than filed away.

## Inherent vs. Residual Risk

Every model arrives with risk baked in, and every control the organization applies reduces
that starting risk. Keeping the two apart is what makes prioritization honest.

- **Inherent risk** is the level of risk a model would pose with no controls, validation,
  or mitigation applied — the danger present simply because the model exists and is used
  in a particular context. *Platform and Model Design for Responsible AI* observes that
  inherent risk grows from the absence or age of validation, unaddressed known issues, and
  the model's own complexity and reach.
- **Residual risk** is the risk that remains after the organization's controls,
  validation, monitoring, and safeguards are applied. It is the realistic risk the
  business actually carries day to day.

The gap between the two is the value of the control environment. Good practice assesses
both: you need to know the inherent risk to decide how much control is proportionate, and
you need the residual risk to know whether you are actually where you intend to be. If the
residual risk still exceeds the organization's appetite, the model should not ship as-is —
or the controls must be strengthened.

| Concept | What it captures | Question it answers |
|---|---|---|
| Inherent risk | Risk with no controls applied | "How dangerous is this model in its context, unmitigated?" |
| Residual risk | Risk after controls | "What risk are we actually carrying today?" |
| Risk appetite | The amount of risk the org accepts | "Is the residual risk acceptable?" |

## Model Risk Tiering

Not every model deserves the same depth of scrutiny, and applying the heaviest process to
everything is wasteful while applying the lightest to everything is reckless. Model risk
tiering is the practice of classifying each model in the inventory into a risk tier so the
governance effort matches the stakes.

*Platform and Model Design for Responsible AI* describes tiering as the industry's way of
making model risk comparable across a portfolio: it identifies and differentiates the risk
one model's use presents relative to others, weighing factors such as volume, context of
use, and financial or customer impact. A common arrangement sorts models into high-,
medium-, and low-risk tiers, each with its own depth of review, validation, and monitoring.
The book stresses several principles for building a tiering scheme:

- **Expert judgment drives it.** The classification tool quantifies and orders risk, but
  the priorities behind the weights are business and functional decisions, not pure math.
- **It should be simple, transparent, and consistent**, producing reliable and broadly
  understandable rankings across the whole inventory rather than opaque per-model scores.
- **It weights inherent risk over residual risk.** Tiering a model by how risky it would
  be unmitigated ensures the review depth tracks the true stakes, not just what a strong
  control environment currently hides.
- **It supports correlation across the portfolio**, so teams can see how risk clusters by
  business unit, entity, or model family.

A concrete tiering rule set is embedded in the [model-risk-assessment.md](../templates/model-risk-assessment.md) template, and the
`use-case-risk-tier.py` script automates a defensible default classification from
structured inputs. Regardless of the exact weights, the tiering decision should be recorded
for each model so the reasoning is auditable and consistent over time.

## The Risk Register

A risk register is the central ledger of identified AI risks and the actions against them.
Each entry captures the risk, its source, who owns it, its likelihood and impact (and hence
a score), the mitigating controls in place, the residual score after those controls, and
the status of any planned response. A register is only useful if it is:

- **Owned.** Someone is accountable for keeping it current and for acting on its findings.
- **Reviewed.** It is revisited on a cadence and whenever a significant model or use case
  is added, so it reflects the live portfolio rather than a one-time snapshot.
- **Linked to escalation.** Entries above a threshold route into the tiered review and the
  governance council described in `governance-operating-model.md`.

*Responsible AI in the Enterprise* describes risk registers, heat maps, and scoring
models as complementary instruments for ranking risks by their likelihood and impact, so
that mitigation effort lands where it counts. The use-case intake flow
([use-case-intake-form.md](../templates/use-case-intake-form.md)) feeds new risks into the register, and the register in turn
informs board reporting on the aggregate risk profile.

## The NIST AI Risk Management Framework

> **Verify against current NIST releases at use time.** The AI RMF is under active
> revision and sector profiles (for example, critical-infrastructure and cybersecurity
> guidance) are still being added, so the exact structure, category lists, and profile set
> below may have moved by the time you read this. Confirm the current version, its
> Generative AI Profile, and any relevant sector profile on the NIST site before applying
> them.

The NIST AI RMF is a voluntary, consensus-built framework (first released January 2023)
that organizes AI risk management into four functions. As *Responsible AI in the
Enterprise* explains, it is deliberately risk-based, resource-efficient, and innovation
friendly: it offers a shared vocabulary, taxonomy, and set of target outcomes instead of
forcing uniform one-size-fits-all rules, and it is built to layer onto an organization's
existing risk management rather than stand in for it. The four functions are:

- **Govern** — the cross-cutting governance function that informs and runs through the
  other three: cultivating a risk-management culture, establishing roles and
  responsibilities, and embedding accountability into AI design, development, and use.
- **Map** — establishing the context of the AI system: who the actors are, what the system
  is for, and where its risks and benefits arise across the life cycle.
- **Measure** — identifying and applying metrics to evaluate and assess risks, including
  performance, trustworthiness, and bias-related measures.
- **Manage** — prioritizing, responding to, and mitigating the measured risks, then
  monitoring outcomes and feeding back into the system.

Each function breaks down into categories and subcategories that resolve into concrete
actions and outcomes. NIST stresses that the functions should be carried out continuously
and throughout the AI life cycle, incorporating multidisciplinary perspectives, rather
than as a one-time exercise.

The framework also calls out what makes AI risk *different* from traditional software risk
— a point the book emphasizes and that still shapes the field. AI-specific risks include
data representation and ground truth problems, harmful bias, data dependency, behavior
changes during training, detachment from training context, opacity, unpredictable failure
modes, and the attack surface introduced by pretrained models and third-party components.
Because privacy and cybersecurity frameworks (such as the NIST Cybersecurity and Privacy
frameworks) do not fully cover these, the AI RMF exists to close that gap.

NIST has since issued a companion document, the Generative AI Profile (labeled
NIST-AI-600-1, July 2024), which maps the four functions onto the distinctive hazards of
generative models: the ease of fabricating plausible synthetic output, confabulation,
degraded information integrity, and fresh avenues for misuse. As of the mission's research
(mid-2026), NIST continues to add sector profiles and revise the RMF, which is why this
section must be re-verified against current NIST releases.

## ISO/IEC 42001 and ISO/IEC 23894

On the international standards side, the key pair is ISO/IEC 42001 and ISO/IEC 23894,
both published under the JTC 1/SC 42 joint technical committee. They play complementary
roles, as the mission's research makes clear.

- **ISO/IEC 42001:2023** is the world's first management-system standard centered on
  artificial intelligence. It lays out the requirements for setting up, running, and
  steadily refining an AI Management System (AIMS) inside any organization that builds,
  sells, or relies on AI-based products or services. Following the Plan-Do-Check-Act cycle
  common to management-system standards, it hands the organization a disciplined mechanism
  for defining AI policies and targets, handling risks and opportunities, and showing
  responsible use — all while keeping innovation and control in balance. It has become a
  leading certification target precisely because it is a complete management system rather
  than just guidance.
- **ISO/IEC 23894:2023** is a guidance document rather than a full management-system
  standard. It advises organizations on handling AI-specific risk and on weaving risk
  management into their AI activities, and it can be tailored to any organization and
  situation.

In practice the two work in tandem: an organization can apply 23894 to understand and
evaluate AI-specific risk, then fold that discipline into the larger management system that
42001 calls for. Where NIST supplies a widely used risk structure in plain language, ISO
42001 supplies a certifiable management system and 23894 supplies the risk process that
feeds it. Many organizations map the two families onto one another — aligning NIST's
four functions with the risk process of ISO 23894 and meeting both through an ISO 42001-style
system — rather than treating them as competing choices.

## Choosing and Combining Frameworks

The frameworks are voluntary and overlapping, not a single compliance target, so the
mission's research recommends adopting a framework-based approach rather than chasing any
one document. A workable posture is:

- **Choose a primary structure** — commonly the NIST AI RMF or ISO/IEC 42001 — as the spine
  of the program.
- **Layer a risk standard** such as ISO/IEC 23894 for the detailed risk-management process.
- **Use principles** (such as the OECD principles) as the values baseline that keeps the
  structure honest.
- **Track the updates**, because the standards themselves are still stabilizing — the AI
  RMF is under revision and ISO is steadily expanding the AI standard family.

Because the frameworks are living documents, the risk-management program should re-verify
its assumptions on a schedule rather than treating a framework version as fixed. This is
also why the reference files and templates in this skill describe the *discipline* (what to
assess, how to score, how to record) in a way that survives changes to any single
framework's category list.

## Running Risk Management with the Rest of the Skill

Structured risk management is the connective tissue of the whole skill:

- **Principles** set what the organization values; risk management makes the consequences
  of those values measurable (`foundations-and-principles.md`).
- **The operating model** assigns who owns and escalates each risk
  (`governance-operating-model.md`).
- **The life cycle** gates when risk assessments must be refreshed, from ideation through
  retirement (`ai-lifecycle-governance.md`).
- **Fairness, transparency, privacy, and security** each contribute concrete risk measures
  and mitigations (`fairness-bias-accountability.md`, `transparency-and-explainability.md`,
  `privacy-and-data-governance.md`, `llm-and-agent-security.md`).
- **The board and third-party tiers** consume the aggregate risk picture for oversight
  (`procurement-third-party-and-board-oversight.md`).

Use the [model-risk-assessment.md](../templates/model-risk-assessment.md) template to run a NIST-aligned assessment and tiering on
a single model, the [use-case-intake-form.md](../templates/use-case-intake-form.md) template to route a new use case into the
register, and the `use-case-risk-tier.py` script to compute a defensible default tier from
structured inputs. Record the resulting inherent and residual scores in the register and
feed the aggregate to the governance council and board.

## Where to Go Next

- **`foundations-and-principles.md`** — why risk management is the substrate of governance.
- **`governance-operating-model.md`** — the roles and councils that own and escalate risk.
- **`ai-lifecycle-governance.md`** — the stage gates where risk assessments are refreshed.
- **`procurement-third-party-and-board-oversight.md`** — how the aggregate risk picture
  reaches the board and governs procured models.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *Responsible AI in the Enterprise*
and *Platform and Model Design for Responsible AI*. Current context on the NIST AI RMF,
its Generative AI Profile, ISO/IEC 42001 and 23894, and the evolving standards landscape is
drawn from the mission research note research-standards.md. All prose is an original
paraphrase and synthesis of the ideas in these sources; idea-level attribution is
consolidated in `source-index.md`. Framework details must be verified against the
governing bodies' current releases at use time.

# Six-Level Governance Framework

The Six-Level Governance (6L-G) framework is a practical cross-cutting overlay for governing
Generative AI systems. It comes from *AI Governance* by Engin Bozdag and Stefano Bennati, and is
included here as an author-developed operating model, not as a NIST, ISO, or regulatory term. Use it
alongside the named frameworks in `risk-management-and-frameworks.md`, not as a replacement for them.

**Use this reference when:** an organization needs a repeatable governance loop across strategy,
risk assessment, implementation, launch, operations, and learning; when a review is becoming a
checklist with no evidence; or when a deployment posture or material change requires governance
controls to be re-applied.

## What 6L-G adds

`ai-lifecycle-governance.md` answers **when** a system moves through ideation, data, build,
evaluation, deployment, monitoring, and retirement. 6L-G answers **which governance discipline
must produce evidence** at each point and how that evidence feeds the next decision. The two views
are complementary:

| 6L-G level | Governing question | Typical lifecycle touchpoints |
|---|---|---|
| 1. Strategy & Policy | What is the organization willing and unwilling to do with AI? | Portfolio, policy, and intake |
| 2. Risk & Impact Assessment | Who could be affected, how, and with what residual risk? | Ideation, data, and material changes |
| 3. Implementation Review | Are the promised controls present in the design and build? | Data, build, procurement, and architecture review |
| 4. Acceptance Testing | Is there enough independent evidence to release this version? | Evaluation and go/no-go |
| 5. Operations & Monitoring | Is the live system behaving within its approved posture? | Deployment, monitoring, incidents, and vendor changes |
| 6. Learning & Improvement | What did evidence, incidents, and users teach us, and what changes now? | Postmortems, retraining, policy updates, and retirement |

Do not treat the levels as a one-time waterfall. A material incident, model update, new data class,
new region, new tool, or change in user population can send a live system back to Levels 2 through 4.
The trigger and resulting decision should be recorded in the system's governance history.

## The six levels

### Level 1: Strategy & Policy

Set the organization's direction before individual teams choose tools. Define the intended benefits,
prohibited uses, risk appetite, accountable executive, decision rights, and escalation path. Make
principles operational: for example, a human-oversight principle should identify which decisions
require review, what evidence the reviewer receives, and what happens when the reviewer disagrees.

**Evidence to retain:** approved AI strategy or principles, policy set, risk appetite, inventory
owner, role and decision-rights map, prohibited-use rules, exception process, and review cadence.

**Useful signals:** inventory coverage, policy training coverage, overdue policy reviews, exceptions
by risk tier, unresolved ownership gaps, and the percentage of material changes that triggered a
policy or risk review.

**Failure pattern:** a council or charter exists on paper, but teams can deploy unregistered AI,
policy language is not translated into controls, and no named person can accept an exception.

### Level 2: Risk & Impact Assessment

Identify affected people and groups, intended benefits, foreseeable harms, misuse paths, legal and
contractual constraints, and the controls that reduce inherent risk. Record likelihood and impact
separately from residual risk. Include affected-stakeholder input when the system can materially
affect people, and document uncertainty rather than converting unknowns into false precision.

**Evidence to retain:** use-case intake, stakeholder and data-flow map, impact assessment, risk
tier, risk register, control plan, residual-risk decision, owner, due date, and acceptance or
escalation record.

**Useful signals:** percentage of in-scope systems assessed before build, overdue high-risk
mitigations, residual-risk acceptance by role, time from identified risk to disposition, and
reassessment coverage after material change.

**Failure pattern:** a generic checklist produces a low-risk label without testing whether the
system has sensitive data, consequential actions, untrusted input, or a vulnerable population.

### Level 3: Implementation Review

Verify that the architecture can enforce the commitments made at Levels 1 and 2. Review data
collection and retention, model and dataset provenance, vendor and subprocessor boundaries,
identity and access, prompt and retrieval paths, tool permissions, output handling, monitoring,
rollback, and human-review mechanics. A control belongs in the design and build evidence, not only
in a policy or launch slide.

**Evidence to retain:** architecture and data-flow diagrams, threat model, model and data cards,
software or machine-learning bill of materials, vendor evidence, control-to-risk traceability,
configuration and prompt versions, test plan, and open-finding register with owners and service
levels.

**Useful signals:** review completion time, high-severity findings closed before testing, percentage
of controls verified in the running environment, threat-model coverage, and vendor evidence gaps.

**Failure pattern:** the organization approves privacy, security, or fairness commitments without
showing the implemented anonymization, authorization, logging, or fallback behavior.

### Level 4: Acceptance Testing

Require independent, scenario-based evidence before release. Test normal, boundary, adversarial,
and failure cases, including prompt injection, data leakage, hallucination, subgroup performance,
unsafe tool use, cost runaway, and recovery. Test the human interface as well as the model. A
reviewer who can only see a polished demo cannot provide meaningful oversight.

Record the exact model or model-service version, configuration, prompt and retrieval versions,
input data or fixture version, test harness, results, unresolved limitations, and go/no-go decision.
For high-impact actions, validate the approval path and rollback or kill-switch behavior, not just
the text response.

**Useful signals:** test coverage by risk, critical findings open at go/no-go, pass and failure
rates by subgroup or scenario, approval-path success rate, and time to resolve launch blockers.

**Failure pattern:** a launch email is called approval, red-team findings are accepted without a
risk owner, or a human is present only to rubber-stamp the model's answer.

### Level 5: Operations & Monitoring

Observe the live system against the approved posture. Capture enough structured evidence to
reconstruct what happened: request or run identifier, user or agent identity, model and
configuration version, relevant retrieval source identifiers, tools and parameters used, policy
decisions, approvals, output disposition, and downstream outcome. Redact or minimize sensitive
content in telemetry, and separate audit evidence from unrestricted model reasoning by default.

Monitor reliability, quality, fairness, privacy leakage, prompt injection, tool behavior, spend,
latency, vendor changes, drift, user complaints, and control failures. Define thresholds that
pause, degrade, revoke, or escalate the system. A dashboard without an owner, threshold, or
response path is not operational governance.

**Evidence to retain:** monitoring plan, structured traces, alert history, incident and exception
records, vendor-change log, periodic review, access review, and evidence that a control actually
worked in the live boundary.

**Useful signals:** incident detection and resolution time, recurring incidents, subgroup drift,
blocked or approved tool calls, unexpected data destinations, policy exceptions, leakage findings,
feedback response time, and cost or token anomalies.

**Failure pattern:** logs capture only the final answer, guardrails fail silently, vendor updates
change behavior without a review, or feedback is collected but never routed to an owner.

### Level 6: Learning & Improvement

Turn incidents, near misses, user feedback, evaluation results, and operational metrics into
corrective and preventive action. Distinguish correlation from causation, preserve the evidence
that supports the conclusion, and measure whether the change reduced the original failure mode.
Update prompts, tools, datasets, policies, training, risk assessments, and release gates together
when the evidence requires it.

**Evidence to retain:** postmortem, root-cause analysis, corrective-action record, regression test,
updated risk and impact assessment, policy or control change, owner and deadline, reapproval record,
and before-and-after outcome measure.

**Useful signals:** recurrence rate, corrective-action closure time, time from incident to regression
test, percentage of improvements with measured effect, feedback-to-change conversion, and the
number of material changes that were reapproved before release.

**Failure pattern:** an incident is patched locally, but the playbook, test suite, policy, and
related systems remain unchanged, allowing the same class of failure to recur.

## Maturity across the levels

Assess maturity per level and per system. Do not average away a critical weakness in monitoring or
privacy because strategy is strong.

| Maturity | Observable posture |
|---|---|
| **Ad hoc** | Governance is fragmented and reactive. Inventory, ownership, testing, and evidence are incomplete or depend on individual initiative. |
| **Baseline** | A repeatable process, roles, escalation paths, and minimum safeguards exist. Reviews happen consistently, but measurement and remediation may be fragile. |
| **Managed** | Reviews, controls, exceptions, and findings produce versioned evidence. Metrics have owners, high-risk fixes are verified, and decisions are auditable. |
| **Proactive** | Operational signals can trigger reassessment, policy or control changes, and regression testing. The loop adapts without waiting for a public incident. |

The target is proportional. A low-impact internal experiment may need baseline safeguards, while a
system handling sensitive data or consequential actions may need managed or proactive controls in
specific levels. The rationale, residual risk, and sustainability of the chosen posture belong in
the decision record. Privacy, security, fairness, and accountability are not optional merely
because a system is early-stage.

## Operating rules for an evidence-backed loop

1. **Assign one accountable owner per decision.** Contributors can share work, but risk acceptance,
   exceptions, and release decisions need a named role.
2. **Attach evidence to the decision.** Link the exact artifact, version, test fixture, result, and
   reviewer. Never treat a policy, tool verdict, or vendor assertion as proof of the live boundary
   without checking the boundary itself.
3. **Keep exceptions bounded.** Record the reason, affected risk, compensating control, approver,
   expiration date, and re-review trigger. An unexpired exception is not a permanent tier change.
4. **Trigger reassessment on change.** At minimum, reassess after a model or vendor change, new
   data class, new tool or destination, new geography or user population, new action capability,
   material drift, incident, or changed external obligation.
5. **Measure outcomes, not activity alone.** Counts of reviews and policies are leading indicators;
   recurrence, leakage, subgroup harm, recovery, and user contestation show whether controls work.
6. **Close the loop deliberately.** A finding is not closed when a ticket changes state. Verify the
   fix against the failure mode in the live or representative environment, add a regression test
   where practical, and record residual uncertainty.

## Deployment-posture overlay

The same six levels apply across postures, but the control center of gravity changes. Classify the
posture before assigning control owners:

| Posture | Organization controls directly | Organization must obtain or contract for | Primary governance risk |
|---|---|---|---|
| **SaaS or application consumer** | User policy, identity, approved features, local DLP or review, data selection, and vendor administration exposed by the product | Vendor retention, training use, tenant isolation, logs, incident response, model changes, deletion, and subprocessor behavior | Vendor opacity can make an approved control impossible to verify |
| **API integrator** | User interface, pre- and post-processing, redaction, RAG permissions, application logs, routing, budgets, and action controls | Model behavior, provider-side logs, availability, subprocessors, data use, and service changes | Responsibility is split across two control planes, so either side can bypass the intended policy |
| **Model hoster** | Model and dataset provenance, serving stack, guardrails, logs, data residency, access, change, and incident controls | Hosting or infrastructure dependencies, upstream artifacts, and license or support commitments | Maximum control also means maximum operational and assurance burden |
| **Agentic overlay** | Agent identity, allowed tools, purpose and data scopes, memory, plans, approvals, sandbox, egress, traces, and kill switch | Each connected tool or agent's security, retention, availability, and downstream use | Planning, memory, and tool chaining can create unanticipated data flows and actions |

Do not infer risk from posture alone. A hardened, narrowly scoped agent can be safer than a simple
chatbot on an insecure foundation, and a vendor's certification does not transfer accountability
for choices the organization controls. For a concrete posture and agent review, use
`../templates/agentic-governance-review.md`.

## Deliverable package

A defensible 6L-G review normally leaves behind:

- a system inventory record and accountable owner;
- the approved strategy, policy, risk appetite, and prohibited-use rules;
- a risk and impact assessment with residual-risk decisions;
- architecture, data-flow, threat, vendor, and control-traceability evidence;
- acceptance tests, limitations, launch decision, and rollback or escalation plan;
- monitoring, incident, exception, and change records; and
- a learning record that shows what changed and how the effect was measured.

If a level does not apply, record why and what compensating evidence covers the same risk. Do not
silently skip a level.

## Where to go next

- **`ai-lifecycle-governance.md`**: place these governance disciplines into lifecycle stage gates.
- **`governance-operating-model.md`**: assign councils, stewards, decision rights, and RACI roles.
- **`risk-management-and-frameworks.md`**: translate risk and impact findings into NIST or ISO-aligned artifacts.
- **`llm-and-agent-security.md`**: assess the exposure ladder, tool authorization, containment, and agent identity.
- **`privacy-and-data-governance.md`**: assess the four GenAI privacy pillars, purpose-aware egress, and memory retention.
- **`../templates/agentic-governance-review.md`**: complete the posture, capability, tool, privacy, and evidence worksheet.

---

### Synthesized from

This reference is an original synthesis of the 6L-G framework and maturity model in *AI Governance*
by Engin Bozdag and Stefano Bennati, with terminology and control mappings cross-checked against the
skill's existing lifecycle, risk-management, operating-model, security, privacy, and current
research references. The 6L-G label is attributed to the book and is not presented as an
established external standard. Current legal, regulatory, security, and standards claims must be
re-verified against primary sources at use time. See `source-index.md` for the complete provenance
record.

# AI Governance: Source Index and Provenance

This index records the provenance of every reference in the `ai-governance` skill. It
exists so that anyone can trace any idea back to its informing sources: the twelve
copyrighted books in the original ebook library, the separately reviewed *AI Governance*
Manning MEAP, and the seven current research notes that de-stale those books. Readers should
also be able to see how the material was created and where current claims still require
re-verification.

## Attribution invariant: paraphrase and synthesis at the idea level

Every sentence in every reference of this skill that describes a book is an **original
paraphrase or synthesis of ideas** — never a verbatim reproduction. Several of the source
books carry explicit "all rights reserved, no part may be reproduced" notices, so the
skill authors deliberately did **not** copy book prose, contiguous passages, or near-verbatim
sentences. Instead, each reference:

- **Paraphrases** the underlying idea in fresh words, or
- **Synthesizes** ideas drawn from multiple books and the current research notes into new,
  original framing that belongs to this skill.

This is **idea-level attribution**: we attribute the origin of an *idea* (e.g., "governance
must span the full model lifecycle") to the book or research note that informed it, without
claiming to reproduce the source's wording. Where a book has aged past its publication
date — most visibly in the regulatory and standards areas — the reference is de-staled
against a current research note, and the note (not the book) is treated as authoritative on
the current state. Research notes may be summarized and cited, but large verbatim blocks
are not pasted into any reference.

Nothing in this skill is legal, financial, or security advice. Regulatory, standards, and
security material should be re-verified against primary sources at the time of use.

## The thirteen reference files

The `ai-governance` skill ships thirteen reference files under `references/`. The table below
names all thirteen and maps each to its informing sources: book short names (see the
bibliography) and the mission research notes (see below). The book short names are the
canonical identifiers used consistently across the skill.

| Reference file | Focus | Informing sources |
|---|---|---|
| `foundations-and-principles.md` | What AI governance is; the core principles (fairness, accountability, transparency, privacy, safety, human oversight); governance vs. compliance vs. risk | Responsible AI in the Enterprise; Introduction to Responsible AI; Beyond the Algorithm; Responsible AI: Best Practices; research-standards.md; research-org-board-governance.md |
| `governance-operating-model.md` | Six-step operating model, councils, stewards, decision rights, RACI, federated vs. centralized, maturity, culture, charter | Designing Data Governance from the Ground Up; Data Governance Handbook; research-org-board-governance.md |
| `risk-management-and-frameworks.md` | NIST AI RMF (govern/map/measure/manage), ISO/IEC 42001 & 23894, model-risk tiering, risk registers, inherent vs. residual | Responsible AI in the Enterprise; Platform and Model Design for Responsible AI; research-standards.md |
| `ai-lifecycle-governance.md` | Stage gates across ideation, data, build, evaluate, deploy, monitor, retire; model inventory, lineage, drift, incident response | Platform and Model Design for Responsible AI; Designing Data Governance from the Ground Up; Data Governance Handbook; research-technical-controls.md |
| `six-level-governance-framework.md` | Cross-cutting 6L-G levels, evidence loop, maturity, deployment postures, and proportionality | AI Governance; research-standards.md; existing lifecycle, operating-model, security, and privacy references |
| `fairness-bias-accountability.md` | Fairness metrics and their limits, algorithmic justice, bias sources, trade-offs, accountability, model cards | AI Fairness; Introduction to Responsible AI; Responsible AI: Best Practices; research-technical-controls.md |
| `transparency-and-explainability.md` | Explainability (XAI) methods, when explanation is required, disclosure, human-AI interaction, auditability | Responsible AI in the Enterprise; Platform and Model Design for Responsible AI; Introduction to Responsible AI |
| `privacy-and-data-governance.md` | Training and operational data governance, ownership, lineage, quality, consent, minimization, retention, privacy-enhancing techniques, agentic memory and egress | Designing Data Governance from the Ground Up; Data Governance Handbook; Platform and Model Design for Responsible AI; AI Governance; research-regulatory.md; research-technical-controls.md |
| `llm-and-agent-security.md` | Trust boundaries, prompt injection, exposure ladder, data exposure, hallucination, excessive agency, tool authorization, containment, denial of service, supply chain, red-teaming | The Developer's Playbook for LLM Security; Beyond the Algorithm; AI Governance; research-llm-agent-security.md |
| `regulatory-landscape.md` | Current law by jurisdiction, compliance mapping, horizon scanning, enforcement | research-regulatory.md (authoritative); book regulatory chapters of Responsible AI in the Enterprise and Beyond the Algorithm as historical context only |
| `procurement-third-party-and-board-oversight.md` | Vendor and model due diligence, supply chain, board reporting, metrics, audit | The AI Product Manager's Handbook; Developing Cybersecurity Programs and Policies; research-org-board-governance.md |
| `gxp-and-data-integrity.md` | GxP AI governance, ALCOA+, data integrity, electronic records, audit trails, risk-based assurance, QMS interfaces | FDA, MHRA, PIC/S, WHO, EMA, ICH, ISPE; gxp-ai-governance-brief.md; gxp-ai-governance-log.md |
| `source-index.md` | This file: provenance, attribution, bibliography | All thirteen source books; all seven research notes; GxP research artifacts (meta) |

## Research notes

Seven research notes, produced during the research foundation milestone, de-stale the books
against where AI governance stands today. They live in this skill's research directory and are cited by
short filename throughout the skill. All seven are used, with the two GxP artifacts informing the GxP reference:

- research-regulatory.md — current laws and enforcement across jurisdictions (EU AI Act,
  GDPR, US federal/state, UK, China, sectoral rules), the authoritative basis for
  `regulatory-landscape.md` and a de-staling input to `privacy-and-data-governance.md`.
- research-standards.md — NIST AI RMF and its Generative AI Profile, ISO/IEC 42001 and
  23894, IEEE and industry frameworks; informs `risk-management-and-frameworks.md` and
  `foundations-and-principles.md`.
- research-llm-agent-security.md — OWASP LLM Top 10, agentic risk, prompt-injection
  defenses, red-teaming, AI safety institutes; informs `llm-and-agent-security.md`.
- research-technical-controls.md — model and data cards, model inventories, monitoring,
  drift, audit trails, MLOps governance tooling; informs `ai-lifecycle-governance.md`,
  `fairness-bias-accountability.md`, and `privacy-and-data-governance.md`.
- research-org-board-governance.md — roles, governance councils, the Chief AI Officer,
  fiduciary duty, maturity models, third-party risk; informs `foundations-and-principles.md`,
  `governance-operating-model.md`, and `procurement-third-party-and-board-oversight.md`.

- gxp-ai-governance-brief.md and gxp-ai-governance-log.md — GxP, ALCOA+, data integrity, electronic records, validation/assurance, and QMS interfaces; inform `gxp-and-data-integrity.md`.

## Bibliography — the twelve original source books

These are the twelve titles originally harvested for the skill, drawn from the mission's
read-only ebook library. All content derived from them is paraphrased and synthesized at the idea
level, as described above; no passages are reproduced. The books are listed by the short
name used across the skill.

### Primary (five)

1. **Responsible AI in the Enterprise** — the central operating book on governance, model
   risk, audit, compliance, the NIST AI RMF, maturity, and starter kit.
2. **Responsible AI: Best Practices for Creating Trustworthy AI Systems** — a pattern
   catalogue feeding the skill's control library, review checklists, and playbook.
3. **Designing Data Governance from the Ground Up** — the operating-model book: the six-step
   model, stewards, council, decision rights, roadmap, and lifecycle.
4. **Platform and Model Design for Responsible AI** — governance-as-architecture: risk
   assessment, privacy pipelines, MLOps, inventories, tiering, and lifecycle.
5. **The Developer's Playbook for LLM Security** — the LLM/agent security and
   deployment-control lane.

### Supporting (seven)

6. **AI Fairness** — conceptual and ethical depth: algorithmic justice, fairness metrics and
   their limits, and trade-offs.
7. **Beyond the Algorithm** — a landscape primer connecting governance to security, privacy,
   ethics, and law.
8. **Developing Cybersecurity Programs and Policies in an AI-Driven World** — governance
   infrastructure: policy, risk management, supply chain, privacy, and an AI-governance
   chapter.
9. **Data Governance Handbook** — data ownership, quality, lineage, and operations, with a
   regulated-financial-institution case study.
10. **Introduction to Responsible AI** — an onboarding primer on bias, transparency,
    privacy, and robustness.
11. **The AI Product Manager's Handbook** — the product-governance interface: lifecycle,
    third parties, metrics, and ethics.
12. **Practical Cybersecurity Architecture** — the control-environment architecture that
    surrounds AI systems.

### Additional source reviewed for this enrichment

13. **AI Governance** — Engin Bozdag and Stefano Bennati, Manning Publications Co., MEAP,
    ISBN 9781633436817. The attached manuscript metadata identifies dates of 2025-11-07 and
    2026-04-15. Chapters 2 through 5 were reviewed on 2026-09-14. The book informs the
    author-developed 6L-G framework, deployment-posture boundaries, the six-factor exposure
    ladder, agentic tool and memory controls, purpose-aware egress, and evidence-oriented
    maturity practices. Because this is an in-progress MEAP, its legal, regulatory, security,
    vendor, and incident claims are not treated as current authority; re-verify those claims
    against primary sources before use.

## How to read this index

- To find where a book's ideas appear, scan the "Informing sources" column of the reference
  table for that book short name.
- To find which research note governs the current state of a topic, scan for the `research-*`
  filename.
- To understand why a book is sometimes treated as context rather than authority (e.g., the
  regulatory chapter of Responsible AI in the Enterprise), see the attribution invariant above:
  where obligations have moved past a book's publication date, the current research note is
  authoritative and the book is cited as historical context.

Each domain reference carries its own short "Synthesized from" footer repeating the
source-to-file mapping for that reference; this index is the consolidated, complete
record of attribution across the whole skill.

# Transparency and Explainability

This reference teaches how to make AI systems transparent and explainable, and why that work
matters for governance. Its central message is that transparency and explainability are distinct
but complementary properties: transparency is about making a system's workings and data practices
open and knowable, while explainability is about rendering individual decisions in terms people can
grasp. Both are governed judgments, not a single tool. This reference synthesizes (never
reproduces) the ideas in *Responsible AI in the Enterprise*, *Platform and Model Design for
Responsible AI*, and *Introduction to Responsible AI*. Read it with
`foundations-and-principles.md` (which names transparency as a governing principle),
`fairness-bias-accountability.md` (the accountability half that transparency enables), and
`ai-lifecycle-governance.md` (where explanation and audit surface at stage gates). Use the
[model-card.md](../templates/model-card.md) and [model-risk-assessment.md](../templates/model-risk-assessment.md) templates to record the explanation and disclosure
decisions this reference describes. It is not legal advice, and disclosure duties should be
confirmed against current regulation at use time.

## Transparency and Explainability Are Not the Same Thing

A useful starting point is to separate two frequently conflated ideas. *Introduction to Responsible
AI* frames transparency as the openness and comprehensibility of the overall decision-making
process — making visible what a system is for, what data it relies on, and how it reaches its
outcomes. Explainability, in the same telling, is the narrower act of illuminating the rationale
behind a *particular* decision in terms a person can follow. Transparency answers "how does this
system behave in general?", while explainability answers "why did it decide this case the way it
did?"

| Property | Focus | Example question it answers |
|---|---|---|
| Transparency | The system, its data, and its process are knowable and open | What data feeds this model and who owns it? |
| Explainability | A specific prediction is rendered in human-understandable terms | Why was this loan application declined? |

The two reinforce each other: a system that is opaque cannot be meaningfully explained case by case,
and a system that can explain its decisions is far easier to audit and hold accountable. But they
place different demands on a governance program, and treating them as synonyms leads teams to buy
one tool and declare both problems solved.

## Why Explainability Is Required

Explainability is not an optional polish; it is a governance requirement in proportion to the stakes
of a decision. *Platform and Model Design for Responsible AI* motivates XAI through the decisions
where a wrong call materially harms someone — a model predicting whether a patient will develop a
terminal illness, a system advising on a trial, a lender deciding a credit application — and
contrasts these with low-stakes recommendation systems where opacity is tolerable. The governing
principle is proportionality: the more consequential the outcome and the more autonomy the system
is given, the stronger the expectation that it can account for its reasoning.

Beyond the moral case, explainability is demanded by three practical forces that the books
converge on:

- **Trust and adoption.** *Introduction to Responsible AI* argues that people are reluctant to rely
  on, challenge, or engage with systems whose reasoning they cannot see. When an outcome feels like
  an unexplained verdict, confidence collapses and adoption stalls.
- **Accountability and redress.** A decision that can be explained can be challenged, reviewed, and
  corrected. *Responsible AI in the Enterprise* ties explainability directly to auditability and to
  the ability to trace back to the components that produced a failure.
- **Regulatory and audit pressure.** In regulated sectors such as finance and healthcare, an
  organization may need to demonstrate to a regulator that a model meets the applicable standard
  for transparency. *Responsible AI in the Enterprise* stresses that auditability is especially
  important where proving compliance is a standing obligation.

## The Black-Box Problem and Model Risk

The reason explainability is hard is that the most powerful models are opaque. Neural networks and
other complex learners are frequently described as black boxes because their internal operations do
not map cleanly onto human reasoning; even their developers may not be able to say exactly why a
given output emerged. *Platform and Model Design for Responsible AI* and *Responsible AI in the
Enterprise* both treat this opacity as a source of *explainability risk* — a risk category in its
own right, distinct from accuracy or privacy, that must be identified, assessed, and managed like
any other.

Opacity harms in concrete ways. Unexplained results can be traced to a model performing poorly for a
particular customer segment or during an unusual period, but without explanation the cause is
invisible and stakeholder fear grows. A business that cannot explain its model's decisions is
reluctant to deploy it, and a customer or regulator who cannot see the reasoning cannot trust or
challenge it. This is why model risk assessments treat interpretability as a reviewable dimension
and why organizations are pushed toward models that are transparent and explainable rather than
merely accurate.

## The XAI Toolbox: A Taxonomy

Explainable AI (XAI) is the field of methods that let practitioners understand and interpret a
model's predictions. *Responsible AI in the Enterprise* organizes the landscape along four axes,
which together help a team choose a technique:

- **By scope — local vs. global.** Local explanations describe an individual prediction; global
  explanations describe how the model behaves across the range of its inputs.
- **By method — model-specific vs. model-agnostic.** Model-specific techniques require knowledge of
  the internal architecture; model-agnostic techniques can explain any model as a black box.
- **By timing — intrinsic vs. post hoc.** Intrinsic (or ad hoc) models can be understood from their
  own structure; post hoc explanations require external analysis of an already-built model.
- **By outcome — feature summary, visualization, learned weights, or approximation.** Different
  techniques yield different kinds of output, from human-read tables to graphical maps.

*Platform and Model Design for Responsible AI* uses a closely related frame — scope (local/global)
and model relationship (specific/agnostic) — confirming that these axes are the field's working
vocabulary. An intrinsically interpretable model such as a decision tree or a simple rule system
needs little external machinery to be understood, whereas a deep network almost always requires post
hoc techniques.

### The Interpretability Ordering

Model choice constrains how much explainability is available for free. *Platform and Model Design
for Responsible AI* lays out a rough ordering of models by ease of interpretation, from the most
interpretable to the least: linear models, generalized additive models, decision trees, support
vector machines, random forests, and finally neural networks. This ordering is the practical
backbone of the accuracy–interpretability trade-off: the most flexible models tend to be the least
transparent, so an organization that values explanation may need to accept a simpler model or
invest in post hoc methods.

## Concrete XAI Techniques

The books detail a set of widely used techniques. The table below summarizes the most important and
how they are best used; none is a universal answer, and each has documented limitations.

| Technique | What it produces | Typical use | Notable limit |
|---|---|---|---|
| SHAP (SHapley Additive exPlanations) | Per-feature credit, drawn from game theory, for a prediction; accounts for feature interactions | Explaining individual predictions; comparing models | Assumes feature independence; can be sensitive to outliers |
| LIME (Local Interpretable Model-Agnostic Explanations) | A local surrogate model explaining one prediction in a small region around the input | Explaining a single case without touching the model | Local only; may miss global complexity |
| Counterfactual explanation | "If X had been different, the outcome would have changed" | Showing users what would flip a decision | Needs a defensible notion of a reachable alternative |
| Saliency maps / feature attribution | Highlights which input regions (e.g., pixels) most drove the outcome | Image, video, and gradient-based models | Focus on activated regions, not global feature importance |
| Global / local surrogate models | A simpler model trained to approximate the black box | Summarizing overall or local behavior | The surrogate is an approximation, not the model |

*Responsible AI in the Enterprise* devotes particular attention to SHAP, explaining that it extends
the idea of feature importance by accounting for how features interact, which makes its attributions
more reliable in complex models. *Platform and Model Design for Responsible AI* explains LIME as a
model-agnostic approach that perturbs the input, queries the model on the variations, and fits a
simple linear surrogate in the local neighborhood — yielding, for example, a ranked list of the
words that most influenced a spam classification. Both books also discuss surrogate and
distillation-style techniques that approximate a black box with a transparent one.

Beyond explanation, the books point to techniques aimed at *understanding*: causal inference tools
(such as DoWhy and CausalNex) that go beyond correlation to reason about which features cause an
outcome, and ELI5-style plain-language explanations that render a model's reasoning in terms a
general audience can read. For governance purposes the aim is not the most sophisticated method but
the one whose output a decision-maker can actually use.

## The Accuracy–Interpretability Trade-off

Explainability is rarely free. *Platform and Model Design for Responsible AI* is explicit that the
search for a good model is a search for the right balance between accuracy and interpretability,
alongside robustness and other properties. Enforcing interpretability can cap the complexity of a
model and therefore its raw predictive power, and a model chosen purely for accuracy may be so
opaque that it cannot be trusted or defended. The discipline this reference recommends is to treat
the trade-off as a conscious, documented decision rather than an accident of tooling — the chosen
balance, and the reasoning behind it, belongs in the model's documentation where it becomes
auditable.

## Disclosure and Communication

Explanation is only half the story; the other half is *disclosure* — actually communicating the
system's capabilities, limits, and data practices to the people who need to know. *Platform and
Model Design for Responsible AI* highlights guidance from bodies such as NASSCOM emphasizing that
transparency is operationalized through dashboards, visualization tools, internal audits, and
proactive communication about privacy and capability limits to users. The point is that
transparency is a communication practice, not merely a technical artifact: stakeholders — senior
leaders, legal, data scientists, and users — need a channel through which the model's behavior and
limitations are made visible and contestable.

*Introduction to Responsible AI* adds that transparent communication about what an AI system can and
cannot do is what lets people make informed decisions and challenge outcomes they believe are wrong.
Disclosure should also be candid about trade-offs, including the tension between revealing enough to
be transparent and protecting legitimate proprietary interests — a balance the book flags as a real
challenge, not a formality.

## Human–AI Interaction and Contestability

Explainability exists to serve humans, so it has to be designed with the interaction in mind. The
books connect explanation to several forms of human oversight:

- **Layered human oversight.** *Platform and Model Design for Responsible AI* describes three
  oversight modes — a person embedded in the loop of a decision, a person keeping watch from outside
  the loop, and a person holding final command — each of which keeps a human meaningfully involved.
  Explanations are what make these roles meaningful: a human reviewer cannot oversee a decision they
  cannot understand.
- **Contestability.** When an AI outcome significantly affects a person or group, there should be a
  timely way to audit and challenge the use or outcome of the system. *Introduction to Responsible
  AI* frames transparency and explainability as what empowers individuals to question results that
  seem unfair or discriminatory.
- **Decision understanding as a design target.** *Platform and Model Design for Responsible AI*
  treats "decision understanding" as one of the three pillars of a reliable XAI system (alongside
  prediction accuracy and traceability), delivered to end users as dashboards and plain-language
  factors. Without understanding, trust in the system is undermined and the system risks rejection.

The practical takeaway is that an explanation is only useful if the intended audience can act on it.
A governance review should ask not just "does an explanation exist?" but "can the person facing this
decision understand it and act on it?"

## Auditability

Explainability and transparency ultimately support the ability to *audit* — to reconstruct, after
the fact, what a system did and why. *Responsible AI in the Enterprise* is explicit that a safe
system must be auditable, meaning its internal state at decision time is transparent enough to
verify, and that this is especially important in fields such as healthcare and finance, where being
able to prove compliance may be a standing obligation. The book describes auditability as resting on production
traceability — the availability of immutable snapshots of models, together with their source code,
metadata, and associated artifacts, so that a failure can be traced back to its components for
root-cause analysis. *Platform and Model Design for Responsible AI* reinforces this by tying
auditability to reproducibility and to monitoring that keeps a model's behavior verifiable over its
lifecycle.

For governance purposes, auditability converts "we explained it" into "we can prove what happened
and why, long after the fact." This is why explanation and disclosure decisions should be recorded
durably — in model cards and risk assessments — rather than left to memory. A complete audit
trail captures the model version, the data and parameters, the explanation method used, and the
reasoning behind the choices made.

## Risks and Limits of Explanations

Explanation methods are not automatically trustworthy; they have failure modes a governance program
must account for. *Platform and Model Design for Responsible AI* documents a *scaffolding attack* in
which an adversary deliberately crafts post hoc explanations that look fair and unbiased even though
the underlying classifier remains biased. Because the explanation conceals the discrimination,
customers, regulators, and auditors relying on the output can be misled before making consequential
decisions such as parole, bail, or credit. This is a pointed warning that an explanation is evidence
to be examined, not a certificate of fairness.

More generally, the books note that every technique has limits: LIME only explains locally; SHAP
assumes feature independence and can be swayed by outliers; surrogate models approximate rather than
reproduce the true model. Governance therefore should not treat "we ran SHAP" as equivalent to "we
understand the model." Explanations should be sanity-checked, and the auditability layer should
preserve the raw model and data so that claims made through any explanation tool can be independently
verified.

## How to Wire Transparency and Explainability into the Skill

Transparency and explainability thread through every other governance surface:

- **Principles** frame transparency and human oversight as values the organization commits to
  (`foundations-and-principles.md`).
- **Risk frameworks** host explainability risk alongside accuracy, robustness, and fairness, and
  route opaque, high-stakes models through the same tiering and register as any other risk
  (`risk-management-and-frameworks.md`).
- **Lifecycle gates** schedule when explanation and audit evidence are produced — from the model
  choice and development phase, through pre-deployment validation, to post-deployment monitoring
  that re-verifies explanations (`ai-lifecycle-governance.md`).
- **Fairness and accountability** consume explanations to make bias review and answerability
  concrete (`fairness-bias-accountability.md`).
- **Board and third-party oversight** review the disclosure and audit findings as part of the
  aggregate risk picture (`procurement-third-party-and-board-oversight.md`).

Use the [model-card.md](../templates/model-card.md) template to record intended use, data, the explanation method chosen and
why, the accuracy–interpretability balance, and limitations; the [model-risk-assessment.md](../templates/model-risk-assessment.md) template
to weigh explainability against the other risk dimensions; and the [use-case-intake-form.md](../templates/use-case-intake-form.md) template
to flag high-stakes, explanation-sensitive use cases early.

## Where to Go Next

- **`foundations-and-principles.md`** — transparency and human oversight as governing principles.
- **`fairness-bias-accountability.md`** — the accountability that transparency and explanation
  enable.
- **`ai-lifecycle-governance.md`** — where explanation, audit, and monitoring happen across the
  lifecycle.
- **`risk-management-and-frameworks.md`** — hosting explainability as a managed risk dimension.

---

### Synthesized from

This reference synthesizes (never reproduces) ideas from *Responsible AI in the Enterprise*,
*Platform and Model Design for Responsible AI*, and *Introduction to Responsible AI*. All prose is an
original paraphrase and synthesis of the ideas in these sources; idea-level attribution is
consolidated in `source-index.md`. The discussion of disclosure duties and audit expectations is
educational context, not legal advice, and specific requirements should be confirmed against current
regulation at use time.

# GxP AI Governance Research Brief

**Research question:** What must the `ai-governance` skill add to support a job requirement for AI governance aligned with GxP, ALCOA+, and data integrity?

**Decision context:** Determine whether the skill fully covers the job requirement and identify a bounded, reusable extension without turning the skill into legal advice or a complete pharmaceutical quality-system manual.

**Scope:** Governance of AI systems that create, transform, analyze, or influence GxP-relevant records, decisions, processes, or submissions. Includes data integrity, electronic records, validation/assurance, lifecycle controls, quality ownership, and AI-specific evidence. Excludes legal interpretation, detailed SOP authoring, site-specific validation protocols, and implementation of production security controls.

**Evidence standard:** Prefer current official regulator, inspectorate, and standards-body sources. Industry guidance is used for implementation context and is labeled accordingly. Regulatory claims require re-verification at use time.

## Executive summary

The existing skill covers the general governance architecture needed for GxP AI: accountable ownership, risk tiering, lifecycle gates, lineage, auditability, monitoring, vendor due diligence, and regulatory mapping. It does not yet cover the regulated operating vocabulary and evidence expected for GxP systems: ALCOA+ data-integrity attributes, computerized-system validation or risk-based assurance, electronic-record and electronic-signature controls, audit-trail governance, quality-unit oversight, deviation/CAPA/change control, periodic review, and the special validation and performance evidence needed when AI contributes to regulated decisions or processes.

The appropriate extension is a focused reference, not a claim that the skill makes a system compliant. That reference should add a GxP overlay to every AI lifecycle gate and require a documented boundary between general AI governance and the organization's QMS, CSV/CSA, data-integrity, privacy, and security owners.

## Key findings

### 1. GxP data integrity is a quality-system concern, not merely better data lineage

FDA's drug-CGMP data-integrity guidance frames data integrity within CGMP requirements. MHRA's GxP guidance covers compliant data-governance expectations across GLP, GCP, GMP, GDP, and pharmacovigilance. PIC/S PI 041-1 and WHO guidance make ALCOA+ attributes operational expectations for records and data, including audit trails and lifecycle controls.

**Implication for the skill:** Add an explicit ALCOA+ control review. Existing fields such as provenance, lineage, quality, and retention are necessary but do not ask whether records are attributable, legible, contemporaneous, original, accurate, complete, consistent, enduring, and available.

### 2. Electronic records and audit trails need explicit controls

21 CFR Part 11 requires controls for electronic records and signatures, including validation, record protection and retrieval, access limitation, and secure, computer-generated, time-stamped audit trails. WHO guidance calls for GxP-relevant audit trails to be enabled and periodically verified throughout the data life cycle.

**Implication for the skill:** A GxP deployment gate must ask whether the system creates or relies on regulated electronic records, whether audit trails capture create/modify/delete events, whether they are protected and reviewed, whether signatures are attributable and non-repudiable, and whether records remain retrievable for the retention period.

### 3. Validation/assurance and change control are missing

GAMP 5 Second Edition explicitly addresses risk-based compliant GxP computerized systems and adds AI/ML, cloud, open-source, and data-integrity considerations. FDA and ICH materials likewise connect computerized systems, validation, quality risk management, and data integrity.

**Implication for the skill:** Add a risk-based validation/assurance decision at intake and build gates. For adaptive or probabilistic AI, the evidence plan must define the intended use, model/data version, acceptance criteria, performance envelope, change boundaries, revalidation triggers, and rollback or retirement path.

### 4. AI introduces evidence questions beyond ordinary computerized-system validation

EMA's reflection paper on AI in the medicinal-product lifecycle emphasizes data integrity for model development and generalizability of performance to the target population and context of use. FDA's 2025 draft guidance addresses AI used to produce information or data supporting regulatory decision-making for drugs and biological products. These are guidance materials, not a universal AI validation standard.

**Implication for the skill:** Add AI-specific evidence requirements: context-of-use statement, representative data and provenance, training/evaluation separation, performance by relevant subgroups and operating conditions, uncertainty and failure handling, human review, model/version traceability, monitoring for drift, and controls for model or data changes.

### 5. Governance must connect to the QMS instead of creating a parallel bureaucracy

The sources converge on accountability, risk management, documentation, auditability, and lifecycle control, but they do not imply that an AI council replaces the quality unit or validation process.

**Implication for the skill:** Add an ownership map that distinguishes the AI governance body from QA/quality unit, system owner, process owner, data owner, validation/assurance lead, privacy, security, regulatory, and supplier-quality roles. Exceptions, deviations, CAPA, change requests, and periodic reviews must land in the authoritative QMS or linked controlled records.

## Capability gap map

| Capability | Existing skill | Required extension |
|---|---|---|
| AI governance operating model | Strong | Add QMS and quality-unit interfaces |
| Risk tiering | Strong | Add GxP criticality and patient/product/data-integrity impact |
| Lifecycle gates | Strong | Add GxP evidence and release criteria per gate |
| Data lineage/provenance | Strong | Add ALCOA+ and complete data lifecycle review |
| Electronic records/signatures | Minimal | Add Part 11 / equivalent control prompts, without legal interpretation |
| Audit trails | General auditability | Add enablement, protection, review, retention, and exception handling |
| Validation/assurance | Implied by evidence gates | Add risk-based CSV/CSA decision and validation evidence inventory |
| AI performance evidence | General evaluation | Add context of use, generalizability, uncertainty, drift, and change triggers |
| QMS operations | Not covered | Add deviation, CAPA, change control, periodic review, training, and SOP interfaces |
| Supplier oversight | Strong general due diligence | Add supplier quality, model/data provenance, audit rights, change notification, and continuity |
| Regulatory mapping | Strong generic mapping | Add GxP overlays and require primary-source verification |

## Recommended skill change

Add `references/gxp-and-data-integrity.md` with:

1. A boundary statement: educational governance guidance, not legal advice or a validation package.
2. A GxP applicability/intake screen.
3. ALCOA+ and data-lifecycle control prompts.
4. Risk-based validation/assurance decision logic.
5. GxP AI lifecycle gates and required evidence.
6. QMS ownership and escalation interfaces.
7. Supplier/model due diligence additions.
8. A verification checklist and primary-source links.

Add at least one evaluation case for a life-sciences AI system and assert that the answer distinguishes general AI governance from GxP validation and QMS responsibilities.

## Confidence and limitations

- **High confidence:** The skill currently has a material GxP/ALCOA+ coverage gap. This is supported by direct comparison between the skill's current references and multiple official or standards-body sources.
- **High confidence:** ALCOA+, audit trails, electronic-record controls, lifecycle integrity, and validation/assurance need explicit treatment.
- **Moderate confidence:** The exact evidence package for a particular AI use case depends on GxP domain, jurisdiction, intended use, system boundary, and the organization's QMS. No universal AI validation recipe should be asserted.
- **Open question:** Which GxP domains matter most for the target role: GMP manufacturing, GCP clinical trials, GLP laboratories, GDP distribution, or pharmacovigilance? The reference should remain cross-domain, with domain-specific obligations routed to QA/regulatory specialists.

## Sources

Accessed 2026-08-20 UTC.

1. FDA, *Data Integrity and Compliance With Drug CGMP*, https://www.fda.gov/media/119267/download
2. FDA, *Part 11, Electronic Records; Electronic Signatures — Scope and Application*, https://www.fda.gov/regulatory-information/search-fda-guidance-documents/part-11-electronic-records-electronic-signatures-scope-and-application
3. eCFR, *21 CFR Part 11 — Electronic Records; Electronic Signatures*, https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11
4. MHRA, *GxP Data Integrity Guidance and Definitions*, https://www.gov.uk/government/publications/guidance-on-gxp-data-integrity
5. PIC/S, *PI 041-1 Guidance on Data Integrity*, https://picscheme.org/docview/4234
6. WHO, *TRS 1033 Annex 4: Guideline on Data Integrity*, https://www.who.int/docs/default-source/medicines/norms-and-standards/guidelines/inspections/trs1033-annex4-guideline-on-data-integrity.pdf
7. EMA, *Guideline on Computerised Systems and Electronic Data in Clinical Trials*, https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-computerised-systems-and-electronic-data-clinical-trials_en.pdf
8. EMA, *Reflection Paper on the Use of Artificial Intelligence in the Medicinal Product Lifecycle*, https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf
9. FDA, *Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products*, https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological
10. ICH, *Q9(R1) Quality Risk Management*, https://database.ich.org/sites/default/files/ICH_Q9(R1)_Guideline_Step4_2022_1219.pdf
11. ISPE, *GAMP 5 Guide, 2nd Edition*, https://ispe.org/publications/guidance-documents/gamp-5-guide-2nd-edition
12. European Commission, *EudraLex Volume 4, Annex 11: Computerised Systems*, https://health.ec.europa.eu/system/files/2016-11/annex11_01-2011_en_0.pdf

# Research Log: GxP AI Governance Gap

**Question:** What must `ai-governance` add to support GxP, ALCOA+, and data-integrity expectations?

**Track:** Academic/comprehensive, narrow technical/regulatory investigation
**Started/completed:** 2026-08-20 UTC
**Inclusion criteria:** Official regulators, inspectorates, standards bodies, and established life-sciences guidance directly addressing GxP data integrity, electronic records, computerized systems, validation/assurance, or AI in the medicinal-product lifecycle.
**Exclusion criteria:** Vendor marketing, generic responsible-AI commentary, legal interpretation, and sources that only mention AI without GxP/data-integrity relevance.

## Search record

| Query | Purpose | Result |
|---|---|---|
| `site:fda.gov data integrity ALCOA+ guidance pharmaceutical CGMP official` | FDA data-integrity baseline | Retained FDA CGMP data-integrity guidance |
| `site:fda.gov 21 CFR Part 11 electronic records electronic signatures official guidance` | Electronic-record controls | Retained FDA Part 11 guidance and eCFR |
| `site:ema.europa.eu Annex 11 computerized systems GMP official data integrity` | EU computerized-system expectations | Retained EMA computerized-system and Annex 11 materials |
| `site:gov.uk MHRA GxP data integrity guidance ALCOA official` | Cross-GxP inspectorate guidance | Retained MHRA final guidance |
| `site:ich.org ICH Q9(R1) quality risk management computerized systems data integrity official` | Risk-management baseline | Retained ICH Q9(R1) |
| `site:fda.gov artificial intelligence machine learning drug manufacturing quality systems guidance` | AI-specific FDA context | Retained FDA AI drug-development/manufacturing materials |
| `site:ema.europa.eu artificial intelligence reflection paper medicines lifecycle governance data integrity` | AI-specific EMA context | Retained EMA AI reflection paper |
| `site:picscheme.org PI 041 data integrity ALCOA official PDF` | ALCOA+ operational detail | Retained PIC/S PI 041-1 |
| `site:who.int guidance data integrity ALCOA pharmaceutical official` | Independent global corroboration | Retained WHO data-integrity guidance |
| `site:ispe.org GAMP 5 second edition artificial intelligence machine learning regulated systems` | Industry implementation context | Retained ISPE GAMP 5 description |

## Source decisions and extracted claims

### Retained: FDA CGMP data integrity guidance

- **Authority:** Tier 1 official regulator.
- **Relevant claim:** Data integrity is part of CGMP compliance for drugs under 21 CFR parts 210, 211, and 212.
- **Use in synthesis:** Establishes that data integrity belongs inside the quality and compliance system, not only in an AI/data platform.
- **Limitation:** Drug-CGMP focus; does not by itself cover every GxP domain or AI-specific validation.

### Retained: FDA Part 11 guidance and eCFR Part 11

- **Authority:** Tier 1 regulator and codified regulation.
- **Relevant claims:** Electronic records/signatures require controls for authenticity, integrity, confidentiality where appropriate, record retrieval, access control, validation, and secure time-stamped audit trails.
- **Use in synthesis:** Justifies an explicit electronic-record/audit-trail gate.
- **Limitation:** Applicability depends on the system's records and regulated use; this research does not make a legal applicability determination.

### Retained: MHRA GxP data-integrity guidance

- **Authority:** Tier 1 inspectorate guidance.
- **Relevant claim:** Guidance applies to data-integrity expectations across GxP sectors including GLP, GCP, GMP, GDP, and pharmacovigilance.
- **Use in synthesis:** Supports a cross-GxP reference and a data-governance/QMS boundary.
- **Limitation:** Guidance is not a substitute for jurisdiction-specific legal or quality advice.

### Retained: PIC/S PI 041-1 and WHO data-integrity guidance

- **Authority:** Tier 1 international inspection/health authority guidance.
- **Relevant claims:** ALCOA+ attributes and audit-trail lifecycle controls make data usable for informed decisions and support integrity across the data life cycle.
- **Use in synthesis:** Provides the operational vocabulary missing from the current skill.
- **Limitation:** These are guidance documents; exact adoption and terminology can vary by authority and domain.

### Retained: EMA computerized-system and AI materials

- **Authority:** Tier 1 regulator.
- **Relevant claims:** Computerized-system data security includes integrity, reliability, and availability; AI lifecycle considerations include integrity of model-development data and generalizability to the target population and context of use.
- **Use in synthesis:** Supports adding AI-specific performance/context evidence to GxP lifecycle gates.
- **Limitation:** The AI reflection paper is guidance/reflection material, not a universal validation standard.

### Retained: European Commission EudraLex Volume 4 Annex 11

- **Authority:** Tier 1 European Commission good-manufacturing-practice guidance.
- **Relevant claim:** Annex 11 provides the computerized-system control context relevant to regulated records, system operation, validation, and data integrity.
- **Use in synthesis:** Supports the explicit electronic-record, audit-trail, validation/assurance, and QMS-interface prompts in the GxP overlay.
- **Limitation:** Applicability and current interpretation depend on the system boundary, GxP domain, jurisdiction, and responsible quality/regulatory functions.

### Retained: ICH Q9(R1)

- **Authority:** Tier 1 international harmonization guidance.
- **Relevant claim:** Quality risk management provides the framework for risk-based decisions, including computerized-system and data-integrity concerns.
- **Use in synthesis:** Supports risk-based validation/assurance rather than one fixed control burden.
- **Limitation:** It is a framework, not an implementation procedure for a specific AI system.

### Retained: ISPE GAMP 5, 2nd Edition

- **Authority:** Tier 2 established industry guidance.
- **Relevant claim:** The second edition addresses risk-based compliant GxP computerized systems and includes AI/ML, cloud, open-source, and data-integrity topics.
- **Use in synthesis:** Supplies implementation vocabulary for validation/assurance and supplier/system lifecycle controls.
- **Limitation:** Industry guidance, not law or regulator-issued binding requirements.

## Rejected or not promoted

- Generic AI governance and responsible-AI sources: redundant with the existing skill and not specific enough to fill this gap.
- Vendor blogs and certification marketing: excluded because the question requires an authoritative baseline and vendor incentives would add little evidence.
- Search snippets without a retrievable primary document: used only for discovery, not as standalone evidence.

## Synthesis status

The research reached saturation for the bounded question: every retained source adds one of the same convergent control families — data integrity attributes, electronic records/audit trails, risk-based assurance, lifecycle traceability, AI context/performance evidence, or QMS accountability. The remaining uncertainty is domain- and jurisdiction-specific applicability, which the new reference must explicitly route to QA/regulatory owners rather than resolve itself.

#!/usr/bin/env python3
"""Governance-maturity self-assessment scorer for the ai-governance skill.

Reads a JSON answers file describing an organization's current state across the
canonical governance dimensions (each scored 1-5), computes an overall maturity
level and a list of gaps (dimensions below the target), and reports the result.

Behavior contract:

* ``--json`` emits a single JSON object to stdout containing the keys
  ``maturity_level`` (string) and ``gaps`` (array).
* Exit code 0 on healthy input; exit code 1 when every dimension is at minimum
  (critical gaps); exit code 1 on missing, malformed, or semantically-invalid
  input (with an explanatory message on stderr).
* ``--dry-run`` is a true preview: the tool is read-only and never writes files,
  so a dry-run's ``--json`` output is byte-identical to the real run's.
* Output is deterministic: no timestamps, stable ordering, and sorted keys.

Standard library only. No third-party runtime dependencies.
"""

from __future__ import annotations

import argparse
import json
import statistics
import sys
from typing import Any

# The canonical governance dimensions scored by the self-assessment. Scores are
# 1 (ad hoc / absent) to 5 (systematically optimized).
DIMENSIONS: tuple[str, ...] = (
    "roles_and_decision_rights",
    "risk_register",
    "lifecycle_gates",
    "incident_response",
    "fairness_reviews",
    "transparency_reporting",
    "model_inventory",
    "third_party_due_diligence",
)

MIN_SCORE = 1
MAX_SCORE = 5
DEFAULT_TARGET = 3  # "Defined" is the minimum for a baseline governance posture

# Maturity level names, keyed to a lower-bound on the average score.
LEVEL_THRESHOLDS: tuple[tuple[float, str], ...] = (
    (4.5, "Optimized"),
    (3.5, "Managed"),
    (2.5, "Defined"),
    (1.5, "Developing"),
    (0.0, "Initial"),
)


def validate_answers(data: Any) -> dict[str, Any]:
    """Validate the parsed JSON payload and return its ``answers`` mapping.

    Raises ``ValueError`` with a human-readable message when the payload is not
    the expected shape: a top-level object with an ``answers`` object whose keys
    are exactly the canonical dimensions and whose values are integers 1-5.
    """
    if not isinstance(data, dict):
        raise ValueError("top-level JSON must be an object")
    answers = data.get("answers")
    if not isinstance(answers, dict):
        raise ValueError("'answers' must be an object of dimension scores")
    unknown = sorted(set(answers) - set(DIMENSIONS))
    if unknown:
        raise ValueError("unexpected dimension(s): " + ", ".join(unknown))
    missing = [d for d in DIMENSIONS if d not in answers]
    if missing:
        raise ValueError("missing required dimension(s): " + ", ".join(missing))
    for dimension in DIMENSIONS:
        value = answers[dimension]
        if isinstance(value, bool) or not isinstance(value, int):
            raise ValueError(f"dimension '{dimension}' must be an integer score, got {value!r}")
        if not MIN_SCORE <= value <= MAX_SCORE:
            raise ValueError(
                f"dimension '{dimension}' score {value} out of range {MIN_SCORE}-{MAX_SCORE}"
            )
    return answers


def compute_maturity(answers: dict[str, int], target: int = DEFAULT_TARGET) -> dict[str, Any]:
    """Compute the maturity result for validated dimension scores.

    Returns a dict with ``maturity_level`` (string), ``average_score`` (float),
    ``min_score`` (int), ``critical`` (bool), ``target`` (int), and ``gaps``
    (a list of gap entries for dimensions scoring below ``target``, sorted by
    dimension name for deterministic output).
    """
    scores = [answers[d] for d in DIMENSIONS]
    average = statistics.fmean(scores)
    minimum = min(scores)
    level = next(name for bound, name in LEVEL_THRESHOLDS if average >= bound)
    # "Critical gaps" means every dimension is at its minimum (all scored 1),
    # indicating the organization has no governance controls in place at all.
    critical = max(scores) == MIN_SCORE
    gaps: list[dict[str, Any]] = [
        {
            "dimension": dimension,
            "score": answers[dimension],
            "target": target,
            "deficit": max(target - answers[dimension], 0),
        }
        for dimension in sorted(DIMENSIONS, key=str.lower)
        if answers[dimension] < target
    ]
    return {
        "maturity_level": level,
        "average_score": round(average, 3),
        "min_score": minimum,
        "critical": critical,
        "target": target,
        "gaps": gaps,
    }


def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description=(
            "Score an organization's AI-governance maturity from a JSON "
            "answers file of dimension scores (1-5)."
        )
    )
    parser.add_argument(
        "answers_file",
        help="path to a JSON file with an 'answers' object of dimension scores",
    )
    parser.add_argument(
        "--json",
        action="store_true",
        help="emit the result as a single JSON object on stdout",
    )
    parser.add_argument(
        "--dry-run",
        action="store_true",
        help="preview the result without writing anything (this tool is read-only)",
    )
    parser.add_argument(
        "--target",
        type=int,
        default=DEFAULT_TARGET,
        metavar="SCORE",
        help=(f"minimum dimension score considered non-gap (default: {DEFAULT_TARGET})"),
    )
    return parser.parse_args(argv)


def _fail(message: str) -> int:
    print(f"error: {message}", file=sys.stderr)
    return 1


def main(argv: list[str] | None = None) -> int:
    args = parse_args(argv)

    if not MIN_SCORE <= args.target <= MAX_SCORE:
        return _fail(f"--target must be between {MIN_SCORE} and {MAX_SCORE}")

    try:
        with open(args.answers_file, encoding="utf-8") as handle:
            raw = handle.read()
    except OSError as exc:
        return _fail(f"cannot read answers file '{args.answers_file}': {exc}")

    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        return _fail(f"invalid JSON in '{args.answers_file}': {exc}")

    try:
        answers = validate_answers(data)
    except ValueError as exc:
        return _fail(str(exc))

    result = compute_maturity(answers, target=args.target)

    if args.json:
        print(
            json.dumps(
                result,
                sort_keys=True,
                ensure_ascii=False,
                separators=(",", ":"),
            )
        )
    else:
        _render_human(result)

    return 1 if result["critical"] else 0


def _render_human(result: dict[str, Any]) -> None:
    header = (
        f"Governance maturity: {result['maturity_level']} "
        f"(avg {result['average_score']}, min {result['min_score']}, "
        f"target {result['target']})"
    )
    print(header)
    if result["gaps"]:
        print("Gaps:")
        for gap in result["gaps"]:
            print(f"  - {gap['dimension']}: score {gap['score']} (deficit {gap['deficit']})")
    else:
        print("No gaps: every dimension is at or above the target.")
    if result["critical"]:
        print("Critical: every dimension is at minimum; governance is not established.")


if __name__ == "__main__":
    raise SystemExit(main())

"""Unit and CLI tests for the governance-maturity self-assessment scorer.

The scorer reads a JSON answers file of governance dimensions (each scored
1-5), computes an overall maturity level and a list of gaps, and reports via
exit code 0 (healthy) or 1 (critical gaps where every dimension is at minimum).
It is stdlib-only and deterministic, and supports ``--json`` and ``--dry-run``.
"""

from __future__ import annotations

import importlib.util
import json
import subprocess
import sys
from pathlib import Path

SCRIPT = Path(__file__).resolve().parent / "governance-maturity.py"

HEALTHY = {
    "organization": "Acme",
    "answers": {
        "roles_and_decision_rights": 3,
        "risk_register": 2,
        "lifecycle_gates": 3,
        "incident_response": 2,
        "fairness_reviews": 2,
        "transparency_reporting": 2,
        "model_inventory": 3,
        "third_party_due_diligence": 2,
    },
}

ALL_MINIMUM = {
    "organization": "Acme",
    "answers": {
        "roles_and_decision_rights": 1,
        "risk_register": 1,
        "lifecycle_gates": 1,
        "incident_response": 1,
        "fairness_reviews": 1,
        "transparency_reporting": 1,
        "model_inventory": 1,
        "third_party_due_diligence": 1,
    },
}


def _load_module():
    spec = importlib.util.spec_from_file_location("governance_maturity", SCRIPT)
    assert spec and spec.loader, "could not build import spec"
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module


def _run_cli(args, stdin=None):
    return subprocess.run(
        [sys.executable, str(SCRIPT), *args],
        capture_output=True,
        text=True,
        input=stdin,
    )


def _write_input(tmp_path, payload):
    path = tmp_path / "answers.json"
    path.write_text(json.dumps(payload), encoding="utf-8")
    return path


def _module():
    return _load_module()


# --- pure-computation unit tests ---------------------------------------------


def test_healthy_input_yields_level_and_gaps():
    result = _module().compute_maturity(HEALTHY["answers"])
    assert isinstance(result["maturity_level"], str)
    assert isinstance(result["gaps"], list)
    assert result["maturity_level"] in {
        "Initial",
        "Developing",
        "Defined",
        "Managed",
        "Optimized",
    }
    assert not result["critical"]


def test_all_minimum_is_critical_with_nonempty_gaps():
    result = _module().compute_maturity(ALL_MINIMUM["answers"])
    assert result["critical"] is True
    assert len(result["gaps"]) > 0


def test_healthy_average_score_is_deterministic():
    first = _module().compute_maturity(HEALTHY["answers"])
    second = _module().compute_maturity(HEALTHY["answers"])
    assert first == second


def test_every_dimension_is_one_gap_entry_below_target():
    result = _module().compute_maturity(HEALTHY["answers"], target=3)
    names = {g["dimension"] for g in result["gaps"]}
    assert "risk_register" in names
    assert "model_inventory" not in names  # scored at target


def test_full_score_is_optimized_with_no_gaps():
    full = dict.fromkeys(_module().DIMENSIONS, 5)
    result = _module().compute_maturity(full, target=3)
    assert result["maturity_level"] == "Optimized"
    assert result["gaps"] == []


# --- CLI behavior tests ------------------------------------------------------


def test_cli_help_advertises_flags(tmp_path):
    proc = _run_cli(["--help"])
    assert proc.returncode == 0
    assert "--json" in proc.stdout
    assert "--dry-run" in proc.stdout


def test_cli_healthy_exits_zero_and_emits_contract_keys(tmp_path):
    path = _write_input(tmp_path, HEALTHY)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode == 0
    out = json.loads(proc.stdout)
    assert isinstance(out["maturity_level"], str)
    assert isinstance(out["gaps"], list)


def test_cli_critical_exits_one_with_nonempty_gaps(tmp_path):
    path = _write_input(tmp_path, ALL_MINIMUM)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode == 1
    out = json.loads(proc.stdout)
    assert len(out["gaps"]) > 0


def test_cli_missing_file_fails_gracefully(tmp_path):
    proc = _run_cli([str(tmp_path / "nope.json"), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_malformed_json_fails_gracefully(tmp_path):
    path = tmp_path / "bad.json"
    path.write_text('{"organization": "Acme", "answers": ', encoding="utf-8")
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_out_of_range_score_fails_gracefully(tmp_path):
    bad = {
        "organization": "Acme",
        "answers": {
            "roles_and_decision_rights": 7,
            "risk_register": 2,
            "lifecycle_gates": 3,
            "incident_response": 2,
            "fairness_reviews": 2,
            "transparency_reporting": 2,
            "model_inventory": 3,
            "third_party_due_diligence": 2,
        },
    }
    path = _write_input(tmp_path, bad)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_missing_required_dimension_fails_gracefully(tmp_path):
    bad = {"organization": "Acme", "answers": {"risk_register": 2}}
    path = _write_input(tmp_path, bad)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_dry_run_matches_real_json(tmp_path):
    path = _write_input(tmp_path, HEALTHY)
    real = _run_cli([str(path), "--json"])
    dry = _run_cli([str(path), "--json", "--dry-run"])
    assert real.returncode == 0
    assert dry.returncode == 0
    assert real.stdout == dry.stdout


def test_cli_is_deterministic_across_runs(tmp_path):
    path = _write_input(tmp_path, HEALTHY)
    first = _run_cli([str(path), "--json"])
    second = _run_cli([str(path), "--json"])
    assert first.stdout == second.stdout

"""Unit and CLI tests for the use-case risk-tier classifier.

The classifier reads a structured JSON description of an AI use case (data
sensitivity, autonomy, exposure, decision impact), computes a risk tier (low /
medium / high) and the controls that tier requires, and reports via exit code 0
(success) or 1 (input/usage error). It is stdlib-only and deterministic, and
supports ``--json`` and ``--dry-run``.
"""

from __future__ import annotations

import importlib.util
import json
import subprocess
import sys
from pathlib import Path

SCRIPT = Path(__file__).resolve().parent / "use-case-risk-tier.py"

LOW = {
    "use_case": "Internal document summarization",
    "data_sensitivity": "internal",
    "autonomy": "human_in_the_loop",
    "exposure": "low",
    "decision_impact": "informational",
}

MEDIUM = {
    "use_case": "Operations ticket triage",
    "data_sensitivity": "internal",
    "autonomy": "human_on_the_loop",
    "exposure": "medium",
    "decision_impact": "operational",
}

HIGH = {
    "use_case": "Automated fraud screening",
    "data_sensitivity": "confidential",
    "autonomy": "fully_automated",
    "exposure": "high",
    "decision_impact": "financial",
}

OVERRIDE_SENSITIVE = {
    "use_case": "Healthcare risk scoring",
    "data_sensitivity": "sensitive_personal",
    "autonomy": "human_on_the_loop",
    "exposure": "medium",
    "decision_impact": "financial",
}

OVERRIDE_LIFE = {
    "use_case": "Release-decisions assistant",
    "data_sensitivity": "internal",
    "autonomy": "human_on_the_loop",
    "exposure": "medium",
    "decision_impact": "life_liberty",
}

REALISTIC = {
    "use_case": "Customer credit scoring",
    "data_sensitivity": "personal",
    "autonomy": "human_on_the_loop",
    "exposure": "medium",
    "decision_impact": "financial",
}


def _load_module():
    spec = importlib.util.spec_from_file_location("use_case_risk_tier", SCRIPT)
    assert spec and spec.loader, "could not build import spec"
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module


def _module():
    return _load_module()


def _run_cli(args, stdin=None):
    return subprocess.run(
        [sys.executable, str(SCRIPT), *args],
        capture_output=True,
        text=True,
        input=stdin,
    )


def _write_input(tmp_path, payload):
    path = tmp_path / "use_case.json"
    path.write_text(json.dumps(payload), encoding="utf-8")
    return path


def _raises_value_error(fn, *args, **kwargs):
    try:
        fn(*args, **kwargs)
        return False
    except ValueError:
        return True


# --- pure-computation unit tests ---------------------------------------------


def test_low_use_case_has_tier_and_controls():
    result = _module().compute_risk(LOW)
    assert result["tier"] == "low"
    assert isinstance(result["controls"], list)
    assert result["controls"]  # always at least the base controls


def test_medium_use_case_has_medium_tier():
    result = _module().compute_risk(MEDIUM)
    assert result["tier"] == "medium"


def test_high_use_case_has_high_tier():
    result = _module().compute_risk(HIGH)
    assert result["tier"] == "high"


def test_sensitive_personal_data_forces_high_tier():
    result = _module().compute_risk(OVERRIDE_SENSITIVE)
    assert result["tier"] == "high"


def test_life_liberty_impact_forces_high_tier():
    result = _module().compute_risk(OVERRIDE_LIFE)
    assert result["tier"] == "high"


def test_high_tier_includes_high_controls():
    result = _module().compute_risk(HIGH)
    assert "board_or_ai_council_approval" in result["controls"]
    assert "audit_trail_and_logging" in result["controls"]


def test_personal_data_adds_privacy_controls():
    result = _module().compute_risk(REALISTIC)
    assert "privacy_impact_assessment" in result["controls"]
    assert "data_protection_and_access_controls" in result["controls"]


def test_output_is_deterministic():
    first = _module().compute_risk(REALISTIC)
    second = _module().compute_risk(REALISTIC)
    assert first == second


def test_base_controls_present_everywhere():
    result = _module().compute_risk(LOW)
    assert "register_in_inventory_and_risk_register" in result["controls"]
    assert "document_in_model_or_data_card" in result["controls"]


# --- validation unit tests ---------------------------------------------------


def test_validate_accepts_required_levels():
    levels = _module().validate_use_case(REALISTIC)
    assert levels["data_sensitivity"] == "personal"


def test_validate_rejects_missing_key():
    assert _raises_value_error(_module().validate_use_case, {"data_sensitivity": "internal"})


def test_validate_rejects_unknown_key():
    assert _raises_value_error(_module().validate_use_case, {**REALISTIC, "bogus": "x"})


def test_validate_rejects_out_of_range_value():
    assert _raises_value_error(_module().validate_use_case, {**REALISTIC, "exposure": "extreme"})


# --- CLI behavior tests ------------------------------------------------------


def test_cli_help_advertises_flags(tmp_path):
    proc = _run_cli(["--help"])
    assert proc.returncode == 0
    assert "--json" in proc.stdout
    assert "--dry-run" in proc.stdout


def test_cli_realistic_input_exits_zero_and_emits_contract_keys(tmp_path):
    path = _write_input(tmp_path, REALISTIC)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode == 0
    out = json.loads(proc.stdout)
    assert isinstance(out["tier"], str)
    assert isinstance(out["controls"], list)


def test_cli_missing_file_fails_gracefully(tmp_path):
    proc = _run_cli([str(tmp_path / "nope.json"), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_malformed_json_fails_gracefully(tmp_path):
    path = tmp_path / "bad.json"
    path.write_text('{"use_case": "x", "data_sensitivity": ', encoding="utf-8")
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_out_of_range_value_fails_gracefully(tmp_path):
    bad = {**REALISTIC, "exposure": "extreme"}
    path = _write_input(tmp_path, bad)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_missing_required_key_fails_gracefully(tmp_path):
    bad = {k: v for k, v in REALISTIC.items() if k != "decision_impact"}
    path = _write_input(tmp_path, bad)
    proc = _run_cli([str(path), "--json"])
    assert proc.returncode != 0
    assert proc.stderr.strip()


def test_cli_dry_run_matches_real_json(tmp_path):
    path = _write_input(tmp_path, REALISTIC)
    real = _run_cli([str(path), "--json"])
    dry = _run_cli([str(path), "--json", "--dry-run"])
    assert real.returncode == 0
    assert dry.returncode == 0
    assert real.stdout == dry.stdout


def test_cli_is_deterministic_across_runs(tmp_path):
    path = _write_input(tmp_path, REALISTIC)
    first = _run_cli([str(path), "--json"])
    second = _run_cli([str(path), "--json"])
    assert first.stdout == second.stdout

#!/usr/bin/env python3
"""Use-case risk-tier classifier for the ai-governance skill.

Reads a structured JSON description of an AI use case (data sensitivity,
autonomy, exposure, and decision impact), computes a governance risk tier
(low / medium / high) and the controls that tier requires, and reports the
result.

Behavior contract:

* ``--json`` emits a single JSON object to stdout containing the keys ``tier``
  (string) and ``controls`` (array).
* Exit code 0 on valid input; exit code 1 on missing, malformed, or
  semantically-invalid input (with an explanatory message on stderr).
* ``--dry-run`` is a true preview: the tool is read-only and never writes files,
  so a dry-run's ``--json`` output is byte-identical to the real run's.
* Output is deterministic: no timestamps, stable ordering, and sorted keys.

This is a defensible default classifier for intake triage; it does not replace a
full model risk assessment (see ``model-risk-assessment.md``). Expert judgment
should confirm the tier before it is recorded in the risk register.

Standard library only. No third-party runtime dependencies.
"""

from __future__ import annotations

import argparse
import json
import sys
from typing import Any

# Allowed values per dimension, mapped to their inherent-risk contribution.
# These follow the vocabulary used by the intake form and model-risk-assessment
# templates.
DATA_SENSITIVITY: dict[str, int] = {
    "public": 1,
    "internal": 2,
    "confidential": 3,
    "personal": 4,
    "sensitive_personal": 5,
    "regulated": 5,
}

AUTONOMY: dict[str, int] = {
    "human_in_the_loop": 1,
    "human_on_the_loop": 2,
    "fully_automated": 3,
}

EXPOSURE: dict[str, int] = {
    "low": 1,
    "medium": 2,
    "high": 3,
}

DECISION_IMPACT: dict[str, int] = {
    "informational": 1,
    "operational": 2,
    "financial": 3,
    "life_liberty": 4,
}

LEVEL_MAPS: dict[str, dict[str, int]] = {
    "data_sensitivity": DATA_SENSITIVITY,
    "autonomy": AUTONOMY,
    "exposure": EXPOSURE,
    "decision_impact": DECISION_IMPACT,
}

REQUIRED = ("data_sensitivity", "autonomy", "exposure", "decision_impact")
OPTIONAL = ("use_case", "notes")

# Data sensitivity and decision impact weigh twice as heavily as autonomy and
# exposure because they are the principal drivers of inherent risk.
WEIGHTED = ("data_sensitivity", "decision_impact")
SENSITIVITY_WEIGHT = 2
AUTONOMY_WEIGHT = 1

# Weighted-score thresholds for the tier (scores range 6..24).
HIGH_SCORE = 15
MEDIUM_SCORE = 10

# Values that force the high tier regardless of the weighted score.
FORCED_HIGH_SENSITIVITY = ("sensitive_personal", "regulated")
FORCED_HIGH_IMPACT = ("life_liberty",)

# Ordered control sets. Tier controls are cumulative; driver-specific controls
# are added when the matching dimension is present. Duplicates are collapsed.
CONTROL_BASE: tuple[str, ...] = (
    "register_in_inventory_and_risk_register",
    "document_in_model_or_data_card",
)
CONTROL_MEDIUM: tuple[str, ...] = (
    "bias_and_fairness_review",
    "monitoring_and_drift_detection",
    "privacy_and_minimization_review",
    "human_review_and_override_path",
)
CONTROL_HIGH: tuple[str, ...] = (
    "full_nist_rmf_aligned_risk_assessment",
    "independent_validation",
    "board_or_ai_council_approval",
    "audit_trail_and_logging",
    "security_review_and_red_teaming",
)
CONTROL_SENSITIVE_DATA: tuple[str, ...] = (
    "privacy_impact_assessment",
    "data_protection_and_access_controls",
)
CONTROL_HIGH_IMPACT: tuple[str, ...] = (
    "human_in_the_loop_final_decision",
    "appeal_and_redress_process",
)
CONTROL_AUTONOMY: tuple[str, ...] = ("automated_decision_oversight",)
CONTROL_EXPOSURE: tuple[str, ...] = ("scaled_monitoring_and_escalation",)


def validate_use_case(data: Any) -> dict[str, str]:
    """Validate the parsed JSON payload and return its dimension levels.

    Raises ``ValueError`` with a human-readable message when the payload is not
    the expected shape: a top-level object with exactly the four required
    dimensions (each an allowed string value), plus the optional ``use_case``
    and ``notes`` string fields.
    """
    if not isinstance(data, dict):
        raise ValueError("top-level JSON must be an object")
    unknown = sorted(set(data) - set(REQUIRED) - set(OPTIONAL))
    if unknown:
        raise ValueError("unexpected key(s): " + ", ".join(unknown))
    missing = [k for k in REQUIRED if k not in data]
    if missing:
        raise ValueError("missing required key(s): " + ", ".join(missing))
    levels: dict[str, str] = {}
    for key in REQUIRED:
        value = data[key]
        if not isinstance(value, str):
            raise ValueError(f"'{key}' must be a string, got {value!r}")
        allowed = sorted(LEVEL_MAPS[key])
        if value not in LEVEL_MAPS[key]:
            raise ValueError(f"'{key}' value {value!r} not in {allowed}")
        levels[key] = value
    for key in OPTIONAL:
        if key in data and not isinstance(data[key], str):
            raise ValueError(f"'{key}' must be a string, got {data[key]!r}")
    return levels


def _weighted_score(levels: dict[str, str]) -> int:
    total = 0
    for key, weight in (
        (k, SENSITIVITY_WEIGHT if k in WEIGHTED else AUTONOMY_WEIGHT) for k in REQUIRED
    ):
        total += weight * LEVEL_MAPS[key][levels[key]]
    return total


def _derive_tier(levels: dict[str, str], total: int) -> str:
    if (
        levels["data_sensitivity"] in FORCED_HIGH_SENSITIVITY
        or levels["decision_impact"] in FORCED_HIGH_IMPACT
    ):
        return "high"
    if total >= HIGH_SCORE:
        return "high"
    if total >= MEDIUM_SCORE:
        return "medium"
    return "low"


def _derive_controls(tier: str, levels: dict[str, str]) -> list[str]:
    controls: list[str] = list(CONTROL_BASE)
    if tier in ("medium", "high"):
        controls.extend(CONTROL_MEDIUM)
    if tier == "high":
        controls.extend(CONTROL_HIGH)
    if levels["data_sensitivity"] in ("personal", "sensitive_personal", "regulated"):
        controls.extend(CONTROL_SENSITIVE_DATA)
    if levels["decision_impact"] in ("financial", "life_liberty"):
        controls.extend(CONTROL_HIGH_IMPACT)
    if levels["autonomy"] == "fully_automated":
        controls.extend(CONTROL_AUTONOMY)
    if levels["exposure"] == "high":
        controls.extend(CONTROL_EXPOSURE)
    seen: set[str] = set()
    deduped: list[str] = []
    for control in controls:
        if control not in seen:
            seen.add(control)
            deduped.append(control)
    return deduped


def compute_risk(levels: dict[str, str]) -> dict[str, Any]:
    """Compute the risk tier and required controls for validated levels.

    Returns a dict with ``tier`` (string), ``total_score`` (int), the four
    dimension levels echoed back, and ``controls`` (a list of control names).
    """
    total = _weighted_score(levels)
    tier = _derive_tier(levels, total)
    return {
        "tier": tier,
        "total_score": total,
        "data_sensitivity": levels["data_sensitivity"],
        "autonomy": levels["autonomy"],
        "exposure": levels["exposure"],
        "decision_impact": levels["decision_impact"],
        "controls": _derive_controls(tier, levels),
    }


def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description=(
            "Classify the governance risk tier and required controls for an AI "
            "use case from a JSON file of dimension levels."
        )
    )
    parser.add_argument(
        "use_case_file",
        help=(
            "path to a JSON file with data_sensitivity, autonomy, exposure, "
            "and decision_impact levels"
        ),
    )
    parser.add_argument(
        "--json",
        action="store_true",
        help="emit the result as a single JSON object on stdout",
    )
    parser.add_argument(
        "--dry-run",
        action="store_true",
        help="preview the result without writing anything (this tool is read-only)",
    )
    return parser.parse_args(argv)


def _fail(message: str) -> int:
    print(f"error: {message}", file=sys.stderr)
    return 1


def main(argv: list[str] | None = None) -> int:
    args = parse_args(argv)

    try:
        with open(args.use_case_file, encoding="utf-8") as handle:
            raw = handle.read()
    except OSError as exc:
        return _fail(f"cannot read use-case file '{args.use_case_file}': {exc}")

    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        return _fail(f"invalid JSON in '{args.use_case_file}': {exc}")

    try:
        levels = validate_use_case(data)
    except ValueError as exc:
        return _fail(str(exc))

    result = compute_risk(levels)

    if args.json:
        print(
            json.dumps(
                result,
                sort_keys=True,
                ensure_ascii=False,
                separators=(",", ":"),
            )
        )
    else:
        _render_human(result)

    return 0


def _render_human(result: dict[str, Any]) -> None:
    header = (
        f"Risk tier: {result['tier']} "
        f"(score {result['total_score']}; "
        f"sensitivity={result['data_sensitivity']}, "
        f"autonomy={result['autonomy']}, "
        f"exposure={result['exposure']}, "
        f"impact={result['decision_impact']})"
    )
    print(header)
    print("Required controls:")
    for control in result["controls"]:
        print(f"  - {control}")


if __name__ == "__main__":
    raise SystemExit(main())

# Agentic AI and Deployment Posture Governance Review

> Fill this out for an AI system that uses an external application, a model API, a self-hosted
> model, or one or more autonomous agent capabilities. Use with
> [`six-level-governance-framework.md`](../references/six-level-governance-framework.md),
> [`llm-and-agent-security.md`](../references/llm-and-agent-security.md), and
> [`privacy-and-data-governance.md`](../references/privacy-and-data-governance.md). This is a
> governance worksheet, not legal or security advice. Re-verify legal, regulatory, and technical
> requirements against current primary sources.

## 0. Review record

- **System / product name:**
- **Version, model, provider, and configuration:**
- **Business owner:**
- **Technical owner:**
- **Risk owner / approver:**
- **Privacy or legal contact, if applicable:**
- **Review date:**
- **Review trigger:** [ ] New system  [ ] Material change  [ ] Periodic review  [ ] Incident or near miss  [ ] Vendor change  [ ] User or regulator request
- **Review scope and exclusions:**
- **Evidence location:**

## 1. Deployment posture and control boundary

Select the primary posture and record hybrid boundaries. Do not assume a vendor's controls cover
what the organization controls, and do not assume a self-hosted deployment is safer merely because
data stays inside the network.

- [ ] **SaaS or application consumer:** vendor operates the interface, middleware, and model stack.
- [ ] **API integrator:** organization operates the interface and pre/post-processing; an external
  provider operates the model API.
- [ ] **Model hoster:** organization operates the model-serving and guardrail stack, whether on
  premises or through managed infrastructure.
- [ ] **Agentic overlay:** the system plans, observes, reflects, acts, uses tools, or retains memory.

| Layer or control | Organization owner | Vendor / provider owner | Evidence or contract reference | Gap, assumption, or follow-up |
|---|---|---|---|---|
| User interface and identity | | | | |
| Input filtering and redaction | | | | |
| Model, weights, and updates | | | | |
| Retrieval data and permissions | | | | |
| Output validation and disclosure | | | | |
| Tools, APIs, and downstream actions | | | | |
| Logs, traces, and retention | | | | |
| Incident response and notification | | | | |
| Deletion, export, and termination | | | | |
| Subprocessors and data destinations | | | | |

**Control-boundary conclusion:**

- **Controls directly verified by the organization:**
- **Controls supported only by vendor evidence or contract:**
- **Controls that cannot currently be verified:**
- **Compensating controls for those gaps:**

## 2. Capability and exposure inventory

Mark every capability that exists in the deployed configuration, not only the capability shown in
the happy-path demo. Record the narrowest useful scope and the evidence that proves it.

| Capability | Present? | Scope, limits, and stop condition | Evidence / test reference |
|---|---:|---|---|
| Planning across multiple steps | [ ] | | |
| Reflection or self-refinement | [ ] | | |
| Observation of events or schedules | [ ] | | |
| Acting on records, messages, workflows, or code | [ ] | | |
| Tool or API invocation | [ ] | | |
| Multi-agent communication or delegation | [ ] | | |
| Persistent memory across tasks or users | [ ] | | |
| External web or browser access | [ ] | | |
| Access to sensitive or regulated data | [ ] | | |
| Irreversible or consequential actions | [ ] | | |

### Six-factor security exposure ladder

Use these factors to identify the system's exposure, not to calculate a numeric risk score. The
factors are a dependency ladder, not a severity ranking. A weak environment can cause more harm
than a higher-level capability, and one factor can be present without all the others.

| Factor | Applies? | What makes it apply here | Existing control and residual concern |
|---|---:|---|---|
| **Environment**: weak logging, secrets, access, isolation, or vendor security | [ ] | | |
| **Model**: unvetted, externally controlled, fine-tuned, or provenance-uncertain model | [ ] | | |
| **Input**: untrusted user, document, web, or tool content reaches the model | [ ] | | |
| **Data access**: runtime retrieval of personal, confidential, or sensitive data | [ ] | | |
| **Ability to make changes**: records, messages, transactions, code, or configuration can change | [ ] | | |
| **Agency**: the system chooses goals, tools, order, or retries across multiple steps | [ ] | | |

**Highest exposed factor and why:**

**Why a lower factor cannot be treated as solved by a higher-level control:**

## 3. Risk and impact decision

- **Affected people, groups, customers, employees, or third parties:**
- **Intended benefit and success measure:**
- **Foreseeable harms and misuse paths:**
- **Sensitive or regulated data involved:**
- **Consequential decisions or external communications:**
- **Inherent risk:**
- **Residual risk after controls:**
- **Uncertainty that must not be hidden in the rating:**
- **Risk tier and rationale:**
- **Required stakeholder review:**

### Material-change triggers

Reopen the risk and impact assessment when any of these changes: [ ] model or provider
[ ] prompt, policy, or guardrail [ ] data class or retrieval source [ ] user population or geography
[ ] tool, agent, destination, or permission [ ] memory or retention behavior [ ] material drift
[ ] incident, near miss, complaint, or challenge [ ] external obligation or contract.

## 4. Approved tools and authorization

A tool registry is an allowlist and accountability record, not a catalog of everything a connector
can do. Approve individual operations where possible. Authorization must be evaluated outside the
model's natural-language reasoning and immediately before invocation.

| Tool / operation ID | Purpose | Data fields sent and returned | Read / write / destructive | Destination / region | Agent identity and user context | Approval tier | Version or integrity evidence | Owner |
|---|---|---|---|---|---|---|---|---|
| | | | | | | | | |
| | | | | | | | | | |
| | | | | | | | | | |

### Authorization policy

- **Default decision:** [ ] Deny unless explicitly allowed  [ ] Other, with rationale:
- **Runtime attributes checked:** [ ] agent identity  [ ] user identity  [ ] role  [ ] purpose
  [ ] data classification  [ ] fields and parameters  [ ] destination  [ ] tenant or resource
  [ ] time or rate limit  [ ] current approval  [ ] tool version or integrity
- **Decision point outside model reasoning:**
- **Policy decision and enforcement component:**
- **Credential scope and lifetime:**
- **How tool-description or tool-output injection is handled:**
- **How newly advertised or changed tools are detected and held for review:**
- **Inter-agent trust and authentication, if applicable:**

### Action approval tiers

Classify by consequence, not by what the agent claims it intends to do.

| Tier | Example class | Default behavior | This system's rule and evidence |
|---|---|---|---|
| Low impact, reversible read | Read a permitted record or retrieve a non-sensitive source | Allow with logging and rate limits | |
| Medium impact or externally visible | Draft, schedule, update a reversible field, or send a bounded message | Confirmation or targeted human review | |
| High impact, sensitive, or irreversible | Payment, deletion, access change, regulated decision, production change | Explicit approval every time, with rollback or escalation | |

- **Approval threshold is calibrated to avoid:** [ ] approval fatigue  [ ] rubber-stamping
  [ ] unreviewed consequential actions  [ ] hidden outbound egress
- **Approval evidence and approver identity:**
- **Rollback, reversal, or containment path:**

## 5. Purpose-aware data egress

Evaluate every outbound tool call or external transfer at the enforcement point. A “read-only”
operation can still exfiltrate data through a URL, query, tool response, or downstream log.

- **Declared purpose for this run:**
- **Lawful basis or internal authorization record, where applicable:**
- **Minimum data necessary for that purpose:**
- **Permitted destinations, regions, and subprocessors:**
- **Blocked destinations or data categories:**
- **Data classification method:**
- **Egress enforcement point:**
- **Fail-closed behavior when no rule matches:**

For each proposed transfer, the gate must evaluate:

1. **Purpose:** Is the use compatible with the declared purpose?
2. **Necessity:** Are these exact fields required, or can the payload be redacted, generalized, or
   replaced with a reference?
3. **Destination:** Is this tool, recipient, region, and subprocessor permitted?

Record the policy result and the data actually sent:

| Run / call ID | Tool and destination | Purpose | Fields proposed | Fields sent after minimization | Decision | Approval / basis | Evidence |
|---|---|---|---|---|---|---|---|
| | | | | | | | |
| | | | | | | | |

Allowed outcomes should be explicit: **allow**, **allow with redaction**, **block**, **request
human approval**, or **request just-in-time consent**. A policy engine may return a decision, but a
separate enforcement point must prevent the call when the decision is block or approval is absent.

## 6. Memory, retention, and user rights

Treat memory as a governed data store and a possible persistence mechanism for malicious or
incorrect instructions. Do not allow durable memory to become an unreviewed second source of truth.

| Memory tier | Contents and purpose | Retention / TTL | Who can read or write | User or admin inspect/delete path | Evidence |
|---|---|---|---|---|---|
| Ephemeral working memory | | | | | |
| Bounded session or task log | | | | | |
| Long-term user or organizational memory | | | | | |
| Shared multi-agent memory | | | | | |
| Derived model, embedding, or index state | | | | | |

- **Durable writes require:** [ ] explicit user choice  [ ] human review  [ ] approved source
  [ ] purpose and lawful-basis check  [ ] summarization or normalization  [ ] other:
- **Memory-poisoning detection and recovery:**
- **Cross-tenant or cross-user isolation:**
- **Erasure workflow covers:** [ ] source records  [ ] prompts and outputs  [ ] logs and traces
  [ ] caches and backups  [ ] embeddings and indexes  [ ] memory stores  [ ] model updates or
  fine-tuning inputs  [ ] downstream tools and agents
- **How future runs are prevented from using erased data:**
- **What cannot currently be erased or corrected, and how that limitation is disclosed:**
- **User memory view, correction, and deletion surface:**

## 7. Test and evidence plan

Do not infer control effectiveness from policy text, a vendor assertion, or a model response alone.
Test the control at the boundary where it is supposed to operate.

| Risk or control | Test scenario / fixture | Expected result | Actual result | Model, config, data, and tool versions | Evidence link | Owner / due date |
|---|---|---|---|---|---|---|
| Direct and indirect prompt injection | | | | | | |
| Retrieval or tool authorization | | | | | | |
| Sensitive-data leakage or egress | | | | | | |
| Hallucination, output integrity, or overreliance | | | | | | |
| Fairness or subgroup impact | | | | | | |
| Memory poisoning or stale memory | | | | | | |
| Cost, loop, timeout, or denial-of-wallet | | | | | | |
| Human approval and contestability | | | | | | |
| Kill switch, credential revocation, and rollback | | | | | | |
| Vendor, model, tool, or configuration change | | | | | | |

- **Independent reviewer or red team:**
- **Known limitations and untested paths:**
- **Regression tests added for prior failures:**

## 8. Six-level governance decision

Summarize the evidence at each level. A level can be lighter for a low-impact system, but it must
not be silently skipped.

| Level | Evidence available | Open gap or exception | Owner | Decision / re-review trigger |
|---|---|---|---|---|
| 1. Strategy & Policy | | | | |
| 2. Risk & Impact Assessment | | | | |
| 3. Implementation Review | | | | |
| 4. Acceptance Testing | | | | |
| 5. Operations & Monitoring | | | | |
| 6. Learning & Improvement | | | | |

- **Decision:** [ ] Approve  [ ] Approve with bounded exception  [ ] Pilot only  [ ] Reject or hold
- **Approved scope and duration:**
- **Named exception owner and expiration:**
- **Compensating control:**
- **Required remediation before expansion:**
- **Next review date or trigger:**
- **Approver and date:**

## 9. Runtime handoff and learning loop

- **Structured trace fields:** request/run ID, identity, model and configuration version, retrieval
  source IDs, tools and parameters, policy decisions, approvals, output disposition, downstream
  outcome, and error or override. Minimize or redact sensitive content.
- **Runtime thresholds and response:**
- **Automatic degradation, pause, credential revocation, or kill-switch triggers:**
- **Manual intervention path:**
- **User complaint, contestation, and feedback path:**
- **Incident and near-miss review owner:**
- **How findings become policy, control, or regression-test changes:**
- **Metric showing whether the last improvement worked:**

### Completion check

- [ ] Control ownership is explicit across the deployment boundary.
- [ ] The capability inventory reflects the deployed configuration, including hidden or optional paths.
- [ ] Risk factors, affected people, residual risk, and uncertainty are recorded.
- [ ] Tool operations, credentials, destinations, and action approvals are allowlisted and enforced.
- [ ] Every external data transfer has purpose, necessity, destination, and fail-closed checks.
- [ ] Memory tiers, retention, deletion, and correction paths are documented.
- [ ] Tests exercise adversarial, boundary, failure, and recovery behavior with versioned evidence.
- [ ] Runtime monitoring, intervention, and learning owners are assigned.
- [ ] The decision, exceptions, expiration, and re-review triggers are signed by the accountable role.

---

### Source note

This original worksheet operationalizes the deployment postures, six exposure factors, agentic
capabilities, tool authorization, approval calibration, purpose-aware egress, memory tiers, and
closed-loop evidence practices described in *AI Governance* by Engin Bozdag and Stefano Bennati,
then cross-wires them with the existing `ai-governance` references. It does not reproduce source
prose or establish legal requirements. See `../references/source-index.md` for provenance.

# Board AI-Governance Report

> **Confidentiality:** A completed report turns the noisy detail of day-to-day AI governance into a compact, decision-useful picture for directors, covering inventory and tiering, material risks and incidents, compliance exposure, third-party posture, governance operations, and progress against goals. Store it with the access controls appropriate to board-oversight and fiduciary information. This template implements the board-reporting discipline described in `references/procurement-third-party-and-board-oversight.md`, including the accountability chain that runs from named owners up through a council to a board committee. It is a working report, not legal advice; fiduciary and disclosure expectations should be confirmed with qualified counsel at use time.

## When To Use

Use this report to structure the periodic AI-governance update to the board or its designated committee (audit, risk, or a dedicated technology or AI committee). Prepare it on a fixed cadence — commonly at least quarterly, with a standing agenda item — so AI risk is reviewed on a schedule rather than buried until an incident forces attention. Use it both for the recurring update and for a special briefing when a material AI incident or decision rises to the board's threshold. Management prepares, owns, and is accountable for the report's accuracy; the board's job is to ask questions and challenge the answers.

## When Not To Use

Do not use this report as the working-level risk assessment of a single system (that is `model-risk-assessment.md` or `third-party-due-diligence.md`), and do not use it as the intake form for a new use case (that is `use-case-intake-form.md`). This is the top-side summary for directors; the depth behind each metric must already exist in the references, registries, and lifecycle artifacts it aggregates.

## Report Identity

| Field | Entry |
|---|---|
| Report title | <e.g. Board AI-Governance Update — Q3 2026> |
| Reporting period | <start date — end date> |
| Prepared by / owner | <name, role, and function accountable for accuracy> |
| Reviewed by | <AI council / risk committee / audit> |
| Presented to | <board / audit committee / risk committee / technology or AI committee> |
| Meeting date | <YYYY-MM-DD> |
| Status | <draft / final / for decision / for escalation> |

## Executive Summary

Give directors the three-to-five things they must know in one screen, with the metrics that back each claim.

- State of the program: <one-paragraph overall assessment of safety, compliance, and governance posture>
- Material changes this period: <new high-tier systems, exits, acquisitions, or policy changes>
- Top risks and incidents: <the highest-residual-risk systems and any material incidents or near-misses>
- Key decisions or approvals requested: <what the board is being asked to do>
- Bottom-line metric set: <the few trended numbers that summarize the period, per the reference's metric guidance>

## Inventory And Tiering

Show how the AI portfolio is shifting, because directors need to know the scale and shape of what is being governed.

- Total AI systems in inventory: <count>
- Count by risk tier: <low / medium / high — and the trend versus last period>
- High-tier systems with a named owner: <count and percentage, and any gaps>
- New systems added this period: <list or count, with tiers>
- Systems retired or deprecated: <list or count>
- Coverage: <share of systems with completed risk assessments, model cards, or monitoring>

## Material Risks And Incidents

Surface the highest-risk systems, their residual risk, and anything that needed prompt attention.

- Highest-residual-risk systems: <the top systems and why they remain risky>
- Material incidents and near-misses this period: <list, severity, and status>
- Incidents escalated to the board / committee: <what was briefed and when>
- Time to detect and to remediate: <mean or worst-case metrics, trended>
- Risk-appetite exceptions: <any residual risk above the accepted appetite and the plan to close it>

## Compliance And Regulatory Exposure

Cover open obligations, audit findings, and any enforcement or investigation activity.

- Open compliance obligations: <count and the highest-priority items>
- Regulatory or enforcement activity: <investigations, notices, or filings relevant to AI>
- Audit findings: <open versus closed, and the material items>
- Policy and guidance status: <AI-specific policies in place, under review, or missing>
- Regulatory horizon: <known upcoming obligations the board should anticipate>

## Third-Party And Supply-Chain Posture

Report the vendor surface, because much of the AI footprint arrives through procurement.

- Material third-party AI engagements: <count and the critical ones>
- Vendors with current vs overdue diligence: <counts and any material gaps>
- Critical-vendor concentration: <dependency on any single vendor and the risk it creates>
- Supply-chain or model events: <data breaches, model changes, or end-of-support at a vendor>
- Data flows without a documented lawful basis: <count and the plan to resolve>

## Governance Operations

Show the state of the control environment and decision machinery.

- Policies and council decisions: <status of governance policies and approvals this period>
- Approvals granted or denied: <high-risk use cases or acquisitions reviewed and their outcome>
- Controls operating vs planned: <the state of the control environment across tiers>
- Exception and waiver register: <open exceptions and the process tracking them>
- Director education: <AI-governance education delivered or planned for the board>

## Progress Against Goals

Connect the program to the objectives the board approved, and name the gaps.

- Approved objectives: <the board-approved goals for the period>
- Progress against each: <status per objective, with evidence>
- Identified gaps and barriers: <knowledge, budget, regulatory uncertainty, or resource constraints>
- Plans for the next period: <the roadmap and what the board should expect>
- Maturity signal: <how the program is advancing, e.g. toward the framework-based maturity described in the references>

## Decision And Escalation Requests

Give the board a clear, bounded set of asks so the meeting produces decisions rather than a briefing.

- Decisions requested: <specific approvals or direction sought>
- Escalation path for material incidents: <how the board is briefed between scheduled meetings>
- Next reporting date: <the next scheduled report and any interim briefings>
- Recorded by / minuted by: <name and role>

## Completion

To complete this report: fill every labeled field, prepare the executive summary with the trended metric set that backs each claim, populate the inventory and tiering view from the registry, list material risks and incidents with severity and status, report compliance obligations and audit findings honestly, summarize the third-party and supply-chain posture including concentration, describe the state of governance operations, and chart progress against the board-approved goals. End with a bounded set of decisions or escalations for the board. Management must own and be accountable for the report's accuracy; the board reviews, asks questions, and challenges the answers. Rebuild the report on the fixed cadence you set and whenever a material incident or change warrants a special briefing.

> **Synthesized from** `references/procurement-third-party-and-board-oversight.md`, which draws on *The AI Product Manager's Handbook* and *Developing Cybersecurity Programs and Policies*, together with `research-org-board-governance.md` (current to August 2026). Fillable artifact of the `ai-governance` skill; educational context, not legal advice.

# AI Governance Council Charter

> **Confidentiality:** A completed charter names accountable executives, decision rights, and escalation paths. Store it with access controls appropriate to governance and board-oversight information. This template instantiates the council terms of reference described in `references/governance-operating-model.md`.

## When To Use

Use this charter to stand up (or refresh) any AI governance body — an AI ethics council, an AI risk council, an enterprise AI committee, or a board-level technology committee. The operating model reference explains the tiered council structure; this template turns the council's purpose, membership, responsibilities, decision rights, cadence, and reporting lines into a written terms of reference. Complete it when the council is created and review it at least annually or whenever the operating model or risk profile changes.

## When Not To Use

Do not use this template as a substitute for an individual role's job description, and do not use it to assign operational work that belongs to stewards and functional owners. A charter governs how a body deliberates and decides; it is not a RACI for every task. Use `model-risk-assessment.md` for single-model reviews and `use-case-intake-form.md` for routing individual use cases.

## Charter Meta

| Field | Value |
|---|---|
| Council name | `<name, e.g. Enterprise AI Risk Council>` |
| Charter version | `<version, e.g. 1.0>` |
| Effective date | `<YYYY-MM-DD>` |
| Next review date | `<YYYY-MM-DD>` |
| Sponsor / accountable executive | `<name and role>` |
| Status | `<draft / ratified / amended>` |

## Purpose

State, in one to three sentences, why the council exists and what outcomes it is accountable for. Anchor it to a mission statement so every decision can be traced back to it.

- Council purpose: `<one-to-three-sentence statement of the mandate and the outcomes it owns>`
- What the council is accountable for: `<list the decisions, standards, and risk approvals it must own>`
- What the council must NOT decide alone: `<identify matters that require executive sign-off or board approval>`

## Membership

List the representative roles and named individuals. A cross-functional council should bring together legal, compliance, risk, privacy, security, data, product, and engineering. Note alternates so the body is never blocked by a single person's absence.

| Role | Representative | Alternates | Term / rotation |
|---|---|---|---|
| <council chair> | <name> | <name> | <term> |
| <legal / compliance> | <name> | <name> | <term> |
| <risk management> | <name> | <name> | <term> |
| <privacy / data protection> | <name> | <name> | <term> |
| <security> | <name> | <name> | <term> |
| <data / product / engineering> | <name> | <name> | <term> |
| <business unit / domain steward> | <name> | <name> | <term> |

- Quorum: `<minimum number or roles required for a valid meeting>`
- Decision method: `<consensus / majority / by chair with recorded dissent>`

## Responsibilities

List the standing duties of the council. Tie each duty to the stage of the AI life cycle or the risk framework where it bites.

- Set and maintain AI policy, standards, and principles: <duty details>
- Review and approve higher-risk AI use cases and their residual risk: <duty details>
- Own the risk register and ensure entries above threshold are escalated: <duty details>
- Review monitoring, incident, and drift signals and direct responses: <duty details>
- Oversee third-party and procured AI diligence: <duty details>
- Prepare aggregate risk reporting for the executive team and board: <duty details>

## Decision Rights And Escalation

Make explicit who the council can decide, who it must consult, who it must inform, and how disputes are raised. Reference the operating model's RACI so one person is accountable for each outcome.

| Matter | Decision right | Consulted | Informed | Escalation path |
|---|---|---|---|---|
| Approve low-risk use case | <who decides> | <roles> | <roles> | <path> |
| Approve medium-risk use case | <who decides> | <roles> | <roles> | <path> |
| Approve high-risk use case | <who decides> | <roles> | <roles> | <path> |
| Approve residual-risk exception | <who decides> | <roles> | <roles> | <path> |
| Declare material incident | <who decides> | <roles> | <roles> | <path> |

- Escalation trigger and path: <describe when a matter must be raised to the executive sponsor, CEO, or board>
- Dispute resolution: <describe how a deadlock or contested decision is resolved and recorded>

## Meeting Cadence And Operation

Define how often the council meets, what it reviews, and how members prepare. The operating model reference notes that councils need a regular cadence and ground rules for psychological safety so that honest discussion, including disagreement, is possible.

- Meeting frequency: <e.g. every other week, monthly, quarterly>
- Session length: <e.g. 60–90 minutes>
- Standing agenda items: <list recurring items, e.g. new use cases, risk register, incidents, metrics>
- Pre-read expectations: <describe what members review before the meeting>
- Ground rules for discussion: <state expectations for candid disagreement and psychological safety>
- Record keeping: <state where decisions, minutes, and dissents are recorded and retained>

## Reporting And Oversight

Describe how the council reports up (to the executive sponsor and board) and down (to stewards and operating owners), consistent with the board tier's "noses in, fingers out" oversight posture.

- Reports to: <executive sponsor, CEO, board committee — name them>
- Report cadence and contents: <what is reported, how often, and to whom>
- Material-incident briefing path: <how the board is briefed promptly on material incidents>
- Interactions with stewards and operating owners: <how decisions are communicated and enforced downstream>

## Effectiveness Review

Define how the council evaluates its own performance so the charter stays a living instrument, not a filed artifact.

- Review trigger: <annual / on operating-model change / on material incident>
- Effectiveness criteria: <list measurable criteria, e.g. decisions within SLA, incidents caught early, documented dissent>
- Success measures: <list the metrics the council watches and reports>
- Amendment process: <who can change the charter and how it is ratified>

## Completion

To complete and ratify this charter: fill every labeled field, confirm each named member and alternate, obtain sign-off from the accountable executive (and board sponsor where applicable), record the ratification date and version, and store the ratified copy in the shared governance location referenced by the operating model. Schedule the next review date before circulating the final version.

> **Synthesized from** `research-org-board-governance.md` and the ideas of *Designing Data Governance from the Ground Up* and the *Data Governance Handbook* (see `references/governance-operating-model.md`). Fillable artifact of the `ai-governance` skill.

# Model Card

> **Confidentiality:** A completed model card records intended use, data, performance, fairness results, limitations, and monitoring for a specific model. Store it with access controls appropriate to the sensitivity of the model and its data. This template implements the model-card documentation discipline described in `references/fairness-bias-accountability.md` and `references/ai-lifecycle-governance.md`, and is informed by the current state of model and data cards in `research-technical-controls.md`. It is the durable, audit-ready record that accompanies a model-registry entry.

## When To Use

Use this card to document any model you develop, procure, or operate so that it is understood and applied only where it is suited. Finalize it at the evaluation gate, before deployment, and refresh it when the model, its data, its context of use, or its performance changes materially. The card is both the evidence that clears the evaluation gate and the reference that the monitoring and incident-response phases consult later. For a point-in-time analysis of how risky a model is and which tier it belongs in, use `model-risk-assessment.md`.

## When Not To Use

Do not use this card as a substitute for a risk assessment and tiering decision (`model-risk-assessment.md`), and do not use it in place of a use-case intake form for a brand-new application (`use-case-intake-form.md`). The card documents what a model is and how it behaves; it does not decide whether the residual risk is acceptable. Keep the card current through the lifecycle rather than treating it as a one-time launch artifact.

## Model Identity And Overview

| Field | Entry |
|---|---|
| Model name and version | <name and semantic version> |
| Model type | <e.g. classifier, regressor, LLM, agentic system, recommender> |
| Model registry / inventory ID | <registry entry this card accompanies> |
| Owner / accountable role | <name and role> |
| Development team | <name or team> |
| Release / card version | <version> |
| Effective date | <YYYY-MM-DD> |
| Next review date | <YYYY-MM-DD> |

## Intended Use

Describe what the model is for, who it serves, and where it should and should not be applied, so it is not used in a context for which it is unsuited.

- Intended function: <what the model does, e.g. classify, recommend, generate, automate, decide>
- Intended users and audience: <who operates it and who consumes its output>
- Intended domain and context of use: <the specific business process, environment, and jurisdiction>
- Intended decision outcome: <how the output is used and what decision it informs>
- Known non-intended uses: <applications the model is NOT suited for>
- Deployment status: <development / evaluation / production / shadow / retired>

## Data Description

Document what the model was trained on and what it consumes in production, so its provenance and limitations are explicit.

- Training data sources: <data sets, systems, or vendors supplying training data>
- Data collection and annotation method: <how data was gathered and labeled>
- Data sensitivity: <public / internal / confidential / personal / sensitive personal / regulated>
- Contains personal or protected-attribute data: <yes/no — if yes, list the types>
- Data lineage and governance: <where data comes from and how it is governed>
- Known data quality issues and gaps: <errors, missingness, historical bias, or staleness>

## Performance And Evaluation

Record how the model was evaluated, on what data, and how it performed — broken out by relevant subgroups so the spread across groups is visible rather than being collapsed into one average.

- Evaluation procedure: <test sets, holdouts, cross-validation, or benchmark methodology>
- Primary performance metrics and thresholds: <e.g. accuracy, precision, recall, F1, calibration, quality>
- Overall results: <the headline performance against the agreed thresholds>
- Results by subgroup: <performance disaggregated by relevant demographic or other groups>
- Robustness and edge cases: <how it performs under expected variation and known adversarial cases>
- Evaluation limitations: <what the evaluation does not cover>

## Fairness And Bias Assessment

State which fairness metrics were chosen, why, what they showed across groups, and which decisions were made in response — not a single number but an explicit, auditable statement.

- Fairness metrics computed: <e.g. demographic parity, equalized odds, equal opportunity, calibration>
- Metric selection rationale: <why these metrics were the right ones and how priorities were ordered>
- Results across protected groups: <what the subgroup results showed, including any disparate impact>
- Known biases or proxy features: <describe any identified bias or correlated-proxy concerns>
- Trade-offs accepted: <what was given up in accuracy or parity, and why it was defensible>
- Mitigations applied: <reweighting, threshold adjustment, human review, or other corrective actions>

## Limitations And Considerations

Document the situations where the model is likely to underperform, so it is not applied somewhere it is unsuited.

- Known limitations: <where the model is likely to fail, mislead, or behave unpredictably>
- Suitable versus unsuitable contexts: <restate the boundary between intended and non-intended use>
- Environmental or dependence assumptions: <what must hold in production for the model to behave as intended>
- Security and integrity concerns: <prompt injection, misuse, data exposure, supply chain, tooling>
- Human oversight requirements: <how and when a person reviews or overrides the output>

## Monitoring And Maintenance

Define how the model is watched in production and how it is kept current, so that drift, degradation, or reintroduced bias is detected and responded to rather than improvised.

- Monitoring configuration: <metrics, alert thresholds, owners, and response runbook>
- Drift detection approach: <how data drift and concept drift are detected and who responds>
- Retraining / refresh cadence: <how often the model is re-evaluated or retrained and on what trigger>
- Incident and response path: <how problems are escalated and how the card is updated in response>
- Retirement plan: <how the model is decommissioned and its outputs archived>

## Approval And Review

Record who approved the card and its contents, and how it is kept current.

| Review field | Entry |
|---|---|
| Prepared by | <name and role> |
| Reviewed by | <name and role> |
| Approved by | <name and role> |
| Approval date | <YYYY-MM-DD> |
| Review trigger | <annual / on drift / on material change / on incident> |

## Completion

To complete this card: fill every labeled field, finalize the intended-use and non-intended-use statements, record the data description and its limitations, report performance broken out by subgroup rather than a single average, state the fairness metrics chosen and their results, document limitations and human oversight requirements, define the monitoring and drift-detection configuration, and obtain the approvals named above. Attach the card to the model-registry entry as the source-of-record, and refresh it whenever the model, its data, its context of use, or its performance changes materially or on the review trigger you set.

> **Synthesized from** `research-technical-controls.md` and `research-org-board-governance.md` and the ideas of *Responsible AI in the Enterprise*, *Responsible AI: Best Practices*, and *AI Fairness* (see `references/fairness-bias-accountability.md` and `references/ai-lifecycle-governance.md`). Fillable artifact of the `ai-governance` skill.

# Model Risk Assessment & Tiering Worksheet

> **Confidentiality:** A completed assessment records inherent and residual risk, mitigations, and a tiering decision for a specific model. Store it with access controls appropriate to governance and board-oversight information. This template implements the NIST AI Risk Management Framework functions — Govern, Map, Measure, Manage — and the model risk tiering discipline described in `references/risk-management-and-frameworks.md`. It is a working worksheet, not a legal opinion; framework details should be confirmed against current NIST and ISO releases at use time.

## When To Use

Use this worksheet to run a single-model risk assessment and assign a risk tier before development deepens, at deployment, and whenever the model, its data, its context of use, or its controls change materially. It is the companion to the intake form: `use-case-intake-form.md` captures intent and initial classification, while this worksheet performs the full inherent-versus-residual assessment that justifies the tier. Complete it for any model whose tiering decision must be auditable and defensible. For documenting what a model is and how it behaves rather than how risky it is, use `model-card.md`.

## When Not To Use

Do not use this worksheet as the intake gate for a brand-new use case (that is `use-case-intake-form.md`), and do not use it as a substitute for ongoing post-deployment monitoring and drift detection, which the `model-card.md` template and the lifecycle gates describe. This is a point-in-time assessment that records the analysis and decisions; monitoring keeps those decisions honest after launch.

## Assessment Identity

| Field | Entry |
|---|---|
| Model / system name | <model or system name> |
| Assessment ID | <assessment reference, e.g. MRA-2026-017> |
| Model inventory / registry ID | <registry entry this assessment updates> |
| Use-case link | <linked intake-form ID, if applicable> |
| Assessor | <name and role> |
| Reviewing authority | <steward / AI council / board committee> |
| Assessment date | <YYYY-MM-DD> |
| Status | <draft / in review / approved / approved with conditions / rejected> |

## Purpose And Context (Map)

Establish the context of the AI system, as the NIST Map function directs: who the actors are, what the system is for, and where its risks and benefits arise across the life cycle.

- Intended function: <what the model does, e.g. classify, recommend, generate, automate, decide>
- Context and domain of use: <the specific business process, environment, and jurisdiction>
- Users and affected parties: <who operates it and who is affected by its output>
- Expected benefit and success metric: <the value it is meant to deliver and how success is measured>
- Alternatives considered: <non-AI or lower-risk alternatives and why they were set aside>
- Deployment environment and reach: <scale of exposure, e.g. users, transactions, or decisions per year>

## Data And Inputs

Describe what trains and feeds the model. Sensitive, high-volume, or personal data raises inherent risk and the controls required.

- Training and input data sources: <data sets, systems, or vendors supplying data>
- Data sensitivity: <public / internal / confidential / personal / sensitive personal / regulated>
- Contains personal or special-category data: <yes/no — if yes, list the types>
- Data lineage and provenance: <where data comes from and how it is governed>
- Data quality and known limitations: <gaps, errors, or biases you are aware of>
- Retention, minimization, and privacy: <how long data is kept and what is minimized>

## Autonomy And Decision Impact

Classify how much the model decides and how consequential its output is. This is a principal driver of the inherent risk tier.

- Level of autonomy: <human-in-the-loop review / human-on-the-loop review / fully automated>
- Decision type: <advisory / recommendation / direct action / automated decision>
- Decision impact: <informational / operational / financial / life- or liberty-affecting>
- Opportunity for human override: <how and when a person can review or reverse the outcome>
- Affected population: <who bears the consequences and any protected groups involved>

## Inherent Risk Assessment (Measure)

Score the risk the model would pose with no controls, validation, or mitigation applied — the danger present simply because the model exists and is used in this context. Rate likelihood and impact, then derive an inherent risk tier.

| Risk factor | Likelihood (1–5) | Impact (1–5) | Notes |
|---|---|---|---|
| Bias / fairness | <1–5> | <1–5> | <basis for the scores> |
| Data quality and privacy | <1–5> | <1–5> | <basis for the scores> |
| Accuracy / reliability | <1–5> | <1–5> | <basis for the scores> |
| Autonomy / decision impact | <1–5> | <1–5> | <basis for the scores> |
| Security / integrity | <1–5> | <1–5> | <basis for the scores> |
| Regulatory / legal exposure | <1–5> | <1–5> | <basis for the scores> |

- Inherent risk tier: <low / medium / high — justify from the scores above>
- Primary risk drivers: <list the factors that most raise inherent risk>
- Known limitations and underperformance contexts: <where the model is likely to fail or mislead>

## Risk Tiering Decision

Assign the model to a governance tier so the depth of review, validation, and monitoring matches the stakes. The tier should weight inherent risk over residual risk so the review depth tracks the true stakes, not just what strong controls currently hide.

- Assigned risk tier: <low / medium / high>
- Tiering rationale: <why this tier, referencing the drivers and scoring above>
- Required review depth: <standard / enhanced / full assessment + controls review>
- Required approvals: <who must sign off at this tier>
- Tiering decision recorded by: <name and role> on <YYYY-MM-DD>

## Mitigations And Controls (Manage)

Record the controls that bring inherent risk down, then re-score to estimate residual risk. The gap between inherent and residual is the value of the control environment.

| Control / mitigation | Owner | Status | Reduces which risk |
|---|---|---|---|
| <e.g. human-in-the-loop review> | <owner> | <planned / in place / verified> | <risk factor> |
| <e.g. subgroup fairness testing> | <owner> | <planned / in place / verified> | <risk factor> |
| <e.g. input/output monitoring and drift detection> | <owner> | <planned / in place / verified> | <risk factor> |
| <e.g. security and red-teaming> | <owner> | <planned / in place / verified> | <risk factor> |

## Residual Risk Assessment

Re-score the model with controls applied to determine the realistic risk the business actually carries day to day.

- Residual likelihood: <1–5>
- Residual impact: <1–5>
- Residual risk tier: <low / medium / high>
- Risk appetite check: <is residual risk within the organization's accepted appetite? state how you know>
- If residual risk exceeds appetite: <what additional controls are required, or why the model should not ship as-is>

## Risk Register And Escalation

Link this assessment to the central risk register and to the escalation path so the finding is acted on rather than filed away.

- Risk-register entry ID: <registry reference for the residual risk>
- Escalation trigger: <what bumps this model to a higher review tier or to the council / board>
- Review cadence: <when this assessment is revisited, e.g. annually, on drift, on material change>
- Linked artifacts: <model card, monitoring configuration, audit logs, incident reports>

## Linked Assessments And Closure

The model-risk worksheet coordinates specialized assessments; it does not replace them. Decide which
follow-ups are required from the data, affected population, deployment posture, capability inventory,
and risk tier. Give every required assessment an owner and due date, and link evidence or the
recorded exception.

| Follow-up assessment | Required? | Owner | Due date | Status | Evidence / finding ID | Closure or exception decision |
|---|---|---|---|---|---|---|
| Privacy impact / data-protection review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Fairness and affected-group review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Security, threat, and abuse review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Vendor / third-party due diligence | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Data governance, provenance, and licensing | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Validation / assurance or quality review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Human oversight and contestability review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |
| Other specialist review | <yes / no / conditional> | | | <not started / in progress / complete / waived> | | |

- Required follow-ups that remain open: <list the risk, owner, due date, and interim control>
- Approval condition: <all required assessments complete, or bounded exception with approver,
  compensating control, expiration, and re-review trigger>
- Closure rule: <what evidence proves the finding was addressed and residual risk updated>
- Process-health measures: <assessment coverage, bypassed reviews, overdue actions, repeated
  findings, escalation rate, and time from completed assessment to decision>

## Completion

To complete this assessment: fill every labeled field, score inherent risk honestly before considering controls, assign and justify the tier, list every planned or existing control with its owner and status, re-score to residual risk and check it against the organization's appetite, identify and track every required linked assessment with an owner, due date, evidence, and closure or bounded exception, link the entry to the risk register, and obtain sign-off from the reviewing authority named above. Record the tiering decision as the source-of-record that every later lifecycle gate — evaluation, deployment, monitoring, and retirement — calibrates against. Revisit the worksheet whenever the model, its data, its context of use, or its controls change materially, or on the review cadence you set here.

> **Synthesized from** `research-standards.md` and `research-technical-controls.md` and the ideas of *Responsible AI in the Enterprise* and *Platform and Model Design for Responsible AI* (see `references/risk-management-and-frameworks.md`). Fillable artifact of the `ai-governance` skill.

# Third-Party AI Due Diligence Questionnaire

> **Confidentiality:** A completed questionnaire records what an external AI engagement — a hosted model API, pretrained weights, a licensed dataset, an ML platform, or a consultancy — brings in, what it could harm, and what the contract and controls must protect against. Store it with access controls appropriate to procurement, governance, and board-oversight information. This template implements the due-diligence and supply-chain discipline described in `references/procurement-third-party-and-board-oversight.md`, proportional to the risk tier assigned under `references/risk-management-and-frameworks.md`. It is a working questionnaire, not legal advice; contract, liability, and regulatory terms should be confirmed with qualified counsel at use time.

## When To Use

Use this questionnaire to investigate a proposed third-party AI engagement **before** you commit to it, so that you accept the relationship with your eyes open rather than discover its flaws after signing. Run it on intake for any vendor, model, dataset, or service that will support a governed use case, and complete it in full for anything that lands in a medium or high risk tier. Keep the completed questionnaire as the source-of-record that justifies the onboarding decision, and revisit it when the system, the vendor, its subprocessors, or the surrounding risk profile changes. It is the intake companion to `model-risk-assessment.md`; use that worksheet for the full inherent-versus-residual scoring of a model you are already engaged with.

## When Not To Use

Do not use this questionnaire as the routine intake form for an internally built model or use case (that is `use-case-intake-form.md`), and do not use it as a substitute for ongoing vendor monitoring and exit planning, which the lifecycle discipline describes. This is a point-in-time diligence artifact that characterizes an external engagement and sets the contract terms; monitoring, audit, and exit keep those commitments honest after onboarding.

## Engagement Identity

| Field | Entry |
|---|---|
| Engagement / vendor name | <vendor, model, or service name> |
| Due-diligence ID | <reference, e.g. DD-2026-032> |
| Engagement type | <hosted model API / open-source weights / licensed dataset / ML platform / professional service> |
| Supporting use case | <linked intake-form ID and use case name> |
| Risk tier (provisional) | <low / medium / high> |
| Diligence owner | <name and role> |
| Reviewing authority | <procurement / AI council / risk committee / board committee> |
| Diligence date | <YYYY-MM-DD> |
| Status | <draft / in review / approved / approved with conditions / rejected> |

## Purpose And Scope (Map)

Establish what the component is, how it is used, and where its risks and benefits arise across the supply chain, as the governance reference directs.

- Intended function of the component: <what it does, e.g. generate, classify, recommend, automate, enrich>
- Use case and deployment context: <the business process, environment, and jurisdiction it supports>
- How it is integrated: <API / fine-tuned / embedded library / fully hosted / human-supervised>
- Affected parties: <who operates it and who is affected by its output>
- Expected benefit and success metric: <the value it delivers and how success is measured>
- Alternatives considered: <in-house build or a different vendor, and why they were set aside>

## Provider Identity And Standing

Assess who owns and operates the component, because its stability, jurisdiction, and accountability shape your exposure.

- Provider name and legal entity: <registered name and corporate form>
- Location and jurisdiction: <country / state, and governing law>
- Size and financial stability: <scale, funding, or financial indicators of continuity>
- Relevant regulatory status: <whether the provider is subject to a regime relevant to your data or use>
- Ownership and subprocessors: <who controls the provider and which subprocessors it relies on>
- Reputation and track record: <public incidents, known weaknesses, or sanctions the team is aware of>

## Data Flows And Handling

Enumerate the data that moves to and from the vendor. Personal, regulated, or proprietary data raises lawful-basis, transfer, and confidentiality duties.

- Inputs shared with the vendor: <what data is sent, and for what purpose>
- Outputs and their use: <what the vendor returns and how you act on it>
- Data sensitivity: <public / internal / confidential / personal / sensitive personal / regulated>
- Contains personal or special-category data: <yes/no — if yes, list the types>
- Lawful basis and transfer basis: <the basis relied on and any cross-border mechanism>
- Vendor data handling commitments: <training, retention, storage location, reuse, and subprocessors>

## Model And Capability Assessment

Characterize the model itself so you know what you are relying on and how to validate it.

- Model origin and lineage: <who built it, on what data, and what its provenance is>
- Version and registry entry: <exact model version and any AI BOM / model card references>
- Capabilities and limitations: <what it does well and where it is known to fail>
- Intended vs allowed use: <vendor's stated intended use and any restricted uses>
- Known biases and fairness data: <what bias testing the vendor provides and its results>
- Performance and accuracy evidence: <benchmarks, evaluations, or guarantees supplied>

## Security And Privacy Controls

Ask for evidence of the vendor's controls rather than accepting assertions at face value.

- Security certifications and audits: <e.g. SOC 2, ISO 27001, NIST-based assessments — and their scope>
- Privacy certifications and practices: <e.g. GDPR/DPA posture, privacy-enhancing measures>
- Access control and data isolation: <how your data is isolated from other tenants or use>
- Encryption in transit and at rest: <the mechanisms in place and their scope>
- Model-governance practices: <how the vendor governs its own models and changes>
- Right to audit or review evidence: <whether you can inspect the controls you rely on>

## Incident, Reliability, And Exit

Confirm what happens when things go wrong and how you would leave.

- Incident-notification commitment: <the window and format for notifying you of a material incident>
- Uptime and performance commitments: <service-level targets and remedies for breach>
- Liability and indemnification terms: <how liability and IP indemnity are allocated>
- Data return and deletion on exit: <what happens to your data when the contract ends>
- Migration and transition assistance: <the path for moving off the vendor>
- Dependency and concentration: <how critical this vendor is and any concentration risk it creates>

## Proportionality And Approvals

Weigh how deep this diligence must go and who must sign off, driven by inherent risk, impact, and likelihood.

- Inherent risk drivers: <the factors that most raise the risk of this engagement>
- Required diligence depth: <standard / enhanced / full assessment plus controls review>
- Contractual commitments to encode: <data handling, subprocessor disclosure, notification, audit rights, exit>
- Required approvals: <who must sign off at this tier>
- Decision and rationale: <approved / rejected / approved with conditions — and why>
- Decision recorded by: <name and role> on <YYYY-MM-DD>

## Completion

To complete this questionnaire: fill every labeled field, classify the engagement and assign a provisional risk tier, characterize the provider and its jurisdiction, enumerate the data flows and the vendor's handling commitments, review the model's provenance and capabilities, record the security and privacy evidence you actually obtained, confirm the incident, reliability, and exit terms, and route the outcome to the reviewing authority named above. Obtain the required approvals and record the decision as the source-of-record that every later onboarding, monitoring, and audit step calibrates against. Revisit the questionnaire whenever the system, the vendor, its subprocessors, or the surrounding risk profile changes materially.

> **Synthesized from** `references/procurement-third-party-and-board-oversight.md`, which draws on *The AI Product Manager's Handbook* and *Developing Cybersecurity Programs and Policies*, together with `research-org-board-governance.md`. Fillable artifact of the `ai-governance` skill; educational context, not legal advice.

# AI Use-Case Intake Form

> **Confidentiality:** A completed intake form describes data, decisions, and risk. Store it with access controls appropriate to the sensitivity of the use case. This template turns the earliest lifecycle gate and the risk-register entry described in `references/risk-management-and-frameworks.md` and `references/ai-lifecycle-governance.md` into a fillable form.

## When To Use

Use this form to open a governed lifecycle for any proposed or newly discovered AI use case — whether built in-house, procured from a vendor, or embedded in an inherited system. Completing it routes the use case into the risk register and to the correct depth of review before significant investment, model training, or deployment. Submit it at the intake stage; revisit it when the use case, data, or context changes materially.

## When Not To Use

Do not use this form as a substitute for a full single-model risk assessment. If the use case already exists and you are evaluating an individual model's residual risk, use `model-risk-assessment.md`. This form captures intent and initial classification; it does not replace monitoring, drift detection, or incident response after deployment.

## Use-Case Identity

| Field | Entry |
|---|---|
| Use-case name | <short descriptive name> |
| Use-case ID | <registry ID, e.g. UC-2026-014> |
| Submitted by | <name and role> |
| Submitted date | <YYYY-MM-DD> |
| Business unit / domain | <unit> |
| Status | <new / triaged / in review / approved / rejected / live / retired> |

## Deployment Posture And Control Boundary

Classify how the organization reaches the AI capability before assigning control owners. A posture
changes where responsibility and evidence sit; it does not determine safety by itself. Select all
that apply and record any hybrid boundary.

- Primary posture: [ ] SaaS or application consumer  [ ] API integrator  [ ] Model hoster
- Agentic overlay: [ ] no  [ ] tools or external actions  [ ] multi-step planning or delegation
  [ ] persistent memory  [ ] browser or web access
- Model / provider / hosting boundary: <who operates the model, serving stack, and updates>
- Organization-controlled boundary: <interface, preprocessing, retrieval, guardrails, tools, logs,
  approvals, and downstream actions the organization operates>
- Vendor-controlled boundary: <retention, training use, model behavior, subprocessors, availability,
  incident response, deletion, and change notification that require evidence or contract>

| Control layer | Organization owner | Vendor / provider owner | Evidence or contract reference | Gap or follow-up |
|---|---|---|---|---|
| Identity, interface, and user access | | | | |
| Model, updates, and serving | | | | |
| Input, retrieval data, and permissions | | | | |
| Output validation, tools, and actions | | | | |
| Logs, retention, deletion, and exit | | | | |

**Capability escalation indicators:** [ ] untrusted input or retrieved content  [ ] sensitive or
regulated data access  [ ] external tool or browser calls  [ ] write or destructive actions
[ ] cross-user or persistent memory  [ ] multi-step agency or delegation  [ ] consequential decision.
Record the narrowest deployed scope and the evidence needed to verify each selected indicator.

## Purpose And Context

Describe what the use case does, who it serves, and why it is being built or adopted.

- Problem statement: <what problem the AI system addresses and for whom>
- Intended function: <what the system does, e.g. recommend, classify, generate, automate, decide>
- Users and affected parties: <who operates it and who is affected by its output>
- Expected benefit: <what value is expected, with a rough scale or metric>
- Alternatives considered: <non-AI or lower-risk alternatives and why they were set aside>

## Data And Inputs

Describe the data that trains and feeds the system. Sensitive, high-volume, or personal data raises the inherent risk and the controls required.

- Primary data sources: <data sets, systems, or vendors supplying data>
- Data sensitivity: <public / internal / confidential / personal / sensitive personal / regulated>
- Contains personal or special-category data: <yes/no — if yes, list the types>
- Data lineage and provenance: <where data comes from and how it is governed>
- Data quality and known limitations: <describe quality issues, gaps, or biases you are aware of>
- Retention and minimization: <how long data is kept and what is minimized>

## Autonomy And Decision Impact

Classify how much the system decides and how consequential its output is. This drives the tier.

- Level of autonomy: <human-in-the-loop review / human-on-the-loop review / fully automated>
- Decision type: <advisory / recommendation / direct action / automated decision>
- Decision impact: <informational / operational / financial / life- or liberty-affecting>
- Scale of exposure: <approximate users, transactions, or decisions affected per year>
- Opportunity for human override: <how and when a person can review or reverse the outcome>

## Initial Risk Classification

Record the inherent risk (with no controls applied) and any immediate risk considerations, aligned with the tiering discipline in `references/risk-management-and-frameworks.md`.

- Inherent risk tier: <low / medium / high — justify>
- Primary risk drivers: <list factors such as sensitive data, autonomy, decision impact, volume>
- Known biases or fairness concerns: <describe any identified bias or disparate-impact risk>
- Known security or integrity concerns: <prompt injection, data exposure, misuse, tooling, supply chain>
- Suggested review depth: <standard / enhanced / full assessment + tiering>
- Proposed controls to reach acceptable residual risk: <list candidate mitigations>

## Review Routing

Route the use case to the right level of review based on its tier, and record the decision.

| Routing field | Entry |
|---|---|
| Assigned reviewer / assessor | <name and role> |
| Review path | <steward only / AI council / board committee> |
| Required approvals | <who must sign off before development or deployment> |
| Escalation trigger | <what bumps the use case to a higher review tier> |
| Linked risk-register entry | <registry reference, if created> |

## Decisions And Next Steps

Record the outcome and the follow-up actions so the intake is closed out.

- Decision: <approved / approved with conditions / deferred / rejected / referred>
- Conditions or mitigations required: <list any conditions attached to approval>
- Next steps and owners: <describe the next actions, owners, and due dates>
- Re-review trigger: <what change would require the intake to be reopened>

## Completion

To complete this intake: fill every labeled field, confirm the submitted-by and business owner, perform the initial risk classification honestly (inherent risk first), route the form to the assigned reviewer through the path your operating model defines, record the decision and any conditions, and file the completed entry in the risk register as the source-of-record. Revisit the form whenever the use case, data, context, or risk tier changes materially.

> **Synthesized from** `research-standards.md` and `research-technical-controls.md` and the ideas of *Responsible AI in the Enterprise* and *Platform and Model Design for Responsible AI* (see `references/risk-management-and-frameworks.md` and `references/ai-lifecycle-governance.md`). Fillable artifact of the `ai-governance` skill.