3 The Six Level GenAI Governance Framework
GenAI governance in this chapter is presented as a lifecycle, not a single policy document or security control. The authors argue that real governance must connect executive intent, risk assessment, technical design, launch approval, live monitoring, and continuous improvement so that decisions made early are actually enforced later. Using examples like unauthorized data exposure, chatbot jailbreaks, and weak operational tracking, the chapter shows that policies only matter when they are translated into concrete artifacts, owners, review steps, logs, tests, and escalation paths.
The six levels build on one another. Strategy and policy define what is allowed, who owns decisions, and which uses need review; impact assessment evaluates a proposed use case and records required conditions; design review checks whether the architecture can truly implement those conditions; launch readiness verifies that the tested release matches the approved design; operations and monitoring watch the live system for drift, misuse, and incidents; and learning and improvement feed findings back into updated policies, tooling, and test procedures. The chapter repeatedly emphasizes that each level needs standard artifacts and metrics, such as inventories, risk tiers, assessments, test plans, and incident records, so governance can be measured instead of assumed.
MediAssist is used throughout to show how this works in practice, especially for healthcare where stakes are high. The chatbot first operates under policy constraints, then undergoes an impact assessment for features like dermatology photo referral, followed by architecture changes, launch review, and monitoring for unsafe outputs or privacy leaks. The chapter ends by extending the framework to agentic systems, arguing that autonomous agents require even tighter runtime controls such as policy-as-code, identity and authorization checks, memory governance, request tracing, human review triggers, and kill switches. Across all of this, the core message is that effective AI governance is a loop: define rules, enforce them technically, observe what happens, and update the system when reality breaks the rules.
The six governance levels before and after go-live. Level 6 routes operational findings back to the policy, assessment, implementation, or test that must change.
How organizational values funnel into day-to-day artifacts, by passing through a fixed chain of ownership that turns values into policies, risk assessments and training. And what's observed in production then feeds back into those same values.
Impact Assessment with the relevant inputs and follow-up actions.
Illustration: MediAssist’s Derm-Cam feature, using image recognition to recommend whether the patient should see a dermatologist.
An approved deployment pattern for AI systems. The technical standard defines the path AI traffic must take: an approved gateway, checks on what enters and leaves the model, a registered set of tools and data sources, and the shared governance services that hold the rules those checks apply. This runtime layer is often called a control plane. Architecture review compares each proposed design against it and records every departure.
Illustrative example of how image EXIF metadata might reveal sensitive information about the user.
Illustration of how image features may lead to re-identification and privacy risks: the fictional Derm-Cam photo captures the user’s personal details from a printed report in the background.
Illustration of prompt injection through an image. Hidden text in the image could reveal system instructions or alter downstream model behavior.
Governance feedback cycle from live findings to changed controls. Once an update is tested and deployed, it starts generating new logs and telemetry of its own, which feeds straight back into the next round of findings.
A policy engine checks each proposed action against the rules before it runs.
Agentic identity and logs: the same action, recorded differently. Without a broker the record names a shared account. With one, it names the agent, names the person it acted for, and the credential it used has already expired
Summary
AI governance usually breaks down during execution, not during policy writing. Most organizations have clear rules, but they lack the technical mechanisms to enforce them in software. For example, as we discussed in the introduction of this chapter, Community Bank maintained a strict policy against unapproved software, yet employees still processed Social Security numbers through unauthorized tools. Sierra built abuse detection features, yet for the GAP instance it was not configured properly. In both cases, valid policies or tooling existed on paper, but they were not enforced or used properly.
The Six-Level Model we introduced in this chapter (Fig 3.1) bridges this gap between policy and engineering. Each level takes a rule from the level above it and translates it into something the next level can physically test, measure, and audit:
- Level 1: Strategy & Policy. Defines what the organization will and will not do with generative AI, its risk classification and risk appetite, who can approve exceptions, and how teams submit new project proposals.
- Level 2: Risk & Impact Assessment. Reviews specific proposals, assigns risk tiers, and defines mandatory operating conditions, including vendor contract requirements.
- Level 3: Design & Implementation Review. Checks proposed software architectures against security standards and design patterns. It ensures that every Level 2 condition becomes a design element tied to a tracked engineering ticket.
- Level 4: Launch Readiness Review. Tests safety and compliance controls on the final release candidate itself, rather than relying on checks from earlier builds.
- Level 5: Operations & Monitoring. Manages the live system through configuration records, request logging, customer complaints and other data points. It is a shared oversight across engineering and legal teams.
- Level 6: Learning & Improvement. Ensures that when a flaw reaches a live system, the team updates the relevant policy, architecture, process, test or guardrail that let it through.
If you have no governance program yet, three artifacts give you a baseline: a one page Responsible AI Charter naming what your organization will and will not do with GenAI, an AI inventory listing every GenAI tool in use, and a risk tier assigned to each entry using Table 3.5. Technical reviews and controls will follow based on what you detect and what you want to set as policy.
Most organizations focus on Level 1 because this is where things get started and many certifications require it. However, underinvesting in other levels such as technical controls at Level 3 or monitoring at Level 5 means that your policies may not be executed in the live system. A clinician’s corrections may be added to patient notes, but never reach the model team. You should start taking two or three metrics from each level and start reporting them. Metrics show whether the level is still working or requires additional work.
The next four chapters turn from how to govern to what to govern. Security (Chapter 4), privacy (Chapter 5), ethics (Chapter 6), and regulation (Chapter 7) each add risks specific to their domain, and the Six Level model stays the frame that tells you how much rigor each one warrants.
AI Governance ebook for free
