Overview

1 Governing Generative

Generative AI is already transforming work by writing, coding, and creating on demand, but the chapter stresses that its speed and flexibility also amplify failures, from hallucinated legal citations to data leaks, prompt injections, and deepfakes. Because these systems can be used instantly across many contexts, traditional governance approaches built for predictable software no longer suffice. The central message is that organizations need a new way to manage risk that recognizes GenAI’s scale, unpredictability, and ability to cause public, legal, financial, and reputational harm very quickly.

The chapter explains that governance, risk management, and compliance for AI are not just about ticking boxes; they are essential for turning AI into something trustworthy and sustainable. Governance sets direction and accountability, risk management continuously identifies and reduces AI-specific threats, and compliance ensures alignment with laws, regulations, and internal commitments. Yet compliance alone is not enough, because a system can be formally compliant and still unsafe or unfair in practice. Effective AI governance must be built into daily work from the beginning, not added after deployment, and it must adapt as models, threats, and regulations evolve.

To make this practical, the chapter proposes a lifecycle-based framework for GenAI governance that runs from strategy and risk assessment through design, testing, deployment, monitoring, and retirement. This model emphasizes layered controls, continuous evaluation, and shared responsibility across model developers, integrators, and users. It also shows why GenAI requires new tools such as threat modeling, red teaming, output filters, monitoring dashboards, and feedback loops for improvement. The overall takeaway is that responsible AI innovation is possible, but only when organizations treat governance as an ongoing operating discipline rather than a one-time review.

Generative models such as ChatGPT can often produce highly recognizable versions of famous artwork like the Mona Lisa. While this seems harmless, it illustrates the model's capacity to memorize and reconstruct images from its training data; a capability that becomes a serious privacy risk when the training data includes personal photographs.
Trust in AI: experts vs general population. Source: Pew Research Center[46].
Classic GRC compared with AI GRC and GenAI GRC
Six Levels of Generative AI Governance. Chapter 2 will expand into what control tasks attach to each checkpoint.

Summary

  • GRC for generative AI is not a dry compliance exercise, but it is an active, risk informed, ongoing discipline that requires involvement from legal to engineering.
  • The Six-Level Governance (6L-G) model spans the AI lifecycle and multiple layers of risk, making sure critical issues aren’t missed. The model gives you that lifecycle structure (strategy, risk assessment, implementation, testing, operations, improvement), catching issues where they're still cheap to fix.
  • Many risks cut across the org chart: hallucination, prompt injection, model theft, and data deletion obligations. These touch security, privacy, legal, and product all at once, so no single team owns the problem on its own.
  • We have also provided a teaser of the tools and practices that can address those challenges, giving you a sense that this is manageable with the right approach.

FAQ

Why does generative AI need governance now more than ever?Generative AI is being adopted extremely quickly and can create high-impact outputs across many domains, but it can also hallucinate, leak data, and be manipulated in ways that traditional controls were not designed to handle. Governance is needed now to reduce reputational, legal, security, and safety risks while still enabling innovation.
What makes GenAI risks different from traditional software risks?Unlike conventional software, GenAI is probabilistic, context-dependent, and harder to predict or fully inspect. It can fail in unexpected ways even when the surrounding system is well secured, and remediation is often more difficult because fixes may require retraining, guardrail changes, or broader redesigns rather than a simple patch.
What does GRC mean in the context of AI?In AI, GRC stands for Governance, Risk Management, and Compliance. Governance sets direction and accountability, risk management continuously identifies and addresses AI-related risks, and compliance ensures adherence to laws, regulations, standards, and internal policies. Together, they create a practical structure for trustworthy AI use.
Why is compliance alone not enough for GenAI?Compliance is only the minimum bar. A system can satisfy documentation or disclosure requirements and still be unsafe, biased, privacy-invasive, or unreliable in practice. GenAI governance aims for demonstrable trustworthiness, not just checklist compliance.
What are the main categories of GenAI risk discussed in the chapter?The chapter highlights several major risk areas: hallucinations and misinformation, prompt injection and jailbreaks, data privacy and model memorization, model extraction and IP theft, bias and discrimination, and compliance failures around data use, consent, and erasure obligations.
What is the six-level GenAI governance model?The six-level model is a lifecycle-based framework that organizes governance across Strategy & Policy, Risk & Impact Assessment, Design & Implementation Review, Acceptance Testing, Operations & Monitoring, and Learning & Improvement. It ensures controls are applied from the earliest planning stage through deployment, monitoring, and decommissioning.
How does the chapter suggest organizations handle AI hallucinations?The chapter says hallucinations cannot be eliminated entirely, but they can be caught early and contained through governance and controls such as confidence scoring, retrieval grounding, selective human review, testing, and monitoring. The idea is to prevent errors from snowballing into larger incidents.
Why are prompt injection and jailbreak attacks so concerning?These attacks exploit the flexible natural-language behavior that makes GenAI useful in the first place. They can hijack outputs or even downstream actions, causing systems to reveal sensitive information, bypass rules, leak data, or perform unintended tasks.
What tools and practices support GenAI governance?The chapter points to a mix of procedural, technical, and cultural tools: charters, impact assessments, threat modeling, red-teaming, AI firewalls, model cards, output scanners, drift dashboards, logging, and post-incident reviews. Good governance combines these tools rather than relying on any single control.
What is the chapter’s overall message about governing generative AI?The core message is that GenAI governance is not a separate end-stage audit but a living program embedded throughout the AI lifecycle. Organizations that build governance early and continuously are better able to innovate responsibly, protect users, and avoid costly failures later.

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • AI Governance ebook for free
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • AI Governance ebook for free