Overview

8 Governing for Uncertainty

Generative AI governance is best understood as managing uncertainty rather than waiting for perfect certainty. The chapter uses cloud computing as a comparison: organizations adopted cloud systems before every risk was fully settled, then made them manageable by assigning responsibility, adding controls, improving logging, and building standards and shared tools. The same lesson applies to AI: the goal is not to eliminate all risk, but to make a system’s authority, evidence, and remaining uncertainty visible enough that a real decision can be made.

Much of what organizations can govern today comes down to the full system around the model, not the model alone. That includes what data it can reach, what tools it can use, who approves actions, what gets logged, and who is accountable when behavior changes. The chapter argues that approval is always conditional and temporary, because once a system is live, new data, new model behavior, or new workflows can change the risk. Good governance therefore means narrowing permissions, isolating sensitive actions, and keeping enough evidence to explain and respond to failures.

Some problems remain genuinely unsolved: prompt injection can steer autonomous systems through ordinary text, deleting data does not always remove its influence from a trained model, model outputs can be inaccurate or misleading even when they cite real sources, and misalignment can lead agents to take unintended actions. Better standards and tools are emerging to help with identity, authorization, tracing, policy enforcement, and auditability, but they do not remove the need for judgment. The chapter closes by urging organizations to make case-by-case decisions about what each system may do, under what conditions, and when uncertainty is high enough to say not yet.

Summary

  • Many AI governance problems can be addressed today by controlling the system around the model: data access, tools, permissions, evidence, monitoring, and reassessment.
  • Some problems remain unsettled even for careful organizations, including prompt injection, machine unlearning, durable evaluations, explainability, provenance, and agent misalignment.
  • Emerging standards and tools may make governance easier, but they will not remove the need for judgment.
  • Organizations should govern for uncertainty by limiting what systems can read and do, collecting evidence, and deciding who may accept the remaining risk.

FAQ

What does it mean to govern an AI system “today”?It means making decisions about a real system with current evidence: who owns it, what data it can use, what tools and permissions it has, what controls constrain it, what evidence supports release, what is monitored, and who can respond when things change.
Why is approving the model not enough for AI governance?Because the model is only one part of the system. Governance also has to cover the workflow around it, including data access, tool use, user reliance, vendors, infrastructure, logging, and escalation paths.
What kinds of AI risks can organizations address now?Organizations can already govern access, permissions, logging, approvals, monitoring, evidence collection, and release conditions. They can also narrow the model’s role so failures are easier to trace and contain.
Which AI governance problems still do not have reliable technical solutions?Examples include prompt injection, machine unlearning, durable evaluations, complete explanations of model outputs, and fully understanding misalignment or hidden behavior in autonomous agents.
Why is prompt injection considered a system problem rather than just a prompt problem?Because malicious text can travel through documents, emails, webpages, connectors, plugins, or external services and influence what the agent reads or does. The risk comes from the whole connected system, not only the prompt itself.
Why is deleting data from an AI system so difficult?Personal data can exist across logs, caches, retrieval stores, dashboards, vendor systems, and training or fine-tuned model weights. Deleting it everywhere is already hard, and machine unlearning is still immature.
Why can’t organizations promise full explainability for AI outputs?Because records of inputs, retrieved sources, tool calls, and outputs do not reveal exactly why the model produced a specific answer. A model may also give an explanation that sounds plausible without faithfully reflecting its internal process.
What can organizations do about AI misalignment right now?They can limit autonomy by capping steps before human review, isolating the system from unnecessary tools and systems, and routing consequential actions through escalation channels that the agent cannot bypass.
How are standards and tools making AI governance easier?Standards and shared tools are beginning to provide common approaches for identity, authorization, observability, evaluation, policy enforcement, and evidence collection, making controls more reusable and auditable across systems.
What is the main governance lesson for organizations using generative AI?Do not wait for perfect certainty, but do not ignore risk either. Decide what each system may do, under what conditions, who accepts the remaining risk, and what would cause the decision to be reopened if facts or rules change.

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • AI Governance ebook for free
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • AI Governance ebook for free