17 AI in policy practice
As AI tools became embedded in everyday workflows, ACME explored where they could help with authorization-related work without being trusted to make access decisions. The chapter argues that AI should not become the policy engine because authorization requires deterministic, explainable, and auditable decisions, while AI is inherently inferential and nondeterministic. Instead, ACME kept a strict boundary: policy engines remain the source of truth for permit and deny decisions, and AI is limited to supportive roles.
Within that boundary, AI proved useful for policy authoring and analysis. It helped engineers explain existing decisions, explore alternative ways to express the same intent, analyze concrete requests, and review logs to spot patterns, edge cases, and surprising outcomes. These uses improved clarity and maintainability, but human judgment remained central, since the AI’s output was always advisory and had to be checked against actual policy evaluation traces and runtime evidence.
The chapter also shows how this principle applies to RAG systems, where authorization must happen before retrieval so unauthorized data never enters the model’s context. By enforcing access control at the retrieval boundary and using decision logs to connect runtime behavior back to governance, ACME made AI-enabled systems safer and more accountable. The broader takeaway is that policy is a living practice: AI can accelerate understanding and review, but people, policy engines, and governance processes must retain authority over enforcement and change.
Nondeterministic AI authorization. When authorization intent is embedded in prompts and inferred by an AI system, identical requests with the same context can result in different outcomes. In this example, the same access request and prompt lead to both permit and deny decisions, each with a plausible explanation. Since the reasoning is inferred rather than evaluated as policy, the results are not repeatable and cannot be used as a reliable basis for access enforcement.
Authorized context for AI prompts. ACME performs authorization before prompt construction, producing an authorized context that defines which data and actions are permitted for the current request. The AI model may receive sensitive data, but only data approved by the authorization layer. The model does not query sensitive data directly or decide what it is allowed to access.
Design-time intent versus runtime reality. Authorization policies are created to express intent at design time, while their effects are revealed through observed decisions at runtime. Comparing these two perspectives (how policies were intended to work and how they actually behave in production) helps teams identify mismatches, unexpected outcomes, and areas that need governance review.
Authorization before retrieval in RAG systems. Authorization constrains retrieval rather than generated output. By enforcing access before data is retrieved, the system ensures that the language model operates only on authorized inputs, making information leakage structurally impossible rather than merely discouraged.
Runtime analysis and governance loop. Authorization outcomes captured in decision logs are analyzed with AI assistance and surfaced as questions, patterns, and reports. These insights feed into governance processes, where humans interpret the evidence and decide whether policy changes are needed. The flow makes authorization behavior observable and actionable without placing AI in the enforcement path.
Summary
- Authorization needs predictability, transparency, and traceability, which AI systems cannot deliver.
- AI works best when it is limited to assistance roles such as drafting, explaining, and analyzing.
- Embedding authorization logic within prompts as text results in unpredictable and unsafe outcomes.
- Clear architectural boundaries stop AI systems from turning into implicit enforcement tools.
- Design-time intent and runtime behavior frequently differ, especially in complex policy systems.
- Decision logs offer the evidence necessary to understand how authorization behaves in practice.
- AI-assisted analysis helps reveal unexpected outcomes, edge cases, and policy interactions.
- Governance relies on observable actions, not guessed intentions or likely explanations.
- In retrieval-augmented generation systems, retrieval serves as the proper enforcement point.
- Authorizing before retrieval prevents unauthorized data from ever entering an AI model’s context.
- Policy residuals can be used to convert authorization results into retrieval constraints.
- Architecture, not prompts, determines whether AI-enabled systems are safe by construction.
- Accountability emerges when decisions are traceable from outcomes back to policy and ownership.
- Policy becomes a practice when authoring, observation, and governance form a continuous loop.
Authorization in Action ebook for free
