Identity and ownership
Every production agent needs a stable identity, business owner, technical owner, purpose, environment, and lifecycle status. Shared service credentials erase accountability.
AI governance and control
AI agent governance: decide what every agent may see, decide, and do.
Governance is not a policy document added after launch. It is the operating design that gives each agent an identity, limits its access and actions, routes consequential decisions to people, and leaves evidence of what happened.
Short answer: maintain an agent inventory and owner; give each agent a distinct identity; apply least-privilege access; separate reading, recommending, drafting, changing, and sending; set risk-based approval thresholds; log inputs, decisions, tools, actions, and reviewers; and test the controls continuously.
Authority matrix
Match autonomy to consequence.
| Authority level | What the agent may do | Example | Minimum control |
|---|---|---|---|
| Read | Retrieve approved information without changing a system. | Find a current policy or account status. | Source-level permission, provenance, access log. |
| Recommend | Analyze and propose a next action. | Flag invoices likely to be duplicates. | Named reviewer, confidence and evidence. |
| Draft | Create a reversible artifact for review. | Prepare a customer reply or purchase request. | Human approval before external effect. |
| Act within limits | Perform bounded, reversible actions. | Update a low-risk internal field. | Value limits, allowed tools, audit receipt, rollback. |
| Act with approval | Prepare a consequential action, then wait. | Issue a credit or send a contract amendment. | Authenticated approver and exact action preview. |
| Never autonomous | Escalate decisions the organization reserves for people. | Employment action, material commitment, safety exception. | Hard policy boundary—not a prompt suggestion. |
Control plane
Governance needs more than human-in-the-loop.
Permissions and data policy
Control which knowledge sources, records, connectors, tools, actions, and destinations are available. Apply the user’s own rights where appropriate and narrower agent rights where needed.
Approval and escalation
Require approval based on consequence, reversibility, confidence, novelty, and value—not because “AI” is involved. Show the reviewer the exact proposed action and evidence.
Audit and observability
Record the agent version, context references, model, tool calls, policy decisions, action receipts, exceptions, and human decisions so work can be reconstructed.
Lifecycle control
Review, publish, change, suspend, and retire agents deliberately. Version policies and prompts. Re-test before broader access or autonomy.
Accountability
Human oversight is not accountability if nobody owns the outcome. Name who watches performance, resolves exceptions, and can stop the workflow.
A review button does not make a system safe. If the reviewer lacks time, context, authority, or a clear decision standard, “human in the loop” becomes ceremonial approval.
Implementation
Start with the work, then encode the boundary.
Map the workflow and identify every read, inference, recommendation, write, message, payment, commitment, and disclosure. For each step, ask who is affected, how reversible it is, what evidence exists, and what happens if the agent is wrong. Convert those answers into permissions, thresholds, approvals, and hard stops. Then test attempted overreach: an unauthorized record, a higher-value action, a manipulated instruction, a missing approver, and an unavailable audit store.
Microsoft’s current governance guidance groups agent identity, role- and attribute-based access, centralized inventory, policy enforcement, knowledge and connector controls, audit logs, runtime risk, and credit governance into one control plane. OpenAI similarly emphasizes explicit permissions, auditable actions, and agent identity. See Microsoft Copilot Studio security and governance and OpenAI Frontier.
Governance review
Questions leadership should be able to answer
Which production agents exist, who owns each one, and which are currently active? Which systems and data can each identity access? Which actions can occur without review, under what limits, and on whose authority? What evidence does an approver see? Which policy is enforced in code and which remains an instruction? How quickly can access be suspended? Can an auditor reconstruct the exact context, version, tool calls, policy decisions, human approvals, and external effects for a consequential run?
Also ask about the people operating the controls. How large is the exception queue? Are reviewers accepting by habit? Are overrides analyzed? Is anyone accountable for the business result after approval? Governance should enable safe production work. If every step needs the same approval forever, either the risk boundary is poorly designed or the workflow is not ready for autonomy.
Enterprise and regulated environments
Governance must be enforceable at runtime.
| Required feature | Control evidence | Required outcome |
|---|---|---|
| Registered agent identity | Inventory record, accountable owner, service identity, approved purpose, risk tier, environment, and lifecycle status. | No anonymous or ownerless agent can access production systems. |
| Least privilege and separation of duties | Role- or attribute-based access, scoped credentials, incompatible-role rules, periodic access review, and emergency revocation. | An agent cannot approve its own high-risk action or exceed the authority of the person and process it represents. |
| Risk-based human oversight | Approval thresholds, qualified reviewers, exact action preview, supporting evidence, override reason, and authenticated decision. | People can understand, contest, stop, or reverse consequential actions rather than merely observe them. |
| Policy enforcement | Hard limits in code or control services, data-loss prevention, allowed tools and destinations, transaction limits, and fail-closed behavior. | Prohibited actions remain prohibited even when a prompt, model, or user requests them. |
| Audit and incident response | Tamper-evident logs, policy decisions, approvals, action receipts, complaints, incidents, notification paths, and preservation rules. | Material events can be investigated, reported, remediated, and used to improve controls. |
| Third-party and change governance | Vendor diligence, model and subprocessor register, contractual controls, release approval, impact reassessment, and decommission plan. | A provider or model change cannot silently alter the approved risk boundary. |
The specific obligation depends on the use. The EU AI Act, for example, establishes risk-dependent requirements and calls for effective human oversight for high-risk systems. NIST’s Generative AI Profile emphasizes governance, pre-deployment testing, content provenance, incident disclosure, and additional tracking or oversight where warranted. Obtain qualified advice for the applicable law and sector.
Primary references: NIST Generative AI Profile, EU AI Act, and Microsoft’s enterprise governance controls.