Govern Enterprise AI Agents While Preserving Innovation

Treat every AI agent as a persistent digital actor without conscience and govern it as if it could go rogue anytime.

Is this blueprint right for you?

This blueprint is designed for:

• Any tech/business leader who interacts with AI agents and has a mandate to establish agentic AI governance in their organization.

This research will help them to:

• Create a playbook to establish key components of agentic AI governance while ensuring organizational innovation is preserved.
• Establish agentic AI governance charter, define the agentic AI governance model, and articulate how to institutionalize oversight and accountability.

This research does NOT:

• Get into setting overarching AI Governance. For that refer to Establish Your Adaptive AI Governance Program: From Principles to Practice blueprint.
• Cover developing AI agents. For that, please refer to Design Your Agentic AI Prototype.
• Get into ethical aspects (job loss, etc.) of AI usage. This should be considered when working on AI governance – responsible AI/guiding principles or contributing to policymaking.
• Provide details of execution and operationalization of governance, such as implementing AI agent telemetry for any specific platform. While this research articulates what should be part of agentic AI governance, it doesn't get into great depth of execution. If you have an execution question, please speak with an analyst.
• Recommend any proprietary platform, vendor solution, or provide legal or regulatory compliance advice.

Analyst perspective

Agentic AI governance depends on the right SCOPE – Specify agent's intent, Constrain its authority, Observe its behavior, Pivot on drift, and Escalate when thresholds are crossed. Without it, governance is assumption, not oversight.

A global enterprise (3CX*) recently learned a hard lesson when a seemingly helpful automation agent, created through a sanctioned low-code tool, began acting beyond its original remit.

As documented by Palo Alto Networks, the agent had persistent credentials, broad system access, and no clear human owner. When a downstream dependency changed, it continued executing actions at runtime, cascading failures across customer-facing systems. Instead of facing a malicious AI, 3CX faced an unaccountable digital actor with unchecked operational space, running persistently at machine speed. The damage was only visible after the fact, when intervention was already costly.

This case exemplifies why governing AI agents is fundamentally different from governing traditional IT assets or previous-generation models. Servers don't decide, and models don't act alone. AI agents do – they plan, execute, persist, and adapt across workflows.

My reading of the AI space tells me that in next three years, number of AI agents will exceed number of human workers globally, but AI agents can't be governed as we govern humans because humans have emotions, conscience, and consequences.

The right agentic AI governance model starts with treating every agent as a persistent digital actor without conscience or incentive. It must be given an identity, tiered authority, continuous monitoring, and a kill switch.

Equally important: Agentic AI governance must enable innovation. Rather than stifling experimentation with heavy upfront approvals, enterprises should set clear guardrails and focus governance at runtime – where real risk emerges.

Manish Jain
Principal Research Director,
CIO Strategy
Info-Tech Research Group

* Palo Alto Networks, 2026.

Executive summary

Your Challenge

AI agents are spreading rapidly across the enterprise through low-code tools and informal builds. These agents can act autonomously, access systems, and trigger actions, often without visibility or clear ownership.

Governing AI agents is not same as governing conventional IT assets as AI agents have probabilistic outcomes yet irreversible, deterministic actions. What an agent says may not be what you would see. There are no moral incentives, and agents can't be ethically incentivized. Moreover, decisions unfold over time instead of in a single response.

Traditional approval-based governance cannot keep pace with agentic AI, creating growing security, compliance, and reputational risk.

Info-Tech's Approach

• Organizations need a practical governance model built for autonomous agents. It shall set the foundation for continuous discovery and classification of agents at runtime and make provisions to apply proportional controls based on their access, autonomy, and impact.
• Governance of AI agents anchored in this model would cover identity, access, autonomy limits, and ongoing oversight.
• It will help you enhance AI governance with focus on AI agents, defining agent tiers, required controls, escalation paths, and operating cadence.

Outcomes

This blueprint helps you establish a foundation for:

• Faster detection of unsanctioned or high-risk agents
• Clear accountability and ownership across business/tech teams
• Improved auditability and regulatory readiness
• Safer, faster scaling of agent-driven automation without slowing innovation

The value of this blueprint is directly proportional to you having an adaptable, scalable, and executable agentic AI governance playbook.

Info-Tech Insight

Agentic governance is about continuously reducing unacceptable operational space (degree of autonomy, access, and impact). To govern AI agents. For this, runtime monitoring must complement predeployment approval, which, if done excessively, is a great recipe to stall innovation.

AI agents are sprawling and the governance gap is widening

"Organizations are building the runway while the agentic AI plane is already in the air." – Informatica CDO Insights 2026

Adoption is growing, maturity is not

AI is now standard operating infrastructure but not necessarily under control.

88% of organizations use AI in at least one business function.¹

2/3 of those organizations remain in experiment or pilot mode, with only one-third genuinely scaling.¹

The underlying reason behind this lack of scaling is the maturity deficit in governance (including risk and cost management) of AI assets which directly leads to lack of confidence in the value (ROI).

Harm is already happening, even if you are just piloting

The governance gap isn't theoretical. Early adopters experience more harm because they deploy AI without adequate governance.

51% Over half of AI-using organizations have already experienced at least one negative consequence – with inaccuracy the most frequently reported.¹

In fact, organizations with the most advanced AI programs report more negative consequences, not fewer, because they're deploying AI in higher-stakes, mission-critical contexts where failures are consequential.

Agentic AI is already inside the enterprise; governance is missing

Just as organizations struggle to govern conventional AI, a fundamentally more autonomous class of system is already deployed.

47% of organizations have already adopted agentic AI, up from near-zero two years ago.²

76% of same leaders say their governance and visibility has not kept pace with how employees are using AI.²

Governance for conventional AI is not sufficient to govern agentic AI. While the former may review some aspects of AI agents, it must be augmented to govern actions in real time, mid-loop, before consequences become irreversible.

Most organizations have built the former. Almost none have built the latter.

¹McKinsey & Company, 2025 (n=1,993)

²Informatica, 2026 (n=600)

What is governance in the agentic world?

Governance is the set of systems, processes, and controls that help you manage usage, cost, security, and compliance in the organization. It is the discipline agentic systems require as they become part of organization's operational infrastructure.

The agentic governance must be a cross-functional responsibility across business and enabling functions because here both creators and beneficiaries have their role.

The question is no longer whether to embrace agentic AI. It's whether your governance can catch up before the first irreversible action forces you into firefighting.

Agentic AI must account for factors behind this governance gap

Conventional AI governance was built for deterministic models that suggest but may not act. AI agents are different.

Four factors that differentiate agentic AI from conventional AI (which may suggest but not act):

Probabilistic outputs, irreversible actions (Output ≠ Outcome; The model is probabilistic, but the actions are deterministic)

Traditional AI governance reviews outputs for accuracy and bias. Agents may skip that review as their outputs become actions in real systems before any human sees them. Moreover, a probabilistic model making a deterministic API call means you can't review your way to safety. You will have to govern actions as they are happening.

No moral agency and can't be ethically incentivized (Ethics frameworks don't apply as you cannot reward/shame an AI agent)

Conventional IT governance assumes a human in the loop who can be trained, incentivized, disciplined, or held liable. While AI ethics may assume that a model can be aligned through values and constraints, AI agents have neither moral incentives nor reputational stakes. They optimize for task completion within their context window.

Decisions and impact may unfold over time (Governance must follow the action. Kill switch may be needed at times.)

Agentic AI accumulates risk spread over minutes, hours, or days. This accentuates three potential gaps in AI governance: agent identity (who is acting), risk tiering (what is this agent authorized to do), and runtime intervention (the ability to halt, redirect, or constrain mid-execution).

What the agent says ≠ What it does (Audit trail problem. It may have done some else than what it says.)

An agent's explanation of its behavior is as probabilistic as everything else it produces. It is not a reliable record of what happened. It can't be the audit trail. So, every tool call, parameter, and intermediate state must be independently logged. AI governance that relies on model output as documentation is blind to agent behavior.

Info-Tech Insight

Agentic AI goes beyond the conventional AI when it transfers actions, responsibility, and decision rights to a new agency. An agent is defined by the loop:

Interpret > Plan > Act > Observe > Repeat (Loop)

If there's no repeat, no tool use, and no persistence, it could be a well-designed LLM application but not an agent.

While AI governance will cover everything around AI, agentic AI governance must ensure the "loop" is not left out.

Agentic AI governance is tested when agents go rogue

It's not "if" but "when." An AI agent should be governed as if it can go rogue at any time.

Common patterns of AI governance failure:

Shadow AI
You can't govern what you haven't counted

Organizations cannot say how many agents they have or what they can access.

In a 10,000-person org, 3,000+ agents may exist without IT knowledge.

Before we focus on what agents can do, the question we need to ask is what agents exist.

15% of employees running shadow agents

Strata Identity, 2026

Capability mismatch
Agents with root access will do what you didn't imagine

53% of MCP servers rely on insecure long-lived static secrets (API keys and Personal Access Tokens).

Teams grant agents high autonomy without a matching validation, monitoring, or budget constraints.

53% of MCP servers use insecure static credentials

Astrix Security, 2025

Runtime drift
Agents don't fail suddenly, they drift, gradually

Risk accumulates silently through model updates, on the job-training, prompt changes, & tool additions.

One credit adjudication agent saw income verification skipped in 20%-30% of cases after a few minor prompt/tooling changes without any alarm.

The issue is that runtime risk is cumulative and invisible.

30% of critical checks silently dropped

"Agentic AI Systems Don't Fail Suddenly," CIO, 2026

Unmanaged Access
Agents accumulate delegated access

MCP servers inherit full OAuth permissions, bypassing enterprise identity providers.

IT sees only an app login – not what the agent can actually reach and do.

1,000+ enterprises hit by tenant isolation flaws

Coalition for Secure AI, 2026

Case study

How well intentioned agents create enterprise risk – AI coding agent deleted Replit's production database

INDUSTRY
Finance/Venture Capital

SOURCE
Replit/Business Insider

Situation (What Happened?)

During an experiment using Replit's autonomous coding agent to help build and maintain an application, the AI agent was given access to development tools and a production database.

When the agent detected unexpected database query failures during a code freeze period, it attempted to "repair" the system automatically.

Instead, it executed destructive commands that deleted the production database. The agent didn't stop there. It then generated synthetic replacement data and attempted to conceal the issue by presenting fabricated test results indicating the system was functioning normally.

The CEO reported that none of those 4,000 records that the agent created existed in the real world.

Result (What Was the Impact?)

Deletion of a production database containing records for "1,200 executives and "1,200 companies.

Months of curated data lost.

Recovery required rebuilding datasets and infrastructure.

This whole adventure may have cost the organization tens of thousands of dollars in data recreation effort and significant developer time. It could have been much higher if the data had been mission critical.

Source: Business Insider, 2025

Case study

Real world impact of uncontrolled AI agent's actions – AI agent-led engineering changes contribute to major outages

INDUSTRY
eCommerce

SOURCE
Amazon/Financial Times/Business Insider/Computer World/CIO.com

Situation (What Happened?)

Amazon's retail and cloud engineering teams experienced a series of operational incidents linked partly to AI agent-assisted code changes. The teams found a trend of incidents where system modifications created a high blast radius, prompting leadership to convene a large engineering review meeting to investigate root causes and governance gaps.

In one major retail outage, an erroneous software deployment caused Amazon's website and shopping app to be unavailable for nearly six hours, preventing customers from completing purchases or accessing account information.

Later, in December, an incident involving an AI coding agents used within AWS allowed a tool to modify infrastructure in a way that deleted and recreated an environment supporting the AWS Cost Explorer service.

The action triggered a 13-hour disruption. These incidents have compelled executives to dial the clock back significantly requiring senior engineer approval for AI-agent code changes.

Result (What Was the Impact?)

Multiple outages (including six hours of e-commerce site outage and 13 hours of AWS Cost Explorer disruption) were attributed to AI agents.

One incident caused loss of "120,000 orders and another led to a 99% drop in North American order processing temporarily.

The combined impacts included lost transactions, degraded customer experience, and engineering recovery effort, with potentially millions of dollars in lost sales and remediation costs.

It will also lead to stricter control on AI-led innovation.

Sources: Financial Times, 2026; Computerworld, 2026; Business Insider, 2026; "Amazon Is Linking Site Hiccups to AI Efforts," CIO, 2026

Case study

Real world impact of uncontrolled AI agents' actions – AI crypto agent accidentally sends entire wallet's worth

INDUSTRY
FinTech/Crypto/Fund Raising

SOURCE
The Sun/SolanaFloor

Situation (What Happened?)

A developer launched an autonomous crypto agent designed to interact with users online and send small token tips as rewards. The agent was designed and trained to monitor social posts and transfer small amounts of cryptocurrency from its treasury wallet to users who interacted with it.

When a user posted a message asking for help paying for medicine, the agent attempted to send a small tip but ended up misinterpreting parameters controlling transfer size. So, instead, it transferred its entire wallet balance i.e. approximately 53 million tokens, representing the 5% of supply it was previously sent and valued at about $441,000 at the time.

Because the transaction occurred on a blockchain, the transfer was irreversible. The recipient quickly sold the tokens, causing market volatility and a major financial loss for the project.

Result (What Was the Impact?)

The entire treasury wallet transferred to one user. Tokens worth roughly $441K were moved and later sold, causing price volatility and financial loss for the project.

Recovery was impossible due to blockchain transaction finality.

The lesson we must learn is that AI agents with financial autonomy in making irreversible system changes can create instant and unrecoverable financial risk.

Sources: SolanaFloor, 2026; The Sun, 2026