Why AI agents could become the next enterprise security blind spot

By Manpreet Singh, Co-Founder & Principal Consultant, 5Tattva

As enterprises rapidly deploy AI-powered support assistants and autonomous agents, many are unintentionally introducing a new class of security risk into their identity and access management (IAM) environments. AI systems are gradually being trusted with operational authority over sensitive account functions without adequate authorisation safeguards.

From customer support systems to internal identity governance workflows, organisations are allowing AI agents to interact directly with validation infrastructure in ways that traditional security models never anticipated. As AI adoption accelerates across customer service, workforce identity management, and digital operations, identity governance is becoming a business-critical issue rather than a purely technical concern.

An Old Security Problem in a New AI Environment
The underlying issue resembles the Confused Deputy problem, a long-recognized security pattern in which a privileged intermediary can be manipulated into performing actions on behalf of an unauthorised actor. AI support agents represent a modern version of the same challenge.

Unlike traditional applications with defined input structures, AI systems operate through natural language interactions, significantly expanding the attack surface. When AI agents are granted access to identity-related workflows, the conversation itself can become a pathway to sensitive operations.

As enterprises integrate AI into customer support, workforce identity management, and access governance systems, authorization failures can create operational, compliance, and reputational risks that scale across the organisation.

Where the Architecture Breaks: Authentication vs. Authorization
The root failure is a conceptual conflation that product teams make with regularity and security teams rarely catch in design review.

The failure occurs when that authenticated session context is treated as implicit authorisation for state-changing operations, specifically, modifications to the account’s own authentication factors. The fact that a session is active tells you nothing about whether that session’s principal should be allowed to atomically change the recovery email and trigger a verification OTP to the new address. Those are separate, high-risk authorisation decisions that require separate, explicit confirmation chains.

The timing of this challenge is particularly relevant as enterprises accelerate AI adoption across customer service, employee support, and digital operations. In sectors such as banking, healthcare, telecommunications, and IT services, AI assistants are increasingly being integrated into workflows that interact with sensitive identity data. Without clearly defined authorisation boundaries, these deployments risk introducing vulnerabilities into some of the most critical enterprise systems.

When an LLM agent holds API write-access to IAM functions, it collapses this distinction by design. The agent was built to be helpful by completing tasks described to it in natural language. If completing a described task requires writing to an IAM endpoint, the agent writes to the IAM endpoint. It has no architectural reason not to. The authorisation check is not a prompt instruction; it is a system boundary and if that boundary was never enforced in the API layer, no prompt engineering will substitute for it.

Why Strong Authentication Must Accompany AI-Driven Identity Workflows
Hardware-backed MFA provides an important safeguard against AI-mediated account takeover risks because authentication remains tied to a physical authenticator rather than a conversational workflow. While AI agents may influence recovery processes, they cannot independently satisfy hardware-based authentication requirements.

The implication for system architects is direct: hardware-bound phishing-resistant MFA should be treated as a security primitive, not an optional enhancement, for any account that an AI agent has recovery capabilities over. If you are deploying an AI-assisted IAM flow and hardware MFA is not enforced for enrolled accounts, you have accepted a structural exposure that no prompt-layer control will close.

Rethinking AI Governance Through the Principle of Least Privilege
An AI support agent needs to read account state to answer questions. It needs to retrieve policy documentation. It needs to navigate a user through a recovery workflow. None of these functions require the agent to hold direct write-access to IAM endpoints. Write operations should require a human confirmation step or a back-end authorization layer that the agent cannot directly invoke.

The architectural model should be: AI advises; humans and verified signals authorize; back-end systems execute. The agent generates a structured request – “user wants to change primary email to X”, which is then queued for OOB verification before any write is committed. The agent never touches the IAM API directly.

AI-integrated IAM should follow the same principle: AI can initiate requests, but authorization must remain independent of the agent.

The Forward Problem
AI agents are moving from customer-facing support functions into internal identity governance workflows: provisioning access, onboarding contractors, managing role assignments.

The Confused Deputy is not a novel bug. It is an old pattern wearing a new interface.

Every organisation deploying an AI agent with any proximity to authentication or identity state needs one specific question answered before that deployment goes live: At what layer is the authorization boundary enforced, and can that boundary be crossed by a natural language instruction?
If the answer to the second part of that question is yes, even partially – even conditionally, the architecture is not production-ready. In the AI era, competitive advantage will come not only from deploying intelligent systems, but from governing them responsibly.