Who is accountable when an AI agent makes a business decision?

By Jeff Marowits, CEO, Keystone

Most Indian companies have spent the past year teaching employees how to deploy AI agents to automate routine tasks. Microsoft found that 32% of India’s workforce are redesigning work around AI agents and another study, from Deloitte, found that 94% of organisations expect their AI spending to increase in the coming year.

The next challenge for business management is mastering how to use Agentic AI the right way. It’s perhaps why half of India’s business leaders are concerned about governance, risk and compliance when it comes to agentic AI adoption, as they fear being non-compliant and having to bear the legal costs of doing so. An AI agent can pursue a defined goal across several steps, call upon approved tools, retrieve information and take actions within the permissions that an organisation sets for its technology.

For instance, customer service AI agents in use today can process refunds, alter deliveries based on customer requests and resolve routine issues. In the financial services industry, AI agents are already in use for loan underwriting, customer onboarding, KYC (Know Your Customer) processes, and more. Each of these tasks is routine, but together, they create a new operating model in which software automates the decisions that employees previously made. When AI agents make decisions across business functions, who is accountable for their actions? This is the accountability issue facing organisations because AI agents are adding new layers of business risk.

So what should enterprises do to navigate this change?

India’s own policy direction recognises the AI accountability issue, offering a model. The India AI Governance Guidelines recommend obligations and liability that reflect an AI’s function, risk and due diligence. The guidelines also call for greater transparency, including audit trails and mechanisms for reporting AI incidents, such as algorithmic bias, data breaches, systemic safety failures and the spread of deepfakes or misinformation.

As a result, corporate boards must actively focus on continuous monitoring, accountability, explainability and human intervention. To achieve this, organisations must move beyond basic compliance measures and conduct rigorous algorithmic audits, deploy proactive techno-legal frameworks and establish cross-functional oversight that aligns agentic AI adoption with strategic and regulatory foresight. Here are some actions organisations can take:.

Adopt early
The scale of India’s AI build-out strengthens the case for early discipline. The Union government has onboarded over 38,000 GPUs through its subsidized compute facility. AIKosh hosts over 9,500 datasets and 273 sectoral models. Wider access to compute and data will support more experimentation across sectors. It will also allow more business functions to connect AI systems with internal workflows, customer records and transaction systems.

Be specific in mapping
Given the natural environment this creates to speed up agentic AI adoption, companies must begin with the decision itself. A team can map each proposed agent use case and separate research, recommendation, execution and approval. An agent that gathers policy documents presents a different risk from an agent that changes a customer record or releases a payment. This distinction helps managers set sensible authority limits. It prevents a broad instruction like “handle customer complaints” from becoming an unclear transfer of responsibility.

Establish clear ownership of the project
Assign a dedicated business-owner for every action an AI agent performs. Leadership must weigh the commercial benefits of an AI agent against the potential financial and reputational fallout of a system error. To ensure true accountability, the human owner must deeply understand the system’s architecture, including its underlying data sources, integrated tools, behavioural boundaries and broader network connections.

Senior management should be in charge of approving what decisions AI can make—and weigh the financial, legal, reputational and customer consequences of these decisions. Clear ownership gives employees an escalation path when an AI agent veers off course. It gives employees a map of when to use AI and when to not use it at all because currently 78% of workers say AI has increased their workload in India. It also makes life easier for audit, risk management and legal teams as they will have a concrete blueprint for review.

Be discerning in deployment
Business leaders in-charge of AI adoption must stop themselves from falling into the pit of “let’s deploy AI everywhere”. While an AI agent could improve productivity in one department, it could add more work in another. Track the entire journey, from the first request or prompt to the final resolution because the real measure of success is in comparing the cost of human review against faster resolution, or even better customer outcomes.

Keep meticulous records
Records matter just as much as permissions. A company must retain the instruction that started an AI agent’s task, the data and tools the agent accessed, the action it proposed or took and any human review. The RBI’s AI Model Risk Management Guidance already calls for auditable logs, explainability and human-in-the-loop mechanisms. This is necessary as a company can’t investigate a disputed decision if its records show only the final outcome.

Test your agents
Organisations also need a way to test agents before deployment and after material changes. Testing must cover ordinary tasks, conflicting instructions, incomplete data, attempted misuse and failures in connected tools. A business team can then judge whether the agent is following governance policies or simply producing responses. Vendor contracts need a lot of scrutiny because third-parties have their own way of processing data, may use different models or even change features that could affect a company’s controls.

Set up a human review system
Always have humans reviewing AI’s actions. Teams can allow AI agents to carry out menial tasks but when it comes to decisions involving money transfer, credit, employment, safety, handling sensitive data, risk management or legal issues, it’s always necessary to have human oversight and approvals.

Boards don’t have to approve every agent. Instead, it’s better to have a clear view of the decisions agents make, the limits placed on their decision-making capabilities and the incidents where the AI governance policies aren’t working so they can decide the way forward.

Companies that make these choices early on will deploy agents with clearer boundaries and fewer surprises. They will also give employees confidence about the decisions that remain theirs to make. Good governance shouldn’t slow down automation but give companies a way to scale without losing sight of accountability.