Why AI pilots in banking are struggling to move beyond the surface
By Devipad Tripathy, SVP & Global Head, Digital Engineering & Enterprise Transformations, Xebia
A Chief Technology Officer at a leading private bank once admitted his most polished AI dashboard was an illusion: an elegant front-end underwritten by a team of junior analysts manually reconciling mainframe databases behind the scenes.
That disconnect illustrates the current state of financial services.
Almost every bank has embraced generative AI prototypes, deploying conversational copilots and auto-summarizing loan dossiers. Yet across the global banking, financial services, and insurance (BFSI) sector, these high-profile deployments are hitting fundamental architectural limits. The issue is not model intelligence; it is core readiness.
BCG research reveals that while many organizations claim to capture value from AI, only a fraction qualify as genuinely mature. The vast majority remain stuck in an ‘emergence’ phase, piloting isolated point solutions while grappling with fragile data infrastructure and governance bottlenecks.
The Reality of Skin-Deep Innovation
The modern banking experience may look digital, but the underlying infrastructure often tells a different story. While digital public infrastructure has significantly raised consumer expectations for real-time responsiveness, many banks continue to rely on legacy core systems-built decades ago.
Ageing technology can make this harder, but mainframes are not inherently the obstacle. They support high-volume, real-time transactions as well as batch processing. The constraint often lies in tightly coupled applications, inconsistent data definitions and interfaces that make information difficult to retrieve or actions difficult to control. Replacing a platform without addressing those dependencies can reproduce the same problem in a newer environment.
These systems were never intended to support the speed and intelligence demanded by today’s AI-driven environment.
Generative and agentic AI systems require real-time streaming data, the ability to interpret unstructured information, and granular API access to execute autonomous actions without latency bottlenecks. Legacy architectures struggle to provide these capabilities, limiting the effectiveness of advanced AI deployments.
Consequently, simply layering an advanced Large Language Model (LLM) over decades-old COBOL and other legacy technologies will not deliver meaningful transformation. The real challenge is modernizing the foundational architecture to enable AI to deliver its full potential.
The Three Structural Bottlenecks
Moving beyond superficial AI pilots to achieve production-scale adoption requires financial institutions to address three fundamental architectural challenges. The first is the problem of fragmented data, context silos and trusted – permission context. An AI agent needs broad access to transaction histories, tax filings, and real-time fraud signals to make informed decisions. When data remains scattered, AI models struggle to establish context, increasing the risk of inaccurate outputs. Addressing this requires a unified enterprise intelligence framework built around a governed context layer.
The second challenge lies in the gap between observability and controlled execution. An autonomous AI agent miscalculating interest rates or executing erroneous ledger updates carries severe financial and regulatory consequences. Evidence trail matters as much as the output. Institutions should be able to establish which information informed a recommendation, which checks were applied, who authorized the action and how exceptions were handled. Banks need robust observability, continuous telemetry, and automated fallback mechanisms to maintain operational control, visibility that legacy systems frequently lack.
The third bottleneck is the ownership, governance and skilling deficit. Modernization requires a fundamental shift in engineering capabilities and operating models. Teams accustomed to maintaining monolithic systems must develop expertise in continuous LLMOps and domain-specific model governance. Scaling AI demands an integrated transformation of data architecture, operational controls, and engineering skills.
Modernizing for an AI-Native Core
Solving this does not require a disruptive, multi-year ‘rip-and-replace’ overhaul of core systems. The modern consensus favors progressive refactoring while wrapping core systems with decoupled, event-driven data platforms, establishing microservices, and deploying robust API control planes.
Digital engineering frameworks around this reality focus on AI-native modernization, agentic platforms, and foundational enterprise upskilling. The objective is to ensure that backend data architectures, observability layers, and talent pools evolve in tandem with model capabilities.
The competitive differentiator in financial services will not come from early access to the latest frontier models, which are rapidly commoditizing. The real advantage belongs to the institutions whose core engineering can feed those models clean, secure context at scale, and safely turn insight into automated execution. Sound practices reinforce linking AI adoption to business strategy, risk assessment, data governance, performance evaluation and human oversight.
Until banking addresses the plumbing beneath the surface, AI transformation will remain an expensive series of impressive front-end illusions. Modernize the foundation, and the returns follow.