Over the past couple of years, PhonePe’s journey toward operationalizing AI began with a search for external platforms capable of automating complex, cognitive business processes. However, as it tested various AI tools for its engineering and operational requirements, it consistently encountered friction, eventually leading it to architect its own internal application layer: Agent Hub.
Navigating the Constraints of External Tools
PhonePe’s initial objective was to use AI to achieve efficiency on internal processes. However, off-the-shelf products repeatedly imposed hard constraints. The team frequently hit request limits in standard AI studios and strict document caps while indexing its internal knowledge bases. For its technical teams who need specialized integrations (for example: 1TB+ GitLab repositories), the expenditure and effort involved proved insurmountable due to multiple factors, both technical and non-technical.
Technical issues it faced included context window overflow, slow processing/ storage limitations and so on.
Given the domain PhonePe works in, a large part of the organisation was governed by strict regulatory frameworks that not only govern data sharing and hosting, but also requires it to scrape sites like the RBI and SEBI for up-to-date information. Standard web scrapers failed to provide the high degree of accuracy and reliability these workflows demanded.
Additionally, many of its operations required human-in-the-loop interventions; forcing teams to toggle between their existing operational consoles and external AI vendor environments would have introduced unacceptable change management friction.
Agent Hub: Engineering for Scale and Security
PhonePe built Agent Hub to provide a highly specialized, secure, and integrated platform tailored to our unique engineering needs. By focusing on deep infrastructure capabilities, the team created an environment that powers complex operational workflows at scale:
Production-Grade Sandboxing and Execution: Agent Hub utilizes ephemeral Docker containers featuring production-grade isolation (no network access, strict PID limits) combined with orchestrator-aware CPU pinning via our internal server (Drove) for precise cgroup-aware resource allocation.
Air-gapped MCP Architecture: PhonePe engineered its system to spawn local MCP servers as subprocesses via pipx directly from its internal Artifactory. This ensures zero public internet dependency, aligning with our strict data privacy postures.
Deep Integrations and Fan-out Workflows: Agent Hub features deep parsing nodes capable of extracting PDF, DOCX, and CSV attachments directly from inbound Email threads, while automatically resolving Google Drive links. The platform supports true bulk execution—allowing a single workflow to fan-out across 10,000 CSV rows or every file within a Drive folder, complete with parent-child tracking.
Advanced Semantic Discovery: Agent Hub utilizes a unified semantic discovery engine where agents, knowledge bases, and tools are all indexed within a single vector collection. This uses LLM-generated summaries to allow natural language cross-entity search.
Toggleable Sequential Reasoning & Tool Curation: PhonePe exposes MCP-based step-by-step reasoning as a dynamically toggleable tool, injected only when the agent specifically requires it. This is supported by an administrative tool factory featuring expert curation and dependency tracking, ensuring stability for tools relied upon by agents in production.
Inline ML Anomaly Detection: Agent Hub provides a native tool that directly queries Grafana dashboards, fetches time-series data, and runs anomaly detection inline—providing a streamlined solution for parsing voluminous metrics across our service architecture.
These engineering choices enable a flexible, mixed-model stack. By avoiding vendor lock-in, PhonePe processes routine document tasks cost-effectively via open-source models hosted on its own infrastructure, while reserving paid frontier models for tasks demanding complex reasoning.
The Impact Inside PhonePe
By focusing its engineering efforts on owning the workflows, integrations, and application layer rather than just the underlying models and data, Agent Hub has profoundly impacted operations across its business units. Today, the platform runs over 200 internal agents that handle more than 1,600 daily queries for 759 Weekly Active Users, saving an average of 5 minutes per automated task.
The depth of these automations has transformed its turnaround times:
Process Acceleration: The turnaround time for New Product Readiness Assessments dropped by 7x from weeks to days.
In compliance, a lending reverse audit agent reduced the time to audit loan documents for violations from 3–4 days to just 2–3 minutes.
Fraud and Risk: AI now drives 80% of its end-to-end fraud investigations. Complex investigations utilizing multi-modal inputs that previously took 90 minutes are completed instantly. By leveraging open-source models, the cost per investigation has plummeted from Rs 50 to between 5 and 15 rupees, while autonomously handling 800 cases previously lacking human bandwidth.