By Vincent Hsu (CTO , Fellow & VP IBM Storage) , Sandeep Patil (Distinguished Engineer, IBM Storage)
Enterprise AI discussions have spent the last few years focused heavily on models, accelerators and compute capacity. But as organisations move AI applications from experimentation into production, another challenge is becoming just as important: how quickly and reliably AI systems can access the enterprise data they need to generate a useful answer.
This shift is expanding the infrastructure conversation beyond models and GPUs. The question is no longer simply how well a model performs, but how efficiently it can access current, trusted and relevant information at the point of inference.
That is bringing databases and storage closer to the centre of enterprise AI architecture. The systems that have powered transactional workloads for decades across banking, insurance, healthcare, retail and manufacturing are evolving to support multiple data types, including the vector embeddings used by AI applications. As these capabilities converge, the infrastructure supporting enterprise databases is becoming an increasingly strategic part of the inference stack.
Why Storage Matters to Tier-1 Databases
Tier-1 databases have traditionally relied on high-performance block storage because transactional workloads require predictable latency, consistent I/O performance and high availability. Fibre Channel-based SAN environments have long supported these requirements, while newer approaches such as NVMe over Fabrics have expanded the options available to infrastructure teams.
The fundamental need for performant primary storage is not changing. Change is happening in the nature and volume of workloads increasingly being placed alongside traditional database operations.
As vector search and AI inferencing begin to converge with transactional workloads, infrastructure must support both without allowing the performance requirements of one to compromise the other. Storage is therefore no longer only supporting the database. It is also influencing how effectively enterprise AI applications retrieve and use data in production.
This shift is enabling primary block storage suitability to AI inferencing workloads in addition to the traditional Tier 1 OLTP/OLAP workloads.
The Database Itself Is Changing
Storage requirements are not changing, but what the database now does on top of storage is transforming.
Several leading enterprise database platforms are incorporating native vector search capabilities directly into their core engines – becoming multi modal databases. This enables organisations to store and search vector embeddings alongside structured business data.
The development reflects a broader shift in enterprise data architecture. AI applications increasingly need access to current business information, and keeping retrieval closer to that data can reduce some of the movement and synchronisation required when information is distributed across multiple systems.
This does not mean standalone vector databases are no longer relevant. The appropriate architecture will depend on factors such as workload scale, latency requirements, data freshness, governance and operational complexity. However, the emergence of native vector capabilities gives enterprises another way to bring AI-oriented workloads closer to their existing systems of record.
From Separate Systems to a More Connected Data Architecture
The appeal of bringing vector search closer to the primary database is largely operational. When structured business data and its vector representations are maintained across separate systems, enterprises may require additional pipelines to synchronise information, preserve data freshness and apply consistent governance. Each additional layer can introduce operational complexity and potentially increase the distance between an AI application and the information it needs.
For appropriate workloads, a more integrated architecture can reduce some of this complexity while allowing vector retrieval to operate alongside existing transactional data.
However, this is not a universal architectural prescription. Enterprises still need to assess the scale, performance, governance and operational requirements of individual workloads before deciding where vector search should reside. The objective should not be consolidation for its own sake, but an architecture that enables trusted enterprise data to be accessed efficiently at inference time.
RAG Is the Bridge to AI Inferencing
The reason in-database vector capability matters is Retrieval-Augmented Generation, or RAG.
RAG retrieves relevant enterprise information through vector search and provides it to a language model as contextual grounding, rather than relying solely on the model’s training data. Gartner has described
RAG as becoming a cornerstone for deploying generative AI applications and predicts that by 2028, 80 per cent of generative AI business applications will be developed on existing data management platforms rather than built on new, standalone platforms.
Every RAG query depends on efficient vector similarity search behind it. When the embedding store resides natively within the same multi-model database as an enterprise’s core transactional data, retrieval can take place closer to the source information.
This brings AI inferencing closer to the infrastructure already supporting mission-critical workloads. It also means that storage performance can directly influence the responsiveness and consistency experienced by employees, customers and AI agents interacting with enterprise applications.
As inferencing becomes a central enterprise AI workload, storage infrastructure is moving from a supporting layer to a strategic part of the AI architecture.
What This Means for Storage Performance
Enterprises should not assume that vector search is a large, sequential and bandwidth-heavy workload simply because embedding datasets can be large.
In practice, vector similarity search can behave more like a close cousin of transactional I/O, involving many small, latency-sensitive reads rather than one large sequential scan. Bandwidth remains important for bulk operations such as index building or large-scale reprocessing. However, the live, interactive query path experienced by users and AI agents is often primarily an IOPS and latency consideration rather than a pure throughput challenge.
This presents a meaningfully different design point from the broad assumption that every AI workload is predominantly bandwidth intensive. Infrastructure teams should therefore validate the characteristics of their own applications and platforms before sizing storage.
The economics of enterprise inferencing will not be determined by compute alone. If an AI system cannot retrieve relevant business data quickly and consistently, additional model or accelerator capacity may not address the underlying bottleneck.
What Infrastructure Leaders Should Evaluate
A few questions are becoming particularly important for CIOs and infrastructure leaders assessing storage for this shift:
Workload isolation: Can vector search and inference workloads scale without affecting the latency and performance of transactional applications sharing the same infrastructure?
Cyber resilience and data protection: As databases support a growing number of AI applications, the underlying data, vector representations and backups become critical assets. How will these workloads be protected, recovered and governed in the event of an operational failure or cyber incident?
Non-disruptive scaling: Vector datasets and AI workloads can grow faster and less predictably than traditional relational workloads. Can infrastructure expand capacity and performance as requirements change without disrupting production applications?
These considerations point to a broader shift. AI infrastructure can no longer be evaluated solely through compute capacity and model performance. The systems supporting access to enterprise data are becoming part of the AI architecture itself.
The Bottom Line
Multi-model databases are changing more than how enterprise data is stored and accessed; they are changing the demands placed on the infrastructure underneath.
As vector and inference workloads converge with Tier-1 databases, storage decisions that were once made primarily around transactional performance will increasingly influence how AI performs in production. It also implies the role of traditional block storage to evolve dominantly for AI Inferencing use cases.
For infrastructure leaders, the challenge is to ensure that these evolving workloads can scale together without compromising performance, resilience or reliability.