In the hyper-competitive world of online travel, speed is not just a feature—it is the experience. For MakeMyTrip, delivering highly relevant recommendations in real time is critical to keeping users engaged at the moment of intent. The company’s latest engineering leap—powered by Databricks’ Real-Time Mode (RTM)—is redefining what personalization at scale looks like.
Real-time expectations meet real-world scale
Every time a user taps the search bar, MakeMyTrip’s platform is expected to instantly surface “last-searched” hotels—tailored to individual behavior and context. At millions of daily users across both B2C and B2B segments, even slight delays can disrupt the journey and impact conversions.
However, traditional architectures struggled to meet these expectations. Apache Spark’s micro-batch processing model, despite extensive tuning, plateaued at one to two seconds of latency—far beyond acceptable thresholds for real-time personalization.
The architectural dilemma
The data team briefly explored Apache Flink, which met latency requirements. But adopting a second streaming engine introduced trade-offs: duplicated business logic, higher operational overhead, increased infrastructure costs, and the risk of inconsistencies between batch and real-time systems.
Rather than fragment the architecture, MakeMyTrip made a deliberate choice—to stay within the Spark ecosystem and wait for it to evolve.
That bet paid off with Databricks’ Real-Time Mode.
Breaking the latency barrier with RTM
RTM, built into Apache Spark Structured Streaming, eliminates the constraints of micro-batching through continuous data processing, concurrent pipeline execution, and streaming shuffle. These innovations allow data to be processed as it arrives, reducing latency to milliseconds.
For MakeMyTrip, the impact was immediate and measurable:
- P50 latency dropped from ~1.23 seconds to 44 milliseconds
- P99 latency reduced from over a minute to ~500 milliseconds
- Click-through rates improved by 7%
“Real-Time Mode gave us the performance we needed with the simplicity we wanted,” said Aditya Kumar, Associate Director of Engineering at MakeMyTrip. “We wanted one source of truth—not multiple engines to maintain.”
A unified pipeline for personalization
At the core of this transformation is a unified streaming architecture. MakeMyTrip merged B2C and B2B clickstream data into a single pipeline, ensuring consistent personalization logic across all user segments.
The pipeline ingests clickstream data, processes it in real time using RTM, enriches it with stateful lookups via Aerospike, and pushes results to Redis for instant UI delivery—all within sub-50ms response times.
Notably, enabling RTM required minimal effort: a single-line configuration change, without rewriting business logic. This allowed the team to transition seamlessly from batch to real-time processing.
Co-innovation with Databricks
As an early adopter, MakeMyTrip worked closely with Databricks to productionize RTM. Together, they introduced enhancements such as stream union for unified pipelines and task multiplexing to optimize infrastructure costs.
Operational challenges—like checkpointing, backpressure, and handling high-throughput spikes—were addressed through iterative tuning, resulting in a stable, scalable system capable of handling millions of events daily.
Powering the next wave of AI
Beyond immediate gains in performance and engagement, the move to RTM positions MakeMyTrip for the next phase of innovation—AI-driven decisioning.
Real-time data pipelines now serve as a foundation for feeding AI agents with the most current context, enabling more accurate and dynamic recommendations.
“As we move into the era of AI agents, real-time context becomes critical,” Kumar noted. “RTM allows us to deliver that without the complexity of multiple systems.”
The bigger picture
MakeMyTrip’s journey highlights a broader shift in data engineering: the move toward unified architectures that eliminate silos between batch and real-time processing. By achieving millisecond latency within a single Spark stack, the company has demonstrated that speed, scale, and simplicity do not have to be trade-offs.
In an industry where every millisecond influences user behavior, that advantage can make all the difference.