Stephen Brobst, Chief Technology Officer, Teradata Corporation, discussed in-memory analytics, Big Data and the latest trends in analytics with Jasmine Desai
Absolutely not! Data volumes are growing faster than memory becomes cheaper. Marketing people say that the price of memory is going down by 30% every 18 months, which is roughly correct. However, data is growing much faster at 50% every 12 months. The Indian market is extreme because the volume of data is very high and the market is price sensitive. It’s economically irrational to put all data in-memory because not all data is accessed equally frequently. Less than 15% of data will be accessed more than 90% of the time. So why would anyone pay the price of in-memory computing for the remaining 85%?
How does an organization take the three Vs of Big Data and create business value from them?
It depends on the nature of the data that an organization is working with. The insight and the customer experience is the most important thing here. Traditional data warehouses analyze transactions but Big Data is about getting below the transactions and into the interactions. Transactions help understand the customer’s value while interactions help understand the customer’s experiences. Interactions would include clickstream data, social interactions etc. If an organization wants to retain its best customers and make the experience more helpful, then transactions need to be analyzed. In most cases, Big Data analytics is focused on optimizing and understanding the customer experience.
Right now, the challenge around Big Data is about the dearth of data scientists in this field. There is not much focus on building a strong foundation for statistics in education. This is what needs to be done as these people are going to be highly-valued. A pipeline is being created but there is already a lot of demand.
What do you mean when you say that Big Data is technology- rather than business-driven?
The early adopters of any new technology tend to be innovators. In the Big Data space, the likes of Google, Yahoo and Stanford University are the pioneers. Then there are the early adopters, companies like eBay, Netflix and Travelocity fall into this category. Then comes what Geoffrey Moore calls the ‘chasm’. On the other side of these are banks, telcos and retailers—these are the pragmatists; the Fortune 1000 companies.
Big Data is primarily in this space right now. The average bank spends $8,000 on Big Data. They send two people to a conference and buy a whitepaper.
There are a lot of folks kicking the tires when it comes to Big Data. Our job is to bridge the chasm. That’s why Big Data is technology- rather than business-oriented right now.
Could you elaborate on the trend of Consumer Intelligence in the field of analytics?
Traditional data warehouses are built for internal business decision makers to make better decisions. Now imagine that an organization has built a data warehouse wherein it lets its customers access this data. BFSI is an early adopter of this. It is rapidly being picked up in the retail sector as well. Healthcare is going to be huge in this space as consumers want to make better decisions about their health; not only cost-wise but also quality wise.
Apart from this, another trend is going to be multi-temperature data management. Therefore, what you want is a small percentage of data in-memory. It will be a multi-tiered storage approach wherein there will be in-memory, SSDs and electromechanical drives. Data migration across these tiers would occur as per data usage patterns. However, the key is that it should be done automatically. Because, an army of DBAs will be required if the process is manual.
Another trend in Big Data will be sensor data that will be far larger than social media. Every building, bridge, vehicle or mobile device will have sensors embedded in it. E.g. sensors on a bridge can detect the vibration patterns and forecast when the bridge is likely to become unsafe. We have customers that have sensors on the ball-bearings on railway tracks. Based on the vibrations detected by these sensors, the railways can determine when they need to do maintenance.
Can you shed some light on Teradata’s acquisition of eCircle?
Teradata has been guiding organizations on acquiring customers through analytics/CRM. We acquired Aprimo for its integrated marketing management capability. eCircle takes that a step forward and lets us execute those campaigns and do the outbound bit. It gives us execution capabilities, particularly in the e-space. For our customers, it will mean a more complete solution.