According to IDC, the digital universe will be 44 times bigger in 2020 than what it was in 2009. That’s 35 zettabytes of data. 80% of the growth in data is expected to come from unstructured data such as audio, video, email, machine-generated data from a multitude of sensors and data from external sources such as the Internet and social media. To retrieve real business value from it, enterprises need the right tools to capture, organize, process and analyze the data meaningfully.
That’s where big data analytics comes in handy. Big data analytics breaks down huge amounts of data to uncover hidden patterns, unknown correlations and other useful information. Such information can provide companies with a competitive advantages over rival organizations and result in business benefits such as more effective marketing and increased revenues.
Before the mainstream entrance of the Big Data concept, companies were building home-grown solutions catering to specific data processing needs. They also built large and expensive data warehouses, wherever possible, to tap into big data. In the structured data space data mining technologies existed.
However, these technologies did not always suit the near-real-time nature of business scenarios. Moreover, for unstructured data, there weren’t any elegant solutions. Rahul Kanodia, Vice Chairman & CEO of Datamatics Global Services, explained, “Until the advent of Big Data analysis techniques, unstructured and semi-structured information was not considered to be a part of mainstream data analysis, which used to be limited to structured data.”
Advanced solutions
Big Data analytics is slowly entering the mainstream of computing as more people access the tools and analyze all the leveraging data from multiple sources. The new tools that are gaining popularity in the market are mostly built around Hadoop, an open source framework that supports the processing of large amounts of data. Originally conceived on the basis of Google’s MapReduce technology, Hadoop has become most popular processing tool today.
According to Venkatachaliah of IBM, Hadoop not only enables the distributed processing of large data sets across clusters of commodity servers but it also helps scale up from a single server to thousands of machines, with a very high degree of fault tolerance.
Abhishek Bhattacharya, Director – Technology, Sapient Global Markets, explained, “Hadoop is not an easy technology to master. Hence, several technology firms are building more developer-friendly solutions around Hadoop. Most large database and analytics vendors are trying to integrate Hadoop solutions into their existing offerings.”
With advances in Cloud computing, Big Data as a service is becoming another trend in this space. Munwar Sharif, CTO- CIGNEX Datamatics, explained, “Cloud-based offerings from companies such as Tresata, 1010data and ClickFox provide advanced visualization and analytics capabilities on Big Data repositories.”
Some key applications
The canvas for Big Data is wide and wider. Almost every company will find the need to understand their data—both structured and unstructured—in order to gain competitive advantage and quickly adapt to business opportunities. Depending on the type of the business, sector and industry it is used in, Big Data analytics can be applied to various key business decisions. Commented Ram of Oracle, “Based on the experience in the global market, Big Data analytics can become a huge success in business verticals with strong Web-based customers. It includes location-based services from telecom, social media analysis, online retailing, Internet banking and so on. Another key application is in the area of national intelligence.”
“The healthcare industry is experiencing data proliferation brought about by the computerization and digitization of medical records. Analyzing large data sets will become increasingly important for the mass healthcare sector as well as for personalized healthcare,” Vaidyanathan added.
Security agencies across the globe are also using Big Data analytics solutions. In supply chain and logistics, GPS information can be analyzed for the smart-routing of delivery trucks in real time using analytics tools. Product feedback posts can be analyzed to understand the efficacy of marketing campaigns and customer sentiment.
Analytics solutions have become a priority for CIOs across industry segments. Purswani added, “We see a lot of interest, with companies trying to get a clear idea of technologies and conducting pilots and proofs of concept to evaluate the same. CIOs have started evaluating how they could make use of the data lying in warehouses and making sense from these stores of data.”
Vendor opportunities
Big Data analytics is creating numerous opportunities for vendors. Companies such as Oracle, IBM, HP and SAP are developing their own Big Data engines and appliances. For example, Oracle has come up with a big data appliance, which can capture data in real-time. Ram said, “We also have Oracle advanced analytical solutions, which is an open source solution with a high degree of visualization as well as a NoSQL database.”
Talking about IBM’s solutions, Venkatachaliah said, “IBM’s platform for Big Data uses state-of-the-art technologies including patented advanced analytics. Built over Hadoop, we have the IBM InfoSphere BigInsights, which processes persistent data, and InfoSphere Streams, which supports streaming data.”
In the Big Data analytics sphere, SAP has taken the lead with its SAP HANA Real-time Data platform. “It can acquire, store, process and report Big Data with varying volume, variety and velocity,” commented Dey.
Apart from the big players, many open source vendors are also actively contributing solutions to the analytics field. Additionally, many niche players have a vast opportunity to develop implementation capabilities to partner with the majors.
Dey explained, “Implementing a Big Data solution that meets the business needs of an organization will require an understanding of the scenarios, evaluating the availability of necessary data and putting together a viable solution with the various tools that are available in the market. Hence, apart from the vendors who are focused on building technology solutions in this space, there is a huge market for services and consulting companies to start their Big Data practice.”
Hurdles remain
Like any emerging technology, stumbling blocks exist when it comes to Big Data analytics. The technology is still developing and has not matured fully. Balasaheb Vadnere, Head, Business Intelligence, Activecubes, pointed out, “As this is a fairly new industry segment, one of the challenges is to build solutions that tap the right balance of domain expertise and technical resources to devise problem statements and apply technology, respectively.”
There is a massive business and technology overlap around analytics. To get the right RoI from Big Data investments, a business needs to work closely with analysts and technologists. However, this does not necessarily occur in every company. Getting the people with that level of technical knowledge to manipulate the data is tough. Also, considering the volume, the data deluge sometimes results in privacy and security challenges.
“Efforts to tackle these are in their infancy. Technology needs to mature in the field of analytics,” felt Vaidyanathan of CSC.
These hurdles will have to be surmounted as experts agree that the next generation of innovation and competitive differentiation will come from Big Data. Which leaves organizations with little choice but to tap into this technology and ensure that it forms an integral part of business strategy.