While the topic of Big Data grabbed a lot of media attention in 2011, mostly around defining “Big Data” and exploring the plethora of emerging technical solutions, we see 2012 as the year that Big Data moves from technical curiosity and pilots to real-life implementation for mainstream organizations. We expect that as organizations adopt innovative new data processing and analytics techniques such as Hadoop to gain insights from vast amounts of highly unstructured data streaming into the enterprise, they will face a variety of business challenges related to the business case and prioritization of analytics initiatives, access to data management and analytics skills, and data confidentiality and privacy.
In 2012, the challenges will move from mostly technical ones such as determining the right solutions and platforms suitable for gathering Big Data-related insights to business challenges of determining how and where to get the optimal business benefit from big data analytics. With so many different sources of data streaming into the enterprise, including new data streams that have become available from external sources, one of the key considerations will be knowing which streams of data can give the greatest payback in terms of real-time insights and corresponding real-time decision making.
As highlighted in my prior blog post, Big Data and Smart Computing – The Road Ahead for IT, Smart Computing is an excellent complement to the Big Data challenge because once CIOs have transformed their information infrastructures to intelligently collect, store and manage this information, they still need rapid and automated ways to sift through the data, identify patterns, and perform intelligent analytics. Our additional predictions in this space therefore fall into two areas: Data Management & Storage and Data Analytics.
Data Management & Storage
We expect that, driven by the need to address exploding data growth and diversity, organizations will enhance their data audit and security measures for compliance, renew their emphasis on storage rationalization, and prepare to transform their information infrastructures. The time is ripe to fundamentally re-think and re-design how enterprise information is collected, stored and managed and to take advantage of today’s lower costs of storage, processing and analytics. The challenge highlighted earlier, however, will be to determine the key business opportunity areas beyond quick wins in certain areas such as click-stream, ad-targeting and customer sentiment scenarios.
In the storage arena, in particular, we expect that with growing data streams driving up storage costs, organizations will look to rationalize their storage strategies to keep data costs under control. We anticipate a two-fold benefit as organizations firstly rationalize storage across their existing tiers of storage infrastructure and, secondly, procure and deploy new storage technologies to further optimize these cost savings.
The ability to gain insight and competitive advantage from Big Data is both an opportunity and a competitive threat. Businesses and government agencies alike are adopting advanced analytics technologies to build innovative new services, improve service levels, and drive greater efficiency. We anticipate that use of Smart Computing and Intelligent Analytics will move beyond well-known areas such as predictive analytics for sales data, and become increasingly adopted within the IT domain in areas such as Data Center Automation, Security Event Analytics, Regulatory Compliance, and Problem Management Automation. This approach will help IT continue to automate critical aspects of its operations such as data center, security and help desk functions and free up staff for more specialized and value-added tasks. The intelligent analytics component will help IT move further along the automation continuum into new areas previously reserved for human decision makers and will help to alleviate manual bottlenecks where throwing more bodies at the problem is no longer tenable.
In summary, we recommend that organizations consider these predictions when formulating their approach to Big Data in 2012, and develop a strategy that takes a lifecycle approach from data acquisition, to access, to availability, and to analytics. Big Data is much more than just a storage issue – it’s really about your strategic approach to information management and the business value you’re able to extract across the full lifecycle from acquisition to analytics. In 2012, the topic of Big Data will not be about the attributes of the data per se (i.e. the much-touted volume, velocity, variety etc.), but about what you do with it – the new attributes such as extreme scale merely drive the business case for transformation and new techniques.