This little ditty was originally posted back in Q1-2014, but we just recently found the need to refresh the message for a new audience.
More and more often, we are being asked this very question by clients looking to ride the next wave in an effort to extract maximum value from their digital businesses. While Big Data is unquestionably a hot topic and term du jour, it is also something worthy of our client’s interest.
According to SAS, Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured. And big data may be as important to business – and society – as the Internet has become. More data ostensibly leads to more accurate analysis that then leads to more confident decision making and better decisions can mean greater operational efficiencies, cost reductions and reduced risk.
According to Webopedia, Big Data is the term used to describe a massive volume of both structured and unstructured data that is so large that it’s difficult to process using traditional database and software techniques. In most enterprise scenarios the data is too big or it moves too fast or it exceeds current processing capacity.
Lisa Brown, veteran technology marketer, author and Forbes contributor, provided the following humorous internal debate just last year:
Big data is new and “ginormous” and scary –very, very scary. No, wait. Big data is just another name for the same old data marketers have always used, and it’s not all that big, and it’s something we should be embracing, not fearing. No, hold on. That’s not it, either. What I meant to say is that big data is as powerful as a tsunami, but it’s a deluge that can be controlled . . . in a positive way, to provide business insights and value. Yes, that’s right, isn’t it?
After much dialogue and debate, Ms. Brown put forth the following simplified definition: Big data is a collection of data from traditional and digital sources inside and outside your company that represents a source for ongoing discovery and analysis.
Excuse the pun, but the “big” distinction for data today comes from the volume, velocity, variety, variability and complexity. The first three were first defined by Gartner analyst, Doug Laney, whereas the others were added on after the fact my other industry players.
- Many factors contribute to the increase in data volume. Transaction-based data stored through the years. Unstructured data streaming in from social media. Increasing amounts of sensor and machine-to-machine data being collected. In the past, excessive data volume was a storage issue. But with decreasing storage costs, other issues emerge, including how to determine relevance within large data volumes and how to use analytics to create value from relevant data.
- Data is streaming in at unprecedented speed and must be dealt with in a timely manner. RFID tags, sensors and smart metering are driving the need to deal with torrents of data in near-real time. Reacting quickly enough to deal with data velocity is a challenge for most organizations.
- Data today comes in all types of formats. Structured, numeric data in traditional databases. Information created from line-of-business applications. Unstructured text documents, email, video, audio, stock ticker data and financial transactions. Managing, merging and governing different varieties of data is something many organizations still grapple with.
- In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Is something trending in social media? Daily, seasonal and event-triggered peak data loads can be challenging to manage. Even more so with unstructured data involved.
- Today’s data comes from multiple sources. And it is still an undertaking to link, match, cleanse and transform data across systems. However, it is necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of control.
While the word “Big” is certainly a noteworthy distinction, acquiring data from all relevant sources and transforming it into actionable information has always been mission critical. It was true back in 1996 when building a B2B eCommerce business unit on behalf of Panasonic and it remains true today as we position clients such as Sony Music for success through the deployment of integrated business intelligence teams.
Now that we’ve provided a better understanding of what big data is, our next installment will focus on keys to succeeding in the Big Data-verse. Stay tuned!