Ah, big data. It’s one of those trendy buzzwords that your boss probably hears in a podcast or at a conference, then comes to you and demands for you to have it right now. What is it? It doesn’t matter. It has to be done.
The thing is, big data is more than a buzzword, and we’re starting to enter an era in which the utilization of data to make smarter decisions is seemingly so invaluable to our future as a society that IBM created a 2,000 employee department just to advance it, and it seems like data driven companies are cropping up all over the place.
But just because you can collect data on something, does it also mean that you absolutely should?
What is Big Data?
A bit of housekeeping: You can take the term quite literally, as it refers to the vast amounts of information in the world and how we interact with it. With the influx of technology, we now have more information than ever, from social media to Google searches to just about any other industry that generates data. This type of data is characterized by:
- Velocity — Real time information
- Variety — Vast sources
- Volume — Sheer volume of information
The best thing about big data is that it can be applied to almost anything to make better decisions, from marketing trends to the way that buyers interact with the market. After all, it’s better to carefully analyze statistics and data in order to make a decision, right?
The Trough of Big Data Disillusionment
According to Gartner’s Hype Cycle, technology adoption follows along a line graph in terms of implementation and industry usage. The chart is as follows:
What this means is that while we’ve seen an influx of big data in the industry, it won’t be that way forever. As inflated expectations start to fall and reality sets in, businesses will start to see that implementation hasn’t gone as expected, and that processes need to be modified. Gartner describes it as the time when “interest wanes as experiments and implementations fail to deliver. Producers of the technology shake out or fail. Investments continue only if the surviving providers improve their products to the satisfaction of early adopters.”
Big data may be in its descent into the Trough of Disillusionment. In the same CIO.com article on big data trends, founder and CTO of Adaptiva, Deepak Kumar, said that this year, business leaders will start to feel the tug of the trough.
“Data usage will become more regulated, as providers won’t be able to keep up with the data demand and businesses won’t be able to keep up with the rising cost,” Kumar says. Furthermore, he adds, the integration of data solutions in the enterprise will “fall short,” as business insights are left in “disconnected silos of data.” In other words, companies will have valuable data collected in their databases, but may not be able to access and make use of it either due to improper management or focusing on the wrong insights.
Maria Bartiromo, a business journalist, seconds that one of the major pitfalls of big data is that it’s, well, just so big. “Information is everywhere. And investors are better off for it. The downside? It’s harder to mine and analyze and digest because there is a lot of it. The onus is on the individual to not get lost in the noise and analyze and act on portions of it that they believe to be important.”
But, Kumar says, it isn’t over for big data, so it will be up to systems managers and data monitoring technology to deal with these challenges — and that’s where IT can step in.
Going Up the Slope of Enlightenment
How do you keep your enterprise from wasting money on the insights that don’t matter? While there’s no tried and true rule of thumb for the specific data you gather, here are some guidelines to make sure that big data is a successful tool in your toolbox:
- Make sure every piece of data is supporting the business objectives, goals and projects — As Oracle noted in their “Five Big Data Mistakes” article, one of the easiest ways to misuse big data in your enterprise is to focus only on what the technology can do, not what it should do. Make sure that, particularly if you work with a vendor, you carefully select the use cases you think are best, not accept every single one because you may use it one day.
- Adapt your business — In the same Oracle article, Subramanian Iyer also emphasized that adopting big data into your organization means that you have to create the right infrastructure for it. This means that it pays to go the extra mile to maintain your data.
- Avoid spurious correlations — i.e., just because you’re gathering data and two sets happen to maybe show a correlation, doesn’t necessarily mean that they actually have any causation or association with one another.