Allow Your Dark Data to See the Light

You can run but you can’t hide from big data analytics. In fact, it’s already a cozy part of your life. Think: Fitbit, Amazon, Waze, Google, Facebook. Data is everywhere and it’s continuing to evolve and play a significant role in the digital transformation of a business. In fact, as the era of the Internet of Things (IoT) continues to move at breakneck speed, there will be considerable challenges for organizations in what to do with managing “dark data.”

What is dark data you ask? According to Gartner (who originally defined the term), dark data is the “information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes.” Much like dark matter in physics (we promise this won’t be a physics lesson), which is invisible matter that makes up most of our universe, dark data comprises the bulk of the company’s universe of information assets (as much as 80%, according to Gartner and IDC estimates) and has mostly lied untapped. Yet, it is within this dark data where business intelligence can be gained.

Reap the Cognitive Benefits

As organizations continue the upward trajectory toward people-centric, there is untapped potential in dark data to extract real-time insights in new customer segments, buying patterns to make better product recommendations, and developing new products and services based on consumer behavior data. This is only a snippet of insights you could gain from trends in marketing, sales, production, and distribution that have previously stayed hidden.

IT has developed and maintained document and data control systems in the past but new systems, software, and infrastructure will need to be developed to be able to move, process and store unstructured data, as well as continue supporting the 20% of traditional, structured data located in data warehouses.

As the need continues for service management to become more people-centric, and the necessity increases for systems to be in place to better capture and interpret the data, IT will stand at the crossroads of cognitive computing. These new systems will generate analytics, insights and advice from multi-structured dark data that will spearhead digital transformation and innovation.

Avert the Risks

Amidst all of the excitement around dark data and the intelligence it can provide, organizations need to realize potential risks. It’s wise to have systems in place that help mitigate any potential issues. For example, there should be a regular cadence of inventory assessments of where the dark data resides, to maintain control and security of the information and insights the data holds. Additionally, there should be periodic security audits to evaluate risks and expose any potential issues that may not have been brought to light earlier. Lastly, encrypt! Any digital data needs to have strong encryption, and this includes dark data.