There’s not a company on the planet that isn’t generating and collecting data in some way. Today alone, the world will generate trillions upon trillions of bits of information. But there’s a problem with gathering all that data: It’s way too much for companies and their chief data officers (CDOs) to handle — and way too much for people to comprehend.
Much of what’s collected is called dark data, information that’s collected but never used. In our experience, dark data makes up most of the information companies collect. Unstructured data — videos, images, emails and other points that can’t be inserted into a spreadsheet — falls into this category, but businesses also collect a lot of information that for one reason or another never gets analyzed.
What’s exciting is that while a lot of valuable data is considered dark, when brought to light it can reveal real insights into the wants and needs of your organization, including around your customers and employees, the quality of your assets and manufacturing and the risks your brand faces on social media.
More dark data ahead
CDOs need to get a handle on their dark data now, as their companies are gathering increasing amounts of information every day. Businesses are putting data-collecting sensors on pretty much everything — industrial equipment, agricultural fields, office building walls and people’s wrists. They’re using drones to measure climate impacts and snap pictures of assets, while artificial intelligence (AI) technology is finding data points in minutes rather than the days, weeks or years it used to take to uncover.
While companies don’t need to analyze every number or statistic that comes their way, they also shouldn’t continue keeping all this data in the dark.
A lot of this is potentially valuable information that can be used to increase productivity and boost growth. And not knowing what’s there can pose significant risks.
Say a utility has sensors on its equipment to detect gas leaks, and a safety incident occurs. Now suppose that there was information hidden in the dark data those sensors collected that could have been used to predict and prevent the incident, but the utility never looked at that data. Having the data available but unused could bring lawsuits or a public relations nightmare.
Or take a bank dealing with a fraud case. An analysis of its dark data might have revealed red flags that would have prevented the breach. Worse, regulators — who now have a much better understanding of the data companies collect — might be unimpressed with a bank defending itself with, “How could we have known that was coming?” Regulators expect businesses to use all the tools they have to prevent fraud and generally crack down if they see something that a company missed.
How can companies make their data more visible? Here are some ideas:
Get a handle on all your information
- Know what’s being gathered and where it’s being stored.
- Then decide whether it’s useful to the business or not.
Understand your risks
- Start by identifying the risks in your business that require you to take action to address. Risks could include becoming more compliant with certain industry regulations, fixing holes in your cybersecurity defenses or better understanding when equipment might break down.
- Once you know the risks and what actions you must take to fix them, start identifying what kind of dark data will help.
Use AI tools to parse data
- Use artificial intelligence tools that can identify patterns in photos, detect irregularities in sensor data and uncover other hard-to-find insights. A lot of data, whether it’s images, weather patterns or vibrations, can’t be interpreted by the average person. The volume of information will be overwhelming, and humans won’t be able to understand all of the anomalies in the information.
Get excited about data
- Understand how data can elevate business performance. Business leaders should take a keen interest in all kinds of data, not just what they can see in front of them.
- Create data lakes where structured and unstructured information gets stored.
The more data that your company can analyze, the better decisions you’ll make. While it’s important to not get overloaded with information — keep going back to your business objectives and the risks you’re accounting for to determine what to observe — you should still be aware of what you’re collecting.
As companies become even more data-focused, and as new tools emerge to help people analyze their unstructured information, it’s only a matter of time before your dark data gets out in the open. The companies that use their data to their advantage can be the ones that get ahead.
This article first appeared on PwC's website here.
