It's going to take how much longer?! It's going to cost how much more?!!
If those sound like all too familiar expressions of frustration, in relation to your data journey (projects), you're in good company.
It seems most corporations these days struggle to make the progress they plan, with regards to building a single customer view (SCV), or providing the data needed by their analysts.
An article on MyCustomer.com, by Adrian Kingwell, cited a recent Experian survey that found 72% of businesses understood the advantages of an SCV, but only 16% had one in place. Following that, on CustomerThink.com, Adrian Swinscoe makes an interesting case for it being more time/cost-effective to build one directly from asking the customer.
That approach could work for some businesses (especially small and medium-sized busineses) and can be combined with visible data transparency, but it is much harder for large, established businesses to justify troubling the customer for data they should already have. So the challenge remains.
A recent survey on Customer Insight Leader suggests another reason for problems in "data project land." In summary, you shared that:
- 100% of you disagree or strongly disagree with the statement that you have a conceptual data model in place;
- 50% of you disagreed (rest were undecided) with the statement that you have a logical data model in place;
- Only 50% agreed (rest disagreed) with the statement that you have a physical data model in place.
These results did not surprise me, as they echo my experience of working in large corporations. Most appear to lack especially the conceptual, data models. Given the need to be flexible in implementation and respond to the data quality or data mapping issues that always arise on such projects, this is concerning. With so much focus on technology these days, I fear the importance of a model/plan/map has been lost. Without a technology independent view of the data entities, relationships and data items that a team needs to do their job, businesses will continue to be at the mercy of changing technology solutions.
Your later answers also point to a related problem that can plague customer insight analysts seeking to understand customer behavior:
- All of you strongly disagreed with the statement that all three types of data models are updated when your business changes;
- 100% of you also disagreed with the statement that you have effective meta data (e.g. up-to-date data dictionary) in place.
Without the work to keep models reflecting reality and meta data sources guiding users/analysts on the meaning of fields and which can be trusted, both can wither on the vine. Isn't it short-sighted investment to spend perhaps millions of pounds on a technology solution but then balk at the cost of data specialists to manage these precious knowledge management elements?
Perhaps those of us speaking about insight, data science, big data, etc. also carry a responsibility. If it has always been true that data tends to be viewed as a boring topic compared with analytics, it is doubly true that we tend to avoid the topics of data management and data modeling. But voices need to cry out in the wilderness for these disciplines. Despite the ways Hadoop, NoSQL or other solutions can help overcome potential technology barriers -- no one gets data solutions for their business "out of the box." It takes hard work and diligent management to ensure data is used & understood effectively.
I hope, in a very small way, these survey results act as a bit of a wake up call. Over coming weeks I will be attending or speaking at various events. So, I'll also reflect how I can speak out more effectively for this neglected but vital skill.
On that challenge of why businesses fail to build the SCVs they need, another cause has become apparent to me over the years. Too often, requirements are too ambitious in the first place. Over time working on both sides of the "IT fence," it is common to hear expressed by analytical teams that they want all the data available (at least from feeds they can get). Without more effective prioritization of which data feeds, or specifically which variables within those feeds, are worth the effort - projects get bogged down in excessive data mapping work.
Have you seen the value of a "data labs" approach? Finding a way to enable your analysts to manually get hold of an example data extract, so they can try analyzing data and building models, can help massively. At least 80% of the time, they will find that only a few of the variable are actually useful in practice. This enables more pragmatic requirements and a leaner IT build which is much more likely to deliver (sometimes even within time & budget).
Here's that article from Adrian Swinscoe, with links to Adrian Kingwell, too.
What's your experience? If you recognize the results of this survey, how do you cope with the lack of data models or up-to-date meta data? Are you suffering data project lethargy as a result?
