I come across a lot of discussions around the (presumed) death of data/information modeling/architecture/management/whatever in the current era of “Big Data” and “Analytics”. Professionals and customers alike are wondering if and how we need to look at these subjects now the focus is shifting towards pure data valorization(monetization) technologies, at the expense of the more ‘traditional’ data modeling skills and activities.
Most professionals and customers don’t realize that from a customer’s perspective technology should not matter, only functionality does. Data abstraction and organization is there to contain and rationalize away from technology choices. And we humans are always doing this, even if it is only in our own mind. All data modeling and data organization aspects are there to make this explicit and manageable to abstract away from implementation details.
All ‘data modeling’ done as part of requirements engineering, or as part of system design should be seem as technology abstraction mechanism’s which are important techniques to facilitate data management in it’s many aspects, from specification, governance, data quality to data logistics. As long as control over information is needed, (good) data modeling has it’s place.
Data and technology
For many reasons technology oriented forces are constantly ignoring, downplaying or opposing this technology agnostic approach to data. There are few whose interest align with this abstract notion of data management. This creates constant pressure on data management/modeling/architecture that is NOT technology focused. Indeed only those benefiting from good data management have an interest. For a lot of software or service providers on the data side of things this does not hold. In fact, it often hinders them in their business. Alas, customers usually presume their technology partners will also provide them with good data management, failing to understand that this is exactly what their partners are NOT providing because it’s essentially not technology.
Technology pressure and data modeling.
To withstand the current pressure from technology and the need for monetization we need strong(er) data -modeling and -management skills. These do not materialize out of thin air. The amount of time and money invested in technology and it’s valorization are immense. The amount of time and money spent on (good) data -modeling and -management is (quite) small in comparison. The current sentiment is that there is not only no money made in these areas of expertise, but that they are even detrimental for data valozrization. This is the trend that needs to be reversed, and this is not easy at all. For an average data modeler this is a tall order indeed. As a profession we need to own up, break with the past and put a brake on the T in IT
Data as Usual?
Does this mean we go back to business as usual? No, because the increased focus on data valorization indeed should trigger us to do better data modeling and data management. In the light of increased technical diversity we should strengthen our data organization and modeling technology, skills and knowledge to better facilitate the increase in data valorization. Alas, where technology is pushed mercilessly by large organization’s, better data modeling and management skills are hard to come by and usually are far more dependent on the commitment of the individual. Now even more so the say 20 years ago.
But it’s also and imbalance between the forces of structuring and organization and the forces of usage and valorization. And an imbalance to these forces, which known to several oriental philosophies as the yin vs yang approach, can make the organism ill. We saw this partially in the totally unexpected financial crisis and the current response by financial supervisors. They focus more and more on actual data management of financial and risk data directly instead of the financial corporations themselves.
“If you see these same patters in your environment, realize that data modeling is of interest to data management as one of it’s prime deliverables and steering aids, and understanding the balance between data management and valozrization is of the essence. Stay tuned for a follow up post on the the art of balance in data management”
ABOUT THE AUTHOR:
Martijn Evers is Chief information architect at and co-founder of I-Refact, delivering top data engineers and architects to high profile organizations helping them to managing their data effectively. He’s an expert facilitator and researcher on a large range of data modeling topics and is regarded as one of the best data architects around.