Information Management: Dividing Responsibilities
Many approach their information management architecture with a philosophy about data modification that seldom gets air time, yet is a vital component of long-term work effort and the ability to utilize information.
That decision has to do with the role of databases and where editing will occur on data in the architecture. On one extreme, in one example, some shops treat their data warehouses like raw dumping grounds for source data. Those data warehouses exist solely because of concurrency issues on the data in its other/source database. Other potential benefits of having a separate data warehouse were not baked into its build.
Yet, as always, there are transformation needs for data quality and there are integration needs. If there are multiple sources in one of these data warehouses, there is no integrating the data models of the various sources. That is left to the consumer of the information.
In this model, at the extreme, it is not thought there can be any common transformations and integrations. It is believed all consumers have unique needs and can come to the database dumping ground and serve themselves. They are not serving themselves business intelligence mind you, just serving themselves data to begin the process of business intelligence. There’s still a lot of hard work to be done, but to be sure, those who built the database have added some value.
At the other extreme end, users self-serve their way to fully baked information. Any integration needs are already handled, not just by housing diverse data in the same database, but also through the data model. Data quality is brought to a standard that is set by someone the end user may not even be aware of – someone from a business area who has brokered the rules. There is a process for any integration requirements to consider incorporating them into the database common to all.
Users in this environment have greater access to usable information. The second time they access something is very easy because the first time was optimized.
If, for example, they utilize a complex customer lifetime value formula, instead of taking all the raw data from the data warehouse and calculating it in Excel, the data team will reverse-engineer the formula, calculate and store the customer lifetime value during the load and make it available not just one department, but also to anyone else who could benefit from it. Excel, and its undisciplined nature, are often in contrast to architected approaches.
This strategy can only work with a level of metadata present in the architecture to communicate the originations of the calculated values so they are used appropriately.
Both extremes are unnecessary. However, consideration of the reuse and self-serve potential of everything we do is a prudent course of action. It does not take much longer to do things this way and it doesn’t matter the size of your organization, but it does take architecture and it does take mindfulness.
This reminds me of a recent tweet response by Merv Adrian of Gartner:
This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.