Analytic Databases and the Boiling Frog
Short transactional requests and more complex, often longer analytic requests, demand different architectures. Analytical databases, though quite diverse, are preferred platforms for the structured-data analytical workload.
However, is a data warehouse workload an analytical workload? And with the analytic capabilities added to most database management systems in the past few years, what is left that is not an analytical database?
I’ll start with the data warehouse question. Large enterprises usually have multiple structures that fit this bill, while midsize organizations tend to have 1 or 2. By the way, I am speaking of the actual database(s) that plays the role of the data warehouse in the data warehouse architecture. Many shops, especially the larger ones, have multiple databases in a data warehouse “ecosystem”. These may be staging, operational data stores and data mart structures. Again, I am referring to the data warehouse database(s).
A data warehouse may or may not be an analytical database, depending on what your architecture expects the data warehouse to do. Taking the 3 demands of a data warehouse ecosystem – intake, distribution and access – it is the access function that determines the analytic nature of the database. More physical tiers can mean more platforms in the ecosystem.
Databases that support data access are analytical unless the demands are limited to reporting. Organizations that are still doing only simple reporting with their data are leaving a lot of value on the table.
It’s a chicken and egg game. If you don’t have the systems to support an analytical workload (high data volume, complex data model, varied traversal patterns, multi-step operations, occasional interim results), often times you don’t pursue an analytic workload as a company so you leave value on the table.
Often this happens because workload and query demands increase modestly in the organization and neither IT nor the application area or the user area sees their workload as needing to add the complexity to the organization of a new platform. To use another analogy, it’s like boiling the frog. We should make ourselves aware of gradual change lest we suffer eventual undesirable consequences.
So many capabilities have been added to previously transactionally-focused databases such as in-database analytics, in-memory capabilities, columnar orientations, new languages, new data types, etc. that arguably we have many more analytic databases than before. One thing is certain however. If these types of capabilities are not being used, it is not, effectively, an analytic database.
Now might be a good time to review your platforms and architecture for current and future suitability.
This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.