A Workload on Hadoop
I recently spoke to a company about their use of Hadoop.
An additional investment in the legacy environment to bring it to standard would have cost millions. The team spent less than $1 million with Hadoop to achieve the goal – and it appears to be much more long-lasting.
Doing some level of Retention Processing is non-negotiable for a company like theirs. Spending less results in a large cost savings. From the TCO perspective, they have achieved a several hundred percent ROI on infrastructure with Hadoop.
All the Retention Processing they want to do across all 50 states, not just one, now runs in a respectable time. This is obviously a repeatable proposition so it is run daily – as opposed to the weekly cycle it had been relegated to.
In the 1 year since migrating Retention Processing to Hadoop, the quality of the processing has gone up as well. The data scientists are “uninhibited” in their ability to explore. They have flipped their time allocation from spending 80% dealing with data issues that the environment was unable to handle, to spending 80% of their time actually analyzing data.
With fewer resources required to get to the same point, they achieve ROI with Hadoop. They save 60% of their data scientists’ time (80% prior, 20% now) for accomplishing the same data preparation tasks and are able to redeploy it towards retention modeling. This win is two-fold since the Retention Processing models are curbing attrition at a much higher pace.
The success of the Retention Processing program on Hadoop has had the knock-on effect of exposing all of the company to Hadoop and consequently several other projects have started using it. Some applications have been completely migrated to Hadoop. They estimate the net effect that Hadoop applications have had, in about a year, is many millions of dollars.
They have already earned enough from their Hadoop implementation to pay the costs. These applications were needed regardless and would have been attempted in the legacy environment. This would have required the legacy environment to be expanded at tenfold the cost of adding Hadoop.
Maybe it’s time to think of how Hadoop can help your organization.
Disclaimer: This post was brought to you by IBM for Midsize Business and opinions are my own. To read more on this topic, visit IBM’s Midsize Insider. Dedicated to providing businesses with expertise, solutions and tools that are specific to small and midsized companies, the Midsize Business program provides businesses with the materials and knowledge they need to become engines of a smarter planet.