My mom has always said “Keith, Important things in life will happen when you least
expect them.” Well, who is one to argue with mom, and that is exactly what happen to me recently regarding the Data Vault methodology? I know, I know the suspense is killing you.
First before we continue the story about my Ah Ha moment. Let me first set the stage. I cannot count the number of times I have been asked “What is Data Vault Keith? And why should we adopt this methodology?” and those answers turn into very technical conversations as many times I am discussing with IT Data Engineers or other IT professionals. But many times, that same question also comes from non-technical people and when those same conversations start to occur, I just see their eyes start to gloss over and I know they are just hearing Charlie Browns teacher saying “Whooomp, Whoomp, Whoomp, Whoomp, Whoooooom” and at that point I have lost them.
Ok, Now back to my story… I was home watching tv with my wife (She controls the remote) and we were watching MasterChef. If you have never seen this, it is a show where amateur home cooks are given challenges to make these elaborate dishes. All to eventually be crowned Master Chef. Oh, and not get yelled at by Gordon Ramsey (I like that part). As the contestants were all given their challenge all took off running into the “pantry”. It happened, right there in front of me ( just as my mom said) a non technical example of a Data Vault. Without delay I said “they look like a bunch of Data Scientist wrangling up some data”. I laughed to myself because I was the only person in the room that thought my comment was anything close to being funny. Nothing like a dad geek joke to kill a room. My wife, fully aware that I am a geek proceeded to roll her eyes instead of laughing (her loss as I thought it was funny).
In the “Pantry” were all the raw ingredients those cooks needed to create their special culinary creations. The food was all organized in a way that these cooks could easily find the ingredients they needed and find them quickly. The vegetables over here, the meats over here and as the chefs traversed this pantry with amazing efficiency, they were quickly back to their stations and started cooking. It was clear to me that there was a standard taxonomy and vernacular that all these cooks understood and the pantry was arranged in the way they could easily navigate. They were able to spend extraordinarily little time getting the ingredients they needed and get back to doing what they truly needed to be doing, COOK.
So, this is exactly what the goal is when creating your data analytics environment. Your Data Scientist and Analysts need to be cooking up some all-important business insight. They need to be able to go into the “Pantry” and get the raw data ingredients they need. The data chefs within your organization can not be wasting time wrangling and scouring for data. Time is too valuable and gaining insight and responding to that insight is what Data Driven organizations strive to do to get that competitive advantage.
Countless times I hear “Putting all this data in a Data Vault is an unnecessary hop”. Now let’s imagine switching things up on the MasterChef show. Watching all the chefs running through those doors and SURPRISE, no pantry. But instead, they ended up in the Stock Room where all these ingredients are sitting just as they were delivered.
Pallets shrink wrapped with Fruits and Vegetables in boxes stacked onto each other. Some items in cases of 24 other items individually wrapped. It would be chaos and probably a bit comical watching all the chefs spend more time trying to wrangle up all the ingredients then it would take them to cook that culinary masterpiece.
So much in the same way with your all-important data assets. If you are only bringing in Raw data in and storing them in their original formats and then asking your all to important data chefs to go and wrangle up that data. You will be losing valuable time gathering data that could be being used building critical insight allowing your organization to be making important business decisions. Because of the lost time you may even be too late to on a decision and losing an important business opportunity.
It is a waste of your Data Scientist and Analysts skills to be rummaging through all that raw data. The cooking show identified the value to spend the time and energy to unpack all those ingredients from the stock room and into the pantry. Because the reward far outweighed the effort to allow their chefs to be able to create their masterpieces in a timely manner and not have an hour show of chefs navigating the stockroom.
Yes, at first glance it may seem unnecessary to make that extra hop to move that incoming data into your raw data vault. But because the Data Vault methodology is highly agile and automatable. It is a methodology that if leveraged properly will get those critical ingredients into that “Pantry” and will be an invaluable investment for your organization. Empowering your data community to democratize that data and gain critical business insight. So, there it is, my non-technical ah ha moment. Happy Cooking Everyone!!
The Key To Unlocking The Power Of Data