Shopping for data is no different than shopping for apparel, shoes or other material things. Let’s take a look at the different events that happen while we go shopping for data and relate it to our routine shopping for our daily needs.
Among the various malls in our area, first we try to figure out which mall has shops for all types of products, age ranges, kid play facilities, restaurants, etc. to cover all our needs at one place.
Comparing this to a data mall – this would be a data marketplace that has data from all different sources (reference data, master data, transactions, reconciliations etc.). Also, all these data sources must be displayed at the main page, hence creating trust. This is similar to marketing banners hanging in front of a shopping mall.
Consumer: Hello, I want to create a predictive model on trade controls. What all data sources you have?
Data Mall: Hello Mr. LAST NAME!
(Creating personalization by identifying a user by his or her name)
Data Mall: Here is the list of all data sources existing with me.
Consumer: I may need 'A'.
Data Mall: Ok, we are receiving 10 data sets (feeds) from 'A'. Frequency as three monthly, two weekly and five daily. It also suggests, the last two receive dates for each data set. This is a visual confirmation of the consistency of the data sets received, and thus the trust factor enforced.
Consumer: I would like to go ahead with 'A' but need some more data sources as well.
Data Mall: Ok, I’ll show you the entire list, but here are some of the combinations of data sources that other users have used with data set 'A'. Is this something you also want?
(The idea here is to create a recommendation model as if you’re going to shop retail at an Amazon or Flipkart store.)
Consumer: Sure, this sounds promising! I'll go with an 'A’ and ‘C' combination. By the way, do you also have any listing of end-user application or models being prepared with these data sets?
(Here comes the curiosity and faith as to why re-invent the wheel if something similar may already exist!)
Data Mall: Yes! Here are the different models created with various combinations of these data sources. Also, you can see the usage details across enterprise.
(Usage details will tell as to how frequently a model is being used across LOBs. This may suggest the effectiveness of respective models).
Consumer: I'll like to go with the model 'Zingo'. Can you give me the business or IT owner details to move this ahead?
Data Mall: Details are as follows: <Name>, <Designation>, <Department>, <Phone No>, <Email ID>
The end objective of any data mall should be to create a data marketplace where any line of business can come and shop for their data needs. The framework should support lineage and traceability across a minimum of three entities, i.e. data source <-> data set <-> end-user application. A data subscription monetary model along with a data access rules engine are required by the data marketplace, as well, to keep running its operations and carry regular maintenance and upgrade activities.
This article is not about any industry domain of telecomm, retail, banking or insurance. Instead, the content applies to any enterprise aspiring to create a data marketplace and not a data landfill.
Crafting this journey from truth to trust is critical. Taking a cue from one of my previous articles, the two can be implemented with a DCQI framework as displayed below. However, to reiterate, the objective is to create trust. The implementation methods may vary.