The 5 Methodology Milestones for Big Data

In the modern world, fashion absolves no one. Cloths, cars, properties, slang, travel, furniture, behavior, etiquette, socializing, art, music, hobbies: you name it, everything is marked by the trend. In the business world, fashion is equally influential: mergers and acquisitions strategies, IT, selling tactics, management styles, changes and optimizations, trainings, profession and knowledge, and growth. Some walk the floor, some follow the trend; in our own different ways, we are all subject to the fashion magnetism.

Presently, one of the fashionable trends on the business market is without doubt Big Data.

From a cursory Google search, you come up with more than 50M… almost the same as when you search for David Guetta.

So, many companies like and want to have Big Data. For some it is a must, for others it is a nice thing to display. However, entering the world of Big Data is more complex than buying a new car or training your selling team to pursue a more aggressive business development practice. It is complex because it needs considerable investment in resources and careful planning with no quantifiable “in time” and money return on investment. Big Data, like any new business initiative, needs a strategy and a plan, otherwise the required investment can go through the roof with results maybe never actually achieved.

The key thing for Big Data or any data-driven initiative to be successful is to have a methodology for implementation and the correct team in place.

If we look back over the history of data evolution, we can see that data has been around forever and that Big Data is actually nothing new to the scientific world. Scientists have always approached data collection, data mining and conclusions in a project based structure working in flexible teams. Also, an important premise is always starting any research with the question already in mind.

In a business environment, successful Big Data initiatives should follow the same pattern:

Be project based (which means that they must have a beginning and an end date), have clearly defined questions (goals, targets) and can count on a multidisciplinary team (data scientists, PMO, subject matter experts, etc.).

There is no clearly defined project methodology for a Big Data project. Many use the Agile approach as data initiatives have strong dependency on IT and heavy computation, others have adopted the SMART Model, some have developed their own internal methodology.

A frequently used approach (especially in SMEs) for a Big Data kick off is resulting in the use of DMADV (originally used in Six Sigma projects) as it encompasses all the important phases for structuring a data initiative but in this case, instead of focusing and analyzing internal data and processes, the attention is on external data or a merge of internal and external data.

The Project Milestones

Define: state goals or the questions we want to find answers to (customer requirements, expectations, behavior trends, interdependencies).

Measure data (internal and external). If data is not in place, classify technical requirements for data collection, storage and analysis, set up collection points, collect data and measure.

Analyze: exploratory data analysis, identify trends and patterns (by using statistical tools: inference, regression models, etc.). Develop best growth alternatives (new offerings).

Design: select best new offering, develop it and the supporting processes to meet customer expectations, run DoE

Verify the design (the product, process, or service output and performance against customer requirements): set up pilot runs, implement the production/marketing/sales/customer care process and hand it over to the process owner(s).

Without a structured methodology which has clearly defined milestones, launching a data-driven initiative is simply inefficient.

Nevertheless, there is one additional and very important aspect to any methodology for Big Data: the training of the people to think, work and deliver in a new way.

Big Data needs a different set of minds, ones that measure in order to decide. And I am not talking about the Big Data stakeholders, the ones who do the number crunching and evaluations. I refer to the employees, the teams, the contractors of the whole organization and the supply chain. Training can have different tactics and scope for different organizational levels, but it is mandatory as Big Data might be trendy now but a type of fashion that will prevail in time.