Big data continues to advance trends in technology promising to allow companies to manage their data in a more effective way. From publishers and advertising marketers to pharma analysts and energy knowledge workers, businesses are now more aware that using the massive amounts of data they have in front of them should be a key part of their everyday activities. But if someone told me they understood all facets of big data, I would not believe them.
Data is more than volume. Yet, enterprises’ strategies seem to stick strictly to the traditional way of managing data, concentrating only on high quantity and common business intelligence tools rather than experimenting with new approaches aiming to manage “the quality dimension” of data and information.
I think that qualifying data still remains the big issue and getting organizations to think differently remains the big challenge.
I was living in San Francisco when the Oakland A’s had that stunning season in 2002, including the record-breaking winning streak as beautifully narrated and portrayed in Michael Lewis' book “Moneyball” that has been described as the simplest way to understand the opportunity that the increasing availability of a vast amount of data presents. The opportunity should really hit home for any organization.
Think about it: there are a lot of sources of data (not just social data) available and accessible to anyone, and I bet there are many smart data analysts out there who would make a big impact even in small organizations. And by the way, if you are finishing high school and concerned about the economic crisis, here is a field that for sure will not suffer in the next 5-10 years!
Data analysis is a major opportunity to level the playing field as today you have more data at your fingertips. But more data per se doesn’t mean anything if you do not understand it. It’s all about accurate analysis that may drive more efficiencies and better decisions, by optimizing operations and mitigating risks.
In “Moneyball,” all the data they needed was right in front of them – they just needed to look at it differently. While statistics had been part of baseball for many years, some stats were ignored, even if they had a high correlation and impact with important data, like number of runs scored. Social media and web statistics are a good place to start, but don’t stop there. Don’t be afraid to experiment with different representations and mixes of data, embrace new approaches and try software tools to aggregate and correlate unstructured content (social media and web pages, typically) and internal structured data such as system logs, etc.
Companies that do not see immediate results often get discouraged. Number crunching is not easy. Combining data from different sources in unique ways makes understanding it even more challenging, and certain skills are obviously required (see the school curriculum of the fictional A’s numbers guy, Peter Brand or the real Peter DePodesta). Big data management may not lead directly to greater profits but discipline, and the willingness to accept a certain degree of failure as you experiment, can lead to interesting and valuable results.
Often organizations suffer from being too focused on the day-to-day activities, but every organization needs leadership – their own Billy Bean – who can set priorities and implement the strategy while embracing the big picture benefits that can be achieved through data analysis.
As with baseball before Billy Bean, there is a lot of information available right now that could bring value to companies in any sector and of any size. That’s why today the ability to look at data in a different way is a core competency, and companies are re-thinking their approach to data, stopping simply gathering data by penalizing its quality.
Technologies that help you analyze unstructured data are affordable, and companies are increasingly adding functionalities that are applicable to specific fields and industries, empowering their business intelligence solutions by adding advanced text analytics features.
The stage is set for any Oakland A‘s type of organization to exploit this information to gain a competitive advantage even against richer, more established organizations. If you don’t do it, someone else will hire a Peter Brand and change the rules of the game. Better to get in the game now.
Article written by Luca Scagliarini
Image credit by Getty Images, E+, shannonforehand
Want more? For Job Seekers | For Employers | For Contributors
