Advanced analytics has more to do with people and our changing attitudes towards data than it does with big data, technologies or machine learning. McAfee and Brynjolfsson (2012) describe big data as a management revolution defined by the use of data to drive business decisions. But we have had business intelligence for decades – what has changed? Fundamentally, I believe that it is our attitudes which have changed. We’re fascinated with data and what it might tell us about our lives!
The story of Moneyball, the triumphs of people like Nate Silva, the rise of Google and Amazon and personalised services have had a dramatic impact on the way we view data. Do you remember high school statistics? I sure do (**cringe**). If my statistics teacher told me 15 years ago that I would put my life on hold to go back to uni to study statistics and data science I would have thought she was crazy! I think we all would have. But look at the world now – it has gone ‘data mad’. It’s a fascination that is driving economies and all aspects of our personal life. And the growth of Internet-enabled devices will only extend the possibilities. Frankly, the ‘data-fication’ of our lives has become something of an obsession as we look to optimise every aspect of the way we live.
What is really driving the “big data revolution”? It is a little bit “chicken or the egg”. Data analysis is certainly not new. There are only a handful of techniques that might be considered genuinely ‘new’, with many simply a restyling or clever combination of existing ideas and technologies. Certainly (in my mind) it is not technology driving this change. It is not Hadoop. It is not Tableau, nor deep neural networks, nor ‘data lakes’, nor… Whilst these products all help to enable change, to me, these technologies are the bi-products of the revolution. And the revolution is us.
Fundamentally, people are at the center of the analytics revolution. Without a doubt, it is our creativity, imagination and intense desire to outperform our peers (I mean this in a business-sense, but increasingly I see personalised data services playing a large role in our daily lives) that is driving the advanced analytics movement. Traditional business intelligence is classically descriptive. Confirmatory statistics has it place. There has to be balance and means to handle uncertainty. But the true fascination has to be in discovery – new insight, new opportunity, new perspectives – which is the very heart of exploratory data mining, data science and advanced analytics.
It truly is fascinating, the way that data and data scientists are perceived right now. Somehow, data has captured imaginations around the world and the stories and level of interest are strikingly similar to the fascination with futuristic worlds portrayed in sci-fi classics such as Star Wars. Scientific discovery has always been the frontier of the human imagination, and that frontier is currently defined by big data. Fascinating times ahead, and what an exciting time to be a part of.
McAfee, A., Brynjolfsson, E. (2012). Big data. The management revolution. Harvard Bus Rev, 90(10), 6167.