Should we aspire to be statisticians, or is data science the evolution of statistics? I came across this amazing debate (Data Science and Statistics: different worlds?) that is just phenominal.
I am somewhat in awe of statisticians. I can’t help but feel somewhat deficient in my statistical knowledge, but being aware of this I drive myself to bridge that gap. What is so typically unlike me, is that there is an underlying vibe that I feel inferior to a statistician. Now that is slightly odd. Normally, with a goal in mind I am single-minded and incredibly driven. But for some odd reason there is this seed of self-doubt when it comes to statistics. For me, it perhaps comes down to two things:
The first is that my early experiences of statistics were truly gruesome. What an insanely boring subject statistics is at school. It is so far removed from the real world and interesting problems that it is incredibly hard to identify with. It took me years to fall in love with statistics.
The second reason that perhaps I fear statistics, is the underlying attitude of ‘traditional statistics’ which is fundamentally rooted in mathematics and has adopted quite a disparaging and unwelcoming persona. This is a terrible stereotype, and I most certainly am not tarring all statisticians with this brush, but they are fundamentally professionally trained skeptics. Statisticians are also highly trained to cut to the bare truth and highlight uncertainties and errors. At times, this comes across as “no, you’re wrong. You idiot”.
Statistics is a hard-won path to mastery. And the conservative skepticism is critical in areas of medicine and public policy where the implications of poor decisions are far reaching and potential life-or-death. But then there is another breed of statistician for whom statistics is all about the primacy of the data. Pioneering statisticians such as John Tukey and Leo Breiman are widely acclaimed as visionaries, and yet their perspectives were at one stage almost heretical within the statistical world.
There seems to be a lot of discussion about the current and future role of statistics in our world. Much of this is centered on a necessary paradigm shift within the statistical community itself, to embrace the wider world and re-emphasise the contribution that statistics is able to have in the modern data era. Perhaps we shouldn’t concern ourselves with this debate, because if statistics does not open its arms to the world, then computer science and information sciences are all too ready to fill that gap.
I will leave with this thought on data science from Thomas Lumley, Professor of Statistics at the University of Auckland and author of the blog Biased and Inefficient:
“Will computer science and informatics eat our lunch? Only if we let them, and it would be bad for data science, too”