Finding the elusive large wisdom in big data was the headline of a post I authored in December 2014. We have come a long way since then in terms of storing, analyzing, and handling vast amounts of data in a digital context, but people still struggle to uncover meaningful nuggets that can improve business outcomes. I am revisiting the concept to evaluate how far we have progressed and how much further we still have to go. Baseball is one of the clearest instances of how data can have a positive impact on a company.
The book and film “Moneyball” depicted how Oakland Athletics general manager Billy Beane revolutionized baseball by adopting advanced statistical analysis instead of relying solely on human scout reports. Baseball is now ruled as much by analysts as it is by experienced players — but is there such a thing as too much data?
According to Alex Spier of the Boston Globe’s Sunday Baseball Notes column, Boston Red Sox manager Alex Cora will have 11 coaches on staff this year for 26 players. You compare that to Terry Francona’s six for 25 in 2011, you can see that the figure has nearly doubled. This, according to Spier, is due in part to the increasing amount of data that teams are collecting, which necessitates more people to watch, understand, and implement a strategy to exploit it. “The result?” wrote Spier. Several clubs now have three hitting coaches, and staffs continue to expand in an effort to condense mountains of data into something that the 26 players on a roster can understand.”
“Startups that analyze and extract insights from fast data have attracted a lot of interest from both public and private investors.” Baseball serves as a testing ground for complex statistical analysis, and businesses may learn a lot from how the sport handles growing statistics.
Companies dump data into data lakes, experiment with machine learning models, and use this data to make decisions, but they still have to make fundamental decisions every day based on their ever-growing mountain of data. Sure, machines and software can assist — and they are improving — but anyone who has dealt with poorly targeted advertising or email marketing knows there is still a lot of work to do.
The difference in play now versus 2014, according to Deepak Jeevankumer, managing director at Dell Technologies Capital, is that firms are increasingly focusing on getting insights to the people who need them faster. “Big Data is no longer as important as ‘quick data.'” “E-commerce customers, streaming media users, gamers, stock/crypto traders, and enterprise marketers all want quick insights and expertise,” Jeevankumer explained. He believes that data should be examined in real-time while flowing and after a query is produced, and that entrepreneurs who build solutions that enable companies (and baseball clubs) to do so would be more successful in the end.
