Nate Silver in Toronto, November 7th, 2013. Photograph: Terry Dawes. The keynote address at SAP’s “Conversations on the Future of Business” conference in Toronto last week was delivered by Nate Silver, author of “The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t” and the now famous “FiveThirtyEight” blog, which correctly predicted 49 out of 50 state races in the 2008 U.S. election and 50 out of 50 in 2012.
Silver started out in the world of analytics in 2002 by developing a baseball stats program called PECOTA (Pitcher Empirical Comparison and Optimization Test Algorithm), rather clunkily named for a middling 1980s Kansas City Royals infielder named Bill Pecota, who often stymied Silver’s beloved Detroit Tigers. A string of baseball forecasting successes gained PECOTA some renown, but Silver’s profile remained high mainly among fellow baseball statheads.
Then came 2008 and the development of FiveThirtyEight. Calling 49 out of 50 states correctly in the 2008 U.S. Presidential Election should have served notice that something new was happening in the world of political forecasting, other than just the pointless shouting of the McLaughlin Group and random guessing of TV pundits. But anyone can get lucky once. Twice looks like accuracy. In advance of the 2012 election, the FiveThirtyEight blog was given a home on the New York Times website, and Silver found himself speaking to a new audience and threatening the game preserves that provide safe haven to the upper pundit echelon.
In the lead-up to the 2012 election, those who stood most to lose from Nate Silver’s newfound prominence spewed a little more vitriol than usual in his direction. Joe Scarborough on NBC lacerated Silver’s prediction of a substantial Obama victory. “Anybody that thinks that this race is anything but a tossup right now is such an ideologue,” he said. “They should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes.”
Silver responded via Twitter in the vernacular that had served him well during his poker playing days. He challenged Scarborough to a bet. This didn’t go over very well with his new Times overlords. All the same, he further humiliated his critics by calling the election results with near 100% accuracy, compared with the “toss-up” prognosticated by the talking heads.
Cantech Letter’s Terry Dawes sat down with Nate Silver in Toronto last Thursday. After some initial banter reminding him that we were speaking on the anniversary of the 2012 election, which must have provided him some satisfaction, we brought the conversation around to business.
Nate, you’re delivering the keynote address at this SAP event, “Conversations on the Future of Business”. Regarding small to medium enterprises, there’s an atmosphere of panic, where buzzwords tend to dominate. It’s a bit like the scouts vs. stat-head paradigm that you discuss in your book, but for baseball 10 years ago. Regarding marketing, there’s a lot of dinosaur-ish behaviour, people who run everything on their gut, basically. For somebody who’s running a 30-employee company, how do you see that panning out in the next five years or so?
It depends on the industry. I certainly wouldn’t say that analytics are a cure-all for every field. I think the broader thing, though, is about accountability almost, right? Where you’re making decisions, high-stakes decisions, at least for something the size of your enterprise, and we should evaluate, in a rigorous way, whether those are smart decisions or not. Whether using your gut instinct or analytics or any other tool, we want to see how smart those are and refine them if they’re not as bright. And of course it’s going to put people in a place where they’re nervous. If you’re being tested on something, and you haven’t been tested before, then it affects attitude and performance, probably in a positive way, if the tests are administered in a smart way. You don’t want to judge people too much on small sample sizes, or predictions that involve more luck than skill. But it certainly would affect the culture, and likewise companies that are successful with this stuff are going to have to have cultures that are amenable to it.
Yeah, with small to medium enterprises, there is a more, I suppose, flexible, or you could call it “agile”, approach to accepting that kind of change. I’m really seeing people essentially in panic mode, watching the buzzwords come down the pike. In your book, you kind of address the possibility that there could be more harm than good done by people using what they regard as correct data interpretation.
Yeah, incorrect usage of data analytics can make things worse. One problem you face is, maybe you’re maximizing some variable in the short term that’s actually quite costly to you in the long term, right? You know, a lot of websites put up articles that get a lot of traffic in the short term, but undermine their brand because of shoddy journalism, or because they have a misleading headline, or whatever else, right? You know, CNN or what have you tweets out incorrect declarations about what a Supreme Court decision might have said or lots of news organizations have trouble with just reporting important facts wrong in the race to be first, and are damaging their brands. Being first is not that hard. Being best is harder.
It requires scale, too, the possibility of effectively addressing a thing, the correct resources applied correctly. Sure, you can see a lot of smaller companies innovate, but if they can’t actually correctly address the problem, it doesn’t matter often…
That’s right. So it’s not that it works on the core business problems. And you’re going to have some firms that I’m sure use analytics as a sideshow, as well, to distract from working on fundamental problems to maximize some very minor component of the business that doesn’t really affect their bottom line very much.
Regarding math and the education system, in the TV show “The Wire” the math teacher Pryzbylewski figures out that he can teach his students probabilistic thinking by throwing dice in the classroom. You made your living for a while as a semi-professional poker player. What do you think?
I think literally, sure. I learned a lot of my math by wanting to create my fantasy baseball league and my NCAA tournament pool. Yeah, I mean, applied knowledge is trying to teach yourself the theory when you’re invested in the problem. It’s very basic. And unfortunately, in the U.S. system at least, math classes seem to miss that. It’s taught as this very abstract thing.
Nate Silver’s breakthrough in bringing data analysis to mainstream prominence in the realm of politics lies in the fact that he’s connected it to sports. In “The Signal and the Noise” he writes, “Baseball offers perhaps the world’s richest data set.”
Business right now, whether it’s developing an internal culture of engagement or struggling with near meaningless cure-alls like SEO, looks a lot like the pre-Moneyball era of baseball, before scouts started working with data analysts to dramatically improve their team’s bottom line, substantially rather than superficially.
Paul Krugman, the Nobel Prize-winning economist who writes for the New York Times, described the last decades of the 20th century as an era in which we saw “vast amounts of theory applied to extremely small amounts of data.” Now the reverse is true.
Silver writes, “The information revolution has lived up to its billing in baseball, even though it has been a letdown in so many other fields, because of the sport’s unique combination of rapidly developing technology, well-aligned incentives, tough competition, and rich data.”
But can business replicate the successes achieved by baseball franchises in their application of Big Data to performance improvement? Perhaps it will require a little humility in admitting that gut feeling is at best a random, if not entirely meaningless, method for evaluation. Even leaders now regarded as visionaries will admit to adjusting their approach in the face of hard evidence.
“I claim not to have controlled events, but confess plainly that events have controlled me,” wrote Abraham Lincoln, once upon a time.
The challenge in finding the signal in the noise is to locate and focus on the point where subjective and objective reality intersect. Easier said than done. Just as each person imagines they’re a better driver than most other people, Silver writes in the conclusion to his book, “our bias is to think we are better at prediction than we really are.”
Even so, talk to even the most arithmophobic sports junky for only a minute, and the numbers come tumbling out. Sports fans instinctively know the importance of statistics, even though very few will profess to an interest in statistical analysis.
Ten years after Big Data revolutionized baseball, Nate Silver’s forecasting models are encroaching on other aspects of life, including business. Looking at his successes in the worlds of baseball and political forecasting, the results ought to speak for themselves.