What Dalessandro made clear is that even well-organized data are not enough to eke out an actionable prediction. That takes -- guess what? -- a real live human essentially playing CSI's Dr. Gil Grissom, sifting through the noisy, dirty numbers for what actually went down.
Taking Mr. Silver as an example, Dalessandro broke it out this way.
"It looks like Nate Silver does a weighted average of polls," he said. He explained that the weights are probably determined by factors such as the freshness of the poll, prior accuracy of the poll, trustworthiness of the organization and perceived partisanship of who is doing the polling.
Silver's "is the ultimate 'wisdom of crowds' methodology, with a handcrafted master mixing the collective wisdom into a potent brew," Dalessandro aptly described.When I pressed him to explain more about what's human and what's not when dealing with data, Dalessandro wrote me a remarkable answer: "You're getting onto a point that is seldom discussed in public -- the subjectivity of data science." It's the story, stupid.
He explained that the basic tools of statistics will always be just that: basic tools. To Dalessandro, data is no savior. A good data researcher is like a jury determining truth based on the evidence at hand, he said. In practice, that role boils down to two basic jobs: making predictions and telling stories. Making predictions is relatively easy, he said. There are well-defined methodologies for making forecasts, and given the assumptions made it's reasonable to expect forecasts to land within the margin of error created. But telling stories with these same numbers? That is an art, Dalessandro said. "In this role," he said, "I usually have to recreate some aspect of the world as seen by the data. Being given a data set is like being given a bloody glove, a ransom note and a bullet and having to recreate the crime. In this capacity I am a detective." He said his tools as a data cop are the classic objective statistical formulas and methods, but success hinges on a hard-to-define creative ability. "Some people can be brilliant at math and computer science," he said, "but suck at this game." That makes the investor lesson from Election 2012 a painfully sobering one: When a bunch of numbers are staring you in the face, passing themselves off as true, be darn sure you get to know the digital Wizard of Oz who is almost certainly making those numbers talk. Pretend otherwise, meaning that by themselves cold, pure data -- as clean as they might taste -- are by themselves anything close to reality, and you'll only wind up like Mitt Romney: Coming up more than a few votes short.