Past performance is no guarantee of future success.
That's what you need to bear in mind the next time an analyst comes up with some historical data that "prove" a certain investment strategy works.
It happens all the time. Analysts scour through reams and reams of data, looking for
patterns or trading rules that have produced superior results over a number of years. They then sell the strategy to investors, saying it has a "proven track record" and should repeat its stellar performance in the future.
The only problem is that seemingly significant discoveries are often little more than chance occurrences.
"If you torture the data, it will confess," said Grant McQueen, professor of finance business management at Brigham Young University.
Russ Francis All-World
One of the more outrageous examples of this so-called data mining is the Super Bowl indicator. According to this theory, a win by a team from the old National Football League (now the NFC) foreshadows a higher market by year-end, while a victory from the old American Football League (or the AFC) suggests stocks will end lower for the year. Over the past 36 years, this indicator has been wrong just seven times.
Despite its accuracy, few people take this seriously, because they realize it's
simply a fluke and has no predictive power. And yet, other strategies that are derived from similar methodologies aren't seen as jokes at all.
Consider the Dogs of the Dow theory, which holds that if you invest equal amounts in the 10 highest yielding components of the
Dow Jones Industrial Average
and keep them for a year, you'll reap big rewards.
In rolling 10-year periods back to 1928 -- when the Dow Jones index was expanded to 30 stocks -- there are only two years in which this strategy did not beat the
, Bear Stearns portfolio manager James O'Shaughnessy said in a recent
10 Questions interview.
To be fair, there is some logic behind the theory. A high dividend yield suggests that the firm is going through a difficult period. As a member of the Dow, however, the firm is assumed by analysts to be large enough to be able to pull out of the slump and turn around fairly soon. But dig a little deeper into this strategy and you'll see things aren't quite what they seem.
"There's no adjustment for taxes or transaction costs," said Tom Taulli, professor of finance at the University of Southern California and author of
The Streetsmart Guide to Short Selling
.The Dogs of the Dow strategy requires investors to rebalance their portfolios annually. "Once you start to account for that, it doesn't give you much of an edge."
McQueen, who has written a paper about data mining, said that when you take into account transaction costs and the risk of owning just 10 stocks in your portfolio, any excess returns "disappear." If you include the tax burden, he said, "the Dogs of the Dow actually underperform a simple buy-and-hold strategy."
Have Your Pets Neutered
Variations of the Dogs of the Dow theory are even more flawed. One method is to take the five lowest-priced Dogs of the Dow and invest equal amounts in each. The Dow Four tweaks this strategy by eliminating the lowest-priced stock. There appears to be no fundamental reason for picking the lowest-priced issues, apart from the fact that this strategy has worked in the past.
Motley Fool creators David and Tom Gardner learned the dangers of data mining the hard way, but not before their investment guide spent several months on the
New York Times
bestseller list. In the mid-1990s they came up with a theory known as the Foolish Four, which advocated doubling the weight of the lowest-priced stock in the Dow Four. Historical data from 1973 to 1993 showed that this strategy would have reaped a 25.5% compound annual return compared with just 11.2% for the Dow. And the Fool suggested this pattern was repeatable.
After more extensive research and back-testing, however, the Fool admitted that its initial sample data didn't go back far enough and that the returns it cited were all just a statistical anomaly.
Data mining shows up in other areas, too. The Hirsch Organization, which publishes the
Stock Traders Almanac
, advises investors to stay out of the market from May 1 through Oct. 31 and return on Nov. 1 through April 30. The historical returns from late fall to early spring are, apparently, infinitely better.
"People notice something in the historical data and say, oh, that must be true all the time," Grant said. "It's probably true in the past just by coincidence."
But how do you know what is coincidence and what is a
valid pattern? Grant said investors should look to see if there's a plausible explanation for the data. And if there's any truth to theories such as the Dogs of the Dow, he said, the rule should be tested on "out-of-sample" data.
"If there's something to the Dogs strategy, let's try it in Japan and England," he said. "Look for proof outside of the data sample and look for a story."
Also, he said, investors should be wary of analysts who don't say how many variables they pored over before coming up with a theory. Terms like "we noted" and "we noticed" should raise a red flag.
"Statistics lie," said Taulli. "And keep in mind, just because that's what's happened in the past, it doesn't mean it's going to apply in the future."
TheStreet.com has a revenue-sharing relationship with Amazon.com under which it receives a portion of the revenue from Amazon purchases by customers directed there from TheStreet.com.