What Investors Need to Know About Historical Data

"A man with one foot in a bucket of ice and the other foot in a bed of hot coals is, on average, very comfortable."

In the most recent Barron's, Michael Santoli took a look at one of my favorite pet peeves: The use and abuse of market data. His column, titled The Limits of History , observed that as the markets have become more volatile of late, the misuse of data has increased:

People who try to handicap the markets for a living practice the art of the plausible. Many trudge from conference room to lunch table to banquet hall lugging PowerPoint decks full of unobjectionable statistical touchstones for commission-wielding clients. At times of investor confusion and market dissonance, such as now, their art is often reduced to carving a slice out of economic history that ratifies their existing outlook.

Mike then proceeds to look at a "dog's breakfast of the kinds of historical analogies making the rounds" and he notes that it seems that for every study which proclaims "Each time X happens, Y has occurred" there is an equal and opposite study incontrovertibly proving that "Y" will not occur. Pick you poison: Since "X" occurred, then we will definitely have/avoid a recession; Stocks are grossly undervalued/overvalued; Markets must rally/fall.

What's an investor to do?

One option is to obtain a college degree in statistics. But that's time consuming and expensive. Rather than spending another four years in school, I suggest a simpler approach to handling statistical data. When confronted with "incontrovertible proofs" based on historical data, ask yourself this short list of questions. (Click here for 15 reader ideas on managing data.)

1. Do we have enough historical examples? All too often, we see broad conclusions drawn because some set of factors have happened a few times in the past. Typically, a dozen examples of an occurrence -- out of decades of market data -- is simply insufficient to draw a firm conclusion
2. Is the data statistically significant? This is similar to the above question. To be significant means that we are not looking at something that is merely the result of chance. Significant means more than important, "significant" means probably true.
3. Causation or Correlation? Does "X" cause "Y" to occur? Or, are we presented with two elements that may have the same underlying causes? Is there even interaction between X & Y?
4. Coincidence? Is it possible is it that these two items are utterly unrelated? The best market related example of spuriousness data is the Super Bowl indicator. (Go Giants!)
5. Look for differentiating elements in different time periods: What factors are similar? What factors are different?
6. Be very wary of single variable analysis. Examples: In 2005, we kept hearing how cheap homebuilders like Toll BrothersTOL and LennarLEN were because of their low P/E ratios -- just as interest rate increases were biting. And, we heard how pricey AppleAAPL and GoogleGOOG were because of their high P/Es.
7. Compare: Compare everything -- interest rates, inflation, dividend yield, P/E contraction or expansion, sentiment, overall market trend, business cycles -- across different eras. Might that account for potentially different outcomes?
8. Any recent market environmental changes? Have we had a regulatory change, or a new financial innovation that may be having an impact? What might these specific changes be doing to the data? Consider: decimalization, change in dividend tax, the rise of ETFs, the spread of online trading, etc.
9. Subjective vs. objective measures: Are the factors under discussion hard numerical data, squishy subjective opinions, or somewhere in between? For example, I recently mentioned the percentage of NYSE stocks over their 200-day moving average - that is an objective measure; On the other hand, I find some chart pattern readings to be somewhat subjective. Earnings can at times be subjective or objective (beware new accounting rules!). BLS inflation measures fall somewhere in the middle.
10. Consider future market action in terms of probabilities, not outcomes. For example, assume that some causative factor resulted in a specific event (X --> Y) seven out of nine times. The most you can say is that when "X" occurred in the past, it has resulted in "Y" approximately 78% of the time.

    But remember, there is a huge difference between historical occurrence and future likelihood. In the example above, this does not necessarily even mean that since "X" has just occurred, there is a 78% that "Y" will happen. Consider: was the first X/Y occurrence really a 100% or zero? Did the second one become 100%, 50%, or 0%, then the 3rd 100%, 66%, 33% or 0%?

11. Contextualize data: It helps to think of data not as a still photograph, but as a frame in an ongoing film. Sometimes a single data point -- even a mean or median -- only tells half a story. A single data point may be part of a trend up or down, or it can be part of a data series reversing directions. Each of these may have differing implications for what comes next. (Examples: Inflation is high, but coming down. Gold is high, and going higher).

Lies, damn lies, and statistics

Using market data is a great way to understand risk and opportunity as investors. The advantages of a methodical, empirically based approach is that it avoids the typical emotional foibles that trip up so many investors. However, there are many ways that data can be misused. These can be expensive errors to learn on the fly.

There are lies, damn lies, and statistics. Smart investors know how to deal with each of these, and here are 15 suggestions on effectively assessing data.