This Day On The Street
Continue to site
This account is pending registration confirmation. Please click on the link within the confirmation email previously sent you to complete registration.
Need a new registration confirmation email? Click here

15 Ways to Check Data

My suggestions for how to deal with data proof are in the main column. I asked readers for their suggestions, and you guys came up with several great additions to the process of checking data:

1. To prove causality, you need three things: co-variation (aka, correlation), time order (to cause B, A has to happen first) and non-spuriousness.

2. Look at the depth of correlation (not just the direction but the amount of movement). What happens when "X" changes 5% -- does it correlate with a 15% change in Y? Do other changes in "X" change with Y? Is there a it tapering off of the effect? Does it invert? Are the increases linearly or non-linearly?

3. Always beware of "ceteris paribus." Translation: "all other things being equal." For the most part, all other things are very rarely equal.

4. As George Soros wrote, Markets are reflexive. Market participants "learn." For that reason alone, be extremely wary of applying past historical patterns to current data. As Mark Twain is reputed to have said, "History doesn't repeat, but it rhymes."

5. Look at outliers. This is very much related to central tendency -- similar to non-recurring costs

6. Extraordinary claims need extraordinary evidence

7. Watch for Distribution. Most stats are based on a normal distribution (i.e., Bell Curve). If you can't provide a basis for the normative assumptions, you must use a distribution-free or non-parametric test; otherwise your conclusions are dubious.

8. Most data patterns are linear, curvilinear or non-linear. Like costs in accounting, it depends on the time period of interest (costs could be fixed, variable or mixed, e.g.)

9. If you are looking to measure a data series central tendency, consider what occurs if you use mean, median or mode. Each has pros and cons and can be "biased," producing different outcomes.

10. The key is to know what someone's interests are. Are they trying to sell you something? They will naturally frame their presentation to compel a certain action. Key question: What's missing? What one thing would have to be true for them to be wrong?

11. Present/Not Present (if X is present; when is Y not present - find an example). not present:present (if X is not present; when is Y present)

12. What kind of logic is being used? Is it Inductive, Deductive, Analogic, etc. There are assumptions with each form of reasoning, and each has potential pitfalls. Inductive can always be wrong -- the "black swan" issue. Deductive: What are the premises? Analogic: Do the compares have materially relevant similarities?

13. Case studies are often wrong. Many stories are built on them and often have compelling detail, but they are often part of a selling frame that is not generalizable to other situations.

14. Observe all points in your analysis where you assumed that events were independent (in the probabilistic sense). Re-examine whether this assumption is still valid given the current market environment. If it is not, how does this effect your calculations?

15. If an indicator has, in the past, been such a good predictor of short term moves in the market as to reach the level of "statistical significance," and if that indicator were to become widely recognized as having reached that elusive threshold of statistical significance, the indicator would cease to be effective once the majority recognized its prior effectiveness. Thus, anyone waiting for statistical certainty is certain to be disappointed.
At the time of publication, Ritholtz had no positions in stocks mentioned, although holdings can change at any time.

Barry Ritholtz blogs at the popular The Big Picture, offering up his macro perspectives on the capital markets, the economy, technology and digital media.

He is CEO and director of equity research of Fusion IQ, a quantitative research firm.

Ritholtz holds a bachelor's degree in political science (with a concentration in philosophy) from the State University of New York at Stony Brook, and a J.D. from the Benjamin N. Cardozo School of Law, where he studied corporate law and economics.

Ritholtz appreciates your feedback; click here to send him an email.

Check Out Our Best Services for Investors

Action Alerts PLUS

Portfolio Manager Jim Cramer and Director of Research Jack Mohr reveal their investment tactics while giving advanced notice before every trade.

Product Features:
  • $2.5+ million portfolio
  • Large-cap and dividend focus
  • Intraday trade alerts from Cramer
Quant Ratings

Access the tool that DOMINATES the Russell 2000 and the S&P 500.

Product Features:
  • Buy, hold, or sell recommendations for over 4,300 stocks
  • Unlimited research reports on your favorite stocks
  • A custom stock screener
Stocks Under $10

David Peltier uncovers low dollar stocks with serious upside potential that are flying under Wall Street's radar.

Product Features:
  • Model portfolio
  • Stocks trading below $10
  • Intraday trade alerts
14-Days Free
Only $9.95
14-Days Free
Dividend Stock Advisor

David Peltier identifies the best of breed dividend stocks that will pay a reliable AND significant income stream.

Product Features:
  • Diversified model portfolio of dividend stocks
  • Updates with exact steps to take - BUY, HOLD, SELL
Trifecta Stocks

Every recommendation goes through 3 layers of intense scrutiny—quantitative, fundamental and technical analysis—to maximize profit potential and minimize risk.

Product Features:
  • Model Portfolio
  • Intra Day Trade alerts
  • Access to Quant Ratings
Real Money

More than 30 investing pros with skin in the game give you actionable insight and investment ideas.

Product Features:
  • Access to Jim Cramer's daily blog
  • Intraday commentary and news
  • Real-time trading forums
Only $49.95
14-Days Free
14-Days Free
AAPL $93.74 0.00%
FB $117.58 0.00%
GOOG $693.01 0.00%
TSLA $240.76 0.00%
YHOO $36.60 0.00%


Chart of I:DJI
DOW 17,773.64 -57.12 -0.32%
S&P 500 2,065.30 -10.51 -0.51%
NASDAQ 4,775.3580 -29.9330 -0.62%

Free Reports

Top Rated Stocks Top Rated Funds Top Rated ETFs