This account is pending registration confirmation. Please click on the link within the confirmation email previously sent you to complete registration.
Need a new registration confirmation email? Click here
Stocks Under $10 with 50-100% upside potential - 14 days FREE!

15 Ways to Check Data

My suggestions for how to deal with data proof are in the main column. I asked readers for their suggestions, and you guys came up with several great additions to the process of checking data:

1. To prove causality, you need three things: co-variation (aka, correlation), time order (to cause B, A has to happen first) and non-spuriousness.

2. Look at the depth of correlation (not just the direction but the amount of movement). What happens when "X" changes 5% -- does it correlate with a 15% change in Y? Do other changes in "X" change with Y? Is there a it tapering off of the effect? Does it invert? Are the increases linearly or non-linearly?

3. Always beware of "ceteris paribus." Translation: "all other things being equal." For the most part, all other things are very rarely equal.

4. As George Soros wrote, Markets are reflexive. Market participants "learn." For that reason alone, be extremely wary of applying past historical patterns to current data. As Mark Twain is reputed to have said, "History doesn't repeat, but it rhymes."

5. Look at outliers. This is very much related to central tendency -- similar to non-recurring costs

6. Extraordinary claims need extraordinary evidence

7. Watch for Distribution. Most stats are based on a normal distribution (i.e., Bell Curve). If you can't provide a basis for the normative assumptions, you must use a distribution-free or non-parametric test; otherwise your conclusions are dubious.

8. Most data patterns are linear, curvilinear or non-linear. Like costs in accounting, it depends on the time period of interest (costs could be fixed, variable or mixed, e.g.)

9. If you are looking to measure a data series central tendency, consider what occurs if you use mean, median or mode. Each has pros and cons and can be "biased," producing different outcomes.

10. The key is to know what someone's interests are. Are they trying to sell you something? They will naturally frame their presentation to compel a certain action. Key question: What's missing? What one thing would have to be true for them to be wrong?

11. Present/Not Present (if X is present; when is Y not present - find an example). not present:present (if X is not present; when is Y present)

12. What kind of logic is being used? Is it Inductive, Deductive, Analogic, etc. There are assumptions with each form of reasoning, and each has potential pitfalls. Inductive can always be wrong -- the "black swan" issue. Deductive: What are the premises? Analogic: Do the compares have materially relevant similarities?

13. Case studies are often wrong. Many stories are built on them and often have compelling detail, but they are often part of a selling frame that is not generalizable to other situations.

14. Observe all points in your analysis where you assumed that events were independent (in the probabilistic sense). Re-examine whether this assumption is still valid given the current market environment. If it is not, how does this effect your calculations?

15. If an indicator has, in the past, been such a good predictor of short term moves in the market as to reach the level of "statistical significance," and if that indicator were to become widely recognized as having reached that elusive threshold of statistical significance, the indicator would cease to be effective once the majority recognized its prior effectiveness. Thus, anyone waiting for statistical certainty is certain to be disappointed.

At the time of publication, Ritholtz had no positions in stocks mentioned, although holdings can change at any time.

Barry Ritholtz blogs at the popular The Big Picture, offering up his macro perspectives on the capital markets, the economy, technology and digital media.

He is CEO and director of equity research of Fusion IQ, a quantitative research firm.

Ritholtz holds a bachelor's degree in political science (with a concentration in philosophy) from the State University of New York at Stony Brook, and a J.D. from the Benjamin N. Cardozo School of Law, where he studied corporate law and economics.

Ritholtz appreciates your feedback; click here to send him an email.

Select the service that is right for you!

COMPARE ALL SERVICES
Action Alerts PLUS
Try it NOW

Jim Cramer and Stephanie Link actively manage a real portfolio and reveal their money management tactics while giving advanced notice before every trade.

Product Features:
  • $2.5+ million portfolio
  • Large-cap and dividend focus
  • Intraday trade alerts from Cramer
  • Weekly roundups
TheStreet Quant Ratings
Try it NOW
Only $49.95/yr

Access the tool that DOMINATES the Russell 2000 and the S&P 500.

Product Features:
  • Buy, hold, or sell recommendations for over 4,300 stocks
  • Unlimited research reports on your favorite stocks
  • A custom stock screener
  • Upgrade/downgrade alerts
Stocks Under $10
Try it NOW

David Peltier, uncovers low dollar stocks with extraordinary upside potential that are flying under Wall Street's radar.

Product Features:
  • Model portfolio
  • Stocks trading below $10
  • Intraday trade alerts
  • Weekly roundups
Dividend Stock Advisor
Try it NOW

Jim Cramer's protege, David Peltier, identifies the best of breed dividend stocks that will pay a reliable AND significant income stream.

Product Features:
  • Diversified model portfolio of dividend stocks
  • Alerts when market news affect the portfolio
  • Bi-weekly updates with exact steps to take - BUY, HOLD, SELL
Real Money Pro
Try it NOW

All of Real Money, plus 15 more of Wall Street's sharpest minds delivering actionable trading ideas, a comprehensive look at the market, and fundamental and technical analysis.

Product Features:
  • Real Money + Doug Kass Plus 15 more Wall Street Pros
  • Intraday commentary & news
  • Ultra-actionable trading ideas
Options Profits
Try it NOW

Our options trading pros provide daily market commentary and over 100 monthly option trading ideas and strategies to help you become a well-seasoned trader.

Product Features:
  • 100+ monthly options trading ideas
  • Actionable options commentary & news
  • Real-time trading community
  • Options TV
To begin commenting right away, you can log in below using your Disqus, Facebook, Twitter, OpenID or Yahoo login credentials. Alternatively, you can post a comment as a "guest" just by entering an email address. Your use of the commenting tool is subject to multiple terms of service/use and privacy policies - see here for more details.
Submit an article to us!
DOW 17,096.20 -17.34 -0.10%
S&P 500 1,988.34 +4.81 0.24%
NASDAQ 4,477.3270 +21.3110 0.48%

Brokerage Partners

Rates from Bankrate.com

  • Mortgage
  • Credit Cards
  • Auto
Advertising Partners

Free Newsletters from TheStreet

My Subscriptions:

After the Bell

Before the Bell

Booyah! Newsletter

Midday Bell

TheStreet Top 10 Stories

Winners & Losers

Register for Newsletters
Top Rated Stocks Top Rated Funds Top Rated ETFs