Putting Cramer's Mad Money Picks to the Fire

Then, for the fun of it, I tested all of Barron's recommendations during the same time period. I'll describe that test later in this article.

Here are the specific rules I wanted to test:

Buy rules:

When Cramer makes a recommendation, wait one week and buy. Why do we wait one week? In his Mad Money book, Cramer states one should always wait "5-10 days." This helps you avoid the "Cramer Effect" -- people buying the stock right after the recommendation and running it up, and then having it come back down again. Waiting helps you filter out that effect.

I chose five days as the moment from which to test.

Sell rules:

Test #1: Sell one month later.

Test #2: Sell two months later.

In other words, if we buy when Jim suggests, hold for one month (or two), do we outperform the market?

With 445 trades we certainly have enough to test a "trading system" to a level where it is statistically significant.

First off, how did the S&P 500 do during this period? It was up 2.68%, or for any random one-month holding period, 0.39%. So 0.39% is our benchmark for the one-month system and 0.78% is our benchmark for the two-month system.

Results:

Test #1: The Mad Money One-Month Trading System

• 445 trades
• 250 successes (56.18%)

Average return per trade: 1.18%

Test #2: The Mad Money Two-Month Trading System

• 445 trades
• 254 successes (57.08%)

Average return per trade 2.50%

Jim's picks have significantly outperformed the market during this period (the first 7 months of 2007). I'll put up all the data as well so you can see the specific trades and results.

Here are the data so you can see the specific trades and results yourself.

• One-Month Data
• Two-Month Data

Some sample Cramer trades:
Apple (AAPL); entry date, Jan 24; exit date, Feb. 22; gain of 3.28%
eBay (EBAY); entry, March 5, exit, April 5; a gain of 8.4%
Akamai (AKAM); entry, March 16; exit, April 16; a gain of 5.67%
Capital One (COF); entry, Jan. 29; exit, Feb. 27; loss of 2.74%

I wasn't initially going to test Barron's picks. For one thing, Barron's is a newspaper, not a hedge fund, and it's never claimed to be a professional investor. However, I changed my mind when I saw that they do, in fact, have an annual roundup which judges their picks.

Furthermore, I was completely dismayed when I saw how it judged its own success. It used as the starting price for its recommendations the Friday before their stories would appear. This means it's tabulating its own effect on the stock after publication; we filter out this effect, just as we did for Cramer's picks.