Picking the Best Experts

Ninety-nine percent of brokerage analysts are giving the remaining 1% a bad reputation. Yet, you have to wonder whether that small percentage actually might provide something of value to investors.

During the great bull run of the late 1990s, a handful of companies staffed with both math whizzes and washouts sprang up like dandelions with plans to rate analysts. Most of the companies were amateurs at both business and analysis, and disappeared in million-dollar clouds of dust.

But one of the few companies left standing offers some real insights into this opaque and sometimes-crooked profession: StarMine, a company that started out in 1998 with a weak and incomplete system for analyst analysis. But the company has steadily improved to the point that some brokers are hiring it to measure their own stock-picking effectiveness. In addition to providing sophisticated tools to institutional investors, the company also offers services to retail investors.

The need for such heuristics becomes evident on a day such as Friday -- a day that contained news of the tentative federal approval of a new psoriasis drug from battered biotech titan Biogen BGEN. It didn't take long for every pharmaceutical analyst in the land to scratch a note on the meaning of the announcement, and opinions ranged from a startling sell/reduce rating from Merrill Lynch to the reiteration of a strong buy from A.G. Edwards.

They can't all be right, of course. As they say in the industry, on average, all analysts are average. But at the same time, it's possible that one or two specialists in any given large-cap stock might consistently get it right -- either in their estimates of future quarters' earnings, or in their timing for upgrades and downgrades, or both.

Anticipating the Anticipators

The StarMine thesis starts with the well-documented fact that one of the few quantifiable events predicting future stock-price movements is analysts' earnings-estimate revisions. The company believes it's possible to go one step further, however, and attempt to predict which companies will be the subject of rising estimates in the future. Essentially, that puts StarMine in the business of anticipating the anticipators, which is even harder than it sounds.

Founder and Chief Executive Joe Gatto -- who started his career as a decision-engineering consultant after graduating from Stanford University -- says that until 2000, it was possible to predict revisions by examining past revisions: Changes in analyst estimates over the past 30 days had a strong, persistent correlation to changes over the following 30 days. But the Securities and Exchange Commission changed all that when it promulgated Regulation FD, prohibiting companies from selectively informing favored analysts or investors about their business prospects before others.

Analyst revisions had worked as a strategy because there was a delay in the diffusion of information. Company A would tell Analyst B (whose firm had helped with its stock underwriting) that it was selling more widgets than expected in the quarter. Analyst B would tell Fund Manager C that he was planning to raise his earnings estimates on the stock, expecting that the tip would encourage the fund manager to buy, say, 100,000 shares of the stock through his brokerage (a percentage of the commissions on the trade would be credited to the analyst).

After buying his fill, Fund Manager C might leak the idea to Reporter D, sagely opining that the company would beat its numbers. Reporter D might then continue the contagion by calling Analyst E for confirmation of the idea. All the while, as the original data bit -- the analyst's upward earnings revision -- diffused through the investment ecosystem, the stock would rise. Of course, it was the Retail Investor who got an F for ultimately buying the stock on the company's actual rosy earnings report, because Players A through E by that time were taking profits -- and sending the stock down.

Regulation FD, for all its many faults, was intended to stop this madness, which resulted in a predictable chain-letter change in consensus as analysts successively attempted to stay in line with their peers so they didn't look stupid. Now when a company publicly declares on a conference call that business is better, all analysts change their estimates from, say, $1 a share to $1.10 at once. The new information is then immediately reflected in the market.

It took a while for guess-guessers such as StarMine to figure out what to do about this change, but, ultimately, it has made the game of expectation-expecting more complex.

StarMine calculates a "smart estimate" for a stock by overweighting the most accurate analysts' estimates and throwing out old estimates as they age. It then compares the smart estimate to the consensus estimate and uses the oxymoron "predicted surprise" to describe the divergence between the estimates.

In other words, if the consensus estimate for a quarter is $1 a share, and the "smart estimate" is $1.10, the predicted surprise is 10%. Gatto says that historical testing in all market-cap groups, sectors, styles and countries (except, curiously, Slovenia), shows that when the difference is 10% or more, the stock does surprise in the predicted direction 75% of the time. StarMine then combines that figure with a handful of other factors to create a StarMine Indicator, a rating of 1 to 5, with 1 being best, to summarize a stock's prospects for positive surprises and higher prices over the next one to six months.

How It All Works, or Doesn't

All of that sounds good, but the indicator seems to have failed the Biogen test on Friday.

First the good news:

A visit to StarMine's professional-level Web site shows that it identified the team of Craig West and Alex Hittle at A.G. Edwards as tops among 16 analysts on Biogen stock, giving the duo a rare five-star rating for accuracy and putting 9.9% of the weight of the smart estimate on their shoulders (a normal weighting would be 1/16, or 6.3%).

StarMine identified David H. Williams, of Fulcrum Global Partners, as the least capable analyst on the stock, 16 of 16, with a one-star accuracy rating and a 3.3% weighting.

The divergence was prophetic because Fulcrum issued a report on April 1 predicting that the Food and Drug Administration's Advisory Committee would not recommend approval of Biogen's psoriasis drug Amevive, and rated the stock sell. In contrast, West-Hittle of A.G. Edwards wrote on April 21 that they liked the "risk/reward of betting on approval of Amevive" and rated the stock strong buy.

On May 23, the FDA panel voted 10-2 in favor of approving Amevive, and Biogen stock shot up 22% -- its best one-day move in five years. The bad news? The StarMine Indicator for Biogen going into Friday was a very negative 5 -- probably because the drift in earnings revisions going into the big day was on the decline.

But one day and one rating does not an analytical system make, so I will withhold judgment and watch the 10 stocks shown in the table below over the next three months. The first five (all from a strong industry, energy) have the largest predicted positive surprises through May 27. All are StarMine 1s. The next five (all from a weak industry, software) have the largest predicted negative surprises. All are StarMine 5s.

5 StarMine Longs
Company name	Smart Estimate	Consensus	Predicted Surprise
Noble Affiliates (NBL:NYSE)	$0.04	($0.03)	66.70%
Devon Energy (DVN:Amex)	0.99	0.81	22.50
Key Production (KP:NYSE)	0.33	0.28	18.90
Penn Virginia (PVA:NYSE)	0.12	0.11	9.90
Remington Oil & Gas (ROIL:Nasdaq)	0.12	0.11	8.60
Source: StarMine

5 StarMine Shorts
Company Name	Smart Estimate	Consensus	Predicted Surprise
SilverStream Software (SSSW:Nasdaq)	($0.36)	($0.32)	-14.0%
JD Edwards (JDEC:Nasdaq)	0.06	0.06	-3.9
Agile Software (AGIL:Nasdaq)	(0.21)	(0.20)	-2.5
MRO Software (MROI:Nasdaq)	0.02	0.03	-2.3
Legato Systems (LGTO:Nasdaq)	(0.05)	(0.05)	-2.0
Source: StarMine

U.S. Bancorp Piper Jaffray, Merrill Lynch, WR Hambrecht and three other sell-side brokerage firms aren't waiting for more evidence, however. They have asked StarMine to rate their analysts privately so their research director can keep track of capabilities.

You would think that, after all this time, the firms already had this capacity in house. But it turns out that the brokerages have much more elaborate systems for tracking how many phone calls the analysts make to clients pitching their research than they do for tracking whether the research makes sense. Gatto also claims to have 80 buy-side clients paying $15,000 to $80,000 per year for his company's richest set of professional-grade analytical tools and factors.

The smart-estimate and predictive-surprise concepts make sense, and they should offer both consumers and pros a way of finding that top 1% of analysts in any sector or stock whose real value is overshadowed by the mediocrity and mendacity of the rest.

But I would use them as one or two tools among many because a clear lesson of the bear market has been that there are no single magic bullets, holy grails -- or lucky stars -- when it comes to the serious, difficult task of investing.