Internet Traffic Reports May Be Pointing the Wrong Way

Those Internet traffic numbers investors pore over each month may be leading them in the wrong direction.

According to a new study, there are wide and unpredictable discrepancies between figures compiled by sites and those put together by the nation's most prominent Internet ratings services. The study, along with a separate new study that hints at difficulties in measuring online advertising, indicate how cautiously investors must look at online usage statistics, which often move stocks.

Conducted under the auspices of an online advertising industry group known as Future of Advertising Stakeholders, or FAST, the Web traffic study examined records from 35 sites -- audited by outside firms -- covering the pages visitors viewed in November and December 1998. The researchers, led by Bruce MacEvoy, director of research at Yahoo! (YHOO Quote), compared those numbers with estimates of page views for those same time periods compiled by NetRatings (now doing business in partnership with Nielsen Media Research (NMR Quote) as Nielsen//NetRatings) and Media Metrix(MMXI Quote) (which acquired another ratings service, RelevantKnowledge, last November).

Among individual sites, the study found that the NetRatings and Media Metrix page-view estimates, which are based on the activity of separate representative panels of Internet users, ranged anywhere from 15% to three times the number recorded by the sites themselves.

"The range in discrepancy is huge," says Jim Spaeth, president of the Advertising Research Foundation and FAST's measurement committee chairman.

The numbers may be alarming not only to investors, but also to advertisers. "I would like them to clarify what that means for advertisers," says Maggie Boyer, media director for Avenue A, an Internet media and data marketing company. Praising the study for its unbiased information, Boyer says it raises important questions. "What am I getting for my money? What does that discrepancy mean? The number is scary."

Internet researchers argue that it's impossible to get figures that are 100% accurate. "Market research is about providing people direction, understanding of a marketplace and trends," says Allen Weiner, NetRatings' vice president of analytical services. "A market research company that says, 'We're going to be exactly the same as your server logs,' is kidding themselves."

And Rich LeFurgy, chairman of the Internet Advertising Bureau, an industry group devoted to promoting online advertising, argues that the problem with Internet ratings information isn't that it's inaccurate -- it's that it's so plentiful. "I don't think everybody appreciates the depth and breadth of what's available," LeFurgy says. "The industry is working together to make it better -- to make it more comprehensible."

The challenge of comparing log file data to user-side panel data "is that they're two different measurements done in two different ways that tell you two different things," says Andy Fessel, senior vice president of Media Metrix. Log files, for example, don't reflect the fact that several people may use a single computer, or that one person may use several computers, he says.

Fessel also questions the relevance of the specific numbers drawn from the year-old data. "There's been a great evolutionary change in both the Web and in Media Metrix's measurement practices," he says.

Other findings of the study, which was reported in preliminary form in May, suggest that the ratings services now operated by Media Metrix and Nielsen//NetRatings favored larger sites -- by showing more traffic than those companies' own records indicate -- but undercounted the traffic to smaller sites in comparison to those sites' own records. Furthermore, the discrepancies between the two numbers tended to be larger for the smaller sites than they were for the bigger sites.

Meanwhile, a second study presented Monday, like the first, at the @d:tech New York conference devoted to online advertising, demonstrated the difficulty of measuring online advertising in its examination of a single banner-ad campaign for the nonprofit Points of Light Foundation.

The intent of the study, conducted by the Internet Profiles subsidiary of Engage Technologies (ENGA Quote), was to demonstrate how much one can learn from advertising research -- for example, judging from the response to the ad, people in Massachusetts are particularly open to volunteering.

But the study also included an advertising-measurement discrepancy: The Internet advertising firm DoubleClick (DCLK Quote) recorded that it delivered 2.1 million advertising impressions, or views of the ad, to 1.8 million Internet users. But the Nielsen//NetRatings service found that about 649,000 impressions of the ad were delivered to 514,000 Internet users -- less than a third of the numbers compiled by DoubleClick. Possible reasons for the difference, says Tim Meadows, NetRatings' vice president of marketing, are that not all the impressions were counted by the ratings service, or that Web users abandon the page before the ad fully loads.

For investors, the message behind these studies is clear, Yahoo!'s MacEvoy says: "When you're making an important decision, you should never rely on a single data source." In other words, before you buy or sell an Internet stock based on a monthly ratings report, get a second opinion.