Michael Schwarz seems out of place in a crowd of hip, breezy Internet entrepreneurs gathered at a San Francisco bar as they celebrate the latest resurgence of all things Internet. Brainy, contemplative and deliberate in his responses, the 36-year-old former Harvard economist stands in contrast to other revelers, many of whom owe their success more to an instinctive, gut-driven penchant for capitalizing on the next Internet trend than to any kind of reflection. After all, the latest wave of Internet success stories -- video-sharing site YouTube, notorious for the oddball video uploads it carries, not to mention the $1.6 billion Google ( GOOG)
shelled out for it in October ; college networking site Facebook.com; or news-voting site Digg.com -- seem more like flukes than deliberate science projects that could benefit from meditative types. But Yahoo! ( YHOO), Schwarz's employer, is betting otherwise. The contributions of Schwarz and other researchers to the development of the Web may be larger than they appear, thanks to a push by the company. The Internet giant is snapping up academics like Schwarz and arming them with reams of data it generates -- about 12 terabytes per day -- in the hope that it will be able to glean new, massively profitable insights into the way that users behave on the Web.
A closer look at Internet user behavior across the vast amount of traffic that Yahoo! receives can help explain seemingly random phenomena to the Web giant, says Prabhakar Raghavan, the head of Yahoo! Research and a former veteran of IBM's research department. Contributions from Yahoo!'s forward-looking research group will be especially important after 2006, a year in which the company lost a third of its value. The stock has rallied so far in the new year -- up $3.59, or about 14%, to $29.20 -- and Yahoo! will need to continue to deliver improved services to its users if that momentum is to continue.
Yahoo! Responds to GoogleYahoo!'s cerebral approach to quirky behavior may have received its first big public vindication in November, when Google shut down its Answers service in what marked the first closure of a major division for the search giant. Meanwhile, Yahoo!'s rival service, called Answers and Knowledge Search, kept its stride with more than 60 million users contributing roughly 160 million answers since it took off in December 2005. Traffic was at levels 24 times what Google was generating when it shuttered its service. The Yahoo! Research team played a leading role in developing the much more nuanced reward system for Yahoo! Answers -- one of the biggest differentiators between its service and that of its competitor.
Google users would simply place a bid indicating how much they would be willing to pay for a correct answer, which would then get passed on to a select group of Google researchers. Yahoo!, meanwhile, is able to tap its enormous community for answers, offering users an elaborate system of status points and rankings in lieu of cash for their research. With its focus on incentives and decision-making in uncertain conditions, Schwarz's research contributes to seemingly disparate areas such as online dating site Yahoo! Personals, job listings and the marketplace for online ads. Yahoo! Research has also played a role in developing Project Panama, the company's heavily anticipated upgrade of its advertising platform, which is scheduled to debut in the first half of 2007. A good part of the outlook for Yahoo!, which has consulted an outside economist on the project along with its internal team, depends on how fine-tuned the new ad system is, and to what degree Panama helps close the gap in the amount the company earns per search compared with market leader Google. But more than immediate issues, the purpose of Yahoo! Research is to think about the way the Internet and the behavior of users are evolving, says Raghavan. The company is as interested in the dynamics of how people use the Web as it is in the underlying technology itself.
"What we think are engineering decisions turn out to be economic decisions, and these result in real revenue-affecting choices," Raghavan says. Though Yahoo! does not disclose how many members it has in its research group, the number appears to be sizable, with research and development playing a key role at the company. Yahoo! spends about $500 million a year on R&D, including the Research group, according to a Yahoo! representative. Along with an office minutes from the company's main campus in Silicon Valley, Yahoo! research has headquarters in Barcelona, Spain; Berkeley and Burbank, Calif.; Chile; and New York. Recently, Raghavan took a trip to Bangalore, India, and announced that the company is seeking talent there as well. The research group divides its efforts between search, machine learning, microeconomics, media experiences and design, and community systems. Along with economists, the company is also adding other social scientists including psychologists, sociologists and anthropologists. At a time that both Yahoo! and Google increasingly move beyond the search market to explore uncharted waters, cogitation will continue to play an important role.