In the beginning there was Yahoo and very little else. There was hardly even a World Wide Web.
The year was 1993. The World Wide Web was brand new. It was pretty much a geek thing then and the geeks were hunched over their keyboards discovering its wonders. The atmosphere was a bit like an old- time rattlesnake rodeo with the hunters showing off what they'd found.
Jerry Yang and Dave Filo were among the hunters in those days and they found lots of interesting Web sites. That part was easy, the hard part was cataloging the sites because more sites appeared every day.
The two young men were graduate students at Stanford University. They set to work to figure out a way to catalog the sites they found and share them with others. The result was something called "Jerry Yang's Guide to the WWW." Jerry says that working on it was a way to avoid writing their doctoral dissertations.
Originally, the catalog lived on Jerry's workstation and the search tool they developed lived on David's workstation. It was the first of a breed, the "search engine." People like me began to use it to find things on the Web.
We also began to tell each other about this great site. That drew lots of traffic to the Stanford network. Soon the search tool, by now named "Yahoo!" was moved over to Netscape communications.
Yang and Filo loved what they were doing. Searchers loved what they were doing, too and the work of running Yahoo got bigger and bigger. They hired people to help with the cataloging. It seemed like they could continue to do what they loved, and perhaps put off writing their dissertations forever, if they figured out a way to make money from Yahoo.
Those were the heady days of the Dot-Com Bubble, so getting money to turn Yahoo into a business wasn't much of a problem. Yahoo went public in 1996. The stock shot up 154% on the first day of trading.
The boys took some of their wads of money and put it into promotion and advertising. That paid dividends. By 1997 Yahoo was drawing more Web traffic than its top three competitors put together. Companies were elbowing each other out of the way to give Yahoo money to advertise there.
Yahoo was the dominant player in what was becoming the search engine business. But there were lots of other new companies out there looking for a piece of the action.
Elsewhere in Silicon Valley an outfit called Architext was starting up. It would become Excite. Infoseek was there, too. It became go.com. The Mining Company became About.com. There was Lycos and Looksmart and AltaVista. Goto became Overture. There was HotBot which, we were told, was "powered by Inktomi."
Some of these were directories like Yahoo. Directories were organized by category. Human beings made the decisions about what category or categories a listing belonged in.
There were also search engines. Search engines used software to go out on the Web and gather information about Web sites. Then they used more software to index the information and display it when someone typed a keyword into the search box.
It didn't matter that much to users, who called them all search engines and used them to explore the Web. Lots of things didn't seem to matter much back then. It didn't seem to matter much to advertisers whether they could track ad performance or not. And profitability didn't seem to bother investors who just "knew" they would make big money someday in this wonderful, new economy.
Meanwhile, back at the ranch, or the Farm as folks call Stanford, two other graduate students were thinking about search engines. Their names were Sergey Brin and Larry Page and the search engine that emerged from their thinking was Google.
They had arrived at Stanford after Yang and Filo and they got to watch as one challenger after another stepped up to take on the mighty Yahoo. It seemed like venture capitalists were willing to throw money at anybody who come up with a search engine idea.
Brin and Page decided that they would concentrate on what Web searchers wanted most: good results. They figured that if they could make the user experience a good one, other good things would follow. They developed a system for evaluating Web pages that was based on two things.
Like everyone else they considered the actual content of the page. What words where there? How were they positioned? Which ones were boldfaced? How many times did certain words occur? Then they went a step farther.
The Google Guys decided to let the Web help them determine which were the really good pages. To do that they would evaluate the links that sent folks to the page Google was evaluating. How many links were there? Did they come from within content or were they part of giant lists of links called "link farms?"
Being good computer science types, they developed an algorithm that would automate the process. By 1998 when Yahoo and the stock market were still flying high, they founded Google. The name was originally Googol, a mathematical term for a number followed by 100 zeroes.
What seems to make Brin and Page different from Yang and Filo is that the Google Guys were actually thinking about running a business, not setting up a business as a way to keep doing things they wanted to do.
They raised private money to get started. Then came funding from venture sources. They don't seem in any hurry to go public. Unlike many of their competition, they've paid attention to costs.
They haggled over rent for the garage they used as the first non-dorm home for their business. They've used the Linux operating system so they don't pay licensing fees. They've used a lot of off-the-shelf equipment. They have hardly advertised at all.
The Google Guys have been careful and conservative in their business management. They know what can happen if they aren't. They don't want to be another AltaVista.
For a while AltaVista looked like the great challenger to Yahoo. It had snappy technology and a great reputation. In 1999, CMGI paid Compaq Computer $2.3 billion for an 83 percent stake in AltaVista. But AltaVista never made it to a public offering. Earlier this year it was bought by Overture.
AltaVista tried to become a portal, a site that's the first place that people go when they go to the Web. That didn't work. It seemed like they quit working on improving search quality while they concentrated on marketing. There are lessons there for Google. And there are the challenges of the final stages of a developing industry.
Developing industries follow the same general pattern. You start with lots of players, lots and lots of players. Each one has a very small share of the market. Then, over time, some companies, usually no more than three or four, split 80 percent or so of the market between them. That leaves 20 percent for everybody else.
Right now it looks like three search engines are lining up to take the market leader prize. Yahoo is still in the running, though its profitability is still suspect. Google is certainly a player. And then there's Overture.
Overture started out as GoTo.com with the idea that it could auction spots in search results to advertisers. This "pay-for-hits" business got started about the same time as Google, but most "experts' at the time thought the basic idea was dumb. They didn't see why advertisers would be willing to guess what to pay and they didn't think that simple listings could be more persuasive than those banner ads.
It turned out that the experts were wrong. Advertisers liked paying for results and having a say in what those results were worth to them. And, guess what, Web searchers liked the simple text messages because they were relevant. Bingo, another innovation.
The search engine business is turning out to be something like television. With television the business isn't entertainment, it's audience delivery. Television produces shows that will draw audiences that advertisers want to reach and are willing to pay to reach.
Today's search engines compete to draw the Web users that advertisers want to reach and are willing to pay to reach. They may make money through other services, too. Yahoo is certainly trying that with a hodgepodge of services.
Another characteristic of developing markets and industries is that the top players become more alike. That's happening here, too. The distinction between search engine and directory is essentially gone. Both Google and Yahoo have both and so do many lesser players. Overture may have started paid-search but Google and Yahoo now offer this as well.
The search engine business is maturing. It's not just Yahoo anymore. And it looks a whole lot more like a business than it did in the distant Internet past of ten years ago. Now we can all wait and see how the story comes out.
