As Yahoo prospered, along came search engines like Altavista, Excite, Lycos and Webcrawler. They had programs which crawled the web, bringing with them whole sets of pages. They would index the words in the pages. Now, the granularity went from searching for a site to a page, which helped get us to the content we were looking for much faster. Or at least that was how it should have been. But faced with a result pool of tens of thousands or even millions of pages which had our search terms, it was difficult to know where to begin (or end). And so, even as the Web grew, the search industry stagnated as it was weighed down by its own weight.
Into this world came Google. Google also crawled the web. What was different about Google was the way it used to present the results. It used a technique called PageRank analysis, which ranked pages based on their incoming links and which pages linked to it. If an important page pointed to another page, then it was likely that the page pointed to was also quite authoritative. This is somewhat akin to people giving references to others. Who gives the references carries a lot of weightage.
The turning point in the industry came when Yahoo decided to replace its outsourced search provider Inktomi with Google in June 2000. Inktomis stock fell 18% on the news. It is interesting to read a Red Herring article of that time:
Up until Monday, Inktomi was the lead search engine provider to the top four Internet portals. Inktomi still provides primary search engine capabilities to Microsofts MSN, America Online and Lycos. And Inktomi has been in a similar situation with one of those companies before.
“I wouldn’t be surprised if Yahoo bought Google,” says Tomas Isakowitz, an analyst with Janney Montgomery Scott. But despite the acquisition rumors, Mr. Isakowitz thinks that Yahoo switched to Google just to distinguish themselves from the competition.
The story three years hence is very different. Yahoos 2003 revenues from all its services are expected to grow nominally to about USD 1.2 billion. Inktomi was bought by Yahoo recently. Privately held Googles 2003 revenues are expected to be USD 750 million, a 150% increase from 2002.
The passing of the baton from Yahoo to Google over the past few years is symbolic of the evolution of search. This is a point made by Danny Sullivan, editor of Search Engine Watch: “In October 2002, Yahoo made the directory secondary to Google. Suddenly the value of getting listed in Yahoo seemed to disappear. Now, if you’re not listed with Yahoo, it may not matter.” Elwyn Jenkins of Microdocs makes a similar point: Yahoo rested on its laurels as a great Internet Directory, not thinking that search would overtake a directory service at some time. However, what searchers seem to want is the immediacy of search rather than the hand documented web that Yahoo gives.
News.com highlights the transition from directory to search engines as the navigation norm on the Internet.
Once the primary road signs to navigating the Internet, directories have moved to the shoulder. They are being displaced by algorithmic search tools and commercial services that many people now believe do a better job in satisfying Web surfers and advertisers. The transformation is bringing to an end an altruistic era of human editors, who once wielded significant clout in driving traffic to Web sites through recommendations made without regard for commercial considerations.
How did Google come to dominate the search industry?
Tomorrow: Googles Domination
TECH TALK Constructing the Memex+T
