It is important to first understand the different between the Reference Web and the Incremental Web. This is from a post by Rich Skrenta of Topix.net from February 2005:
Search has become the dominant navigational paradigm for goal-directed reference queries. But search is a poor way to stream new developments around a topic.
Google searches the reference Internet. Users come to google with a specific query, and search a vast corpus of largely static information. This is a very valuable and lucrative service to provide: it’s the Yellow Pages.
Blogs may look like regular HTML pages, but the key difference is that they’re organized chronologically. New posts appear at the top, so with a single browser reload you can say “Just show me what’s new.”
This seems like a trivial difference, but it drives an entirely different delivery, advertising and value chain. Rather than using HTML, the delivery protocol for web pages, there is a desire for a new, feed-centric protocol: RSS. To search chronologically-ordered content, a relevance-based search that destroys the chronology such as Google is inappropriate. Instead you want Feedster, PubSub or Technorati. Feed content may be better to read in a different sort of client, such as Newsgator, rather than a web browser.
While there’s been considerable deployment of goal-directed services, there has been little technology development around automated aggregation of relevant topic streams. Until now, this hasn’t been a problem. Most of the growth on the web over the past 10 years has been reference services. But now we’re seeing an explosion in the number of sources publishing new incremental content every day. Blogs certainly — but other sources too, such as news organizations, companies, and our increasingly web-enabled governments are pumping out gigabits of fresh news online every day. There is a vast proliferation of new incremental content underway.
The Reference Web started off more than a decade ago. Various publishing tools made HTML publishing easy. It took a second-generation search engine like Google to convert the publishing that had taken place over the years into rich material that we could browse on-demand. One can thus think of Google as an information refinery for the Reference Web. This Web has grown to billions of pages and has all kinds of stuff that one could spend a lifetime looking over. This Web is now going multimedia with broadband networks now supporting the transmission of video.
Tomorrow: The Potential
TECH TALK The Now-New-Near Web+T