Bosworth’s Web of Data
ONLamp.com has an article by Daniel Steinberg about Adam Bosworth’s keynote at the MySQL Users Conference 2005:
Google’s Adam Bosworth suggested that we “do for information what HTTP did for user interface.”
As a result of a simple, sloppy, standards-based, scalable platform, we have information at our fingertips from Google, Amazon, eBay, and Salesforce. Bosworth’s own company, Google, gets hundreds of millions of hard queries a day. He said they see it as putting Ph.Ds in tanks to drive through walls rather than around them.
In addition to the advantages in software, there have been great gains in hardware. Bosworth said that one million dollars buys you five hundred machines with 2TB of in-memory data, a PetaByte of on-disk data, and a reasonable throughput of fifty thousand requests per second. This amounts to one billion requests per day.
Having this sort of power changes the way you think. For example, organizing things into folders declines in importance. You can’t remember which folder you put something in, and searches are more efficient ways of finding things. The challenge is to take a database and do for the web what was done for content. Bosworth explained that you “need a model that allows for massively linear scalability and federation of information that can spread effortlessly across a federated web.”
Bosworth predicts that RSS 2.0 and Atom will be the lingua franca that will be used to consume all data from everywhere. These are simple formats that are sloppily extensible. Anyone who wants to can use these formats to consume content or to author content. Contrast this with the Semantic Web, which requires that you get a large group of people to agree on the schema of everything.
Dare Obasanjo has a comment: “What I find very interesting is using RSS is the data access format for the Web. RSS gained popularity as a way to syndicate blog posts and news sites but its turned out to be a lot more versatile than that. Sites like Feedster and Amazon’s OpenSearch technology show you can use RSS as a mechanism for providing search results and integrating search engines respectively. Podcasting shows you can use RSS to syndicate digital media content instead of just plain old text or HTML. With Amazon’s syndicated feeds one can keep abreast of when new CDs, books and more are released…In situations where one simply wants to expose read-only data via a service on the Web, it’s looking like RSS is the technology to beat. As more and more information is exposed as RSS feeds, there will be even more interesting things people will be able to do with this technology.”