TECH TALK: Constructing the Memex: Unstructured Content

The problem of information overload has been with us for a long time, and is getting worse. Ray Ozzie puts the situation in context:

Just as the first generation of personal computers was mostly about personal productivity, the first generation of the Internet has largely been about centralized Web sites, used for publishers, transactions and e-mail. For the most part, all seems well and good. At a personal level, however, many of us are overwhelmed. We’re chained to e-mail and the Web, drowning in an information flood that leaves us feeling more and more like human message-processing machines.

Unfortunately, mainstay tools are falling behind our needs. Software was conceived in an era with substantially different requirements. For example, e-mail emerged 30 years ago, when computer viruses, spam and e-mail overload weren’t even on the radar screen. That era could not conceive of a future in which we’d deal daily with online documents and presentations, e-mail and instant messages, Web sites and blogs.

Each of us will soon face hundreds, thousands, or tens of thousands of “inputs” that we’ll need to continuously absorb and coordinate. A world with complex social, economic, organizational and personal interdependencies is inevitable. And as we near this linked future, systems and technologies must evolve or we will simply be unable to cope.

Ozzie believes that Personal productivity tools will become joint productivity tools designed for online use instead of a paper-only world. A rich cadre of collaborative online writing, media management, presentation and consumption tools will move to the forefront of our daily electronic lives.

The problem the Memex solves is that of rapid retrieval of relevant content from a humungous pool of unstructured content on the Web. Esther Dyson puts this in perspective in the Jaunary issue of Release 1.0:

To start, lets just consider how the Webs unstructured information can be organized. The two leading approaches are exemplified by Yahoo! and Google. Yahoo! has created a single, very broad taxonomy; although it has not in fact organized everything (!), it offers a directory (taxonomy) structure that in theory should be able to classify any content that shows up. By contrast, Google organizes the Web dynamically: Tell us what you want, and well put it at the center of the world and find you the surrounding informationTheres a trade-off between depth and breadth; the directory offers fine-grained, carefully vetted material, while the search engine offers access to everything
Yahoos Srinija Srinivasan says: Directories make most sense when you are browsing, when you want to discover something. Whereas you use search when you know what you are looking forWe cant possibly manage the entire range of what people might be looking for. The directory was never intended to cover every word of every page out there.

Google arose from the perspectivethat the Web is simply too vast for anyone to define or structure it properly: Best to let each query define its own neighborhood, and to start each search from the query outwards, rather than from some mythical top down, to where the answer lives.

As Yahoo!s Srinivasan notes, users have turned from browsing directories to searching, from exploring to going after specific results.

Between the two extremes of the centralised approaches of Yahoos directory and Googles search is the individual, ant-like, emergent Memex. To construct the Memex needs the active participation of each of us. As we have seen, the tools to bring to life Vannevar Bushs 1945 vision are only now becoming available. As writing and self-publishing becomes easier, individuals are starting to provide a shape and form to information on the Web, and embellishing it with their thoughts and ideas. This is creating for a richer, two-way web, built around the blogs, RSS and OPML ecosystems.

Next Week: Constructing the Memex (continued)

TECH TALK Constructing the Memex+T

Published by

Rajesh Jain

An Entrepreneur based in Mumbai, India.