Protocols for the Two-Way Web

Dave Winer is giving a talk on May 9. In his abstract, he asks “how can we connect rich editing tools like outliners, word processors and graphics programs with content management systems running on the back end, so that it’s easy for users, and gives them the best tools that we know how to design?”

Dave outlines the 4 key protocols:

1. XML-RPC and SOAP for connecting tools to back-ends.
2. RSS for syndication and aggregation.
3. OPML for hierarchic navigation.
4. The MetaWeblog API to allow tools a way to work with all back-ends.

In fact, this is the topc I am exploring in more detail in my Memex series – not as much for designers, but for all of us in terms of information management.

Routing for Web Services

Phillip Windley writes about Level 5 Routing – “the advent of standards for application integration has brought us to the point where applications can be put together by scripting calls to existing services.” He envisions the routing architecture to do the following functions:

– Service Call Switching
– Filtering Based on Context
– Event Monitoring
– Logging
– Service Facades
– Business Rules Repository

Search Engine Visibility has a discussion on the importance of showing up at the top of the results done via search engines. A quick look at access logs for many sites will show an increasingly significant portion of the traffic coming via search engines. That is why, especially for SMEs, search engine optimisation (especially for Google) is important.

Real-Time Intelligence

InfoWorld writes how “BI (business intelligence) and enterprise application vendors are turning to analytic tools in an attempt to energize the supply chain.”

Companies including SAS, Cognos, Business Objects, and Informatica have unveiled supply-chain analytics designed to expose data otherwise isolated in the enterprise supply chain.

In an environment where cost control and profitability are dominating IT decision-makers agendas, analysts observe that BI companies are aiming to leverage history in managing supply-chain performance.

Continue reading

TECH TALK: Constructing the Memex: A Personal View

For much of the period from 1997-1999, I too was a player in the directory and search business. My company, IndiaWorld Communications, had launched Indias first search engine, appropriately titled khoj in March 1997. (Since November 1999, khoj has been part of Sify, following its acquisition of IndiaWorld.)

The problem I set out to solve in March 1997 was that of India-centricity in search. Yahoo was then the de facto king. It would take a long time to get sites registered into its directory. When one did a search, it was difficult to get India-centric results Yahoo covered the world, but there were times when one wanted to limit the results to ones local context. I also realised then that search was one of the key attractors on the Internet. As new people came online, they needed to know which sites to visit. As new sites get launched, they needed a place to list them to tell the surfers. This is what khoj set out to solve.

We launched khoj on the second anniversary of the launch of IndiaWorld. We positioned it as the Indian alternative to Yahoo. Heres an extract from our press release (sourced from Google Groups):

Finding Indian Web sites just got easier. IndiaWorld, India’s largest Web site, has launched khoj, an online directory of over 800 India-related Web sites. khoj catalogues the Web sites into 11 primary categories, and has a multi-level classification system for business, education, entertainment, news and government. “Khoj” is a Hindi word which means “search.”

“Think of khoj as the Indian alternative to Yahoo. It will help people worldwide find Indian resources, information and companies much more easily. khoj is the first Asian venture on such a large scale,” said Rajesh Jain.

I remember sitting up for about two weeks prior to the launch going through a catalogue of Indian sites and classifying them one-by-one on a slow link to the Internet. In fact to make classification easier, we had written a program to get the top pages of various sites and store them offline in our office so that classification did not necessarily need a real-time connection to the Internet.

It was this crawling of pages that gave us the idea to add a search engine to the khoj directory. This way, people had three ways to find sites: navigate the directory, search the website descriptions in the directory, and get results from the actual cached pages of the Indian sites. This combination is what helped khoj become extremely popular and made it the top-ranked Indian search engine.

Tomorrow: Whats Missing

Continue reading

XML’s Growth wonders about XML: “With standards adoption spreading, some wonder if a messaging glut will lead to infrastructure overload.”

the whole idea of business activity monitoring (BAM) and the real-time enterprise is predicated on being able to pick off and interpret XML data in midstream. That’s why the technology, with all its shortcomings, can’t be written off over the long haul. The consensus around XML makes it a coming priority to address as opportunities arise.

As more users are educated on the opportunities that come with real-time information, it could have a tidal wave effect on IT. Planning, says AMR’s Austvold, means thinking through your business segment’s most relevant real-time information needs, counting the current volume of transactions and then multiplying by at least ten. “Proctor & Gamble went through and documented 8,000 touch points between internal applications and estimated that was probably half of what they really had,” Austvold relates. “They see a storm brewing in the next four years but they don’t want to XML tag every piece of information, they’ll pick off the top 10 percent and prioritize that.”

RSS Feeds Likes and Dislikes

Sven-S. Porst discusses RSS feeds in more detail. There are some good tips on how to make RSS feeds more useful. Recently, I’ve added full post and HTML support into my RSS feed, so people don’t need to necessarily come to the blog to read what I’ve written – they can do so right in their RSS Aggregator. An interesting thought by Porst:

Feeds grouping several small entries together. This can make them a bit harder to reference, but mostly you tend to reference only the more elaborated pieces anyway. This grouping of mini-posts keeps the aggregator uncluttered and gives me more content for one entry in the aggregator. I think this can be used in many cases where the real-time aspect doesnt play a big role which in my opinion it rarely does.

RSS Aggregators as Archivers?

Paolo Valdemarin has a discussion on RSS Aggregators and suggests using them as archivers, and not just as instant readers.

If an aggregator is meant as a way to take a snapshot of what’s going on on hundreds of sources and quicky present it to us, I believe that presenting news in reverse chronological order is the way to go.

But I also think that aggregators could be an interesting way to archive content, to let somebody quickly retireve something wrote sometime in the past.

Archiving by author, again, does not make sense: most weblogging applications already do that, if I’m looking for something and I know who wrote it, I can simply look on the author’s site.

There are search engines, which are of course a good way to find information, but not always very efficient. There are cases when a directory might be more useful.

We believe that archiving by topic in a directory could be a solution, and this is what we are trying to do. It’s not for daily instant reading, it’s to archive content.

I feel that RSS readers should stick to providing the viewing capability. What is needed for archiving articles of relevance is a personal blogging tool, to which items can be posted with a drag-and-drop capability. I’ll discuss this in greater detail when I talk about how to construct the Memex.

In a different but related context, Infoworld describes about how RSS could be used to counter spam:

When aggregators become widespread, many b-to-c newsletters will switch to RSS and drop now highly unreliable e-mail. I wrote three months ago that ISPs such as Hotmail and Yahoo, trying to stop spam, shunt to a junk folder or simply delete 25 percent of newsletters requested by subscribers.

The spam tsunami is forcing many e-mail recipients to build “whitelists,” accepting messages without question only from approved senders. Interestingly, RSS subscriptions work exactly like whitelists. By design, spammers have no way to push their material into anyone’s RSS reader.

Continue reading

really Personalised Advertising

Jason Kottke writes about a cool idea by Greg Elin: “He wants a way to dump calendar items, tasks, and the like out of his calendaring system (iCal, Outlook, etc.) and have those items display as ads on the web sites that he visits. So, when he goes to Slashdot, a banner ad tells him to stop for orange juice on the way home. When he goes to, there’s an ad telling him that his mother’s birthday is coming up.”

It’s one of those “wonder I didn’t think of it” ideas! Takes relevant advertising to its logical end.

TECH TALK: Constructing the Memex: DMOZ and Microsoft

Another development in the search and directory industry has been one which is diametrically opposite to Overture in terms of process and business model.

DMOZ (also called the Open Directory Project or ODP) is, according to the website, the largest, most comprehensive human-edited directory of the Web. It is constructed and maintained by a vast, global community of volunteer editors It provides the means for the Internet to organize itself. As the Internet grows, so do the number of net-citizens. These citizens can each organize a small portion of the web and present it back to the rest of the population, culling out the bad and useless and keeping only the best content.

ODP has over 3.8 million sites, 56,429 editors and over 460,000 categories. It is the most widely distributed data base of Web content classified by humans. Its editorial standards body of net-citizens provide the collective brain behind resource discovery on the Web. The Open Directory powers the core directory services for the Web’s largest and most popular search engines and portals, including Netscape Search, AOL Search, Google, Lycos, HotBot, DirectHit, and hundreds of others.

At present, according to Business Week, Yahoo boasts the biggest audience, Overture the most advertising, and Google has the leading search technology. The stage is set for a battle royal, with Microsoft as the dark horse. Microsoft Research has been looking at ways to improve search, according to

While search tools exist today, a major focus of Microsoft’s research will be to allow for a freer flow of associations between data and to expand how searches can take place. Currently, data on computers is largely stored in a hierarchical fashion: A picture or document gets a file name and is stuffed into a folder. To find a document, people largely hunt and peck, a technique that also gets used on search engines.

People, however, don’t think that way, Rashid said. To find a vacation shot from Australia using newer tools, for example, a person could ask a computer to pull up pictures that feature an ocean background or family members. A search engine inside an application would then comb through the visual images to get matches.

“The problem with hierarchies is this conceit that all knowledge has a place, but no single thing fits in one space,” he said. “They become very cumbersome.”
Microsoft’s “Sapphire,” another lab experiment, exemplifies the difference. The application lists associations with a word in a document. Scroll over a person’s e-mail address, and Sapphire will pop up a balloon listing the person’s instant message address, work title, recent publications, and lists of e-mail exchanges and meetings you’ve had with this person.

Tomorrow: A Personal View

Continue reading

Wireless Electricity for Villages?

Slashdot discusses a Sunday Times story about how “scientists have successfully applied the technology used in microwave ovens to beam electricity without the need for unsightly pylons and overhead cables.” A prototype has illuminated a handful of light bulbs and they expect to be able to power a remote village within three years, according to the story.

This could be extremely useful for rural areas in India, where power shortages are the norm.

Watch Samsung

NYTimes writes about South Korea’s leading company, Samsung:

Since 1996, Samsung has been run by Yun Jong Yong, a soft-spoken engineer who seems to believe that a good dose of crisis and chaos will keep Samsung, South Korea’s No. 1 company, agile and resourceful, fighting as if it were No. 3 when in many areas it has in fact become No. 1. And in many ways, Samsung’s success is emblematic of Korea’s rapid entry into the club of developed nations.

From a mass producer of cheap electronics a decade ago, Samsung is now the world’s largest producer of memory chips and flat screens. Its market capitalization is now bigger than its Japanese rival, Sony. Last year, Samsung was the world’s third-most-profitable electronics company, after General Electric and Microsoft.

Sony’s sales volume still dwarfs Samsung’s, but Sony’s profitability is far lower. During the first quarter of this year, Samsung Electronics earned $942 million in net profit on $8 billion in sales. By comparison, Sony earned a similar amount, $963 million for the full fiscal year ended March 31, but that was on $62.3 billion in sales. On Thursday, Sony announced a profit on sales of 1.5 percent for the fiscal year, compared with 17.5 percent for Samsung last year.

Niche products like graphics chips for game consoles and flash-memory chips for cellphones and hand-held computers have helped Samsung make money in an industry plagued by overcapacity and declining demand.

Management by Blog

Jimmy Guterman writes on what it takes to succeed: “The internal weblogs I’ve seen work are those that track an idea’s progress from offhand notion to fully matured proposal. I have seen three such blogs, always-on virtual whiteboards that have sped development and kept the status of projects clearer than they’d been before. They don’t attempt to capture an organization’s mood…The internal blogs that succeed will be safe, clean, well-lit virtual places in which diverse opinions are welcome and ideas — not people — are judged.”

Tim O’Reilly on Killer Apps

ExtremeTech writes that according to Tim O’Reilly, the future killer apps share a common thread: hacker geeks. Four trends that he recommends watching: Web Services; BARWN, or the Bay Area Research Wireless Network; hardware hacking; and multi-player gaming.

“There’s a common thread a hacker culture that ties together all of these four activities on the O’Reilly radar today,” said O’Reilly said. “Essentially it is being able to recognize the alpha geeks in society and leveraging their enterprise.”

“An invention has to make sense in the world it finishes in, not in the world it started,” O’Reilly said. “We are beginning to see the rise of interconnected networks, the technology uptake is accelerating, there are people with passion like the hacker guys and people with professional experience like professional programmers, so it’s not in the danger of being one really cool party. The most important thing is that this is bottom up it has grassroots support,” he said.

Slashdot thread

WiFi Future

WiFi coverage is all over the place – at least in the media! Barron’s adds its views, summarising that “wireless networks are proliferating, but making money will be tricky”:

After a relatively slow start, Wi Fi seems to have reached the tipping point. “The great value of Wi Fi is not that it is fantastically advanced technology,” explains Kevin Werbach, a consultant and commentator who spent four years at the Federal Communications Commission, working on technology policy issues, and later edited the newsletter Release 1.0. “It has many limitations, and it’s not near the cutting edge. But it’s being widely adopted. You can buy a Wi Fi card for $50. And every time someone puts up a hot spot, I benefit from that. Every added bit of connectivity adds to the collective benefit.” That’s a variation on Metcalfe’s Law — the idea proffered by 3Com founder Robert Metcalfe that a network’s utility increases exponentially as the number of users grows.

That dynamic drives growth. In 2004, according to Oyster Bay, N.Y. based tech market research firm Allied Business Intelligence, more than half of all computer notebooks sold worldwide will be Wi Fi capable; by 2008, the total is expected to reach 93%. Although lagging a bit behind PCs, the same thing is happening with personal digital assistants, or PDAs — 50% of handhelds should be Wi Fi enabled by 2006, according to ABI, up from 3% in 2002. ABI figures all the Wi Fi enabled hardware will drive up the number of paying hot spot customers to 5.5 million in 2006, from an estimated 24,000 last year. Rich Beyer, CEO at Intersil, the leading producer of Wi Fi chips, contends there will be at least 100 million Wi Fi capable systems in the market by year end 2004.

“As the costs come down, it becomes virtually free to put Wi Fi radios in a wider and wider variety of devices,” Werbach says. “Pretty soon, you won’t be able to buy [the devices] without Wi Fi. Right now, $10 $20 for a Wi Fi chip doesn’t add much to the cost of a laptop, but it’s significant for a PDA. At $2, it doesn’t add much to a PDA’s price. At 50 cents, every consumer electronics device will have a Wi Fi chip.”

John Patrick contends that Wi Fi really might be the next big something the Valley desperately needs. “We’re very much in parallel with where we were with the Internet almost 10 years ago,” he says. “I remember looking at the Internet at IBM in 1994 and thinking: ‘This is really cool, but where’s the money?”’ The questions people have about Wi Fi now are the same ones we had in ’93 and ’94 about the Internet. Skeptics say it doesn’t scale, it’s not secure, it’s not industrial strength. It’s the same things people said about the Internet. But there’s no stopping Wi Fi. It’s a grassroots technology, totally distributed, standards based, global, with nobody in charge. Those are the reasons the Internet has flourished. And the implications are huge.”

A related story talks about WiFi as the “Venture Crowd’s New Quarry”.

“There are a lot of different ways to play this game,” says venture investor Tom Blaisdell, a partner with Doll Capital Management of Menlo Park, Calif.

While there are some similarities between the Internet boom and the Wi-Fi wave, the money men have come to expect delays and may be more prepared to wait for their returns this time around. But they do expect their Wi-Fi start-ups to preserve cash and focus on business models that can generate revenue.

“I think all venture investments now have an eye toward capital efficiency and are targeted toward more well-defined problems,” says Blaisdell.

What’s more, when start-ups are developing breakthrough technologies that don’t lend themselves to self-sustaining business models, venture capitalists are realistically guiding their companies to plan to be bought, “as opposed to trying to go public,” Blaisdell says.

Continue reading

Sunday Mornings

For the past couple years, my Sundays have followed a distinctive routine. Much of the morning is spent writing out the Tech Talk for the week. I find my mind at its most creative and freshest on Sunday morning. With little else to clutter my thoughts, the focus is entirely on the task at hand. It takes about 2.5-3 hours to write out the week’s Tech Talks. At times, if I know that I am not going to be free the following Sunday (travelling or otherwise), then I will use up the rest of the day writing the additional Tech Talks for the week after next for the simple reason that I find it very difficult to write the Tech Talks on any other day.

Perhaps, it is the pressure of going to work or the lack of a chunky period of time. Sunday morings are indeed special for me now – I let the mind roam free, sit in front of the computer, catch those ideas and convert them to words. Of course, the side-effect of this is that I hate to go out anywhere on Sundays!

Yesterday, as I started writing the Memex series, I got into a good groove and the ideas just flowed. I spent most of the day writing (interrupted by a two-hour sleep in the afternoon). By the end of the day, I had put together 14 columns. Needless to say, I was quite happy with the work.

When I start to think and write, I have an outline of what I want to write. But, more often than that, it is just the act of sitting at the computer and starting to write which lets the ideas and words flow. They seem to take on a life of their own. Quite fascinating how the mind works. I find that all my reading is very helpful as I try and connect threads. I am not a natural writer, but I do love writing and making connections. It is as if the ideas are there – lying waiting to be discovered. Almost like the Memex itself.

TECH TALK: Constructing the Memex: Overture

One of the amazing commercial success stories in Internet search so far has been that of Overture, which has focused on paid search placements, and ended 2002 with revenues of USD 668 million. More from a story:

Overture began as the brainchild of Bill Gross, whose start-up investment company, Idealab, incubated one-time Internet highfliers like eToys. He founded the company as in September 1997 and a year later, launched its search advertising service with results appearing on and partners including Netscape Communications.

The company compares its service to the Yellow Pages, the phone book that offers a useful resource even as it serves the marketing goals of its advertisers.
The company claims its goal is to create a win-win situation for customers and Web surfers, enforced by self-interest. Because advertisers are required to pay a fee each time someone clicks on one of their links–a practice known as pay for performance–companies are discouraged from misleading readers.

In a world driven by advertising revenues, the importance of what Overture started and others have followed is highlighted by a recent Business Week story:

Placing ads near search results offers the simple appeal of the Yellow Pages, but with different economics. Search-engine companies such as Overture, Google, Ask Jeeves, and LookSmart charge most advertisers by the click. These ads can be presented among the search results, looking like any of the other Web links that have been rounded up. That’s known as paid inclusion. More often, other search-related ads are featured as “sponsored listings” at the top or side of the search results. Advertisers say that search-related ads, whether overt or camouflaged, attract far more interest than regular scattershot Internet ads. Why so? They give people what they’re already looking for.

Search advertising is also cheap. At an average of 35 cents a click, paid search undercuts the $1-per-lead average for Yellow Pages ads. The money is split between the portal, which generates the traffic, and its search-advertising provider.

Changes in Internet usage also power this trend. As Web surfers grow more sophisticated, they focus on specific tasks, such as checking mail or finding a recipe. More are using search engines to hurry through their to-do lists. The percentage of Web site visitors who arrived via search engines nearly doubled in the past year, to 13%, says analytics firm WebSideStory. Increasingly, says Jupiter Research analyst Gary Stein, “people are tuned out on banner ads and tuned in to search results.”

A May 2002 Fortune article on Google, now Overtures main competitor, puts the battle between the two for revenues in perspective:

Overture and then Google started selling something called sponsored links, which is a fancy name for a classified ad with an Internet link. Sponsored links cost nothing to produce, load easily through a narrowband connection, and make a more subtle pitch than banner ads. They’re also more popular with advertisers, which pay based on how many times people actually click on the ad. With banners, advertisers have to pay based on how many times the ads were displayed, which gives no indication of how the ad is doing. Google took the model a step further, marrying the text-based ads with its search results, something Overture did not have. In other words, if you do a search on Google for, say, Botox, an ad and link for comes up alongside your search results. The upshot was something that Website operators had been trying to accomplish since the beginning of the Internet: meaningful search results accompanied by relevant advertising.

Tomorrow: DMOZ and Microsoft

Continue reading

TypePad will lead CMS revolution

Content Management Systems are becoming mass-market, according to this story by Ben Hammersley about TypePad after an exclusive peak:

The features are remarkable: there is a very powerful, but extremely simple, template builder. Users can redesign their weblogs and create fully compliant XHTML pages, with out knowing what that last phrase means. There is a built-in photo album, built-in server stats, so you can see who is coming to visit you and from where, built-in blogrolling (listing the sites you like to read), and built-in listing for your music, books and friends, producing a complete friend-of-a-friend file for every user.

In short, with Typepad, SixApart has embraced almost every advance in weblogging over the past year, and wrapped it into a product my dad could use. It raises the bar for the personal publishing world in a way that the Blogger/ Google buyout promised but has yet to deliver.

TypePad is the first new consumer-grade weblogging product in more than a year, but it shows a change in the marketplace: grabbing the new middle ground of users who want all the advanced features of a self-hosted weblog, but none of the tears of having to learn about Linux or Perl or FTP. This should elevate the standard of weblogs in general, as it does away with any correlation between technical skill and artistic merit. We will no longer be reliant on geeks for top quality weblog reading. It takes the seething masses and pulls them up to the same technical level as the best Movable Type tweakers and hackers.

By creating content management systems with professional features, for around one thousandth of the price of the systems the large sites of the dotcom era were forced to use, the weblogging industry is rapidly creating new possibilities for people to make a living writing for the web.

Continue reading

Small World

The recent Gulf War which was fought as much in Iraq as in the media. It was our first war in the Internet era. The protests against the war in different parts of the world wer co-ordinated through the Net. Now, another development highlights our “small world”. Writes David Kirkpatrick in Fortune: “SARS demonstrates yet again how tiny our interconnected world has become. The epidemic, and our perception of it, are both very much a function of the Internet age.”

There is also a silver lining to it all. Adds David: “Even as certain risks grow greater in a connected world, so are remedies more at hand. The same Net-enabled system that set off the extra-loud alarms also allowed researchers worldwide to identify and decode the genetic sequence of the culpable virus in mere weeks. Unprecedented cooperation among labs in many countries-all linked together by the Internet-was the key to this quick work. It’s likely that similar cooperation will lead to effective treatments and perhaps a vaccine-much more quickly than in times past.”

On a related note, the NYTimes has a detailed look at how the SARS virus originated and spread:

In early January, alarmed health departments in Shunde, Heyuan and Zhongshan all reported the strange pneumonia clusters to Guangdong provincial authorities, who concluded that they were facing a highly infectious pneumonia caused by a previously unknown agent.

It is unclear whether that conclusion was passed on by provincial officials to the Ministry of Health in Beijing, or ever reported to international health agencies that might have conducted an early investigation into the problem. Instead, it would be another two and a half months before the strange pneumonia had a name, coined only after an Italian doctor working in Hanoi, Vietnam, alerted the World Health Organization about a similar new pneumonia he was seeing there.

And it would be three and a half months before China’s leaders would admit that their country had an epidemic of SARS. From January through the middle of March, doctors in Asia and Canada were encountering patients carrying a virulent and highly contagious germ, unaware that they were facing potentially lethal infection.

During that period, hundreds of health workers fell ill. During that period, well-meaning doctors were placing SARS patients in ordinary wards as they would patients with normal pneumonia and those patients were passing the infection on to hundreds of others.