Month: May 2001

The two defining themes of the New Internet will be Pervasive Connectedness and Real-Time Infrastructure. The Internet will be like an envelope around us, creating an always-on, always-connected world. It will also provide us – at home, at work, and everywhere else – with information and services on demand and in real-time. In doing so, the Internet becomes invisible and yet provides visibility across the value chain.

Pervasive Connectedness: Broadband IP-based networks built using a mix of optics for the backbone and wireless for the last mile will ensure that there is ubiquitous connectivity. Today’s cellular networks ensure reach through in most parts of the world. Tomorrow, a combination of the high-speed cellular networks like 2.5G and 3G, and community networks using technologies like 802.11b (Wi-Fi, which uses the free 2.4 Ghz spectrum) will provide high-speed connectivity everywhere.

Real-Time Infrastructure: When software and data moves into the network “cloud”, it becomes possible to deliver just the information one needs – based on location, time and context. Sensors embedded in places and machines originate information which gets carried over wireless networks into the network for distribution to users based on interests and preferences. As information increases, there will be a need for automation through agents (or “knowledge bots”) which can not just filter, but also process the information in real-time.

There are two outcomes: the Connected, Communicative, Consumer (C3) and the Electronic, Extended Enterprise (E3). The successful integration of the two results in the Digital Dividend (D2) – sustainable competitive advantage and superior profits.

For Consumers, the New Internet means universal connectivity to information, application and services via personal, wireless devices. Instead of just pulling up and reading information, the experience now becomes interactive through streaming video. We are already seeing this today as people in countries like India use the Instant Messaging platform combined with multimedia computers have “phone” conversations with family and friends elsewhere in the world – all for the cost of a local call.

For Enterprises, the New Internet extends the enterprise value chain to include its suppliers, partners and customers. Self-service forms, integrated databases in which data is entered only once, event-driven notifications, corporate portals ensure visibility of information across the value chain. An immediate by-product of this will be in the reduction of inventory. The Internet ensures cheaper communications and the reduction in transactions costs by streamlining business processes. Outsourcing will increase as companies think hard about which of their processes add value and which do not.

Among all this change, there are some constants: the need for Vision, Strategy, Capital and Innovation to build the successful businesses of tomorrow.

Tim Berners-Lee, James Hendler and Ora Lassila, writing in Scientific American (May 2001, http://www.sciam.com):

The essential property of the World Wide Web is its universality. The power of a hypertext link is that “anything can link to anything.” Web technology, therefore, must not discriminate between the scribbled draft and the polished performance, between commercial and academic information, or among cultures, languages, media and so on. Information varies along many axes. One of these is the difference between information produced primarily for human consumption and that produced mainly for machines. At one end of the scale we have everything from the five-second TV commercial to poetry. At the other end we have databases, programs and sensor output. To date, the Web has developed most rapidly as a medium of documents for people rather than for data and information that can be processed automatically. The Semantic Web aims to make up for this.

The Semantic Web is not a separate Web but an extension of the current one, in which information is given well-defined meaning, better enabling computers and people to work in cooperation. The first steps in weaving the Semantic Web into the structure of the existing Web are already under way. In the near future, these developments will usher in significant new functionality as machines become much better able to process and “understand” the data that they merely display at present.

A view from Forrester (http://www.forrester.com):

Web browsers have brought Internet services to millions of people. As a result, Internet usage has boomed. However, the Web’s days are numbered as the Internet moves to a second round of expansion beyond the browser. Two new waves of innovation will eclipse the Web: an executable Net that greatly improves the online experience, and an extended Net that connects the real world.

The first stage in the X Internet is an executable Net. Users will get real-time, interactive experiences over the Net through disposable code — programs you use once and throw away — downloaded to their PCs and handheld devices. These quick downloads will allow users to carry on extended conversations with Net services.

An extended Internet is also emerging through Internet devices and applications that sense, analyze, and control the real world. With cheap chips and a worldwide Internet backbone, nearly every device that runs on electricity will have an Internet connection, through both wired and wireless networks. The result? The number of Internet devices will boom from today’s 100 million to more than 14 billion in 2010.

John Doerr from Kleiner Perkins (http://www.kpcb.com), quoted in International Business Week:

The Net [in the future] will be very, very different than the Net we know today. It will be what I like to call an Evernet. It’ll be always on, it’ll be on all kinds of devices, not just on PCs. It’ll be on set-top computers, and it’ll be on tables in kitchens, and God hope, in all the classrooms. And it’ll be wireless everywhere. Lots and lots of Net everywhere, pervasive Net, Evernet, always on. And it will be a very high-bandwidth Net. It won’t be static little pictures and text. Or if it is a picture when you click on it, you’ll get a movie behind it. It’ll be highly personal also. It will know that you are there in the room, and it will know a lot by virtue of your portal or your provider. It’ll present the stuff you’re interested in. It won’t show you ads you’re not interested in.

Forrester calls it the “X Internet”. Tim Berners-Lee calls it the “Semantic Web”. John Doerr calls it the “Evernet”. Everyone’s in search for it, and yet no one can lay claim to it. This is the Internet that is emerging – an Internet which will go beyond the Web of hyperlinks and documents that we know today, an Internet which will be more than just a way of allowing people to access information put up on computers through browsers.

This is the New Internet and it being built around two themes: Pervasive Connectedness and Real-time Infrastructure.

The Internet of tomorrow will be powered by changes taking place today in computing, communications and software.

Computing: The first trend is Moore’s law, which continues to power the rapid progress in semiconductor development. Each new chip doubles in power every 18 months. This has led to an exponential increase in processing power (coupled with falling costs) in the past three decades and is likely to continue for the foreseeable future. The second trend is the increasing proliferation of non-PC devices. Cellphones and PDAs are at the forefront of the race to create smaller but powerful devices which are truly “personal”. The third trend is that of embedded computing. An embedded processor does one task with a larger machine. From cars to elevators, embedded computers are today everywhere. Over 1.6 billion embedded microprocessors are reported to have been shipped.

Communications: The first trend is Gilder’s Law which holds that bandwidth increases exponentially – doubling every 6-9 months. This is being largely driven by fibre optics. Undersea cables with huge capacities now circle the globe. Adding new bandwidth means lighting up more fibre. The second trend is Wireless. GSM and CDMA networks connect a billion cellphones into the telecom network.

Broadband wireless is also becoming a reality to bridge the “last mile”. To bring the connection to the “last few feet” are emerging protocols like Bluetooth, and 802.11b (Wi-Fi) and 802.11a. The third trend is the building of the farms of the technology age – Data Centres. These data farms make technology a utility – offering computing and applications on demand.

Software: The first trend is that of Web Services – software becoming a service. Instead of paying large upfront licencing costs, software will increasingly become available as a utility service in the network “cloud”. Components from different vendors can be assembled on the fly from across the network through standards such as XML, SOAP, WSDL and UDDI. XML allows data to be exchanged, SOAP provides the wrapper for exchanging data, WSDL is the language to describe web services while UDDI provides the directory to locate web services on the Internet. The second trend is open-source, where developers work worldwide on projects which are in the public domain, and can be extended by anyone. Linux and the Apache web server are two examples of open-source software. The third trend is the creation of zero-defect software. Software is at the heart of real-time applications and mission-critical. At the same time, it has become increasingly complex. Yet, it simply cannot fail.

These are the trends which serve as the foundation for the creation of the Internet of tomorrow.

What began as a mechanism to allow researchers to communicate with each other over 30 years ago has become the foundation for the biggest revolution in modern-day business. The Internet, piggybacking on the rise of the computer, has connected people, computers and businesses worldwide, and spurred innovation on an unprecedented scale. Today, nearly 100 million computers and 500 million people are connected by the Internet worldwide.

The defining moments for the dramatic increase in the Internet penetration and usage worldwide were the creation of the Web, hyperlinking and HTML by Tim Berners-Lee at CERN (in Europe), followed by the development of Mosaic by Marc Andressen and his team at University of Illinois, Urbana Champaign (in the US). HTML and its simplicity allowed the publishing of documents by anyone, and enabled documents to be linked to each other, creating what became the “World Wide Web”. Mosaic (and then Netscape) gave the Web a graphical interface and made accessing these documents as easy as just clicking on blue-coloured underlined words.

The IPO of Netscape followed by the subsequent IPOs of Yahoo, Excite, Lycos, Amazon and eBay and their monumental valuations fired the imagine of entrepreneurs worldwide and unleashed perhaps the greatest modern day “gold rush”. The race for eyeballs was on. As money flowed first from venture capitalists and then from the public markets, companies raced to build websites for every conceivable offering for consumers and for businesses.

A new industry was being created. Old business models meant little as companies were valued based on multiples of their traffic. The New Economy was here. Speed and time-to-market mattered. Profits didn’t. Youth was in, Experience out. When AOL bought Time Warner and PCCW took over Hong Kong Telecom, the “revolution” was complete.

In many ways, this “irrational exuberance” perhaps shortened the adoption period and acceptance of the Internet, the Web and eCommerce dramatically. As “Old Economy” companies faced competition from upstarts, there was a frenzy to adopt the Internet in many aspects of business. But the Internet by itself was not a disruptive technology. Consumers and business processes change slower than technology. Relationships and practices built over years do not simply go away just because of the Internet.

The last fifteen months have seen a dramatic, almost unbelievable, change in the world of technology and in people’s attitudes towards technology and new-generation entrepreneurship. As the stock markets collapsed taking with it many listed and private companies, it seemed like the end. But it was only the end of the beginning. The change being brought about by technology and the Internet is for real. As we shall see in the next few columns, a new Internet is being built – an Internet that is helping glue together great progress in computing, communications and software. Only, this time around, profits do matter.

News Pages are the Ore. We are then setting up a complete refining process to remove the junk from these pages so we can get to the “metal”. The components of the Refinery are:

• PageBot: URLs are fed into a bot: . Output of the Bot is an HTML page, which contains the headlines. Need to also check if page has changed. Alert if page cannot be botted for 3 successive time periods. Queue pages for processing. The bots keep running constantly.
• PageQueue: This queues the pages, and sends them to the HeadlinesExtractor. Needs to prioritise pages based on their importance if queue gets big. May also do checks like if page has not changed.
• HeadlinesExtractor: Gets the headlines from the HTML page. A mix of auto-manual techniques. Output is a collection of headlines: . The description comes if it is available on the page with the headlines. NumberOnPage indicates the importance of the headline (the no. in the sequence). Also needs to compare if headline/URL already exists and then overwrite if necessary.
• StoryBot: Takes the URL of the story and gets the full-text page. In case of there being multiple pages for the story, one should get the “printer-friendly” page – most sites offer such a page.
• StoryExtractor: Takes the story page and then extracts the actual story from it, stripping away the unnecessary stuff. Also tries to extract the author of the story, and the summary (first 20-odd words).

Thus, at this time, we have 3 databases: the URL database for botting, Headlines, Stories.

Then, we come to the Analytics.

• Classifier: Classifies the headline/story by topic/concept and puts it in a hierarchy.
• Analyser: can do more detailed analysis, based on aggregated browsing histories.
• Miner: look for other types of trends in what people are reading and doing.

Business Model

I have not thought of how one makes money from this. But the most likely thing is to make into a subscription service (a News Portal) for use of the advanced features. It’s a service which will grow on people. Like Google. No one likes to miss out on a story. People also like to share stories and be the first to share them. We all email interesting stories to friends. After email and instant messaging (both falling in the communications category), news and information is what we access most often on the Web. There is so much of it out there, but so little structure to it all.

Here then is a features summary of the News Refinery:

Aggregation	Collection of all the headlines from multiple sources
Customisation	Creating my newspaper, with multiple pages
Database / Filing	Classifying and filing (archiving) headlines and stories in folders
Search	By date, source, author, content (keywords), concept
Alerts	Notifications on keywords, breaking stories; multiple device support
Summaries	Quick, brief story summaries, especially if headline is short or cryptic
Daily Email	Just in case there is no web access; also email of marked articles
Similar Stories	Stories like the one I am reading
Classification	Directory of Headlines for navigation by category
Recommendations	Based on what other people are reading and overlapping interests
Offline Access	Sync stories with my computer (store locally) so I can read them offline
Sharing	Putting stories in common folders to allow others to access them
Email / Forward	To friends; also keep track of what I am forwarding to whom
Discuss / Weblog	Comments on stories; Use to create user-generated content and communities
Table of Contents	Single Page ToCs for magazines so I can see all stories on 1 page
Translation	For reading stories in other languages
Filters	Remove duplicates (e.g., same story from wire service)
My Addition	Allow for addition of new source by giving URL
In-Context	Place the event / story in context, provide perspective, chronological events (timeline), related stories from other sources
Favourites	What others are reading and where, like Amazon’s Circles
Learning	See what I am doing and learn, become smarter, monitor interests
Repeat Search	Search for certain keywords at specific frequencies of time; email results
News Browser	Custom front-end to better navigate news; add filters to narrow
News Plus	Like VCR+, link online-offline; easier access to stories in print
Extensions	To Discussion Forums, Press Releases, etc.
Visual Appeal	Create a 2D/3D Map of stories; use colour to depict age of stories
NewsBar	Like the Google Toolbar which sits on the desktop for alerts, quick links
Search Others	A single window search on various news sites; email results from all

16. It should also learn my interests by seeing what stories I click on. By comparing my interests with others, it can (a) refer me to other people like me and their comments, and (b) point out stories which I may otherwise have missed.
17. It should be able to make summaries of stories automatically so as to save me time.
18. It should be able to compare stories so as not to give me multiple copies (eg. a story from a news wire like Reuters may actually show up in multiple sites).
19. Headlines should also be pre-classified into a directory, allowing me to navigate by topic.
20. I should be able to add a new news source easily – just by giving a URL.
21. The software and intelligence in the system should adapt to my interests and learn from the collective. It should evolve.
22. I should be able to place the story in “context” with other such developments.
23. I want to get a sense of favourites – what are people reading in different parts of the world (like Amazon’s Circles).
24. I want to be able to chronologically follow certain stories / topics, or even writings by an author.
25. A “Similar Stories” Search: show me other stories like this.
26. Repeat specific “Searches” once a week and send me all new stories.
27. I can pick up articles I want to read, and these can either show up on a single page, or can be emailed to me. On a slow link as I have, it takes 30-45 seconds per link I click on. (My solution to this has been to turn images off for browsing and open articles in multiple windows using the right mouse button.)
28. A useful service: email me a specific news site page at a fixed time daily. Like I want the Red Herring top page, WSJ Tech news page, News.com top page daily since its hard for me to search for stories on that page once they get “overwritten”.
29. We also need to think of how this can be extended to multimedia news (images, audio, video) – these types are increasing.
30. Imagine if every news story published had a unique identifier, like a book’s ISBN number. This would allow one to reference it easily forever. In some ways the URL is this code number. This code could also show up in the print edition at the end of an article, allowing easy access on the web in case I wanted to email it or download it for my future reference (like the physical acts of cutting and filing). This allows for closer linkages between offline and online.

What does the News Refinery do?

1. It collects all the headlines and perhaps brief summaries from all news sites globally on a continuous basis.
2. It allows me to set up preferences so that I can make my own daily newspaper by putting together these headlines – it may have multiple pages which reflect my different interests.
3. I can reference these pages or the headlines for a site by date: which means there is a closure of an edition that happens from one day to next. This ensures that if I am on vacation and cannot access the web, I can always go back and look at the headlines of that day. (This also requires an archival facility for the headlines.)
4. I can create my own “taxonomy” to classify stories into folders. Ideally, I would want to extend an existing taxonomy, which means stories may need to be pre-classified.
5. I want the stories downloaded on to my computer so I have access to them offline also.
6. I should be able to search stories: not just the headlines but also the full-text of the stories. This means that the headlines will need to botted and get the full-text also. It also means a sophisticated search engine. [A bot is an automated program which fetches a page, given a URL]
7. I want to get alerts based on certain keywords. Alerts should be deliverable on multiple devices.
8. I want to access stories by: source (publication), date, author and keywords/people in the content of the story.
9. I should be able to set up on-line folders for stories so classified so that I can share these with others. The current alternative is to email stories to friends and colleagues. In fact, by monitoring my activities, it should be able to recommend which story goes to whom. It should also keep track of which stories I have forwarded to whom.
10. I should be able to comment on the stories (like a Post-It). This can make for a discussion thread, or allow me to maintain a Weblog. This can also serve as the basis of creating communities.
11. For magazines, I should be able to create a Table of Contents (ToC) for all the articles in that issue. Have multiple ToCs on one page to allow me to easily scan a new interesting source that I may have discovered.
12. In many cases, an article not linked from one of the top pages of a site is lost forever (unless discovered by Search). So, a listing of all recent stories (say of the past week, and on a single page) would be very useful.
13. I want my personal newspaper emailed to me daily.
14. It should work in English and other languages. For other languages, it should be able to do translations.
15. It should strip out the news from the story the unnecessary “clutter” (ads, unnecessary images) and focus only on the content of the story, thus reducing the size of the page.

We spend a lot of our time reading everyday – newspapers, magazines, websites. We are in search of news and information. This “information absorption” activity is perhaps the largest consumer of our time after email. The Web allows us to increase manifold the sources of news and information that we can now tap. These sources are not only updating us with what is happening in the world in specific areas, but are also serving as the source of new ideas. It is therefore important to be able to get the news and information without missing anything and from all over, and perhaps faster. Being able to see a lot of sources together can also help us identify trends which we may otherwise have missed. Also, as business becomes more global, it is not enough to just read the local or national newspapers.

We all have our own ways of keeping up-to-date with what is happening. Here is my way of scanning news and managing articles of interest. I just have to read everything possible (written in English) and like to, at least, have a general sense of knowledge of what is happening in most areas. Look at the way I do things. I go to various websites daily (about 6-7), scan the headlines, click on some stories, and if they are of interest, I either read them and/or email the story to myself for later reading and filing. Then, once every few days, I sit for a stretch, read the stories in detail, and make notes which allow me to think and reflect, and then file the electronic stories in appropriate folders (in Outlook Express). This also means I am limited to Outlook Express’ search techniques to search the stories later.

This process helps me cull out new ideas, and perhaps apply what has worked for someone else to my business. It’s a process which is also painstaking. It takes time – going to all these sites and hunting out what I want. Also, if I miss a day, then it is very difficult to get the stories of that day from the Web because the website is a continuum – there is no concept of an edition for a specific day. This means that unless I actually subscribe to the print edition, it is possible to miss out on stories. The print editions have little or no linkage to the website – one cannot, for example, easily look up a story one has just read I the print edition on the web for filing or forwarding.

All in all, even though news is critical for our lives and drives a lot of decisions we make, the way we process news leaves a lot to be desired. One can of course pay for certain services which can allow search but considering that most sites put up their regular editions on the web for free, it should be possible to create a site/service which leverages this to help us derive greater value for news.

Sites like Lexis-Nexis and Factiva provide access to stories, but they all cost a lot of money. Considering that news sites publish freely, it doesn’t make sense to pay for their archives – they actually charge me for the past, which seems kind of counter-intuitive! This forces me to maintain my own filing system. Moreover.com collects headlines from 1700 sources, but is a very primitive form of what we would like. They have done a good job on the “grabbing” and classification. But one needs to go way beyond that. PurpleYogi too has an interesting tool but the free, consumer version is too rigid and limited.

What is needed is a News Refinery – to create a better system to navigate, search and derive value from news.

Continuing the Rays of Hope in today’s India:

11. An increasing number of Indians are returning back from abroad. While some are being forced to do so (due to the slowdown in the USA), there are many others who are doing so out of choice. Even among NRIs, the frequency of visits to India is on the increase. This is a good starting point for building a local base in technology and other areas.
12. Availability of Venture Capital funding has never been better for
    entrepreneurs. Success stories of IP-driven companies like Switch-On
    Networks and Sasken offer a positive outlook for the future. A recent
    issue of Business Today (http://www.business-today.com) has a cover
    story on India’s next big opportunity: BioInformatics.

13. India’s democracy remains its greatest strength, its politicians perhaps its biggest weakness. But there is a new generation of younger leaders coming up who offer hope of doing things differently (and are very media-savvy). The Indian Express talked about some of them in a recent article. Examples: Arun Jaitley, Sushma Swaraj, Pramod Mahajan (BJP); Jairam Ramesh, Madhavrao Scindia, Jaipal Reddy (Congress); Sitaram Yechury (Left Front).
14. India’s free press is a great asset, and along with India’s democracy something Indians can be proud of.
15. There is an increasing recognition of the threat from China among Indian industry. This in itself is a good sign. Businesspeople are realizing that the result of doing nothing will be death. Their answer is not just running to the government asking for greater protection, but looking for partnerships, joint ventures, and even acquisitions.
16. Recognition among India’s foreign policy mavens that a strong relationship with the US is critical to India’s future is an important change. The US has also been quick to reciprocate. During his recent US visit, Jaswant Singh had an unscheduled meeting with the US President George W. Bush, something unthinkable a few years ago.
17. The success of Indians abroad and their desire to contribute back to India is a strong positive going ahead. What India needs is not just their money, but also their contributions in setting up companies locally which can help foster a culture of entrepreneurship.
18. There has been an increase in the number of Indians traveling abroad. As they do so and see the good things abroad, their desire to have the same things in India (the tangibles and intangibles) will make a difference.
19. India’s recent tragedies (the Gujarat earthquake, the Orissa cyclone) drew a huge response in terms of contributions from the international community. This concern and sympathy is a sign of India’s improved standing. Even the domestic and NRI contributions in the wake of the Kargil aftermath were unprecedented.
20. India is now 4th world wide in terms of Purchasing Power Parity, after USA, Japan and China.
21. The media and entertainment industry in India has been growing well. India already makes the largest number of films in the world. What is needed now is for Indian film-makers to create products which can do well internationally – this potential has not yet been tapped. An example is the success of “Crouching Tiger, Hidden Dragon” which grossed over USD 100 million worldwide.

These are but a few Rays of Hope. To build the new India will require a lot more than this – an efforts and sacrifices from every Indian – in India and outside. Change is not going to happen overnight, but over a generation. We owe it to our children to give them an India which is smarter, richer, and more competitive – an India which is a leader in the world.