Making Connections

Dan Gillmor writes in his column in the San Jose Mercury News entitled Building harmony through the Internet about communications and collaboration as the two technologies that are defining the Internet: Digital hardware, software and communication technologies are creating, as one technologist puts it, a kind of “power steering for human interaction.”

Online Communities

Internet navigators think small: An article on which talks about the formation of web communities. One of th questions it tries to address: “The Web follows a power-law distribution meaning there are a lot of sparsely connected sites and a very small number of highly connected sites. Can the rise of small-scale communities, such as those that grow up around Weblogs, change that picture?”

The web communities are already there…its just that we cannot see them, because we are at “street level”. Rise up a little, and we’ll already see the pockets existing, exemplified in the weblogs, their blogrolls and links.

Blog Experiment

Mark Bernstein’s experiment on simulating blogger behaviour. He has created 600 artificial bloggers (in software) and is checking out impact of various factors on bloggers. Initial conclusions: “Responsible webloggers need to avoid getting into a rut of reading and citing the same sites all the time. Reading widely, and changing your weblogging habits often, are the best ways to keep your own weblog and the entire weblog community fresh and lively.”

It’s a good point. Even while blogging, we all get into a comfort zone — the same blogs, the same look, “friends”. This needs to change every once in a while — we need to expose ourselves to new ideas, like reading a new book. Since blogging mirror our interests, it means bloggers need to come out with innovative new thoughts every once in a while to avoid becoming repititive.

BlogStreet Beta

We’ve made available our blog directory and search engine at Its a very, very preliminary effort, and there will be a lot of things which we will be doing in the coming weeks. But, it’s a start. We’ve cataloged about 150 blogs so far, but this number will go up in the coming days.

A word on the search as it is now: we bot the top pages of the blogs cataloged and do a keyword search on it. Even though this is quite primitive, it can be quite helpful to track what bloggers are talking about — unlike Blogdex and Daypop which focus only on the links. We’ll also be adding our own link analysis soon.

The view of BlogStreet in the coming months is to create a site which can help in identifying clusters of bloggers and people. We want to use some of the ideas of “Emergence” and see what can come out of these applied to blogs.

Some initial thinking: Blogs have a combination of “friends” (blogrolls), their entries (the blog posts) which have a mix of links and words, and in some cases, comments. Google looks at the pages as a whole, and its PageRank technology does not (at least from what I have read) distinguish between the links on a page.

In the context of Blogs, this differentiation needs to be done: links at the top of a BlogRoll are more “friendlier” than the ones at the bottom. Also, stories “decay” as one moves lower down the index page — they become older. A search engine for Blogs will need to keep in mind the unique structure of blogs, and make use of the archive pages also (with the permalinks).

Blogs are what the Web should have been in the first place: about people talking about their interests and areas of expertise, on a regular basis. Its been a fascinating ride for me, personally, in the last few months because blogs (more so, bloggers) have helped bring about a richer understanding of the various happenings and innovations that are going on around us.

Blogs give the underdogs a chance to get known — using the power of ideas and knowledge. Websites never did create a level playing field; Blogs can.

Blog Predicament

Steven Johnson captures the challenges facing blogs and bloggers (this is again from his speech at the O’Reilly Emerging Technologies conference) — the note is via Doc Searls:

  • Problems forming higher-level groups
  • Managing overpopulation
  • The tyranny of time
  • Little passive organization
  • Readers lack input

    Finding solutions to the above is what we hope to do via BlogStreet.

  • Blogs and Cities

    An interesting discussion by Steven Johnson at the O’Reilly Emerging Technologies conference, according to Rael:

    What are cities good for? Clusters. Public Spaces/Diversity: you need to plan that these spaces will remain public; you need those unexpected, entertaining things to happen; top-down planning and bottom-up participation. Optimal Density: short blocks (related to the small pieces, loosely joined idea); not have areas be too hot or too cold; Chelsea, NY’s long blocks funnels the vast majority of the foot traffic to large avenues and limits this ways you have of getting from one place to another.

    So how do cities do it? Bottom-up interactions. Strollers create as much as shopkeepers: analagous to blogging. Passive organization. The swerve.

    The Blog Predicament. Problems forming higher-level groups: if you follow links, you can just start to see how they’re related to one-another, but it’s not obvious. Managing overpopulation: pretty soon I’ll have 500 blogs I follow and have trouble managing that.

    This is a follow-on from Johnson’s article in Salon, which I discussed earlier.

    These are very interesting ideas. If you, like me, believe that blogs are the start of something new and big on the Internet, then it will need a different set of tools, search engines and navigators. For the first time, we have tens of thousands of people talking about what they like and dislike, acting as filters on the web, putting forth their interests, and making connections — just like our brain does.

    Searching Blogs

    One of the things I’ve been thinking about is the effectiveness of searching in Blogs. Search for a phrase specifically within blogs on Daypop or Blogdex and it shows the link to a page. Problem then is searching for the entry on the page. Since blog pages (especially the index pages) are dynamic, the entry may even have disappeared from the page (into the archives).

    The problem is the notion of Search on a Page. This has to change for Blogs — the granularity is a single post, an Entry. That is what the search results should be ferreting out.

    Emergent Blogs

    Steven Johnson, author of Emergence, writes in Salon about blogs and their potential to provide an alternative search-cum-filter into information: “The collective future of blogs lies not in dethroning the New York Times — but in becoming a force that can make sense of the Web’s infinity of links.”

    He talks about how blogs could perhaps dethrone Google for finding useful information. He says: “The beautiful thing about most information captured by the bloggers is that it has an extensive shelf life. The problem is that it’s being featured on a rotating shelf.”

    We’ve been doing some thinking of our own as part of BlogStreet over the past month or so. Blogs are very interesting because there is much more structure to what’s being written: there are pages (index, archives, comments, categories) which have posts (which can have links, comments and a permalink) and blogrolls (links to other blogs).

    There are “Hublogs” — what Johnson calls as “guardian angels”. These are at the centre of “Interest Clusters”. Blogrolls and the story links are the spokes emerging out of the hublogs. Today, blogs are like independent ants — everyone doing their own thing. But there are patterns forming. These are dynamic. Looking at the blog-level, we see some connections emanating out of a blog.

    But go up a level, and perhaps the world of blogs will look just like neighbourhoods in cities, with highways, avenues and bylanes making the connections. The neighbourhoods are the ones which we either live in or visit, depending on our interest. This is emergenc at work, where the whole is much greater than the sum of the parts.

    The challenge is to build a blog indexing and search engine which recognises these connections, and automatically forms the clusters among blogs. The feedback also comes in from readers who click on various links and strengthen or weaken the connections.

    The way to look at this is to separate the process into two: take the blog pages and then represent them into a standardised format (perhaps as nodes and links). Then, apply existing mathematical theory to see which nodes are “stronger” or more central, and how “thick” are the links and what is their direction.

    Do this, and we’ll have a very different insight into the information on the web (and the people out there). Because blogs represent people, we will find ourselves coalescing around a certain set of people (our favourite blogs). Websites could never do this because they represented collections of views from different journalists. Most blogs are focused because they reflect a single individual. This will also help us find other people like us, content and opinions on information we are more likely to read. If all this sounds like collaborative filtering, it is — one of the possible by-products of the link analysis and blog lists.

    What does it all mean for Readers? One view is projected by Johnson:

      You define a few “guardian” Bloggers, perhaps by checking a box when you visit their site. You also instruct your software to watch the activity on sites maintained by “friends” of those key bloggers. You tell the software that you want a medium level of intrusiveness: In other words, you want the system to point out useful information to you, but you don’t want it constantly bombarding you with data at every turn. And then you start using your computer as you normally do: surfing, writing e-mail, drafting Word documents.

      Behind the scenes as you write or read, the software on your machine scans the last few paragraphs for high-information text, the six or seven words that make that paragraph distinct from the average paragraph sitting on your machine. If there’s a URL included in the text, it grabs that too. The software then sends a query to the blogs maintained by your guardian Bloggers, as well as those maintained by their friends — say 20 blogs in all — and searches for posts that include those keywords….Let’s say Jason Kottke has linked to a related article; if four other bloggers you’re following have also linked to that URL, Jason’s description of the article pops up beside the paragraph you’ve just written.

      This wouldn’t be a recommendation engine so much as a connection machine, tracking the flow of words across your screen and linking them fluidly to other text residing on the Web.

    Bloggers are, in a sense, information filters. The additional advantage is that they have their own opinions and insights. This leads to new ideas and innovations in a way which would previously have been unimaginable. There’s much more to Blogs and Bloggers than we’ve perhaps thought of as so far. Johnson gives us a peep into a possible future. We need to out there and build it.

    Blogging in the News

    Three articles on Blogging in the past few days in the new media — two in Salon [1 2], and one in Wired.

    Writes Scott Rosenberg in Salon [2] in a very balanced article on the debate between bloggers (amateur journalists) and pros (professional journalists):

      Blogs can do some things the pros can’t. For better and worse, they air hunches and speculations without the filter of an editorial bureaucracy (or the legal vulnerabilities of a corporate parent). They trade links and argue nuances, fling insults and shower acclaim. The editorial process of the blogs takes place between and among bloggers, in public, in real time, with fully annotated cross-links. This carries pluses and minuses: At worst, it creates a lot of excess verbiage that only the most fanatically interested reader would want to wade through. At best, it creates a dramatic and dynamic exchange of information and ideas.

    Blogs are only as good as the person writing the blog. Blogs are about people, in most cases, a single individual with views to express. Bloggers may lack the variety and breadth of mainstream media, but they do have depth in specific topics which they use to articulate their viewpoint and present a lens on the world.