GlobalSpec is a domain specific (or vertical) search engine. It got its start eight years ago as a classic IT play – take all the catalog-based information about engineering parts – sensors, transducers, etc. – and roll it into a huge, cross-referenced database, which you then distribute over the web. Make money by connecting customers to parts suppliers. Simple.
Over the years GlobalSpec has evolved into a robust community of a million or so engineering types who use it to find and spec parts. That alone is pretty cool (I mean, a million engineers!). But the coolest stuff was just launched: They call it “The Engineering Web” and it’s a domain-specific crawl of the web for engineering information. And not only have they crawled the web (about 100K engineering related sites, so far), they’ve also surfaced invisible web databases not found in mainstream search engines – patent and standards sites, for example, which are walled off by registration and business considerations. Anyone can use the service – it’s not limited to registered users. In essence, GlobalSpec has built a portal that drives traffic and intent through their original database business, in the process building an intelligent island of engineering information that lives in the public sphere. Of course this means they can add AdWord-like functionality, which of course they are working on.
Here’s why GlobalSpec points to some exciting developments in search. Because of its limited domain, GlobalSpec can use relatively simple keyword-based algorithms to surface lists of ideas or terms related to your search. This allows you to refine your search in ways that simply don’t scale in the Googleverse. These related ideas are inferred from the results of your initial query. For example, if you search on “aerodynamics”, you will get “aircraft , Flight Mechanics, Helicopter Aerodynamics, computational fluid dynamics and Theoretical Aerodynamics” as related searches.
The GlobalSpec guys outlined a useful trio of attributes shared by domain-specific search engines like GlobalSpec. First is organization. This is the basic premise of domain specific engines – through organization comes efficient search. Schneiter calls his engine “parametric” – everything in the index is organized against the standards and parameters of the engineering field, making “parametric search” a reality. Second is context. Domains engines are by definition contextual, but GlobalSpec has a drop down menu next to its search box that allows you to contextualize the search even more, across a bunch of subdomains like Products & Manufacturing, Company Name, Application Notes (engineers care about this), Suppliers, and Standards. And third is access. Vertical search products, by their exclusive nature, can provide domain-specific access to the invisible web, in fact, they can enable commercial transactions that otherwise would be impossible (as GlobalSpec does, see here – at the bottom is the option to purchase an engineering standard from a company with a deep database of standards, a database which cannot be accessed via mainstream search engines). To that end, I’d argue that a fourth attribute of vertical search engines is commerce – these engines enable serious, highly efficient closed loop markets.
Next, Battelle on Topix:
Topix is an internet media play. More specifically, its a local advertising media play. The service takes a crawl-and-index approach to a vast array of internet news sources, then runs the resultant stew through a metadata engine which tags every news story with location and subject data. Topix then builds more than 150,000 topic- and location-specific pages, pages that live comfortably between the great gunky mass of search results, on the one hand, and the impersonal morass of most news sites on the other.
At the risk of getting mired in academic debate, one could argue that Topix is a proof point in the semantic web. Topix is not interested in every web result that might be rendered for a search news kentfield, for example. Instead, it searches a limited set of web pages in this case thousands of news sites and then annotates the content of those pages with semantic tags – latitude and longitude, for example. It then machine-generates something of an encyclopedia page for Kentfield, cobbling together news, weather, advertising, police blotters a local newspaper in real time.
This is a sign of things to come. As the Web gets bigger, the backlash against the monoculture of Google will give rise to specialised, vertical search engines. There is an opportunity to create the next-generation information portal on which vertical portals and marketplaces can be built quickly.