ACM Queue had a special issue dedicated to Enterprise Search in April. One of the articles (by Anna Patterson) talked about the difficulties in writing one’s own: “There must be 4,000 programmers typing away in their basements trying to build the next ‘world’s most scalable’ search engine. It has been done only a few times. It has never been done by a big group; always one to four people did the core work, and the big team came on to build the elaborations and the production infrastructure. Why is it so hard? We are going to delve a bit into the various issues to consider when writing a search engine. This article is aimed at those individuals or small groups that are considering this endeavor for their Web site or intranet. It is fun, but a word of caution: not only is it difficult, but you need two commodities in short supplytime and patience.”
Anna should know – she has written two search engines.