A few weeks ago I had lunch with John Lervik, CEO of Fast Search and Transfer, the company behind alltheweb.com. Alltheweb.com is the consumer show piece for Fast, which is competing with Google for search engine business among big portals like Yahoo and companies like IBM. Sheepishly, I have to admit that I don't test alltheweb.com as much as I should. Like many people I know, I have a tendency to be lazy and stick with Google. But after talking to Dr. Lervik, I'm resolved to give alltheweb.com more of my time.
Dr. Lervik talked to me about what he called the third generation of search technology. (Actually, "third generation" isn't just his term -- many search technologists use the same language.) According to this world view, we have already come through two generations of search.
The first was fixated on keywords. Crawlers, those little online robots that comb the Web, were designed to find pages that contained the same word that was typed into the query box.
The second generation of search tools used what is called "link analysis," in which crawlers not only identified pages with matching keywords but also determined which pages linked to the pages with the matching keywords. The idea was based on the theory of citations -- the more a page was cited, the more likely it was relevant to the searcher. And if a page was cited by an authority (a page that was already ahead of the heap in citations), it was marked as even more useful. Google's PageRank system is best known for this strategy.
The third generation, according to Fast, focuses not only on words and not only on citations, but on the meaning behind those words. The theory goes that if a search tool could actually understand the context of a page, instead of simply cataloging the words upon it, it would be all the more intelligent and all the more useful. This is Fast's strategy. I'm still digesting the specifics, the potential impact and how different this may be from other upstart search companies. Come back later for more about the details of their quest.
9:27:20 PM 
|
