MetaCrawler

» see metacrawler.com as it lived, on the wayback machine

~ the goodbye ~

metacrawler's good years, roughly '97 and '98, line up almost exactly with my first years online in turkey. but i came into search through a single box, altavista and then very fast google, not through a box that quietly asked five other boxes on my behalf. metasearch was a clever answer to a real problem, and metacrawler got there before any of the names people remember better. it deserves the stone more than most, because it was first.

it started as coursework. Erik Selberg built it in 1994 at the university of washington, with professor Oren Etzioni, as his phd qualifying project, and it went live to the public on 7 july 1995. the idea was plain and, for the moment, correct. no single engine in the mid-90s had indexed much of the web, and they all returned wildly different results for the same query. so metacrawler asked several of them at once, altavista and lycos among them, then merged and re-ranked what came back into one list.

it caught on. about thirty thousand people a day used it in early 1997, and by the middle of 1998 that had climbed past a quarter million. the founders could not figure out how to make money off it, so in january 1997 they sold it to a startup called Go2Net, where microsoft cofounder paul allen later took a controlling stake. Go2Net itself got swallowed by InfoSpace in july 2000 in a deal valued in the billions, back when a search brand was priced like the future.

~ the chronicle ~

~ how it landed ~

metacrawler's whole reason to exist depended on no engine being good enough on its own. once google indexed nearly the entire web and ranked it well, that assumption collapsed under it. asking six weaker engines and blending their guesses was no longer clever, it was just slower and worse. by the early 2000s metacrawler was a brand coasting on a name, the same fate that caught dogpile, which go2net had also owned. in 2014 InfoSpace merged it into another of its engines, Zoo.com, and metacrawler stopped being its own thing entirely.

~ what replaced it ~

the shape did not die, only the product. every time a travel site checks a dozen airlines in one go, or a tool fires the same question at several models and stitches the replies together, that is the metacrawler move, one query fanned out and pulled back into a single view. metacrawler.com still resolves in 2026, parked with the same graveyard company that holds the leftover names of this whole era, serving ads dressed as a search engine. it lies with [[webcrawler]] and [[dogpile]], the engines whose epitaph is that google did their job so completely nobody needed the middleman.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »