Wikia Search

» see search.wikia.com as it lived, on the wayback machine

~ the obit ~

the first time i heard about wikia search was a tech blog item in early '08. jimmy wales had announced an open-source search engine that would let users vote on result rankings and edit the algorithm's behavior in the open. the framing was wikipedia's framing applied one floor down the stack, search instead of encyclopedia. i clicked the link and ran a query. the results ranged from bad to embarrassing. by '08 google was the only search box i used regularly, with yahoo on the side as a backup. live search and ask were still alive but losing share monthly.

wales launched the public alpha on january 7 2008 at search.wikia.com. the indexed corpus was thirty million pages, a fraction of a percent of what google was indexing at the time. the pitch was that the algorithm would be transparent and the rankings would be editable by humans, a wiki layer on top of crawled web. the catch was that ranking signals are exactly the part of search engines that work because they cost a billion dollars in infrastructure to compute correctly. the open editing layer did not fix bad rankings, it surfaced spam faster.

the numbers came in immediately and they were brutal. by august the engine held a 0.000079% share of US search against google at 70.77%. ten thousand unique users a month over the alpha lifetime. wikia inc. was running the project on outside investor money and the recession had hit. on march 31 2009 wales posted that wikia search was not making its numbers. the servers stayed up another six weeks and went off on may 14.

~ the rap sheet ~

~ how it landed ~

the wikia search idea was good on a whiteboard. wikipedia had worked, google's pagerank was an opaque black box, the obvious next move was a search engine where the algorithm sat in the open and the community improved it. the practical problem nobody on the project solved was that quality search depends on crawl coverage and index freshness more than on ranking philosophy. wikia search crawled a thin slice of the web and ranked it with a thin slice of signals. by the second query you noticed.

~ the cracks ~

google's reaction was not to react. the public alpha got polite coverage in the tech press, then nothing. wikia search did not pull any users from any incumbent. the people who clicked it through were tech reporters and search nerds writing comparison posts, who almost universally concluded that the search results were not good. wales kept the project going for fourteen months on a slowly shrinking budget. in march '09 the wikia board pulled the plug as part of a broader recession-era cost cut at the company.

~ what survived ~

the lesson, which the tech press wrote up and then nobody learned, is that crowdsourcing does not substitute for capex when the product is something like web search. wikipedia worked because the unit of contribution (a sentence in an article) had clean ownership. search ranking does not break down that way. the inheritance of wikia search is mostly a footnote in jimmy wales' career and a cautionary chapter in books about why google's search monopoly held. wales went back to running wikipedia. wikia kept growing as fandom.com. the search domain reverted to wikia and quietly redirected away.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »