DMOZ

» see dmoz.org as it lived, on the wayback machine

~ the last words ~

the open directory project was the volunteer reaction to yahoo's directory turning into a paid-listing business. it started in june 1998 as gnuhoo, a side project by rich skrenta and bob truel at sun microsystems. the gnu project objected to the gnu prefix. the team renamed it newhoo. five months in, netscape bought the project, renamed it the open directory project, and put the data under a license that any search engine or portal could use freely.

the deal was the engine of dmoz's first decade. google used dmoz data for its own directory listings. lycos and ask jeeves used it. dozens of smaller portals shipped a category browser that was just a skinned dmoz. the volunteer editor base ballooned. by 2007 there were around 75,000 contributing editors. by 2013 the number was 97,584, with five million sites listed in over a million categories.

and yet the directory model was already obsolete by then. google had won search by 2003. nobody was browsing categories anymore. the value of a dmoz listing for seo cratered with each algorithm update that demoted directory backlinks. submission queues grew while editors quit, and the politics around editor permissions calcified. aol kept the lights on more from inertia than commitment, and on march 17, 2017 aol turned them off.

dmoz was a name i ran into through seo discussions on turkish webmaster forums in the broadband era. donanimhaber and frmtr threads about how to get a site listed. the consensus was always the same: submit and wait six months. emailing the category editor was pointless because nobody would answer. i never tried to become an editor myself. the application process had its own reputation.

~ the record ~

~ how it actually worked ~

dmoz organized the web into a tree of categories. an editor was a volunteer responsible for a specific branch. you applied to edit a category. the existing editors reviewed your application, and if accepted you got the right to add and remove sites within that branch. the bigger branches had hierarchies of editors and meta-editors with escalation paths for disputes.

that structure scaled remarkably well for a few years and then stopped scaling. high-traffic categories attracted editors with vested interests in promoting or excluding certain sites. accusations of corruption became a chronic background noise. the meta-editors tried to police it but the policing itself became contentious. by 2010 most active branches had a faction problem.

~ why it stopped scaling ~

the open directory was a 1998 idea about how to organize the web. it assumed the web could be classified hierarchically and that classification would be useful to readers. neither assumption survived the 2000s. wikipedia replaced reference and google replaced lookup. blogs and rss replaced the kind of webmaster site that wanted a directory listing in the first place. by the time aol pulled the plug there was nothing the directory was actively useful for, only a permissive data dump that other people could fork.

~ the long death ~

after the shutdown a group of former editors launched curlie.org with the same hierarchical structure and most of the dmoz data. it is still online and still maintained, just at a much smaller scale and with a tiny fraction of the original editor base. the dmoz data dump remains downloadable from archive.org, which is how seo tools that needed the original taxonomy kept working through the 2020s. the open directory's permissive license is the reason its bones are still everywhere.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »