Blogdex

» see blogdex.net as it lived, on the wayback machine

~ the eulogy ~

Blogdex asked a question that was only answerable for about four years: what is the internet passing around right now. Cameron Marlow, then a PhD student at the MIT Media Lab, built a crawler that watched every weblog it knew about, collected the links each one made since the last visit, and ranked the results by how fast they were spreading. The front page was that list. Nothing else.

He built it with Elizabeth Wood, a high school student who was at MIT for a summer research program, which is a detail worth keeping because of how small the whole thing was. Jimmy Wales handed over the blogdex domain names for free. The earliest copy of the front page in the archive is from July 2001, sitting at blogdex.media.mit.edu, and the project's own description at the time compared a hyperlink to a tag clipped to a wild animal so you can follow where it goes.

None of this crawl started from Turkish. Whatever was happening on the Turkish side of blogging in those years was happening on blogcu and in forum threads, and it was invisible to an index whose seed list came out of English-language blogrolls. That is not a complaint about Blogdex, it is what a crawler is: an argument about where to start.

~ the dossier ~

~ how it worked ~

The mechanism was deliberately narrow. Blogdex did not read your posts or try to work out your topic. It counted links and weighted them by how recent they were, and the ranking was the whole output. Marlow's stated design rule was to do one thing and do it correctly, borrowed openly from the way Google had won its argument, and the result is that Blogdex was legible in a way its successors were not. You could look at the number and know exactly what it meant.

It also exposed something nobody had a word for yet. Once a list of the fastest-spreading links exists in public, it becomes an input to the thing it measures. People checked Blogdex in the morning and then wrote about what was on it. Marlow ended up studying that feedback loop rather than only building the instrument, and he later went to Facebook to work on exactly this question at a scale where the loop is the entire product.

~ what came after ~

Daypop and Technorati arrived in the same window doing variations on the same count, and by 2005 the idea had escaped research entirely. Digg turned the ranking into the site instead of a report about the site. Reddit did the same with a different vote. Every feed anyone reads in 2026 is running a version of this loop with the arithmetic hidden.

Blogdex itself just stopped. It had been shaky for a while, the crawl could not keep up with a blog population growing faster than a single research machine, and in May 2006 it went offline. There was no post explaining it, which for a project whose entire purpose was tracking what got announced is a quiet kind of ending. blogdex.net does not resolve at all now.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »