Tailrank

» see tailrank.com as it lived, on the wayback machine

~ the eulogy ~

Kevin Burton built Tailrank to solve a problem that only existed for about four years. Between roughly 2005 and 2009 there were enough blogs publishing enough posts that no human could read the relevant slice of them, and not yet enough centralization for a single feed to tell you what mattered. Twitter was barely open. Facebook was for students. The place where opinions were being formed was several thousand independent sites linking at each other, and nobody could see the whole shape of it.

Tailrank went into alpha on 21 September 2005, announced on TechCrunch under the headline "A Tool for the Long Tail". It crawled the blogs registered with it and watched which outside stories they were citing, then ranked those stories on two variables. Recency pushed new items up. Citation count pushed popular ones up. Underneath every entry it listed the blogs that had linked to it, so you could see the shape of the conversation and not just its winner.

I was at university for most of its run, reading feeds the way an engineering student reads feeds, which is to say badly and too many of them. The memetrackers were genuinely useful for that, because they gave you an excuse not to open the reader at all. The catch for anyone outside the English-speaking web was that these things indexed only the blogs they had been told about, and the Turkish blogosphere of 2007 was not on anyone's crawl list. Whatever donanimhaber and the personal blogs around it were arguing about that week, a memetracker in San Francisco had no idea.

~ the dossier ~

~ the math ~

The interesting thing about a memetracker is that it is a citation index wearing a news site's clothes, which is why the good ones were built by people who understood crawling rather than by people who understood editing. Burton's crawler was the actual asset. It had to fetch a very large number of feeds very often, work out what was new, resolve links to canonical targets, and do all of that fast enough that the front page meant something at 9am.

That infrastructure became Spinn3r, a firehose of blog and social content sold to researchers and companies. The ICWSM 2009 conference ran on a Spinn3r dataset of 44 million blog posts collected between August and October 2008. The site people visited died. The pipe that fed it did not.

~ what it was up against ~

Gabe Rivera's tech.memeorandum.com had launched a few days earlier in that same September, and it renamed itself Techmeme the following year. For the first stretch all three of them were fully automatic, and Tailrank and Megite traded second place behind it. Then in 2008 Rivera hired a human editor to sit on top of the algorithm and clean up what the scoring got wrong, which looked like an admission of defeat and turned out to be the winning move. In a field producing a few hundred stories a day, one person's judgment beats a ranking function by more than anyone wanted to admit.

tailrank.com resolves today to a WordPress site full of generic filler about interesting places to visit. Whoever took the domain kept the page title, so the tab still reads "Ranking the World One Blog Post at a Time" above content that ranks nothing at all.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »