BlogScope

» see blogscope.net as it lived, on the wayback machine

~ the obit ~

BlogScope was a search engine for blogs that answered a different question than the others did. You could type a term and get posts back, ranked by relevance or by what the system called influence. What it was actually built for was the shape of a term over time: a popularity curve with the bursts marked on it, next to the other words that spiked in the same hours. It called that correlation discovery, and the demo video on the about page led with it.

It came out of the Department of Computer Science at the University of Toronto, started in 2005, run by the professor Nick Koudas with the PhD student Nilesh Bansal and a rotating group of students and collaborators. The output was academic, and the paper list on its own site runs through VLDB in 2007, 2009 and 2010, WWW 2007, SIGMOD 2008 and WSDM 2009. One of the last ones was called "Remembrance of Things Past: Towards Computational History".

The index got big. Wikipedia records over 725 million posts by April 2009. The final version of its about page claims 56.92 million blogs and 1.307 billion posts in the database, with a Firefox search plugin offered right underneath the figure.

~ the rap sheet ~

~ the machine ~

The about page described the hardware without any coyness: Java, running on four Sun V40z servers under Red Hat Linux AS4. A multi-threaded crawler with a spam analyzer in front of it, then indexing and search, then a statistics layer, a popularity curve generator, a correlation module and a natural language processor. Underneath it was Lucene, MySQL, Apache and Tomcat, with the browser interface drawn in Dojo loaded from AOL's content delivery network.

That is four rack machines holding the searchable part of a billion blog posts in 2011. The entire public conversation that everybody was worried about fit in a university lab's budget line.

~ reading distance ~

In 2008 I was on TTNet broadband, reading donanimhaber and maybe a dozen blogs through a feed reader. A system that ranked a billion posts by influence was built for researchers and for brand monitoring teams, not for someone with twelve subscriptions. The part of the blogosphere I was actually in was small enough to follow by hand, and most of it was not in English anyway.

~ the spinout ~

Sysomos was founded in 2007 as a spinoff of the project, by Bansal and Koudas, selling the same intellectual property to enterprises as social media monitoring. Marketwire acquired it in July 2010. BlogScope carried on separately at the university after that, as the free research version of a thing that now had paying customers somewhere else. It ran in that position for another twenty-one months.

~ the notice ~

On 4 April 2012 blogscope.net was still serving the full search interface, dropdowns and advanced options and all. By 15 April the whole site had been replaced by a page of about 900 bytes: the wordmark with the magnifying glass in it, then one heading, "BlogScope has been discontinued."

Under the heading, two sentences explaining that it was started in 2005 as an academic project at the University of Toronto and had since been channeled into commercial products at Sysomos. Then: "Thank you for visiting." That page outlasted the search engine by years. The logo file next to it was still being crawled in December 2013.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »