Bulkfeeds

» see bulkfeeds.net as it lived, on the wayback machine

~ the goodbye ~

It started on 30 October 2003 as a directory rather than a search engine, and the footer of every page it ever served carried the date like a build stamp: Bulkfeeds r1752 since 2003.10.30. Three weeks in it listed 13,327 Japanese blog feeds. Full-text search over the items inside those feeds came later, and the mobile version arrived on 10 December 2004.

Tatsuhiko Miyagawa was not doing this for an employer. He had been at On The Edge, which became livedoor, since 2000, and in the same month Bulkfeeds launched he was made an executive officer there; in January 2005 he left for Six Apart. Bulkfeeds sat on his personal domain the whole time, hosted on SAKURA Internet, with AdSense on the front page and an API signup address that went to his own mailbox. It outlived his job at livedoor.

~ the chronicle ~

~ two weeks of memory ~

The thing to remember about the design is that it forgot on purpose. Feeds arrived from the blog hosts' own update streams and from a weblogUpdates ping endpoint anyone could call, with plain manual registration as the fallback. The crawler went round every fifteen minutes. Then the site policy took over, which was that anything older than two weeks got discarded from storage so it would stop turning up in results. A search engine for the Japanese blog boom with no memory of most of the boom.

What it gave away was unusual for 2005. The API was free and documented, with no rate limit and no key required for the core search. You could query feeds and get RSS or OPML back, run full-text search over items, ask for pages similar to a permalink, report a splog, or push arbitrary text at an endpoint called terms.xml and get morphological analysis and feature terms scored against the whole corpus. There was an OpenSearch descriptor and a DNS blacklist for spam blogs on separate infrastructure.

The front-page counter read 6,574,312 feeds registered and 7,172 items indexed. Against the figures from May 2005, the feed count had grown more than sixfold and the item index had fallen by a factor of around 450. The crawler was still collecting addresses and had very nearly stopped reading them.

~ what the api was for ~

The part of this that reached outside Japan was FeedDemon. It was the Windows aggregator a great many people were running around 2005, and one of the ways you extended it was by dropping an XML file into a folder called Data\Search to add a search channel. Bulkfeeds shipped as one of those files. So somebody on a broadband line in Turkey could have had a Japanese hobbyist's index wired into their reader and never once looked at what it was.

The heavy consumers had left before the end. GREE used the auto-discovery endpoint to pull a feed URL out of any blog address, and Blogmap took the new-feed stream while Bulkfeeds took Blogmap's rankings back for its own buzzwords page. Both were listed on the applications page in 2004 and both had been commented out of it by March 2006, sixteen months before the hardware went. The thing still pointing at the corpse afterward was not a company's product. It was Miyagawa's own Plagger plugin for reporting spam blogs to a blacklist that no longer existed. It is still on CPAN.

~ a hardware problem ~

On 6 July 2007 the archive still holds a working Bulkfeeds, 2,212 bytes of it. By 28 July the server was answering 403 with 293 bytes in the body: sorry, it is under a temporary maintenance due to a hardware problem, we are working hard to fix this soon. Nothing after that. The announcement feed on Miyagawa's blog, still being archived four years later in 2011, ends on 15 December 2005 with a post about a Google Homepage search module, so there is no closing statement to find because none was ever written.

By January 2012 the domain was a for-sale page, and today bulkfeeds.net answers from two different addresses depending on which one you draw. One of them serves 244 bytes explaining that the IP is shared among many domains. The other serves 7,774 bytes of invented blog prose, a day at the library and some thoughts on Buzz Aldrin, with webcam affiliate links threaded through it under a footer reading copyright Bulkfeeds.net. A service built to index other people's writing now hosts writing that nobody wrote.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »