PubSub

» see pubsub.com as it lived, on the wayback machine

~ the last words ~

PubSub had nothing to search. There was no archive behind the box and no crawl sitting on disk waiting for you. You typed a query, and instead of running it against a store of documents, the service kept the query and ran every document that arrived afterward against it. Its own word for this was prospective search, and the plain version is that an ordinary engine answers about the past while this one answered about whatever happened next.

Bob Wyman and Salim Ismail started PubSub Concepts in New York in 2002 on about $2.5 million of angel money. The matching engine was the asset. It claimed more than three billion query matches a second, which is the number you need when a hundred thousand standing subscriptions have to be checked against every blog post the moment it pings. The inputs were deliberately odd: blogs and press releases first, then Usenet, SEC filings, USGS earthquake alerts and the FAA flight delay feed.

By 2005 it looked like a real company. It shipped LinkRanks in September, a measure of how fast a blog was gaining or losing links rather than how many it had, and it signed Business Wire that June to push press releases into the system as they crossed the wire. The subscription output arrived as a feed, so anything that read RSS could carry it.

The two founders never got on and the argument stopped being private. Ismail lost the fight and was pushed out as chief executive in March 2006. Gus Spathis took the chair. On 6 June that year the staff were laid off in a single day, with a last-minute merger with KnowNow collapsing in the background and roughly $3 million of debt on the books. The site went dark on 15 January 2007.

Prospective search asks something of you before it gives anything back: you have to already know what you are watching. That is a job description more than a preference. A public relations officer watching a brand name has a query worth leaving somewhere for a year, and so does an analyst watching a ticker. In 2005 I was a student, and a student's questions arrive one at a time and get answered the same afternoon. There was nothing I needed to be told about the instant it happened.

~ the record ~

~ searching forward ~

The inversion is easy to state and was hard to build. A normal index is optimized to find documents fast given a query. PubSub had to find queries fast given a document, and it had to do it while holding every subscription any user had ever left in memory at once. That is a different data structure and a different cost curve, and the company's patents were the actual thing on the balance sheet.

The idea did not die with the company. A saved search that emails you and a webhook that fires when a document matches are both the same shape, and by the early 2010s that shape had stopped being a product and become a feature of everything. Wyman ended up at Google working on an internal system that also carried the name PubSub.

~ three billion matches a second ~

The number was in the marketing and it was doing real work there, because the pitch was latency. Google indexed the web on a crawl schedule and a page could sit unseen for days. PubSub was wired into the ping infrastructure that blog software already used to announce itself, so a post could be matched and delivered before the writer had finished rereading it.

That speed was the argument for the odd inputs too. An earthquake alert or a flight delay is only worth anything in the first few minutes. Putting them next to blog posts in one engine looks scattered on a feature list and it was the demonstration: any stream of documents with timestamps could go in, and the same matching machine handled all of it.

~ the founders stopped speaking ~

This is the part that finished it, and it is unusual only in how visible it was. The dispute between Wyman and Ismail ran into public posts and comment threads while the company was still selling. TechCrunch's June 2006 write-up drew a correction from Wyman in its own comments the same day. Conference talks about crisis communications used the whole episode as material afterward.

No product failure caused any of this. The technology worked and the deals were real, and the market for real-time alerting turned out to be enormous once other people got to it. What ran out was the ability of two people to sit in the same room, and there is no funding round that fixes that.

~ the second pubsub ~

Ian Bell's company Something Simpler bought most of the assets in 2007 from the investors who had taken control, along with a million dollars of new money, and told the press it planned to relaunch inside six months as a friendlier version of Yahoo Pipes. The site came back on 15 August 2007 and the rebuilt version, PubSub v2.0, went public on 18 November 2009.

Almost nobody noticed the second one, which is the part worth recording. By 2009 the standing-query idea had been absorbed by every service that could email you, so a site whose entire proposition was that idea had no proposition left. It ran quietly for another four years and stopped in August 2013.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »