Yahoo Pipes

» see pipes.yahoo.com as it lived, on the wayback machine

~ the eulogy ~

yahoo pipes launched on 7 february 2007. it was a browser-based visual programming environment for combining web feeds. you opened the editor and saw a blank canvas. on the left was a palette of modules covering feed fetching, filtering, sorting, regex operations, url building, http requests, and location parsing. you dragged modules onto the canvas and drew connections between them. each connection was rendered as a literal pipe. the output of the pipeline was a feed in rss or json that you could subscribe to or embed elsewhere. the tool was named for unix pipes and the resemblance was not accidental. pasha sadri's team built it explicitly as a graphical analog to the command-line tradition of stitching small tools together.

the timing was the rss-everywhere moment. blogs and news sites both published rss. flickr published rss for tag searches, and yahoo's own search results came back as feeds too. pipes let you take any of those feeds, filter them, transform them, merge them with other feeds, and output the result as a new feed. you could build a personal aggregator that watched ten sources and surfaced only items mentioning a specific keyword. you could combine the rss output of a craigslist search with a google maps geocoder and produce a map of new listings in your city. the example pipes shipped on the gallery showed that range and made the system feel like a serious tool from day one.

yahoo never made money on pipes. it was free, it required server-side resources to run user-submitted pipelines on schedules, and there was no obvious advertising surface inside a tool used mostly by developers. the company moved pipes into reduced-investment mode around 2009 after the bartz-era reorganizations, kept it running with a small team, and finally announced shutdown in june 2015. the editor went read-only on 30 august 2015. the runtime stopped executing pipes on 30 september 2015. for many power users the loss was bigger than the user count would suggest, because most of them had pipelines doing daily work that no other tool could replicate.

i used yahoo pipes a couple of times in graduate years when i was building feed aggregators for my own reading. the visual canvas felt like the closest thing to a spreadsheet for the live web. i remember thinking that it should have been the model for a generation of consumer programming tools and that yahoo did not seem to understand what it had. by 2015 i had moved off rss for most reading anyway, but i remember reading the shutdown announcement and feeling like a small piece of the engineer-friendly web had ended.

~ the dossier ~

~ what it was ~

the editor was the product. the entire tool ran in a single browser tab, no install, no account beyond a yahoo id. modules had typed inputs and outputs that prevented you from wiring incompatible types together. each module had its own configuration ui inside the canvas, so a regex filter showed a pattern field directly in the node. running a pipe streamed the output below the canvas in real time, which was the real magic: you could see the data flowing as you built the pipeline, debug by clicking through nodes, and tweak filters with immediate feedback. very few visual programming tools before or since have given that feedback loop with that little ceremony.

the published-pipe gallery was the second product. once you saved a pipe, it had a public url. other users could clone it, modify it, and republish their own version. that meant pipes accumulated a library of starter pipelines for common tasks. how to merge multiple craigslist feeds. how to watch a wikipedia page for changes. the gallery was its own learning resource, and beginning users often started by cloning a working pipe rather than building from scratch.

~ why power users loved it ~

the unix-pipe metaphor was the thing. for engineers who lived on the command line, pipes mapped a familiar mental model onto the live web in a way that no other tool ever quite matched. the inputs and outputs were typed so the operators composed cleanly, and the result was a hosted feed url anyone could subscribe to. the tool turned the open web into a data plane that you could query and reshape, without a server of your own.

the second thing power users loved was the extension surface. a custom http-fetch module could call any rest api. a custom regex module could parse arbitrary text. a yql module (yahoo's web-database query language) let you treat the entire web as a sql-style database for simple queries. those building blocks combined into pipelines that read more like programs than configurations. for a class of user who wanted to automate small web workflows without managing a server, pipes was unmatched at the time.

~ what replaced it ~

ifttt and zapier ate most of the use cases. ifttt launched in 2011 with a single-trigger-single-action model that was cognitively simpler than pipes' graph-based canvas, but also strictly less powerful. zapier extended the same idea with multi-step "zaps" and paid tiers. neither tool ever supported the kind of branching and merging pipeline that yahoo pipes routinely handled. for casual users that did not matter. for the original pipes audience it always did.

the modern descendants are scattered. node-red carries the visual-programming dna into iot and home automation. n8n and pipedream offer pipeline-style automation with code escape hatches for engineers. none of them are pipes. none of them have the gallery effect or the casual sharing pattern that pipes pioneered. the closest thing in spirit might be the one you build yourself in a saturday afternoon when you realize the alternative is paying $20 a month for an integration service.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »