AWS Snowmobile

» see aws.amazon.com/snowmobile as it lived, on the wayback machine

~ the last words ~

This was a product for organizations that had an exabyte problem, and I have never worked anywhere near that room. The engineering I do generates data continuously and all of it has always fitted through a wire. Snowmobile belongs to a tier of computing that ordinary practitioners only ever see as a photograph in a keynote recap, which is the honest distance here: not a matter of region or era, just a scale of company I have never been employed by.

Amazon announced it on 30 November 2016, at the end of the re:Invent keynote, by having a 45-foot shipping container towed onto the stage. Inside was storage, roughly one hundred petabytes of it, in a ruggedized and climate-controlled box measuring about eight feet wide and nine and a half feet high. It connected to the customer data center over multiple high-speed links, filled up over a few weeks, and then drove to the nearest AWS region to be unloaded into S3.

The security arrangements read like a bullion transfer. GPS tracking throughout, with alarm monitoring and video surveillance running the whole time. Dedicated personnel traveled with the vehicle, and an escort car was available if the customer asked for one. Everything on the disks was encrypted with keys the customer controlled. The threat model was not somebody hacking your data, it was somebody stealing a truck.

It did get used. DigitalGlobe, which had sixteen years of high resolution satellite imagery adding up to about a hundred petabytes, loaded the whole archive onto one in the first quarter of 2017 as part of closing down its own data centers. That was the case the product was designed for and it is the case that keeps appearing in every retelling, partly because it is a great story and partly because there were not many others.

~ the record ~

~ the physics ~

Andrew Tanenbaum wrote the line everybody quotes about a station wagon full of tapes having enormous bandwidth, and Snowmobile was that joke funded and insured. A truck has terrible latency and preposterous throughput, and if your data set is large enough the throughput wins by a margin nothing else can touch.

AWS did the arithmetic in public. Moving an exabyte over a dedicated ten gigabit connection takes about twenty-six years. With a fleet of Snowmobiles it takes about six months. Once you accept those two numbers there is nothing left to argue about, which is why the product never needed a sales pitch beyond a photograph of the container.

~ the funny part ~

It was billed like storage, by the gigabyte and by the month, so on the invoice it sat next to the S3 buckets as though it were the same kind of object. It was not. It weighed tons and it had a driver, and it drew enough power that the customer site either supplied a feed or had a generator brought along with it.

The marketing value ran far ahead of the revenue. Every technology publication on earth ran the photograph, and for a few years the truck was the shorthand for how big the cloud had become. The number of organizations that could actually place an order was small enough that AWS never published it.

~ the actual end ~

Nothing was announced. In March 2024 the Snowmobile pages came off the AWS site and the old links started redirecting to Snowball, the briefcase-sized version that ships by courier. The press noticed in April and asked, and Amazon confirmed it had stopped offering the service, saying that everything released since 2016 had made moving data easier and that customers were choosing the newer transfer tools instead.

The plainer version is in the same coverage: customers worked out that uploading was usually cheaper than hiring a truck. Network capacity got cheap faster than corporate archives grew, and the window where a semi trailer was the rational choice turned out to be about seven years wide. What is buried here is not a service anybody misses. It is the last moment when the physical size of data was something you could see from across a parking lot.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »