SDCH

~ the goodbye ~

Gzip compresses a response against itself. It finds repetition inside the file it is given and nothing outside it, which means that every page on a site ships the same navigation and the same inline template over the wire again, compressed just as well and just as uselessly as the last time. SDCH was the attempt to fix that gap. The server published a dictionary of the strings its pages actually share and the browser downloaded it once. After that, every response could be sent as a delta against that dictionary instead of as a whole document.

The proposal came from Google and carried four names: Jon Butler, Wei-Hsin Lee, Bryan McQuade and Kenneth Mixter. It went to the IETF HTTP working group's list in September 2008, the same autumn Chrome itself appeared. The delta format underneath it was not new at all, since VCDIFF had been specified as RFC 3284 back in 2002, so the hard part was never the math. The hard part was the bookkeeping: a browser advertising Accept-Encoding: sdch, a server answering with Content-Encoding: sdch, and a hash prefixed to the payload so both sides could prove they were talking about the same dictionary.

When it worked it worked well. LinkedIn ran it in production and wrote up the implementation, and Yandex maintained an nginx module for it. The early measurements were double-digit page-load improvements on sites with heavy shared markup. This is exactly the saving that mattered on the lines I learned the internet on, where a 33.6k modem turned every repeated kilobyte into seconds you sat through. By the time anybody shipped it I was on TTNet broadband at home, where a few kilobytes of repeated markup cost me nothing I could measure.

~ the chronicle ~

~ the maintenance nobody signed up for ~

A shared dictionary is a snapshot of a site, and sites move. Deploy a redesign and the dictionary your users are holding describes the old markup, so the deltas get bigger and the compression ratio decays toward plain gzip while you keep paying the cost of the dictionary fetch. Keeping SDCH profitable meant regenerating and reissuing dictionaries as part of every release, which is a permanent line item in a deployment process.

Brotli, which arrived in 2015, solved the same problem the boring way. Its static dictionary ships inside the browser, so there is nothing to version and nothing for an operations team to forget.

~ the attacks that grew up around it ~

CRIME in 2012 and BREACH in 2013 established the shape of the problem: if a secret and attacker-controlled text are compressed together, the size of the output leaks whether the guess was right. SDCH's model made that worse than usual, because a dictionary could be fetched from one origin and applied to responses from another. Cross-origin dictionaries cannot be reconciled with the same-origin policy, and the specification never set out how they were supposed to be.

~ the unship ~

The specification was finally filed as an IETF Internet-Draft, draft-lee-sdch-spec-00, on 28 October 2016. Twenty-five days later, on 22 November, Ryan Sleevi posted "Intent to Unship: SDCH" to blink-dev. Security came second in the reasoning. The first items on the list were that the implementation had nobody assigned to it and no second engine anywhere, after eight years of almost no standards activity. The post also raised a question the specification had never answered: how any of this was supposed to behave alongside Service Workers and the Fetch API.

Unshipping meant Chrome would simply stop advertising the encoding and stop processing it, which it did in Chrome 59 on 5 June 2017. Mozilla's bug asking for SDCH support had been open since 2011 and was never implemented, so when Chrome stopped asking, no browser was left asking.

~ what came back ~

The idea was too good to stay buried. Compression Dictionary Transport is now a published standard and shipping in Chrome again, doing what SDCH did with the holes closed: a dictionary has to come from the same origin as the responses that use it, which removes the cross-origin trick that made the side channels so hard to bound. The second attempt took sixteen years and inherited the better half of a dead protocol's homework.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »