Hypertable

» see hypertable.com as it lived, on the wayback machine

~ the eulogy ~

Hypertable was a wide column store written almost entirely in C++, built to the design Google had published in its Bigtable paper and released under the GPL. You pointed it at a distributed filesystem, HDFS or GlusterFS, and it handed you a table that kept working past the size where one machine stops helping. The pitch was narrow and honest: Bigtable's architecture, outside Google, in a language with no garbage collector to apologize for.

Doug Judd started it inside Zvents, a local events search company, and it went out as open source in 2008. In January 2009 Baidu became a project sponsor and took it into production, which for infrastructure software is the only credential that really counts. Hypertable, Inc. was spun up around it in Burlingame, California to sell support subscriptions and commercial licenses.

It ran for eight years as a project and just under six and a half as a business. The ending was not a collapse or a sale. It was a post on the company blog on a Monday in March, written by the founder, explaining the arithmetic.

~ the dossier ~

~ the C++ years ~

The technical argument for Hypertable was performance, and it was aimed squarely at HBase, the Java Bigtable clone that had arrived inside the Hadoop ecosystem. Hypertable published benchmarks, kept a page on the site titled Hypertable vs HBase, and made the case that a C++ implementation with tight memory control would beat a JVM one on the same hardware.

That case was largely correct and it was not enough. HBase came bundled with the thing everybody was already installing. Choosing Hypertable meant choosing it deliberately, then maintaining a C++ dependency in a shop whose other ten pieces of data infrastructure were Java. Being faster has never been sufficient to overcome being the second thing somebody has to decide to adopt.

~ the mind share war ~

Judd named the cause directly in the closing post, and the phrase he used was mind share rather than revenue. "Given the sheer number of competing technologies and the massive amount of aggregate investment that has been made in the space, it's become very difficult for the company to compete for mind share." The company had customers. The attention of the people choosing databases was the thing it never got.

The decision underneath that deserves recording, because the post volunteers it: "Early on, we made the decision to forgo external funding and bootstrap the company on customer revenue." This gets reported as a Baidu-funded project, which it was not. Baidu sponsored the open source work. The company itself never took a round, by choice, in the exact years when MongoDB and the Hadoop vendors were raising enormous amounts and spending it on conferences and developer evangelism.

I have never run a wide column store in production. My work sits at the opposite scale, on single devices doing vision inference, where the entire question is how little hardware you can get away with. The 2010s argument about which petabyte store to standardize on was a real argument with real stakes, and I followed it from outside.

~ the last release ~

Version 0.9.8.11 shipped on 14 March 2016, the same day as the goodbye. The post explains what was in it: the diagnostic and recovery tools, previously commercial, released so that anyone still running Hypertable could get their data back out of it. A final release whose entire purpose was helping users leave.

Judd said the project would continue to exist without active development and that he would keep answering the mailing lists. Both hosts, hypertable.com and hypertable.org, now answer 503 at 714 bytes, and the .org has been pointed at an unrelated AI tools company. The GitHub repository and the Wikipedia article are what is left of a database that Baidu once ran at scale.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »