Code Spaces

» see codespaces.com as it lived, on the wayback machine

~ the obit ~

On Tuesday 17 June 2014 Code Spaces took a distributed denial of service attack, which the company said afterward happened often enough that they normally absorbed them without customers noticing. This one was cover for something else.

Code Spaces was a code host in Coventry, England, trading since 2007. It has nothing to do with GitHub Codespaces, which arrived six years after this company was dead and took more or less the same name. This one sold Subversion hosting first and Git hosting later, bundled with bug tracking and work items, aimed at small software shops that wanted a managed repository without running the server themselves. The homepage sold it on redundancy.

While the flood was running, somebody who already had access to the company's Amazon EC2 control panel left messages in it asking to be contacted at a Hotmail address. What followed was an extortion demand: pay a large fee and the attack stops. Code Spaces did not pay. They moved to lock the intruder out by rotating passwords, and discovered a set of spare logins to the panel had been created in advance. The moment the lockout started, the deleting began.

The intruder worked through the account for twelve hours. The EBS snapshots went, then the S3 buckets, then every machine image, then a number of the running instances. The private keys were never taken, so no server was ever logged into. Console access alone was sufficient. Most of the data went with the offsite backups, because the offsite backups lived inside the same panel. On 18 June the company published a statement saying it could not operate beyond this point and had no alternative but to cease trading.

~ the rap sheet ~

~ twelve hours ~

The statement is still readable in the Internet Archive and it rewards reading, because people wrote it from inside the thing. It walks the chain of events in order and sets out which repository URL structures had been deleted outright against which could still be exported. It also misspells the company's own domain as codesapces.com in the middle of a technical instruction, which tells you what sort of day was happening.

The closing line hoped that one day they would be able to reinstate the service and the credibility Code Spaces once had. It never came back. codespaces.com today sells online courses.

~ the lesson ~

I was working as an engineer by 2014 and I had never touched Code Spaces. It changed how I think about backups anyway, the way it did for a lot of people, because it is the cleanest demonstration anybody has of one specific failure: a backup your production credentials can delete is not a backup. Multi-factor authentication on the root account and a backup account that production cannot reach are ordinary advice now, and part of the reason they are ordinary is that a seven-year-old company in Coventry stopped existing in a single afternoon for lacking them.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »