McColo

» see mccolo.com as it lived, on the wayback machine

~ the eulogy ~

At about half past one in the afternoon, Pacific time, on Tuesday 11 November 2008, a hosting company in downtown San Jose stopped being reachable from the internet. There was no court order and no police raid. Two companies that sold McColo its connection to the rest of the network, Global Crossing and Hurricane Electric, had each been shown a file of evidence by a reporter, and each decided separately that it no longer wanted the business.

What McColo rented out was rack space and bandwidth, which is the dullest product in the industry. The customers were the interesting part. Security researchers had spent most of 2008 documenting that the control servers for several of the largest spam botnets were sitting in that building: Srizbi, Rustock, Mega-D, and Cutwail by other accounts. A machine infected in Ankara or Ohio would phone home to San Jose for its daily list of addresses.

Brian Krebs, then writing the Security Fix column at the Washington Post, assembled four months of that research and took it to the two upstream providers directly. His paper reported that McColo's tenants ran everything from networks of compromised PCs to counterfeit pharmacies, and noted in the same piece that McColo had not been charged with any crime. No charges are on record since either.

In November 2008 I had TTNet at home in Turkey and an inbox where the real mail was a minority of the arrivals, which was the normal condition of email everywhere and not a Turkish problem. Nothing about that week was visible from the receiving end. A folder just got quieter for a while, and the cause was a conversation between two American network operators that no user anywhere was told about.

~ the dossier ~

~ how big the drop really was ~

IronPort told Krebs the volume fell by almost two thirds. The same engineer told Computerworld that October had averaged roughly 190 billion messages a day and the 24 hours ending that Tuesday came in at about 112 billion, which is a 41 percent fall. Both numbers are correct and they measure different windows, which is why the figures have never agreed in retellings. Spamhaus put the drop at around 60 percent of raw delivery attempts and noted that published estimates ran anywhere from 5 to 90 percent depending on where in the pipeline you took the measurement.

Richard Clayton at Cambridge measured the effect inside a single ISP of roughly 150,000 customers rather than through a global sensor network, and confirmed a real and immediate fall. His more useful finding was where the fall landed: the mail McColo's customers sent was the kind that gets rejected early, by connection-level heuristics, so the drop showed up unevenly depending on which stage of filtering you were counting at.

~ the Moscow car ~

The company's origin story is thin and the two versions of it disagree. Krebs, in the book he later wrote about the spam industry, describes a 23-year-old he calls Nikolai McColo, nicknamed Kolya, the youngest member of a family hosting business that carried his surname. A malware researcher who mined the underground forums in November 2008 described him instead as a 19-year-old Moscow student who founded the firm himself.

Both accounts end the same way. He was a passenger in a BMW racing a Porsche Cayenne through Moscow in September 2007 when the driver hit a lamppost, and he was killed. Krebs dates it to Sunday 2 September. A partner told the customers that service would carry on, and it did, for another fourteen months.

~ thirteen hours in Estonia ~

Srizbi had a fallback. Its bots were built to compute new control-server domain names on a schedule, so when San Jose went dark the infected machines started hunting for addresses that did not exist yet. FireEye registered those domains ahead of the criminals and held the botnet in limbo until the registration bills got too large to keep paying. Control then moved to servers reached through Estonia, and The Register reported that up to around 100,000 machines got back in touch for roughly thirteen hours before the Estonian authorities took the machines down.

Google's Postini measured spam up 156 percent from its post-McColo low by late January 2009. MessageLabs had spam back at 74.6 percent of all email that same month. Symantec called it fully recovered by April. A year later Krebs went back and found McColo's old address blocks sitting on none of the hundred-plus blocklists that track such things, the space quiet enough to look like a hole in the routing table where a company used to be.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »