Surgisphere

» see surgisphere.com as it lived, on the wayback machine

~ the obit ~

For about five weeks in the spring of 2020 a small company in Chicago was the most influential source of clinical evidence on earth. Surgisphere said it operated a platform called QuartzClinical that pulled de-identified electronic health records in near real time from 1,200 hospitals across six continents, and on the strength of what came out of it the World Health Organization paused an arm of a global drug trial and several national regulators changed their guidance.

The company had been founded in 2008 by Sapan Desai, then a surgical resident, and for most of its life it sold medical textbooks and educational material. There is no point in the record where it visibly became a large-scale data business. It simply started describing itself as one. In June 2020 it had eleven employees, and The Guardian, going through them one at a time, found a science fiction writer and an adult model listed among the staff of the company supplying the world's pandemic evidence. Its Twitter account had posted nothing between October 2017 and March 2020.

The New England Journal of Medicine published a Surgisphere-based paper on 1 May 2020. The Lancet followed on 22 May with the one that mattered: 96,032 hospitalized COVID patients across 671 hospitals, showing hydroxychloroquine and chloroquine associated with higher in-hospital mortality and more arrhythmia. The WHO suspended the hydroxychloroquine arm of its Solidarity trial within days. On 28 May an open letter signed by more than two hundred researchers laid out why the numbers could not be right. Desai would not release the data for audit, his co-authors said they could not verify it, and both papers were retracted on 4 June 2020. Surgisphere's website went offline on 15 June and its social accounts were deleted.

I have no standing in medicine and I followed this the way everybody did in 2020, from a chair, reading about a drug I had no business holding an opinion on. The part I can speak to is the engineering, and the engineering is the entire story. Someone claimed a live feed of records out of 1,200 hospitals on six continents, and two of the most demanding journals in the world published on it without anyone requiring a sample export or the name of a single contributing hospital. In my work a dataset you cannot trace back to how it was collected is not a dataset, it is a claim. This is what happens when a claim clears review because it arrived in a hurry.

~ the rap sheet ~

~ what it claimed to have ~

The claim was harder than it sounded and that is why it should have been questioned immediately. Hospital record systems do not agree with each other on anything. Field names differ and coding standards differ by country. Getting two hospitals in the same city onto a shared schema is a project measured in months with lawyers involved. Surgisphere was asserting that it had done this across continents, continuously, with automatic machine-learning mapping, and that the resulting stream was current enough to answer a question about a drug being prescribed that same week.

An operation like that has a shape you can check without seeing the data. It needs signed data-use agreements at every site and an engineering team large enough to keep the mappings from rotting. Reporters went looking for the hospitals afterward and found administrators who had never heard of the company. In Australia, five hospitals were supposedly contributing, and the country's own reporting could not account for them. Nothing that should have existed around the database existed.

~ the numbers that gave it away ~

The first crack was geographic. The Australian figures in the Lancet paper did not line up with what Australia had publicly recorded, and the explanation offered was that a hospital in Asia had been miscategorized, which is the kind of correction that raises more questions than it settles. From there the letter writers worked through the rest. Dose distributions were implausibly uniform across health systems that share no prescribing culture. Patient counts appeared on continents where record digitization at that scale did not exist yet.

What is worth keeping is how it was caught. No regulator did it and no journal did it. It was done by researchers with a spreadsheet and public case counts, in the open, over about a week, because the paper's own numbers were printed where anyone could add them up. The peer review that mattered happened after publication and by people who were not asked.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »