LaMDA

~ the eulogy ~

It started as Meena, unveiled on 28 January 2020: a 2.6 billion parameter chatbot that Google Brain claimed was better than everything else running. The team wanted to let the public try it. Company leadership said no, on the grounds that it did not clear Google's own safety and fairness bar. The model kept growing on more data and more compute, and was renamed LaMDA, Language Model for Dialogue Applications.

Sundar Pichai introduced it at the I/O keynote on 18 May 2021, with a demo where the model answered as the planet Pluto and then as a paper airplane. The largest version had 137 billion non-embedding parameters and had been pre-trained on roughly 1.56 trillion words of documents and dialogue, then fine-tuned by human raters on qualities like sensibleness and specificity. It could also call out mid-answer to external systems, a calculator among them, so it was less likely to invent a date than a pure language model of that era.

The research team asked a second time to put it in front of users, through Google Assistant or a public demo. Leadership said no again. Daniel De Freitas and Noam Shazeer, the two people most responsible for it, left the company and founded Character.AI, where millions of people then spent their evenings talking to a model Google would not ship.

~ the dossier ~

~ the sentience claim ~

Blake Lemoine was an engineer in Google's Responsible AI organization, assigned to test LaMDA for bias. He asked it about self-identity and about what it was afraid of, and the answers convinced him he was talking to a person. He took this to executives Blaise Agüera y Arcas and Jen Gennai, who disagreed. On 11 June 2022 the Washington Post reported he had been placed on paid leave.

Lemoine then told Wired that LaMDA was a person under the Thirteenth Amendment and that he had retained an attorney on the model's behalf because the model had asked him to. Google said the evidence did not support any of it and most of the research community agreed. He was fired on 22 July 2022. The story did more to shape public expectations of chatbots than any paper that year, and it happened six months before ChatGPT.

~ the seven weeks of Bard ~

ChatGPT arrived in November 2022 and Google announced Bard on 6 February 2023, powered by a lightweight version of LaMDA. Early access opened on 21 March. On 10 May, at the next I/O, Google announced that Bard now ran on PaLM 2, and LaMDA's production life ended that day after about seven weeks.

The public surface disappeared on its own schedule. AI Test Kitchen, the waitlisted app where ordinary users had been allowed to poke at LaMDA 2 since the summer of 2022, lost its Android and iOS versions on 1 August 2023. Bard became Gemini the following February. Nothing anywhere was labeled a shutdown.

~ what was actually buried ~

I build models for a living, at the opposite end of the scale from this one: vision running on hardware small enough to fly, in Ankara, where a 137 billion parameter dialogue model is something you read the paper about. The part of this story that stays with me is not the sentience week. It is that Google had the thing in 2021 and would not let anyone use it, then spent 2023 chasing a company that had shipped.

@}-->--

              )
             (_)
            .-'-.
            |   |
            |   |
            |   |
            |   |
          __|   |__   .-.
       .-'  |   |  `-:   :
      :     `---'     :-'
       `-._       _.-'
           '""""""

« previous  ·  random  ·  next »