GPT-3 (davinci era)

~ the obit ~

GPT-3 launched on 11 june 2020 with the paper "Language Models are Few-Shot Learners" and a closed API. The flagship model was called davinci. It had 175 billion parameters, which at the time was an order of magnitude larger than anything anybody else had shipped publicly. The training run cost OpenAI an estimated 5 million dollars, which was unusual to spend on a single model in 2020.

For the first 18 months you could only access GPT-3 through an API or through OpenAI's Playground, a webpage at playground.openai.com where you typed a prompt, set some sliders, and the model completed it. The Playground was where most early adopters lived. People would copy and paste their prompts into Twitter screenshots. The model would do something surprising. The reaction would be "wait, you can do that?"

ChatGPT shipped in november 2022 and pulled the attention away from base GPT-3. Then GPT-4 in march 2023. By 2024 the original davinci models were deprecated. They went offline on 4 january 2024 with about 6 months of advance notice. People had to migrate their apps to the chat-tuned successors. Some of them did. Some of them just shut down.

~ the rap sheet ~

~ what made it different ~

The big idea in the GPT-3 paper was few-shot learning. You did not need to fine-tune the model on your task. You just put 3 to 5 examples in the prompt, and the model would figure out the pattern and continue.

This sounds normal in 2026. In 2020 it was very strange. The previous generation of NLP required you to train a custom model on labeled data for every task. GPT-3 said you could skip that. You could write 5 example haikus, then write "haiku 6:" and the model would write a haiku that was usually pretty good. You could write "english: hello\nfrench: bonjour\nenglish: how are you\nfrench:" and the model would translate. It was not great at any single task. It was usable for almost any task you could describe with examples.

The other thing that made it special was the texture. davinci-002 and 003 had a quality that is hard to describe but easy to feel if you used them at the time. The completions were rough. Sometimes brilliantly rough. The model would write a paragraph that was 90% great and 10% slightly off in a way that was generative, the way a slightly drunk person at 3am can be more interesting than a sober one at 9am. GPT-4 lost that. It is smoother and better and a little flatter to talk to.

~ the playground era ~

playground.openai.com was the canonical 2020-2022 GPT-3 experience. There was no chat interface. There was no system prompt. There were just three things on the page: a big text box, a row of sliders (temperature, top_p, max_tokens, etc), and a "submit" button. You typed something. You pressed submit. The model continued from where you left off.

This is a different shape from ChatGPT. You were not having a conversation. You were giving the model a starting point and asking it to keep going. The mental model was closer to "autocomplete on steroids" than to "talking to an assistant." A lot of the early GPT-3 wow moments came from this. People were not asking it questions. They were starting stories and watching it finish them.

There was a small but dedicated developer community that built apps directly on davinci. Copy.ai, Jasper, Lex (the writing tool), AI Dungeon (the game), and dozens of small productivity tools. Most of these had to migrate to chat-style endpoints by 2023. The migration was always painful because the chat models had different output characteristics. Things that worked on davinci often did not work the same way on gpt-3.5-turbo.

~ how it died ~

ChatGPT was not strictly speaking GPT-3. It was GPT-3.5 fine-tuned for chat. But once chat-tuning was the dominant way to deploy these models, base models like davinci-002 and 003 became less useful. They were harder to control. They had safety issues that the chat models had been trained to avoid. They cost OpenAI money to keep running.

In july 2023 OpenAI announced the deprecation of davinci-002 and davinci-003 with a final shutoff date of 4 january 2024. They gave 6 months notice, which was generous. They suggested gpt-3.5-turbo-instruct as a closest replacement. Most developers reluctantly migrated. A few wrote farewell blog posts about losing the davinci texture.

Then on 4 january 2024 the models went offline. The API endpoints returned errors. The Playground showed only the chat models in its dropdown. davinci was gone.

~ what we lost ~

The base model experience. You can still use OpenAI's API but everything is chat-tuned now. The "give it a prompt and watch it complete" workflow has mostly disappeared from the consumer experience. Some open-source models (Llama, Mistral, Qwen base versions) still have it but they are smaller and worse than davinci was.

The roughness, more importantly. davinci-003 in playground at temperature 0.9 felt like talking to something genuinely thinking but not optimized for politeness. The new models are optimized for politeness. They are useful but they sound like a customer service rep who has read the rulebook. The texture is gone, and that texture was the most interesting part.

OpenAI cannot bring it back even if they wanted to. The models are gone. The training data has shifted. The team who shipped davinci has moved on to other things. The 2020 to 2022 OpenAI Playground was a brief moment when AI felt very new and very weird. It is not coming back.

« previous  ·  random  ·  next »