All Segments
The inside story of how ChatGPT was built – OpenAI cofounder John Schulman
6 segments available
Full Episode: https://youtu.be/Wo95ob_s_NI Apple Podcasts: https://podcasts.apple.com/us/podcast/john-schulman-openai-cofounder-reasoning-rlhf-plan/id1516093381?i=1000655679622 Spotify: https://open.spotify.com/episode/1ivzHH9RWciXe4O1rKtldf?si=53503781e05f4d8f Transcript: https://www.dwarkeshpatel.com/p/john-schulman/ Me on Twitter: https://twitter.com/dwarkesh_sp/
Segments Timeline
The Genesis of ChatGPT
John Schulman discusses the early realization at OpenAI that large language models (LLMs) were the future, leading to the development of ChatGPT. He explains how initial instruction-following models were created to make prompting easier and how these models evolved into more conversational agents, setting the stage for ChatGPT's capabilities.
"you let the creation of Chad jbt at what point do you did you realize first of all these llms are the pat to go and then a Chad bot would be or some way to instruct them would be a useful thing to do ..."
The Shift to Conversational AI
Schulman elaborates on the transition from basic models to conversational AI, highlighting the importance of follow-up questions in chat interactions. He shares insights from his previous project, WebGPT, which emphasized the need for a chat-based approach to enhance question answering and user engagement.
"thinking about um chat so uh so Google had some papers uh like they had uh Lambda and um earlier Mina so they had these chat Bots and it was more like um uh like you had a it was more like a base mode..."
Building on GPT-3.5
In this segment, Schulman reveals how the development of ChatGPT was built on GPT-3.5, which excelled in language and coding tasks. He discusses the features considered during development, including browsing capabilities, and the decision to focus on the model's internal knowledge rather than external browsing.
"because um you always want to ask follow-up questions or sometimes you need a clar the the model should ask a clarifying question because the question is ambiguous so it was kind of clear after we did..."
The Evolution of Instruction Following
Schulman explains the evolution of instruction-following models at OpenAI, detailing the training of GPT-4 and the challenges faced with model reliability. He discusses the excitement around the new models and their occasional hallucinations, emphasizing the need for further refinement before public release.
"um and then uh we were thinking about we had it out for beta testing or to friends and family for a while and we were thinking about doing a public release um but um at that time uh actually GPD 4 fin..."
Creating a Coherent Chat Personality
In this segment, Schulman discusses the advantages of chat models over instruction-following models, particularly in terms of user expectations and model behavior. He highlights how the intuitive understanding of chat interactions led to a more coherent personality for ChatGPT, making it easier for users to engage with the model.
"outputs so it was clearly not quite ready for prime time but it was like obviously very good um and uh yeah so I guess that um people forgot about chat for a little while after thatc about this like a..."
The Challenges of Fine-Tuning
Schulman addresses the complexities of fine-tuning models like ChatGPT, explaining that while it was possible to create similar models using publicly available APIs, achieving the same level of performance required iterative supervised fine-tuning. He emphasizes the importance of human feedback in refining model outputs to ensure quality and reliability.
"um I think people had an intuitive sense of uh like what a helpful robot should be like so I think it was uh just much easier to tell people uh like uh to to get for people to get the idea of what wha..."