All Segments
Why GPT-4 is much smarter than it was a year ago – OpenAI cofounder John Schulman
4 segments available
Full Episode: https://youtu.be/Wo95ob_s_NI Apple Podcasts: https://podcasts.apple.com/us/podcast/john-schulman-openai-cofounder-reasoning-rlhf-plan/id1516093381?i=1000655679622 Spotify: https://open.spotify.com/episode/1ivzHH9RWciXe4O1rKtldf?si=53503781e05f4d8f Transcript: https://www.dwarkeshpatel.com/p/john-schulman/ Me on Twitter: https://twitter.com/dwarkesh_sp/
Segments Timeline
The Shift to Post-Training
John Schulman discusses the evolving landscape of AI training, emphasizing the increasing importance of post-training methodologies over traditional pre-training. He highlights how the quality of output generated by models like GPT-4 has improved significantly, suggesting that future efforts will focus more on refining post-training processes to enhance model performance.
"with the fraction of Compu that is spent on training that is pre-training versus post training change significantly in favor of post training in the future yeah there are some arguments for that I mea..."
Measuring Improvement: ELO Scores
In this segment, Schulman explains the significance of ELO scores in evaluating AI models, specifically noting that GPT-4 has achieved a score 100 points higher than its predecessor. He attributes this improvement largely to advancements in post-training techniques, which encompass various factors such as data quality and iterative processes.
"that was released and is that all because of what you're talking about with these improvements that are brought on by post training yeah I would say that we've um I would say that most of that is post..."
The Complexity of Model Development
Schulman elaborates on the complexities involved in developing AI models, emphasizing the need for skilled personnel and organizational knowledge. He discusses how the intricate nature of post-training creates a barrier to entry for new competitors, while also acknowledging that some smaller players may attempt to leverage existing models to accelerate their own development.
"operation and there's uh so it takes uh you have to have a lot of skilled people doing it and uh so there's a lot of tacet knowledge and uh um there's uh a lot of organizational knowledge uh that's re..."
Curiosity and First Principles in AI Research
In this insightful segment, Schulman shares his perspective on what makes a successful AI researcher. He emphasizes the importance of having a comprehensive understanding of the entire stack of AI technologies, coupled with a curiosity-driven approach. Schulman advocates for a balance between empirical experimentation and first-principles thinking to optimize data collection and model training.
"what what makes for somebody who's really good at doing this sort of R research uh I hear it's super finicky but like what is the sort of intuitions that you have that enable you to find these ways to..."