All Segments
Why GPT-4 is much smarter than it was a year ago – OpenAI cofounder John Schulman
4 segments available
Full Episode: https://youtu.be/Wo95ob_s_NI Apple Podcasts: https://podcasts.apple.com/us/podcast/john-schulman-openai-cofounder-reasoning-rlhf-plan/id1516093381?i=1000655679622 Spotify: https://open.spotify.com/episode/1ivzHH9RWciXe4O1rKtldf?si=53503781e05f4d8f Transcript: https://www.dwarkeshpatel.com/p/john-schulman/ Me on Twitter: https://twitter.com/dwarkesh_sp/
Segments Timeline
The Shift to Post-Training
John Schulman discusses the evolving landscape of AI training, emphasizing the increasing importance of post-training methodologies over pre-training. He highlights how the quality of output generated by models like GPT-4 is now superior to much of the content available on the web, suggesting a paradigm shift in AI development that prioritizes independent model reasoning.
"with the fraction of Compu that is spent on training that is pre-training versus post training change significantly in favor of post training in the future yeah there are some arguments for that I mea..."
Improving ELO Scores
In this segment, Schulman explains the significant improvements in GPT-4's ELO score compared to its predecessor, attributing these gains primarily to advancements in post-training techniques. He outlines various factors contributing to these improvements, including data quality, quantity, and iterative processes that enhance model performance.
"that was released and is that all because of what you're talking about with these improvements that are brought on by post training yeah I would say that we've um I would say that most of that is post..."
The Complexity of Model Training
Schulman elaborates on the complexities involved in training AI models, emphasizing the need for skilled personnel and organizational knowledge. He discusses the challenges of creating models that meet user expectations and the significant R&D investment required to achieve effective post-training outcomes, highlighting the competitive landscape of AI development.
"operation and there's uh so it takes uh you have to have a lot of skilled people doing it and uh so there's a lot of tacet knowledge and uh um there's uh a lot of organizational knowledge uh that's re..."
Curiosity and First Principles in AI Research
In this insightful segment, Schulman shares his perspective on what makes a successful AI researcher. He emphasizes the importance of having a comprehensive understanding of the entire AI stack, a curiosity-driven approach to experimentation, and the ability to think from first principles to optimize data collection and model training processes.
"what what makes for somebody who's really good at doing this sort of R research uh I hear it's super finicky but like what is the sort of intuitions that you have that enable you to find these ways to..."