All Segments
$10B OpenAI & Microsoft Cluster in 2025 – Dylan Patel & @Asianometry
4 segments available
Full Episode: https://youtu.be/pE3KKUKXcTM Transcript: https://www.dwarkeshpatel.com/p/dylan-jon Apple Podcasts: https://podcasts.apple.com/us/podcast/dylan-patel-jon-asianometry-how-the-semiconductor/id1516093381?i=1000671564456 Spotify: https://open.spotify.com/episode/6q1XODE2L5bqqBwe7434S7?si=seXQ6K_LQZeAV6776H6MhQ Me on Twitter: https://twitter.com/dwarkesh_sp
Segments Timeline
The Future of AI Training Infrastructure
Dylan Patel discusses the evolving landscape of AI training, emphasizing the shift towards using synthetic data and advanced search techniques across multiple data centers. He highlights Microsoft's significant investments in fiber connections to enhance their data center capabilities, indicating a future where AI training becomes more efficient and scalable.
"you could imagine that the training regime becomes much more paralyzable where it's like most of the compute for training is used to come up with synthetic data or do some kind of search and that can ..."
Massive GPU Clusters on the Horizon
Patel elaborates on the scale of GPU clusters being built by OpenAI and Microsoft, predicting the deployment of 100K clusters and the potential for even larger setups in the near future. He details the power consumption of GPUs and the implications for AI processing capabilities, suggesting a significant leap in computational power by 2025.
"a gwatt right uh which is like close to a million gpus uh well the each GPU is getting more power higher power consumption too right like it's like you know the rule of thumb is like GPU h100 is like ..."
Elon Musk's GPU Cluster Challenge
The conversation shifts to Elon Musk's claims regarding the largest GPU clusters, with Patel questioning the definitions and metrics used to measure these clusters. He speculates on the future of multi-site clusters and the efficiency challenges they may face, setting the stage for a competitive landscape in AI infrastructure.
"instead of like the total capacity at each data center then you're still like north of multi- gwatt right so um and so they're spending 10 North 10 plus billion dollars on these fiber deals with a few..."
Funding the AI Revolution
Patel discusses the financial requirements for scaling AI infrastructure, suggesting that OpenAI may need to raise between $50 to $100 billion to support their ambitious plans. He emphasizes the importance of funding in driving technological advancements and the role of leadership in securing these investments.
"year 300 to 500,000 um depending on whether it's one side or many right 300 to like 700,000 I think is the upper bound of that but anyways like you know there's it's it's it's about like when they tee..."