All Segments
$10B OpenAI & Microsoft Cluster in 2025 – Dylan Patel & @Asianometry
4 segments available
Full Episode: https://youtu.be/pE3KKUKXcTM Transcript: https://www.dwarkeshpatel.com/p/dylan-jon Apple Podcasts: https://podcasts.apple.com/us/podcast/dylan-patel-jon-asianometry-how-the-semiconductor/id1516093381?i=1000671564456 Spotify: https://open.spotify.com/episode/6q1XODE2L5bqqBwe7434S7?si=seXQ6K_LQZeAV6776H6MhQ Me on Twitter: https://twitter.com/dwarkesh_sp
Segments Timeline
The Future of AI Training Infrastructure
Dylan Patel discusses the evolving landscape of AI training, emphasizing the shift towards using synthetic data and advanced search techniques across multiple data centers. He highlights Microsoft's significant investments in fiber connections to enhance their data center capabilities, indicating a future where AI training becomes more efficient and scalable.
"you could imagine that the training regime becomes much more paralyzable where it's like most of the compute for training is used to come up with synthetic data or do some kind of search and that can ..."
Massive GPU Clusters on the Horizon
Patel elaborates on the scale of GPU clusters being developed by OpenAI and Microsoft, predicting the construction of 100K clusters and even larger setups in the near future. He provides insights into the power consumption of GPUs and the implications for AI processing capabilities, suggesting a significant increase in computational power by 2025.
"a gwatt right uh which is like close to a million gpus uh well the each GPU is getting more power higher power consumption too right like it's like you know the rule of thumb is like GPU h100 is like ..."
Elon Musk's GPU Cluster Challenge
The conversation shifts to the competitive landscape of AI infrastructure, particularly focusing on Elon Musk's claims regarding the largest GPU clusters. Patel discusses the metrics for measuring cluster size and efficiency, speculating on the future of multi-site clusters and the potential for efficiency losses in scaling operations.
"instead of like the total capacity at each data center then you're still like north of multi- gwatt right so um and so they're spending 10 North 10 plus billion dollars on these fiber deals with a few..."
Funding the AI Revolution
Patel addresses the financial requirements for scaling AI infrastructure, suggesting that OpenAI may need to raise between $50 to $100 billion to support their ambitious plans. He emphasizes the importance of funding in driving technological advancements and the role of leadership in attracting investment in the AI sector.
"one cluster is like mean the wishy-washy definition right multi- sight right can you do multi sight what's the efficiency loss when you go multi sight um is it possible at all I truly believe so what ..."