All Segments
Energy, not compute, will be the #1 bottleneck to AI progress – Mark Zuckerberg
3 segments available
Full Episode: https://youtu.be/bc6uFV9CJGg Apple Podcasts: https://podcasts.apple.com/us/podcast/mark-zuckerberg-llama-3-open-sourcing-%2410b-models-caeser/id1516093381?i=1000652877239 Spotify: https://open.spotify.com/episode/6Lbsk4HtQZfkJ4dZjh7E7k?si=GOqj7hUdSaWSgi7ULWXjMA Transcript: https://www.dwarkeshpatel.com/p/mark-zuckerberg Me on Twitter: https://twitter.com/dwarkesh_sp
Segments Timeline
The GPU Supply Challenge
Mark Zuckerberg discusses the historical challenges in GPU production, highlighting how even financially capable companies faced supply constraints. He emphasizes the shift towards increased investment in GPU infrastructure, while foreshadowing the impending energy constraints that will become the primary bottleneck in AI development.
"over the last few years I think there was this issue of um GPU production yeah right so even companies that had the money to pay for the gpus um couldn't necessarily get as many as they wanted because..."
Energy Constraints: The New Bottleneck
Zuckerberg elaborates on the energy requirements for AI training, comparing them to the output of a nuclear power plant. He explains the regulatory hurdles in building energy infrastructure and the long-term nature of such projects, suggesting that energy availability will soon overshadow financial investment as the key limitation for AI progress.
"getting energy permitted is like a very heavily regulated government function and if you're talking about building large new power plants or large build outs and then building transmission lines that ..."
Investing in Future Infrastructure
In this segment, Zuckerberg reflects on the exponential growth of AI and the necessity for substantial investment in energy infrastructure. He acknowledges the uncertainty of future scaling but asserts that overcoming energy bottlenecks is crucial for unlocking the next level of AI capabilities, emphasizing the need for strategic planning in this rapidly evolving field.
"bunch of companies are running at stuff like that um but then when you start getting into building a data center that's like 300 megawatts or 500 megawatts or a gwatt I just I mean just no one as buil..."