All Segments
Segments Timeline
The Value of Ideas vs. Execution
Ilya Sutskever discusses the Silicon Valley adage that 'ideas are cheap, execution is everything.' He challenges this notion by highlighting a paradox: if ideas are so cheap, why are there so few new ideas? He explores the historical bottlenecks in research, particularly the limitations of computing power in the past, and how they have evolved over time.
"the Silicon Valley saying ideas are cheap, execution is everything. And there is truth to that. But then I saw I saw someone say on Twitter, if ideas are are so cheap, how comes no one's having any id..."
The Value of Ideas vs. Execution
Ilya Sutskever discusses the Silicon Valley adage that 'ideas are cheap, execution is everything.' He challenges this notion by highlighting a paradox: if ideas are so cheap, why aren't more people generating them? He explores the historical bottlenecks in research, particularly the limitations of computing power in the past, and how advancements in computing have shifted the landscape of idea viability.
"the Silicon Valley saying ideas are cheap, execution is everything. And there is truth to that. But then I saw I saw someone say on Twitter, if ideas are are so cheap, how comes no one's having any id..."
Computing Power and Research Viability
Sutskever elaborates on the evolution of computing power and its impact on research. He provides an analogy comparing the computational resources used for early models like AlexNet to modern transformer models. He emphasizes that while sufficient compute is necessary for research, it is not always clear that the largest amounts of compute are required to validate new ideas.
"compute to [music] prove some idea. Like I'll give you an analogy. Alexet was built on two [music] GPUs. The transformer was built on 8 to 64 GPUs. No single transformer paper experiment [music] used ..."
Computing Power and Research Viability
In this segment, Sutskever provides an analogy comparing the computational requirements of past and present AI models. He notes that while significant computing power is necessary for research, it is not always clear that the largest amounts of compute are essential for proving new ideas. He references the development of AlexNet and transformer models to illustrate the evolution of computational needs in AI research.
"compute to [music] prove some idea. Like I'll give you an analogy. Alexet was built on two [music] GPUs. The transformer was built on 8 to 64 GPUs. No single transformer paper experiment [music] used ..."