
14 segments available
Full episode with Jim Keller (Feb 2020): https://www.youtube.com/watch?v=Nb2tebYAaOA Clips channel (Lex Clips): https://www.youtube.com/lexclips Main channel (Lex Fridman): https://www.youtube.com/lexfridman (more links below) Podcast full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 Podcasts clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41 Podcast website: https://lexfridman.com/ai Podcast on Apple Podcasts (iTunes): https://apple.co/2lwqZIr Podcast on Spotify: https://spoti.fi/2nEwCF8 Podcast RSS: https://lexfridman.com/category/ai/feed/ Jim Keller is a legendary microprocessor engineer, having worked at AMD, Apple, Tesla, and now Intel. He's known for his work on the AMD K7, K8, K12 and Zen microarchitectures, Apple A4, A5 processors, and co-author of the specifications for the x86-64 instruction set and HyperTransport interconnect. Subscribe to this YouTube channel or connect on: - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - Medium: https://medium.com/@lexfridman - Support on Patreon: https://www.patreon.com/lexfridman
Jim Keller defines Moore's Law, explaining its foundational principle of doubling transistor counts every two years. He discusses the evolution of performance metrics and the current trends in transistor scaling, emphasizing the significance of this law in the context of computer engineering.
"for over 50 years now Moore's law has served for me and millions of others as an inspiring beacon of what kind of amazing future brilliant engineers can build no I'm just making your kids laugh all of..."
Keller reflects on the persistent belief that Moore's Law is nearing its end, sharing his experiences over the decades where predictions of its death have repeatedly been proven wrong. He argues that despite challenges, the law continues to inspire innovation in technology.
"transistor count a shrink factor just getting them smaller small as well well as you use for a constant chip area if you make the transistor smaller by 0.6 then you get 1 over 0.6 more transistors so ..."
Keller discusses the multitude of innovations that contribute to the ongoing relevance of Moore's Law. He highlights how advancements in various fields, including materials science and manufacturing techniques, continue to drive performance improvements in computing.
"decided not to worry about that particular product on occasion for the rest of my life which is which is fun and then I joined Intel and everybody said Moore's law is dead and I thought that's sad bec..."
Exploring the limits of transistor miniaturization, Keller explains the potential for future innovations, including quantum computing. He discusses the physical constraints of current technology and the exciting possibilities that lie ahead in semiconductor design.
"that's just normal so then there's the observation of how small could a switching device be so a modern transistor is something like a thousand by a thousand by thousand atoms right and you get quantu..."
Keller elaborates on the challenges of designing computer architectures as transistor counts increase. He emphasizes the need for strategic thinking in architecture to manage complexity and leverage the benefits of smaller transistors effectively.
"yeah so so the innovation stock is pretty broad you know there's yours equipment there's optics there's chemistry there's physics there's material science there's metallurgy there's lots of ideas abou..."
Discussing the limitations of human intelligence and team dynamics, Keller highlights the challenges faced by design teams as they scale up projects. He underscores the importance of abstraction layers in managing complexity in modern computing.
"like imagine you're you build built brick buildings out of bricks and every year the bricks are half the size or every two years well if you kept building bricks the same way you know so many bricks p..."
Keller reflects on the evolution of computational tasks from simple arithmetic to complex AI algorithms. He discusses how advancements in computing power have enabled new forms of computation, particularly in the realm of artificial intelligence.
"understand you know we're really good in teams of ten you know up two teams of a hundred they can know each other beyond that you have to have organizational boundaries so you're kind of you have thos..."
Keller explores the implications of AI on future computational tasks, discussing the shift from traditional algorithms to more complex, data-driven approaches. He emphasizes the transformative potential of AI in reshaping how we approach computation.
"of sort of enforcing given parallelism or like doing massive parallelism in terms of many many CPUs you know stacking CPUs on top of each other that kind of that kind of parallelism or you kind of wel..."
In this segment, Keller debates the concept of search in AI, contrasting traditional search methods with the complex processes involved in neural networks. He discusses the intricacies of training AI models and the challenges of understanding their inner workings.
"mathematical graph right and then the computations the both computation and data sets support going up that graph yeah the kind of computation of my I mean I would argue that all of it is still a sear..."
Keller discusses the foundational mathematical operations that underpin computing, emphasizing the enduring relevance of basic operations like addition and multiplication. He reflects on how these operations continue to evolve alongside advancements in technology.
"this difference between chess and the space the incredibly multi-dimensional hundred thousand dimensional space that you know networks are trying to optimize over is nothing like the chessboard databa..."
Keller speculates on the future of quantum computing and its potential to revolutionize computation. He discusses the challenges and opportunities presented by quantum technologies and their implications for the future of computing.
"things now a given algorithm may say I need sparse data or I need 32-bit data or I need you know like a convolution operation that naturally takes 8-bit data multiplies it and sums it up a certain way..."
Keller connects the ongoing relevance of Moore's Law to advancements in AI research, discussing how improvements in hardware continue to influence algorithm development. He emphasizes the importance of anticipating future changes in technology.
"know my my guess is it's gonna keep happening so your senses yeah if you focus head down and shrinking a transistor let's not just head down and we're aware about the software stacks that are running ..."
Keller reflects on the unpredictable nature of technological advancements and the societal implications of these changes. He discusses the role of engineers and researchers in shaping the future and the responsibilities that come with it.
"so so can you just linger on it I think you've answered it but it just asked the same dumb question over and over so what why do you think Moore's Law is not going to die which is the most promising e..."
In this concluding segment, Keller shares his philosophical thoughts on the impact of technology on society. He contemplates the broader implications of technological advancements and the ethical considerations that arise as we move forward.
"law so for a long time those mainframes minis workstation PC mobile Moore's law drove faster smaller computers right and then we were thinking about Moore's law Rogers godori said every 10 X generates..."