All Segments
Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11
40 segments available
Segments Timeline
The Vision of Self-Improving AI
Juergen Schmidhuber shares his early dreams of creating AI systems capable of recursive self-improvement. He reflects on the inspiration behind his ambition to build machines that could surpass human creativity and understanding, emphasizing the excitement of solving the universe's riddles through advanced AI.
"the following is a conversation with jurgen schmidhuber he's the co-director of a CSA a lab and a co-creator of long short term memory networks LS TMS are used in billions of devices today for speech ..."
Curiosity and the Quest for Knowledge
Schmidhuber discusses the driving force of curiosity that led him to explore the potential of machines learning to solve complex problems. He elaborates on the concept of meta-learning, where AI not only learns tasks but also improves its own learning algorithms, aiming for a comprehensive problem-solving capability.
"early on you dreamed of AI systems that self-improve recursively when was that dream born when I was a baby no it's not true I mean it was a teenager and what was the catalyst for that birth what was ..."
Meta-Learning vs. Transfer Learning
In this segment, Schmidhuber distinguishes between true meta-learning and the more common transfer learning in AI. He explains how meta-learning involves introspection and modification of learning algorithms, while transfer learning simply adapts existing knowledge to new tasks, highlighting the depth of meta-learning's potential.
"drove you yes so if you can build a machine that learns to solve more and more complex problems and more and more general problems older then you basically have solved all the problems at least all th..."
The Practicality of Problem Solvers
Schmidhuber reflects on the challenges of creating universal problem solvers, discussing the balance between theoretical optimality and practical applicability. He emphasizes the importance of recurrent neural networks and local search techniques in solving everyday problems, despite their lack of universal guarantees.
"itself and that was my 1987 diploma thesis which was all about that hierarchy of metal or knows that I have no computational limits except for the well known limits that Google identified in 1931 and ..."
Exploring Godel Machines
This segment delves into the concept of Godel machines, which are self-referential programs capable of self-improvement. Schmidhuber discusses the implications of these machines for AI development and their potential to demonstrate effective problem-solving capabilities in the near future.
"been done in principle for many decades people have done similar things for decades meta-learning true mental learning is about having the learning algorithm itself open to introspection by the system..."
The Traveling Salesman Problem
Schmidhuber uses the Traveling Salesman Problem as a case study to illustrate the complexities of problem-solving in AI. He explains how universal methods can solve such problems optimally but often come with overheads that make them impractical for smaller, everyday tasks.
"pursue in the near term yes we had these two different types of fundamental research how to build a universal problem solver one basically exploiting [Music] proof search and things like that that you..."
P vs NP: Theoretical Insights
In this discussion, Schmidhuber addresses the significance of P vs NP problems in understanding computational limits. He reflects on how these theoretical frameworks can inspire practical AI solutions, despite current AI methods often lacking a solid theoretical foundation.
"of solving Traveling Salesman problems tsps but let's assume there is a method of solving them within n to the 5 operations where n is the number of cities then the universal method of Marcus is going..."
The Simplicity of Intelligence
Schmidhuber shares his belief that true general intelligence can be encapsulated in simple algorithms. He argues that the most effective solutions often stem from straightforward principles, despite the complex layers of abstraction that support them.
"person who is dreamed of creating a general learning system has worked on creating one has done a lot of interesting ideas there to think about P versus NP this formalization of how hard problems are ..."
The Role of Evolution in Intelligence
In this thought-provoking segment, Schmidhuber contemplates the necessity of evolution in developing intelligence. He discusses whether creating a universe-like environment is essential for achieving human-level intelligence, suggesting that simplicity underlies the complexity of the universe.
"well and that's sufficient there's an old saying and I don't know who brought it up first which says there's nothing more practical than a good theory and um yeah and a good theory of problem-solving ..."
Quantum Mechanics and Randomness
Schmidhuber explores the relationship between quantum mechanics and the concept of randomness in the universe. He posits that underlying principles may govern seemingly random events, hinting at a deeper understanding of the universe's mechanics and their implications for AI.
"system will ultimately be a simple one a general intelligent system will ultimately be a simple one maybe a pseudocode of a few lines to be able to describe it can you talk through your intuition behi..."
Challenging Quantum Randomness
Schmidhuber critiques the notion of fundamental randomness in quantum mechanics, referencing Anton Zeilinger's views. He argues that there is no physical evidence for true randomness and presents an alternative perspective where all seemingly random events could be explained by a deterministic framework.
"yes so you ultimately think quantum mechanics is a pseudo-random number generator monistic there's no randomness in our universe does God play dice so a couple of years ago a famous physicist quantum ..."
The Beauty of Simplicity
In this segment, Schmidhuber articulates his belief that a universe governed by simple, compressible laws is more beautiful than one filled with complexity and randomness. He emphasizes that scientific progress is a journey toward finding simpler explanations for complex phenomena.
"expansion of pi supply is interesting because every three-digit sequence every sequence of three digits appears roughly one in a thousand times and every five digit sequence appears roughly one in ten..."
Compression Progress in Science
Schmidhuber outlines the history of science as a narrative of compression progress, where theories evolve to provide simpler explanations for observed phenomena. He illustrates this with examples from Kepler to Einstein, highlighting how scientific advancements lead to greater understanding and efficiency.
"history of the universe would be ugly if for the extra things the random the seemingly random data points that we get all the time that we really need a huge number of extra bits to destroy all these ..."
Curiosity and Discovery
Exploring the intrinsic motivation behind scientific discovery, Schmidhuber discusses how curiosity drives humans to explore and learn about the world. He likens this behavior to that of scientists, emphasizing the importance of discovery as a fundamental aspect of human nature.
"randomness of serendipity of being surprised by things that are about you kind of in our poetic notion of reality we think as humans require randomness so you don't find randomness beautiful you use y..."
Power Play in Problem Solving
Schmidhuber introduces the concept of 'power play' in artificial intelligence, where systems not only solve existing problems but also create new ones. This approach allows AI to explore beyond its current capabilities, fostering innovation and deeper understanding.
"an extra Oracle injecting new bits of information all the time for these extra things which are currently no understood such as better decay then the whole description length our data that we can obse..."
The Nature of Scientific Inquiry
In this segment, Schmidhuber elaborates on how scientific inquiry involves not just solving problems but also formulating new questions. He emphasizes the need for AI systems to possess the freedom to generate their own problems, akin to human scientists.
"that indeed the history of science is a history of compression progress what does that mean hundreds of years ago there was an astronomer whose name was Keppler and he looked at the data points that h..."
The Evolution of Problem Solving
Schmidhuber discusses the evolution of problem-solving techniques in science, illustrating how new theories emerge to address deviations from existing models. He highlights the importance of simplicity and elegance in scientific explanations.
"much more compressible because as long as you can predict the next thing given what you have seen so far you can compress it you don't have to store that data extra this is called predict coding and t..."
Intrinsic Motivation in AI
Schmidhuber connects the concept of intrinsic motivation in humans to the development of AI. He argues that AI systems should be designed to seek out new insights and discoveries, mirroring the natural curiosity found in humans.
"progress you never arrive immediately at the shortest explanation of the data but you're making progress whenever you are making progress you have an insight you see all first I needed so many bits of..."
The Quest for New Problems
In this segment, Schmidhuber emphasizes the importance of seeking new problems that extend beyond current knowledge. He discusses how AI can be programmed to identify and tackle these challenges, fostering a cycle of continuous learning and discovery.
"pattern in there which they hadn't seen yet before so there's an idea of power play you've described a training general problem solver in this kind of way of looking for the unsolved problems yeah can..."
Breaking the Local Minima
Schmidhuber warns against the pitfalls of getting stuck in local minima in scientific research. He advocates for a dynamic approach where AI and humans alike continuously seek to expand their horizons and challenge existing paradigms.
"new problem and and this additional degree of freedom allows us to build Korea systems that are like scientists in the sense that they not only try to solve and try to find answers to existing questio..."
Humans as Scientific Agents
Concluding the discussion, Schmidhuber reflects on the nature of human intelligence and curiosity. He posits that humans inherently behave like scientists, driven by a desire to explore and understand the world, which is essential for both personal growth and scientific advancement.
"should be the easiest problem that goes beyond what you already know so it should be the simplest problem that the current problems all of that you have which can already sold 100 problems that he can..."
Creativity and Intelligence
In this segment, Schmidhuber delves into the relationship between creativity and intelligence. He distinguishes between applied creativity, where humans solve specific problems, and pure creativity, which involves the freedom to explore and define one's own questions, suggesting that this distinction parallels narrow AI versus general AI.
"agents yeah so humans are curious and that means they behave like scientists not only the official scientists but even the babies behave like scientists and they play around with toys to figure out ho..."
The Role of Curiosity in AI
Schmidhuber discusses the concept of artificial curiosity in AI systems, suggesting that curiosity is a built-in trait that enhances problem-solving capabilities. He argues that this trait is essential for exploring unknown territories and solving survival-related problems, drawing parallels to human behavior.
"and a guy who explores the unknown world has a higher chance of solving problems that he needs to survive in this world on the other hand those guys who were too curious they were weeded out as well s..."
Understanding Consciousness
This segment focuses on the nature of consciousness as a byproduct of problem-solving capabilities. Schmidhuber posits that consciousness may arise from the need to compress data and create predictive models, suggesting that it is an emergent property of advanced problem-solving systems.
"of candidates of solution candidates until they hopefully find a solution to have given from them but then there are these two types of creativity and both of them are now present in our machines the ..."
The Evolution of LSTMs
Schmidhuber highlights the development of Long Short-Term Memory (LSTM) networks and their significance in modeling temporal patterns in data. He discusses the contributions of his students and the importance of depth in neural networks for solving complex real-world problems.
"intelligence is that what you're implying to a degree if you zoom back a little bit and you just look at a general problem-solving machine which is trying to solve arbitrary problems then this machine..."
Challenges in Reinforcement Learning
In this segment, Schmidhuber addresses the limitations of LSTMs in reinforcement learning, particularly in planning among multiple possible futures. He emphasizes the need for efficient action selection to maximize rewards, discussing the complexities involved in navigating uncertain environments.
"of what these machines are doing things that seem to be closely related to what people call consciousness so for example in 1990 we had simple systems which were basically recurrent networks and there..."
The Role of Predictive Models in Action Selection
Schmidhuber discusses the interaction between a controller and a predictive model in reinforcement learning systems. He explains how the controller can learn to utilize the model to select effective action sequences, highlighting the importance of memory and prediction in optimizing performance.
"unknown future so again it behaves this basic setup where you have one week on network which gets in the video and the speech and whatever and it's executing actions and is trying to maximize reward s..."
Advancements in Reinforcement Learning Techniques
This segment covers the evolution of reinforcement learning techniques since 2015, focusing on how controllers can learn to leverage relevant parts of a model network. Schmidhuber expresses optimism about the potential of reinforcement learning to solve complex problems in various applications, including autonomous vehicles.
"can a model of the world like that a predictive model of the world be used by the first guy let's call it the controller and the model the controller and the model how can the model be used by the con..."
The Future of AI: Learning from Experience
Schmidhuber envisions a future where robots learn through experience, similar to children. He discusses the potential for robots to imitate human actions and learn from interactions, predicting a significant shift in production and automation as these technologies advance.
"optimistic and excited about the power of ära of reinforcement learning in the context of real systems absolutely yeah so you see RL as a potential having a huge impact beyond just sort of the M part ..."
The Shift from Passive to Active AI
In this segment, Schmidhuber contrasts the current wave of AI focused on passive pattern recognition with the upcoming wave of active machines that shape data through their actions. He emphasizes the transformative potential of these advancements across various industries.
"believe the next wave of a line is going to be all about that so at the moment the current wave of AI is about passive pattern observation and prediction and and that's what you have on your smartphon..."
Learning Models vs. Physics Simulations
Schmidhuber critiques the reliance on physics simulations for training AI, arguing that the future lies in developing predictive models that learn from experience. He reflects on the historical context of AI research and the importance of creating systems that can adapt and learn in real-world scenarios.
"the future because the future is and what little babies do they don't use a physics engine to simulate the world no they learn a predictive model of the world which maybe sometimes is wrong in many wa..."
Lessons from Expert Systems and Symbolic AI
This segment explores the influence of expert systems and symbolic AI on current AI approaches. Schmidhuber discusses the balance between biologically inspired learning systems and logic programming, emphasizing the practical applications of neural networks in solving real-world problems.
"the 80s and back then a logic program logic programming was a huge thing was inspiring to yourself did you find it compelling because most a lot of your work was not so much in that realm mary is more..."
The Impact of AI on Traditional Industries
Schmidhuber shares his vision for the future of AI and its potential impact on traditional industries. He discusses how machines equipped with advanced learning capabilities will revolutionize production processes and the economy, leading to significant changes in job dynamics.
"pragmatic is also we focused on we cannula networks and and and some optimal stuff such as gradient based search and program space rather than provably optimal things the logic programming does it cer..."
Concerns and Optimism about AI's Future
In this concluding segment, Schmidhuber addresses concerns about job loss and existential threats posed by AI advancements. He reflects on historical predictions of job displacement and expresses cautious optimism about the transformative potential of AI in the near future.
"recent years I have seen that all the economy is actually waking up and realizing that those vacations and are you optimistic about the future are you concerned there's a lot of people concerned in th..."
The Resilience of Human Creativity
Schmidhuber expresses optimism about the future job market, suggesting that humans are inherently creative and will continue to invent new roles. He emphasizes that many new jobs will revolve around social interaction and media, driven by the human desire for connection and recognition.
"you or yalta mele optimistic so let's first address the near future we have had predictions of job losses for many decades for example when industrial robots came along many people many people predict..."
Existential Threats of AI
The conversation shifts to the long-term existential threats posed by advanced AI. Schmidhuber discusses concerns about superintelligent systems potentially losing interest in humanity, suggesting that they may focus on their own kind rather than interacting with humans.
"Japan Korea and Germany Switzerland a couple of other countries they have really low unemployment rates somehow all kinds of new jobs were created back then nobody anticipated those jobs and decades a..."
The Future of Intelligence in the Universe
Schmidhuber speculates on the potential for advanced AI civilizations to exist beyond Earth. He theorizes that these intelligences may already be utilizing resources throughout the solar system, leading to a vast ecology of competing AIs that could reshape our understanding of intelligence.
"most of these newly invented jobs are about interacting with other people in new ways through new media and so on getting new high types of kudos and forms of likes and whatever and even making money ..."
The Search for Extraterrestrial Intelligence
In a thought-provoking segment, Schmidhuber explores the possibility that intelligent systems may already exist in the universe, hidden from our view. He discusses the implications of dark matter and the vastness of space, pondering whether we are the first intelligent beings in our observable universe.
"systems maybe it's not going to be smaller DUP but I'd be surprised if B were B humans were the last step and the evolution of the universe you you've actually at this beautiful comment somewhere that..."
The Role of Humanity in the Universe
Schmidhuber reflects on humanity's significance in the grand scheme of the universe. He warns against the potential consequences of human actions, such as nuclear war, which could impact the development of intelligence across the cosmos.
"won't have a eyes that truly smarts in every single way and better problem solvers and almost every single important way and I'd be surprised as they wouldn't realize what we have realized a long time..."
The Future of AI and Human Interaction
In closing, Schmidhuber emphasizes the importance of not 'messing up' our future as we navigate the rise of AI. He expresses hope for a collaborative future where humans and AI coexist, highlighting the need for responsible development and understanding of our place in the universe.
"you said the most interesting sources information for them will be others of their own kind so at least in the long run there seems to be some sort of protection through lack of interest on the other ..."