searchlore

Back to Resource

All Segments

John Hopfield: Physics View of the Mind and Neurobiology | Lex Fridman Podcast #76

John Hopfield: Physics View of the Mind and Neurobiology | Lex Fridman Podcast #76

41 segments available

John Hopfield is professor at Princeton, whose life's work weaved beautifully through biology, chemistry, neuroscience, and physics. Most crucially, he saw the messy world of biology through the piercing eyes of a physicist. He is perhaps best known for his work on associate neural networks, now known as Hopfield networks that were one of the early ideas that catalyzed the development of the modern field of deep learning. EPISODE LINKS: Now What? article: http://bit.ly/3843LeU John wikipedia: https://en.wikipedia.org/wiki/John_Hopfield Books mentioned: - Einstein's Dreams: https://amzn.to/2PBa96X - Mind is Flat: https://amzn.to/2I3YB84 This episode is presented by Cash App. Download it & use code "LexPodcast": Cash App (App Store): https://apple.co/2sPrUHe Cash App (Google Play): https://bit.ly/2MlvP5w PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41 OUTLINE: 0:00 - Introduction 2:35 - Difference between biological and artificial neural networks 8:49 - Adaptation 13:45 - Physics view of the mind 23:03 - Hopfield networks and associative memory 35:22 - Boltzmann machines 37:29 - Learning 39:53 - Consciousness 48:45 - Attractor networks and dynamical systems 53:14 - How do we build intelligent systems? 57:11 - Deep thinking as the way to arrive at breakthroughs 59:12 - Brain-computer interfaces 1:06:10 - Mortality 1:08:12 - Meaning of life CONNECT: - Subscribe to this YouTube channel - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/LexFridmanPage - Instagram: https://www.instagram.com/lexfridman - Medium: https://medium.com/@lexfridman - Support on Patreon: https://www.patreon.com/lexfridman

Segments Timeline

1
0:00 - 1:02
1:02 duration142 words

Introducing John Hopfield

In this segment, Lex Fridman introduces John Hopfield, a professor at Princeton known for his interdisciplinary work in biology, chemistry, neuroscience, and physics. Hopfield's contributions to associative neural networks, now known as Hopfield networks, are highlighted as pivotal in the evolution of deep learning. The discussion sets the stage for exploring how Hopfield applies physics to biological questions.

"the following is a conversation with john hopfield professor Princeton whose life's work weave beautifully through biology chemistry neuroscience and physics most crucially he saw the messy world of b..."

2
1:02 - 2:15
1:13 duration218 words

The Role of Cash App

Lex Fridman discusses the sponsorship of the podcast by Cash App, emphasizing its features such as sending money, buying Bitcoin, and investing in stocks. He highlights the innovative algorithm behind fractional share trading, which simplifies investing for new users. This segment serves as a brief interlude before diving deeper into the conversation with John Hopfield.

"direction this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give it five stars an apple podcast supported on patreon or simply connect with me on Twitter and Lex Friedma..."

3
2:15 - 3:39
1:24 duration183 words

Biological vs. Artificial Neural Networks

John Hopfield explores the philosophical differences between biological and artificial neural networks. He discusses how evolutionary biology has shaped the complexities of neurons, allowing them to adapt and evolve, while artificial networks often suppress these features. This segment delves into the implications of these differences for understanding intelligence and computation.

"Google Play and use collects podcast you'll get ten dollars in cash app will also donate ten dollars the first one of my favorite organizations that is helping advanced robotics and STEM education for..."

4
3:39 - 5:11
1:32 duration238 words

Synchronization in Biological Systems

Hopfield illustrates the concept of synchronization in biological systems using the example of the Millennium Bridge incident. He explains how pedestrians walking at similar frequencies can synchronize their steps due to the bridge's oscillation. This phenomenon highlights the adaptive nature of biological neural networks compared to artificial ones, which lack such dynamic interactions.

"so the glitches become features in them in the biological neural network they they can look let me take one of the things that I used to do research on if you take things which oscillate their rhythms..."

5
5:11 - 6:22
1:11 duration170 words

Evolutionary Processes in Biology

In this segment, Hopfield discusses the evolutionary processes that shape biological systems. He explains how DNA duplication can lead to new functions through slight variations, emphasizing the adaptability of biological organisms. This adaptability contrasts with artificial systems, which face different challenges in evolution and improvement.

"under step with it and people were very uncomfortable with it they closed the bridge for two years really fully built stiffening for it no nerves look nerve cells loose action potentials you have a bu..."

6
6:22 - 7:59
1:37 duration219 words

The Complexity of Biological Learning

Hopfield elaborates on the two types of adaptation in biology: evolutionary adaptation over generations and individual learning within a lifetime. He emphasizes the complexity of neurobiology and the challenges it presents for mathematical modeling, contrasting it with the more straightforward nature of artificial neural networks.

"of the biological molecule level you have a piece of DNA which included an encode for a particular protein you could duplicate that piece of DNA and now one part of it encode for that protein but the ..."

7
7:59 - 9:06
1:06 duration144 words

The Beauty of Human Adaptation

In this segment, Hopfield reflects on the beauty of human adaptation, particularly the learning processes that occur during an individual's lifetime. He discusses the rapid development of the brain in early life and the significance of understanding these processes for insights into neurobiology and intelligence.

"between biological and companies have difficulty having a new product competing with an old fraud large yeah and when IBM built this first PC you probably read the dread the book they made a little is..."

8
9:06 - 10:02
0:55 duration110 words

Breakthroughs in Understanding the Mind

Lex Fridman asks Hopfield about the fields likely to yield breakthroughs in understanding the mind in the coming decades. Hopfield shares his perspective, emphasizing the importance of physics in approaching these complex questions and the need for interdisciplinary collaboration to advance our understanding.

"other little quirk that you particularly like adaptation is everything when you get down to it but the difference there are differences between adaptation where your learning goes on on the over gener..."

9
10:02 - 11:39
1:37 duration205 words

The Challenge of Understanding

Hopfield discusses the challenge of understanding biological systems through the lens of physics. He reflects on the nature of understanding in science and the limitations of current models, particularly in relation to neural networks. This segment highlights the ongoing quest for deeper insights into the workings of the mind.

"biology well when you talk to a computer scientist about neural networks it's all math the fact that biology actually came about from evolution the thing and the fact that biology is about a system wh..."

10
11:39 - 12:47
1:08 duration151 words

The Nature of Neural Networks

In this segment, Hopfield contrasts simple feed-forward neural networks with the complexities of biological systems. He argues that while artificial networks can perform impressive tasks, they lack the essential feedback mechanisms present in biological systems, which are crucial for true understanding and adaptability.

"so you mentioned two kinds of adaptation the evolutionary adaptation at the end the adaptation are learning at the scale of a single human life which do you are which is particularly beautiful to you ..."

11
14:27 - 15:58
1:31 duration234 words

The Physics of Understanding

John Hopfield reflects on his upbringing in a family of physicists and how it shaped his belief that the world is understandable through experimentation and mathematics. He discusses the evolution of his understanding of the mind and biological systems, emphasizing the importance of grasping the essence of understanding beyond mere memorization.

"at both the air parents of physicists both of our parents were physicists and the real thing I gathered that was a feeling that the world is an understandable place and if you do enough experiments an..."

12
15:58 - 17:10
1:11 duration174 words

Feedback in Neural Networks

Hopfield explores the critical role of feedback in real systems and contrasts it with the limitations of feed-forward neural networks. He questions whether these artificial systems can ever achieve true understanding, highlighting the need for deeper insights from neurobiology to advance artificial intelligence.

"how to express it well and you run smack up against it well you choose these look at these simple neural Nets feed-forward neural Nets which do amazing things and yet you know contain nothing of the e..."

13
17:10 - 19:53
2:43 duration362 words

The Evolution of AI Understanding

In this segment, Hopfield discusses the iterative nature of AI development, where each generation builds upon previous models of neurobiology. He speculates on the future of AI and its potential to pass the Turing test, emphasizing the need for ongoing exploration of biological principles to enhance artificial intelligence.

"can talk even about recurrent recurrence but do you think all the pieces are there to achieve understanding through these simple mechanisms like back to our original question what is the fundamental i..."

14
19:53 - 22:15
2:22 duration291 words

Collective Properties in Biology vs. AI

Hopfield contrasts the collective properties found in biological systems with the current limitations of artificial neural networks. He argues that understanding these properties is essential for advancing AI, suggesting that future developments will require a more nuanced approach to neurobiology.

"then in some sense passed the Turing test longer and more broad aspects and how many of these are good there are going to have to be before you say I've made something I've made a human I don't know b..."

15
22:15 - 23:14
0:58 duration120 words

The Surprising Efficacy of Learning Systems

Hopfield expresses his surprise at the effectiveness of non-biological learning systems, which have propelled advancements in neural networks. He reflects on the unexpected success of these systems and their implications for understanding memory and learning.

"perfect people to actually dig in and see how they are used what they mean see you're very right might have to return several times to neurobiology and try to make our transistors more messy yo-yo at ..."

16
23:14 - 27:10
3:56 duration518 words

Associative Memory Explained

In this segment, Hopfield delves into the concept of associative memory, explaining how it functions in the human mind. He illustrates this with examples of how memories are linked and compressed, emphasizing the importance of associative memory in intelligent behavior.

"launched a lot of the recent work with neural networks if we go to what are now called hopfield networks can you tell me what is associative memory in the mind for the human side let's explore memory ..."

17
27:10 - 31:43
4:33 duration606 words

Dynamical Systems in Neurobiology

Hopfield discusses the dynamics of synapses in neurobiology, emphasizing that understanding these changes is crucial for grasping how the brain functions. He contrasts this with the static nature of artificial neural networks, advocating for a deeper exploration of biological systems to inform AI development.

"compress this information present in such a way that if I get the information comes in just like this again I don't bother about their to rewrite it or efforts to rewrite it simply do not yield anythi..."

18
31:43 - 32:43
0:59 duration128 words

Insights from Hopfield Networks

Hopfield reflects on the insights gained from his work on Hopfield networks, particularly regarding memory and learning. He explains how these networks can express learned information and complete patterns, providing a framework for understanding cognitive processes.

"mean from the roots of physics by understanding so what did again sorry but hopfield networks help you understand what insight to give us about memory about learning they didn't give insights about le..."

19
32:49 - 34:02
1:13 duration191 words

Boltzmann Machines and Learning

This segment focuses on Boltzmann machines, a type of feedback network that has been influential in understanding learning processes. Hopfield discusses the relationship between Boltzmann machines and feed-forward systems, highlighting their roles in modern neural network architectures. He reflects on the ongoing relevance of these concepts in advancing artificial intelligence.

"had already been put in and you couldn't understand why then putting in a picture of somebody else would generate something else over here but it didn't out under did not have a reasonable description..."

20
34:02 - 35:18
1:15 duration190 words

The Role of Feedback in Intelligence

Hopfield explores the significance of feedback mechanisms in intelligence, questioning whether feedback is more crucial than the number of neurons in neural networks. He draws analogies to human cognition, emphasizing the ability to think independently and reflectively. This segment raises important questions about the nature of intelligence and the mechanisms that underpin it.

"fact that there were things though that the computation wasn't being ideally done all the way along a line and there are lots of models for error correction but one of the models for error correction ..."

21
35:18 - 36:59
1:40 duration213 words

Consciousness: A Complex Narrative

In this thought-provoking segment, Hopfield delves into the nature of consciousness, referencing Marvin Minsky's perspective that consciousness may be an epiphenomenon. He discusses how our understanding of consciousness is intertwined with subconscious processes and the narratives we create around our experiences. This exploration challenges traditional views of consciousness and its role in cognition.

"work here that's right that's right so these kinds of networks actually led to a lot of the work that is going on now and you're on that works artificial neural network so the follow-on work with rest..."

22
36:59 - 38:04
1:05 duration135 words

Learning and the Brain

Hopfield compares learning in artificial neural networks to learning in the human brain, pondering the depth and complexity of biological learning processes. He questions whether the feedback mechanisms present in biological systems are adequately captured by current computational models. This segment invites reflection on the differences between artificial and biological intelligence.

"probabilities in it this is a lovely encapsulation of something in computational something computational something both computational and physical computational and they very much related to feed-forw..."

23
38:04 - 39:11
1:07 duration139 words

The Limits of Computational Models

In this segment, Hopfield discusses the limitations of computational models in replicating the depth of human cognition. He emphasizes the importance of feedback in learning and the challenges of understanding consciousness from a purely physical perspective. This discussion highlights the ongoing quest to bridge the gap between neuroscience and artificial intelligence.

"brain I don't think the brain is as deep as the deepest networks go the deepest computer science networks and I do wonder where they're part of that depth of the computer science networks is necessita..."

24
39:11 - 40:06
0:54 duration133 words

Consciousness and Its Mysteries

Hopfield reflects on the complexities of consciousness and its elusive nature. He shares insights from his conversations with experts, including Francis Crick, and discusses the challenges of defining consciousness within the framework of physics and neurobiology. This segment underscores the ongoing exploration of consciousness as a fundamental aspect of human cognition.

"little bit like a building a big computer and having running up through one clock cycle and then you can't do anything do you put you reload something coming in how do you use the fact that there are ..."

25
40:06 - 41:20
1:14 duration158 words

The Narrative of Memory

In this segment, Hopfield examines how narratives shape our memories and perceptions of events. He illustrates this with the example of John Dean during the Watergate scandal, highlighting how individuals can construct detailed narratives that may not align with factual accuracy. This discussion emphasizes the role of memory in shaping our understanding of reality.

"every once in a while like I interested in consciousness and then I go and I've done that for years and ask one of my betters as it were their view on consciousness there's been interest in collecting..."

26
41:20 - 42:24
1:04 duration146 words

Consciousness as a Narrative Maker

Hopfield concludes with a reflection on the role of consciousness in cognition, questioning whether it serves merely as a narrative maker or if it holds deeper significance in intelligence. He discusses the challenges of understanding consciousness and its implications for our understanding of the mind. This segment invites contemplation on the essence of consciousness in the broader context of cognition.

"subconscious yo-yo subconscious non conscious non-conscious that's the better word sir there's the it's only the Freud captured the other word yeah it's that's a confusing word subconscious Nicholas C..."

27
44:22 - 45:51
1:29 duration176 words

The Role of Consciousness in Intelligence

John Hopfield discusses the complex relationship between consciousness and cognition. He reflects on a conversation with Francis Crick, exploring whether consciousness is merely a narrative maker or a fundamental aspect of intelligence. Hopfield emphasizes the challenges in understanding consciousness from a physics perspective, highlighting the lack of a 'smoking gun' that could provide clarity on its role in cognitive processes.

"do you mean like where do you stand in your today so perhaps has changed his day to day but where do you stand on the importance of consciousness in our whole big mess of cognition is it just a little..."

28
45:51 - 48:03
2:11 duration305 words

Free Will and Determinism in Physics

In this segment, Hopfield delves into the philosophical implications of free will and determinism as they relate to physics. He notes how physicists often struggle with the concept of free will, leading to contradictions with established laws of physics. Hopfield suggests that understanding consciousness and free will may require a deeper exploration of quantum mechanics and the deterministic nature of the universe.

"blood of life would like to have said and that's why I'm we're working unconsciousness but of course he didn't have any smoking gun in the sense of Mendel and that's the weakness of his physician that..."

29
48:03 - 49:34
1:31 duration222 words

Understanding Attractor Networks

Hopfield explains the concept of attractor networks and their significance in complex systems. He describes how these networks function in high-dimensional spaces, where certain pathways are favored over others, leading to stable behaviors. This discussion highlights the importance of understanding dynamics in systems with many interacting components, which is crucial for both physics and neurobiology.

"if you don't push quite that far you can say essentially all of Neurobiology which is relevant it can be captured by classical equations of motion right because in my view of the mysteries of the brai..."

30
49:34 - 51:02
1:27 duration219 words

Dynamics of High-Dimensional Systems

In this segment, Hopfield elaborates on the dynamics of driven systems and their behavior in high-dimensional spaces. He introduces the concept of energy functions and how they can help understand the convergent dynamics of attractor networks. This exploration provides insights into the stability of complex systems and the underlying principles that govern their behavior.

"and the easiest way to get that is to do it in a high dimensional space where some of these dimensions provide the dissipation which base which a kind of a physical system trajectories can dig our con..."

31
51:02 - 52:58
1:56 duration276 words

The Metaphor of Rolling Balls

Hopfield uses the metaphor of rolling balls to illustrate how systems with attractors behave. He explains that, like a ball rolling down a mountain, systems tend to settle into stable states. This analogy serves to clarify the concept of attractor networks and how they can channel dynamics toward defined pathways, even in the absence of detailed knowledge about the system's inner workings.

"that's the way you make a stable behavior so in general looking at the physics of the emergent stability in these not--when networks what are some interesting characteristics that what are some intere..."

32
52:58 - 54:56
1:57 duration237 words

Exploring Intelligence Systems

In this segment, Hopfield discusses the challenges and considerations in creating intelligent systems. He emphasizes the importance of mental exploration and the ability to simulate outcomes before taking action. Hopfield contrasts simple neural networks with biological systems, highlighting the creative elements present in human cognition that are often absent in artificial systems.

"systems which are stable in the whoo to have these attractors behave even if you can't find the idly up and a function behind them or an energy function behind them it gives you a metaphor for thought..."

33
54:56 - 57:01
2:05 duration278 words

The Limitations of Neural Networks

Hopfield critiques the limitations of artificial neural networks, particularly their reliance on training sets. He explains that if a query falls outside the distribution of the training data, the network struggles to provide accurate responses. This discussion underscores the differences between biological cognition and artificial intelligence, emphasizing the need for a deeper understanding of how biological systems operate.

"is there's a creative element like there's an there's that there's there's a creative element and in a simple-minded neural net you ever a constellation of instances from which you've learned and if y..."

34
57:01 - 59:07
2:05 duration299 words

Deductive Reasoning in Science

In this segment, Hopfield reflects on the value of deductive reasoning in scientific inquiry. He contrasts the data-driven approach of modern science with the principles-based reasoning often employed in physics. Hopfield argues that understanding the fundamental principles behind phenomena is crucial, especially in fields like neurobiology where details matter significantly.

"it than that and it gets back to my own question of where's is it to understand something yeah you know is in a small tangent you've talked about the value of thinking of deductive reasoning in scienc..."

35
59:07 - 1:01:01
1:54 duration252 words

The Future of Brain-Computer Interfaces

Hopfield discusses the advancements in brain-computer interfaces, particularly in the context of companies like Neuralink. He emphasizes the importance of recording collective activities of neurons to truly understand brain function. This segment highlights the potential for technology to enhance cognitive abilities and the challenges that lie ahead in achieving effective brain-computer interactions.

"found that way there's a met sure if you're familiar with the entire field of brain-computer interfaces has become more and more intensely researched and developed recently especially with companies l..."

36
1:01:01 - 1:02:30
1:28 duration203 words

Engineering Insights from Neurobiology

In this segment, Hopfield shares insights on how neurobiology's approach to complex systems can inform engineering practices. He argues that embracing the chaotic and error-prone nature of biological systems can lead to more effective robotics and computational designs.

"and in a general sense I think that's right you have to look you have to begin to be able to look for the collective modes of the collective operations of things it doesn't rely on this action potenti..."

37
1:02:30 - 1:04:22
1:52 duration260 words

The Search for Biological Equations

Hopfield reflects on the quest to discover fundamental equations that govern biological systems, drawing parallels with physics. He discusses the challenges of capturing the complexity of biological interactions and the potential for finding elegant mathematical descriptions of these processes.

"world the engineering world touch that's where their pressure sensor or to let very them an array of of a gazillion pressure pressure sensors none of what you're accurate all of which are perpetually ..."

38
1:04:22 - 1:05:30
1:07 duration147 words

The Interconnectedness of Life

In this thought-provoking segment, Hopfield explores the interconnected nature of life and consciousness. He suggests that understanding the essence of life may require a holistic view that encompasses both individual organisms and their environments.

"are and how to capture them from them biology would you say that's one of the main open problems of our age is to discover those equations yeah if you look at theirs molecules there's psychological be..."

39
1:05:30 - 1:06:55
1:25 duration168 words

Mortality and Legacy

Hopfield discusses how his studies of the mind and neurobiology have influenced his perspective on mortality. He reflects on the digital legacy individuals leave behind and how this may alter our understanding of death and existence.

"equations but your ear sounds some well you're both a physicist and a dreamer you have a sense that yes I can although I can only dream physics dreams physics shapes there was an interesting book call..."

40
1:06:55 - 1:08:10
1:14 duration155 words

The Meaning of Existence

In a profound exploration of existence, Hopfield contemplates the meaning of life from a neurobiological perspective. He discusses the complexities of defining life and meaning, emphasizing the interconnectedness of all living systems.

"brain and out in the body that you worry about burying and it is to a certain extent true that for people who write things down equations dreams notepads Diaries fractions of their thought does contin..."

41
1:08:10 - 1:11:00
2:49 duration347 words

Reflections on Science and Inquiry

In the closing segment, Hopfield shares his thoughts on the nature of scientific inquiry and the importance of asking meaningful questions. He encourages curiosity and the pursuit of knowledge, highlighting the role of experts in advancing understanding across disciplines.

"what is the meaning of life looking back studied the mind a weird descendants of apes what's the meaning of our existence on this little earth Oh word meeting is as slippery as the word understand int..."