"the following is a conversation with jay mcclelland a cognitive scientist at stanford and one of the seminal figures in the history of artificial intelligence and specifically neural networks having w..."

"you are one of the seminal figures in the history of neural networks at the intersection of uh cognitive psychology and computer science what do you have over the decades emerged as the most beautiful..."

"neural networks is how they allow us to link biology with the mysteries of thought and um you know in the when i was first entering the field myself in the late 60s early 70s cognitive psychology had ..."

"and i didn't agree with that i i always felt oh look i'm i'm a physical being i from dust to dust you know ashes to ashes and somehow i emerged from that um so that's really interesting so there was a..."

"fundamentals of the human mind yeah i used to think um where i used to talk about the idea of awakening from the cartesian dream so descartes you know thought about these things right he he was walkin..."

"that allowed the physical contact with the stone to cause water to flow in various directions which caused water to flow under the statue and move the statue and he used this as the beginnings of a th..."

"cause action so he had a mechanistic theory of animal behavior and he thought that the human had this animal body but that some divine something else had to have come down and been placed in him to gi..."

"things that you could directly measure when you stimulated neurons and stuff like that and um the study of cognition was something that you know was tied in with abstract computer algorithms and thing..."

"to be pretty continuous so let me uh let me step back to neural networks for for another brief minute you wrote parallel distributed processing books that explored ideas of neural networks in the 1980..."

"early days i'm going to start sort of with my own process in the mid 70s and then into the late 70s when i met jeff hinson and he came to san diego and we were all together in my time in graduate scho..."

"rummelhart had written a book together with another man named don norman and the book was called explorations in cognition and it was a series of chapters exploring interesting questions about cogniti..."

"i was also you know still trying to get from the neurons to the to the cognition and i realized at one point i i got this opportunity to go to a conference where i heard a talk by a man named james an..."

"me this was a bridge between the mind and the brain and i just like stuck and i i remember i was walking across campus one day in 1977 and i almost felt like saint paul on the road to damascus i said ..."

"his phd dissertation showed up uh in an applicant pool to a postdoctoral training program that dave and don the two men i mentioned before remember heart and norman were administering and rommelhardt ..."

"to kind of really resonate with some of rommel hart's um own thinking some of his reasons for wanting something other than the kinds of computation he'd been doing so far so let me talk about ronald h..."

"and uh the word parallel is really interesting so it's it's almost like synonymous from a computational perspective what how you thought at the time about neural networks that is parallel computation ..."

"little computer in and of itself so the idea is that each you know our brains have oh look you know a hundred or hundreds almost a billion of these little neurons right um and they're all capable of d..."

"running on a single computer that's right you're saying wait a minute what what why don't we take a really dumb very simple computer and just have a lot of them interconnected together and they're all..."

"capabilities of deep learning and all these kinds of things but if i could just play this out a little bit a a convolutional neural network or a cnn which you know many people may have heard of is a s..."

"right um and then we think okay at the bottom level there's an array of things that are like the photoreceptors in there in the eye they respond to the amount of light of a certain wavelength at a cer..."

"and get her money so she can buy herself an ice cream it's a huge amount of inference that has to happen to get those things to link up with each other and and he was interested in how the hell that c..."

"of good old-fashioned ai wasn't giving him the answers to these questions yeah and by the way that's called good old-fashioned ai now it was called that well it was it was beginning to be called that ..."

"so he wrote a paper that just really first time i read it i said oh well you know yeah but is this important but after a while it just got under my skin and it was called an interactive model of readi..."

"have these little tiny uh elements that represent each of the pixels of each of the letters and then other ones that represent the line segments in them and other ones that represent the letters and o..."

"a completely interactive bi-directional parallel distributed process that is somehow because of the abstractions is hierarchical so like yeah so there's different layers of responsibilities different ..."

"so there so what ended up happening was that remote heart and i got together and we created a model called the interactive activation model of letter perception which is takes these little pixel level..."

"the algorithmic side the optimization side those are all details like when you first start the idea that you can get far with this kind of way of thinking that in itself is a profound idea so do you l..."

"um but it's not there there's the word time isn't written anywhere inside the bottle it's only written there in the picture we drew of the model to say that's the unit for the word time right yeah and..."

"open my eyes and say oh that's lex or um oh you know there's my own dog and i recognize my dog which is a member of the same species as many other dogs but i know this one because of some slightly uni..."

"you cannot you don't read the contents of the connections the connections only cause outputs to occur based on inputs yeah it's it's and for us that like final layer or some particular layer is very i..."

"how do you square those two like do you think the connections can contain the depth of human knowledge and the depth of what uh dave romohart was thinking about of understanding well uh that remains t..."

"into the cognitive it's like okay so if the under if the substrate is parallel distributed connectionist um then it doesn't mean that the contents of thought isn't you know like abstract and symbolic ..."

"start looking very quickly like organisms that you forget that the forget how the actual thing operates they start looking like they're moving around they're eating each other some of them are generat..."

"that's so you know we won't try to figure out what it is we'll just accept it as given that that that occurs and um you know but he was still on to the magic of it yeah yeah we won't try to really rea..."

"progressively more and more affected so i'm going to talk about the disorder and not about remember heart for a second okay sure the disorder is something my colleagues and collaborators have chosen t..."

"texture of distributed representation in a very nice way i've always felt but at the same time it was extremely poignant because this is exactly the condition that romal heart was undergoing and there..."

"is a celebration yeah yeah yeah and but just to say something more about the scientists and and the back propagation idea that you mentioned um so in in 1982 hinton had been there as a postdoc and org..."

"change their weights from the input and that's why it's called back problems yeah but so it came from hinton having introduced the concept of you know define your objective function figure out how to ..."

"in in in the full space of ideas here at the intersection of computation and cognition well so um jeff has said many things to me that had a profound impact on my thinking um and he's written several ..."

"sort of the idea that um when you when you call a subroutine you need to save the state that you had when you called it so you can get back to where you were when you're finished with the subroutine a..."

"is that he doesn't write too many equations and people tell stories like oh in in the hints and lab meetings you don't get up at the board and write equations like you do in everybody else's machine l..."

"yeah the there's certain people like that here's an example some kind of weird mix of uh visual and intuitive and all those kinds of things feynman is another example different style thinking but very..."

"early to mid 80s on something called the boltzmann machine was his way of connecting with that boolean tradition and bringing it into the more continuous probabilistic graded constraint satisfaction r..."

"some of these things well you're right with the bullying lineage and the the dream of computer science is uh somehow i mean i certainly think of humans this way that humans are one particular manifest..."

"um you know not limited in the ways that we are by our own biology perhaps allowing us to scale the very mechanisms of human intelligence just increase its power through scale yes and and i think that..."

"about uh the properties and relations among uh sets of idealized objects and um uh you know the the mathematical notation system that we unfortunately focus way too much on is um just our way of expre..."

"the insights that human mathematicians have had is a combination of the kind of the intuitive kind of connectionist like knowledge that makes it so that something is just like obviously true so that y..."

"i came across this quotation from i'll replace while i was um walking in the in the woods with my wife in a state park in northern california uh late last summer and what it said on the bench was it i..."

"you know the ability of somebody like hinton or newton or einstein or romal heart or poincare to um archimedes is another example right so suddenly a flash of insight occurs it's it's like the constel..."

"and so i feel like the the kinds of things that we're beginning to see um deep learning systems do of their own accord kind of gives me this feeling of of um i don't know hope or encouragement that ul..."

"um and there's there's kind of like a sense that they've somehow synthesized something like novel out of the you know all of the particulars of all of the billions and billions of experiences that wen..."

"know if you find them as captivating is you know on the deep mind side with alpha zero if you study chess the kind of solutions that has come up in terms of chess it is it there's novel ideas there it..."

"calculation which is the search taking a particular set of steps down the line to see how they unroll but there there is moments of genius in those systems too so that's another hopeful illustration t..."

"here is one part of what i like to emphasize about mathematical cognition at least is that philosophers and logicians going back three or even a little more than 3 000 years ago began to develop these..."

"the kind of the touch point of a of a coherent document that sort of laid out this idea of an actual formal system within which these objects were characterized and the um the system of uh inference t..."

"uh in in experience thinking in that way that you know we now begin to think of our understanding of language as being right so we immerse ourselves in in a particular language in a particular world o..."

"think that um systematic thought is the essential characteristic of the human mind as opposed to a derived and an acquired characteristic that results from acculturation in a certain mode that's been ..."

"it is important to draw that line but then to come back and look at it again and see some of the subtleties and interesting aspects of the difference so if we think about chomsky himself he was born i..."

"the product of this sort of immersion and enculturation uh that is what i believe so and that's limiting it's it's something to be aware of does that limit you from uh having a good model of some of c..."

"i think you're you're right i think that's a great way of characterizing it and um i also think that um it's related to um the concept of beginner's mind uh and um another concept called the expert bl..."

"you know the basic fundamentals of discrete quantities being countable and innumerable and you know indefinite in number um was was not something that had to be discovered um but he was wrong it turns..."

"who still exist who you know who don't have those systems so so this is just an example to me and where you know a certain mode of thinking about language itself or a certain mode of thinking about ge..."

"i'm i although i don't want to dismiss biological individual differences completely i i find it much more interesting to think about the possibility that um you know it was that difference in the dinn..."

"um actually part of the causal process that caused that behavior to occur or even valid observations of the set of constraints that led to the outcome but they are post-hoc rationalizations that we ca..."

"would you give to young people today in high school and college about um how to have a career or how to have a life they can be proud of finding the thing that you are intrinsically motivated to engag..."

"like for me personally i've always been excited about love and friendship between humans and just like the actual experience of it since i was a child just observing people around me and also been exc..."

"for me i was a little bit disillusioned because of how much prescription medication and biochemistry is involved in the discipline of psychiatry as opposed to the dream of the the freud like use the m..."

"well there's some aspect to and sorry to romanticize the whole philosophical conversation about the human mind but to me psychiatrists for time held the flag of we're the deep thinkers in the same way..."

"the i admire the richard feynman ability to do great low-level mathematics and physics and the high-level philosophy yeah i think it was uh frohm and young more than freud that was sort of initially k..."

"speaking which you happen to be a human being who is unfortunately not immortal that seems to be a fundamental part of the human condition that this riot ends do you think about the fact that you're g..."

"interesting and important ones for sure what do you hope your legacy is you've done some incredible work in your life as a man as a scientist when the aliens and the human civilization is long gone an..."

"labeled as a non-theorist right by this uh and um i could have like succumbed to that and said okay well i guess my job is to just go on and do experiments right but but uh that's not what i wanted to..."

"um hold you back well let me ask the big question um when you look at into you said it started with colombia trying to observe these humans and they're doing weird stuff and you want to know why are t..."

"stories on top of each other and eventually we'll colonize hopefully other planets other solar systems other galaxies and will tell even better stories but all starts uh here on earth jay year speakin..."