0:00
The very rigid objective culture that we live [music] inside has enormous downsides and flaws.
Search this video
0:04
It's not to say we should never have an objective, [music] but just that sometimes it's not the best way to do things.
0:10
>> The creativity, the discovery, all of that which children naturally have.
0:16
That's the original firmware.
0:16
And then we spend all of our time beating it out of them.
0:22
Serendipity is something that you can increase the [music] probability of happening, but you just have to recognize you don't know where you're going.
0:29
Like we did pickreeder because we thought it was interesting.
0:31
We don't have an exact hypothesis or objective, but we thought something really interesting [music] would come out of this.
0:37
When I make that identification of something being interesting, I'm using my entire life experience, all of my intellect to make that designation of interesting.
0:46
[music] If you think about it, there's more information informing those kinds of decisions than inform objective decisions.
0:58
Well, [snorts] hello everybody.
0:58
It's Jim Onessy with yet another Infinite Loops.
1:03
I have been looking forward to today's guest.
1:06
Well, maybe before I even had this podcast because his book, Why Greatness Can't Be Planned, just hit me dead center because I'm like, that's the one I think.
1:17
[laughter] My guest is Ken Stanley, who has decades of work showing that complex creative domains from biology to technology often show that ambitious objectives can actually blind us to the stepping stones that make breakthroughs possible. Ken, welcome.
1:38
And my first question is, are we deterministic thinkers living in a probabilistic world? >> Well, thank you.
1:45
That's a good good start.
1:47
Uh, thank you for having me.
1:47
Um so are we deterministic thinkers in a probabilistic world?
1:51
Um that is a deep philosophical question.
1:58
Um I think uh I think for practical purposes I would not think of us as deterministic.
2:06
Um but um maybe you're maybe you're getting at a uh at a deeper point.
2:08
So maybe I should let you elaborate the question. >> Yeah. Yeah.
2:13
I I say often we are deterministic thinkers living in a probabilistic world and often valericity or tragedy ensue and and and I guess by that I mean I've always kind of naturally been a nonlinear kind of experimental guy and I caught a lot of health for it when I was like a teenager.
2:34
I used to go to Tower Records and like buy dozens of CDs at random just to see what I was missing.
2:41
And I would go home [laughter] and my parents would say, "What? What are you doing?"
2:49
And and they were nonpluc uh to say the least.
2:51
Um, I I I very much have guided most of my life by the kind of stepping stone path that you embrace and and do such a great job of uh explaining to people and and I often think that uh we we let objectives or um you know we must achieve A, B, and C.
3:21
We let those blind us to all the other letters in the alphabet.
3:27
>> Ah, yeah, I see where you're going.
3:29
Yeah, I I missed the opportunity there to connect.
3:31
Um, yeah, I see what you mean now when you say we're deterministic thinkers.
3:34
Um, it's true that at least culturally I think um we do tend to try to make the world deterministic in our thinking.
3:42
Um, and we set objectives uh as just a matter of routine and uh we think that kind of a tidy laying out of objectives over time is going to get us to where we want to go.
3:55
Um, but the world doesn't actually work that way.
3:57
Um, and so it can lead to trouble and that is kind of a theme of a lot of the work that I've done.
4:02
Um, and so I I think we're aligned on that point. Yeah.
4:06
Uh, that is kind of how the world is.
4:08
Another thing that has always kind of worried me, and maybe that's not the right term, but um desire for certainty to me seems to be like one one of the biggest bugs in human OS.
4:25
And it leads us to vote for the candidate who proclaims things are A, B, and C.
4:31
and and has a very clear, easy to understand uh platform, whatever.
4:39
I mean, CEO, politician, all of the above.
4:43
And your work, you bring receipts, uh you you've changed a lot of this to code, like neat, um and and all of your other work, hyper neat, etc.
4:57
if you wouldn't mind for listeners and viewers who aren't very familiar with your work, let's start there.
5:03
Let's let's start uh with your early work and then we'll get to uh the the newest work that I'm really uh intrigued by. >> Uh sure. Yeah.
5:13
And and and by the way, I totally agree with what you're saying about certainty or it's just actually I think it comes out in certitude um online.
5:21
you see so much certitude.
5:21
Um and and that that is to me frustrating that like people can't express uncertainty.
5:29
Uh everything has to be definitely the way things are and this is definitely the correct opinion.
5:33
Um it's it's probably not great for public discourse.
5:37
Um so so okay, just to transition to my own work then uh where do where do I come from and where does it all begin?
5:45
Um I was uh originally interested in um AI but in a particular aspect of AI.
5:52
I mean this goes back to basically like age eight something like 8 years old.
5:56
Um I was always interested in AI and but the particular aspect of AI I was interested in was I was hoping that the computer would do something surprising.
6:04
That was always what interested me.
6:06
Like early on around age 8, I would write code in basic and I would get the computer to have a conversation with me.
6:13
Uh, it definitely wasn't chat GPT.
6:15
Um, but it would say things like, "Hey, what's your name?"
6:19
And I'd say, "Ken," and he say, "Hi, Ken."
6:20
And I impressed myself for a short time that I could do that.
6:23
But what really started to annoy me was just that I knew what it was going to say.
6:27
And I wish it would say something interesting.
6:31
Um, and that kind of grew into um an interest in what kinds of processes lead to unpredictable interesting types of innovation um over time.
6:40
Um, and it's it's both like from the perspective of like how do humans over time innovate and invent things that are not necessarily predictable?
6:50
Um, but also how did things like humans get invented um over the evolutionary process that produced us or biological evolution.
6:57
Um, and so by the time I got to grad school, um, the thing that I was zeroing in on was how do how does a process like evolution, uh, succeed in creating what appears to be increasing complexity over eons?
7:16
Um, and of course, not every evolutionary lineage necessarily is increasing in complexity.
7:20
Um, but some lineages, including our own, have astronomical increases like up to the 100red trillion connections in the human brain.
7:27
And I was trying to understand algorithmically because I'm in computer science. I'm not a biologist.
7:32
I wanted to understand algorithmically how can a process like that actually happen inside of a computer because at that time I believed that it's probably the best way to get to um humanlike AI is to understand the process that generates things like us rather than trying to reverse engineer how we work.
7:50
That was sort of what how I saw things.
7:52
I thought reversa engineering a 100red trillion component machine is probably really really hard.
7:59
Um but just trying to recapitulate something like evolution which looks relatively more simple to explain might be easier to get off the ground faster.
8:07
Um and so that's what led to inventing this thing called a neuroeolution of augmenting topologies or need which is basically just an algorithm that through uh an evolutionary algorithm would evolve neural networks which are the same things we have today in in in large language models but much smaller back then um but could evolve them of increasing complexity.
8:30
So over generations of this evolutionary process in the computer, the neural networks would get more and more complex and the need algorithm was uh an attempt to abstract that kind of a process into the computer and and it could indeed evolve increasingly complex uh artificial neural networks.
8:47
So that that's sort of where I got started.
8:49
That was my um PhD dissertation was on these increasingly complex networks.
8:53
There's a long story from there to get to like today or or through how did we end up having a book um uh which is sort of an anti-objective manifesto.
9:03
Um but maybe I should pause there since we we got to the beginning at least. [snorts] >> Yeah.
9:08
I think uh what's super cool to me about neat uh to more fully explain it for everyone listening and watching is you you trained it for people to focus on what caught their eye, right?
9:22
And stop me if I'm if I'm butchering the explanation here, but out of just that focus of what caught their eye, really incredible images like skulls, like a variety of things appear, if I'm remembering correctly, I played around with it quite extensively.
9:43
>> Um, and I just was so fascinated and I personally saw I'm sure you're familiar with the game of life.
9:49
Uh, that was I think that was in the 70s, right?
9:55
Um, and I saw a clear connection there.
9:59
I also, uh, I don't know if you're familiar with the work of Howard Bloom, who wrote the book The God Problem.
10:02
Um, if not, you might like to read it because he he basically spends the entire book trying to answer your question, how he of course assumes a godless cosmos, right?
10:16
That's his stipulation one.
10:18
And he's like, "Okay, if you grant me just for sake of argument that the there is no watch maker, uh, how on earth did the cosmos do what it did?"
10:31
And the parallels to your work I find, you know, extremely fascinating because he's of a a very similar mindset uh to you.
10:40
He also does he's kind of wacky, which is one of the reasons I love him.
10:45
uh but he he studies uh nature, bees, ants, termites um and you know after reading his the Lucifer principle and the God problem and a couple others global brain his thesis is is pretty simple.
11:01
It's without exploration, without the what he calls bohemian bees, uh none none of it would work.
11:10
And the the idea that it me was like I wonder if that's one of the reasons why we went through what we now call the dark ages.
11:22
I I wonder if when the the Catholic Church became the most powerful institution on the planet and really didn't take kindly to heretics and uh Bohemians and burned them at the stake while we got a thousand years of darkness.
11:41
How would you how would you react? >> Yeah.
11:45
Um it's true that um there's a connection between this like fundamental question of how can things like us exist if nothing is really in charge like deciding how the process should work um and the kinds of algorithmic principles that I'm interested in.
12:02
Um they're they're kind of trying to answer the same question.
12:05
Um, you know, the the the lack of supervised guidance in evolution is really interesting, you know, because it's like evolution looks like a creative genius in some ways, but but it's not a person.
12:16
It doesn't have a mind or initiative of its own.
12:18
Um, but it's creating things that that that even today like defy engineering in our ability to produce.
12:25
Um, and so explaining that is very important I think from an algorithmic perspective like what kind of algorithm is this?
12:30
like what kind of algorithm is this? Um, and you're you're observing uh are you mentioning the the pictures that you saw with with neat and I just wanted to to to make that connection that that is that is also relevant to this but also that's a few that that happened
12:46
a little bit after neat a couple years after neat when that started to happen that need originally was not about pictures it was just using these evolved neural networks to do usually reinforcement learning tasks like get some kind of robotic system to have some behavior that you want. Um, and it was
13:02
Um, and it was it was a couple years later that people started to have this idea that you could take these evolving neural networks which are increasing in complexity and have them generate an image and you could be the one to decide which ones should breed and create the next generation.
13:20
So, it introduces humans in the loop.
13:22
This actually wasn't my idea and it goes back to evolutionary art which is like a whole area that precedes me by many years.
13:28
Um but people's idea basically was let's combine evolutionary art with neat.
13:33
Um and they started doing that in in standalone apps which are not which are little known today.
13:38
Hardly anyone knows that these existed but they were called neat based genetic art apps.
13:44
Um >> and that that's what I was you're right.
13:46
My bad for yeah I should have made that more specific. >> Yeah.
13:50
So so I started playing with those.
13:52
I mean I was I was amazed by them in some way because like it it was neat was my idea but the idea of generating art with with it was not my idea.
13:58
Um and it was so cool like it was something I hadn't thought of doing with it because one of the things about it is that because these neural networks are increasing in complexity you can actually see the increases in complexity visually like through the evolutionary process with the art.
14:15
So it's like it gives you kind of an intuitive lens down to what's going on inside like how is complexity increasing over time um by seeing the images on the surface.
14:24
And so for me it was like very educational and many people maybe saw it more like a toy like it's a way of getting some pretty image or something or just exploring spaces of possibilities.
14:34
But I felt like I was learning something really deep um when I was playing with it.
14:39
And it led us to um have an idea which was initi initiated by Jimmy Serretton who was a student at the time who was uh in my research group uh at the University of Central Florida.
14:52
Um, and he said, you know, maybe we should maybe we should actually create a crowdsourced version of this, like not just a standalone app, but like put it on the web for everybody.
15:01
And then like if you evolved something cool, I could see it online and I could continue breeding it from there.
15:08
Um, and I thought that was just an amazing idea.
15:11
Like it's so interesting because it means the evolutionary process goes basically forever.
15:16
Um, and maybe we could see something analogous like like very very uh very different and very very less grandiose than say evolution on earth but in some ways an analogy to evolution on earth which is this continual divergent process of exploration that never ends um and increases in complexity and it would just be image space but it sounded really amazing.
15:38
I mean this never been done before.
15:40
So that's what led us to create this pickreeder system which is the system that most people are familiar with this kind of neat batbased genetic art through was the pickreeder.
15:50
Um so we put together Pickreeder and put it out on the web and indeed um many people uh it turned into a couple thousand people probably which isn't huge by any modern standard but it was enough to get this large evolutionary kind of tree of life to grow uh inside the system on the internet and we saw like an an unbelievable flowering of discoveries.
16:15
And you might you might think why is it unbelievable?
16:17
What's so surprising about this?
16:19
And the reason it's surprising is because the space that you're in, which is basically just a certain kind of special neural network that we called a CPN, but it was a kind of neural network space, the vast majority of things that it could draw as images are just total garbage. Like 99. 99999% of the space.
16:37
of the space. And this relates back to this kind of question you know that that you framed around this like from the last question you asked of like how can there be all this complexity in the universe without any kind of guidance when you know most things just like random constellations of particles are
16:54
just garbage um just space waste and so where does all this interesting stuff come from and picker sort of begs this question too because like you're seeing over and over again astonishingly meaningful discoveries like a butterfly or a skull or something that looks like
17:12
the planet Jupiter or a car um in a space where those are like this this infinite decimal tiny part of what's possible and you have to explain and it makes us want to explain even though they're humans in the loop and that's a big difference from evolution but even with that how could they possibly have
17:30
consistently found these unbelievably unlikely needles and hay stack over and over and over again there's got to be some reason for that and so that led to I think a lot of insights about how systems like this really work >> and then specifically let's keep going on that. So what let's go on to the
17:46
So what let's go on to the insights and and then what the additional hypotheses and thesis you came up with. >> Yeah.
17:56
So there was a really there were there was a really cool insight actually recently.
18:00
There's been another insight.
18:01
There's actually two really deep insights, but but the the more famous one that's that's been around for a while is this insight about objectives.
18:10
Um, and so it it it's happened because, you know, after the after picker was around for a couple years.
18:16
Um, you know, and it's worth noting that like we didn't really have a hypothesis like what's the scientific hypothesis?
18:22
Like a lot of people think the scientific method is you have hypothesis, then you do an experiment.
18:28
This didn't work that way.
18:30
This is more like let's just do the experiment then we'll figure out what the hypothesis should be because I was sure that like if you just crowdsource the whole world in this complexifying space something interesting will happen.
18:40
I wasn't exactly sure what but I'm I was sure we would learn from it.
18:42
Part of the reason is because like a real challenge with evolution on Earth is that you have to go into the fossil record to actually know anything about what actually happened.
18:52
It's like digging in the dirt.
18:54
enormous amounts of effort is involved and not everything in the chain is even findable.
19:00
And so like there's an there's a poverty of information to really reconstruct the entire chain of events.
19:04
But what's really cool about pickreeder is that we do know everything.
19:07
There's no digging in the dirt.
19:09
Everything every single step of every single choice has been recorded.
19:13
And so that made me think we're going to be able to extract like a really deep lesson here.
19:17
I don't know what it is, but we will.
19:18
Um and indeed that that's what happened.
19:21
Um, and it happened in a way that I didn't expect.
19:23
But basically what happened was um I was um playing with Pickreeder myself and I uh I I I saw an alien face.
19:32
It looked like ET from that movie ET that ' 80s movie if anybody knows that.
19:38
>> It looked a lot to me like ET.
19:38
Um and I thought and someone else had had bred this face so it wasn't me but it was on the site.
19:45
So I thought oh this would be fun because I can get more aliens because that's kind of how it works.
19:50
It's like once somebody finds something, you can get variations of it.
19:51
So I was like, let's get variations of ET.
19:53
Um, and I was playing this game, but then this really weird thing happened, which is that his eyes, they started to descend like down down the face.
20:01
And I suddenly noticed it look like wheels.
20:07
It's like he has a really kind of flat face or thin.
20:08
And so when the eyes went to the bottom, it looked kind of like a car with wheels underneath.
20:13
And it struck me that I was in car space now, no longer in alien face space.
20:17
And so after that I could steer it to something that looked pretty pretty well like a car.
20:24
Um, and you may think, well great, good for you. You found a car.
20:27
Like what's so exciting about this?
20:29
But actually to me it was a huge epiphany, lifealtering, actually.
20:33
It turns out I maybe didn't know at the time it would be lifealtering, but it was it was it was huge because I started to become obsessed with the the observation that I wasn't trying to get a car, you know, like I hadn't been thinking about cars.
20:47
And and it occurred to me that if I had been like if I had wanted to get a car, I would not have chosen that alien face.
20:54
Um you know, I would have thought there's no way that's going to become a car cuz it's a face.
20:57
Um but it turned out I did need it.
20:59
And the only reason I get it is because I wasn't trying to get a car.
21:02
So I was thinking like how could this be?
21:04
Like because it seemed to defy a lot of lessons I'd learned like in engineering and computer science that the way you actually accomplish things in this world is that you set a goal and then you move towards it deliberately.
21:16
Um, and this is like a really deep cultural thing.
21:18
It's not just in computer science, but it's like across our culture.
21:21
In in fact, I would say it's the world culture.
21:23
It's not just like western culture.
21:25
It's the whole world is like this now.
21:26
Um, and I was thinking here's a case where the only way to achieve this thing would be to not be trying to do it.
21:32
And so I thought, okay, let's see now I'm curious like how often does this occur in Pipreer?
21:38
like in the history of all the images that have been discovered um you know we've got butterflies and skulls and planets and and other animals and faces and there's all kinds of stuff there.
21:48
So, so we said, "Okay, let's go back and look at the history."
21:49
And it turns out that almost every single time this is the story.
21:54
this is the story. Like you might you would think I mean I would probably thought that this is some kind of like lottery win like where it's like extremely lucky like this would never happen again like a huge coincidence but actually it's always what happens um is
22:08
that somebody finds something that does not resemble the ultimate discovery but then the person who makes the ultimate discovery branches from that serendipitous discovery and realizes that this leads to something else and then is opportunistic and gets that other thing. Um, and so it's not an
22:22
Um, and so it's not an exception, it's a rule like that the the only way to find things in Prebraer is to not be looking for them.
22:30
And so that kind of generalized this this observation that this is actually a general principle.
22:37
general principle. So then I'm I became obsessed with this after no after we saw that like because I was like is is this this must mean something deeper than just picker like what does this mean because it violates everything I've ever been taught and it seems to be a
22:51
principle and I started imagining weird things like I was thinking like imagine trying to learn how to race a car because a lot of neuro evolution or evolving neural networks would need was involved stuff like trying to teach cars to go around simulated racetracks and stuff like that. So I was thinking what
23:04
So I was thinking what if instead of trying to get it to go around the whole track like that's the objective like it we just imagine that we just say keep trying to do new things.
23:14
So it's like you crash into one wall but you say don't do that again so it crashes into another wall and it but I was thinking if you just kept playing that game it would eventually drive around the track even though you never told it to drive around the track you never rewarded it for going farther.
23:27
It's just that in the process of trying to find interesting new stuff, it's inevitable that eventually you have to learn something about the world because eventually all the silly things to do that are uh that are that are just trivial to find would would be exhausted and then you're going to be forced eventually to find something interesting.
23:43
And I think that's sort of a description in a way of what's happening in pickreeders.
23:47
people were finding things like like circles and lines and curves and things that are not very interesting.
23:53
But as they increase in complexity over time, they're they're just trying to look for anything more interesting.
23:58
And then when they find something, it becomes a stepping stone and then more interesting things can be discovered from there, just like the alien face leads to the car.
24:04
Um, and so after several months of probably obsessing over this, I started to think there's an algorithmic basis like we could actually like describe this process.
24:16
And we ultimately called it novelty surge.
24:18
Um I did that work with Joel Layman.
24:20
It's actually in fact his like first day of grad school.
24:22
I remember I was it was like a few months after this revelation and I just like poured this whole point onto him.
24:28
Like I just overwhelmed the poor guy.
24:30
He's like this his first day and I was like we don't need objectives like we could just do amazing things like just looking for novelty and like this is so exciting.
24:38
And so he he he was he was very nice and kind of like grabbed on to this point and see if you can actually implement this as an algorithm.
24:45
Um and it turns out it was it turned into the novel search algorithm.
24:50
Um which was which was impactful within its field.
24:52
Um and in some ways though I I I view it as as a validation of a deeper point philosophically rather than just like an algorithm that has utility.
25:00
It's more important, I think, for the deeper point it makes, which is that novelty search often discovers a better solution to something than the alternative version, which would be trying to solve the problem.
25:13
Um, which is so paradoxical.
25:16
And so some people think that, oh well, you know what?
25:18
What are you you're saying basically that the best way to solve your problem is to put novelty search on it.
25:22
But that's not my point because novelty search clearly without any clear distinct objective cannot be guaranteed to solve any particular problem.
25:31
it has no particular goal.
25:33
Rather, I think the lesson is that it's extremely embarrassing and concerning that often it does solve the problem better than something that actually knows what it's trying to solve.
25:41
And it should make us all very concerned about the actual downside of being too obsessed with pursuing a singular objective.
25:52
>> Again, philosophically, I'm a huge fan of Daoism and the Da Jing.
25:54
And uh if you've read any about uh of the philosophy, there's a principle called wooi which is actionless activity and it parallels quite nicely with what you're talking about here.
26:10
The the challenge I like so I'm in business right [laughter] and business loves its or cha tables and people fight like hell to be somewhere special on that list.
26:25
So I I ran into a lot of troubles with trying to encourage this type of thinking.
26:33
A lot of my earlier employees would look at me like I had three heads like what what are you talking about? You're crazy.
26:41
Why why would we why would we just randomly put things together?
26:47
And so we I I came up through asset management and quantitative analysis and literally I was fascinated in a similar way with just throwing uh algorithmic factors together and seeing what emerged.
27:02
And just like you said, 99% of it was It was just like nothing.
27:09
But the 1% that wasn't was astounding.
27:16
And it's like I would have never ever thought of that.
27:20
And you know right now for example I'm fascinated by smart swarms.
27:26
I think that uh you know with the intercon connectivity that we now have uh globally uh with the diversity of minds that are on that network uh a lot of this stuff is now happening just naturally.
27:42
But if you if you took a lot of people who were playing with smart swarms and everything else and you told them what you just said, they would also kind of look at you like, "Oh, no, no. I have objectives."
27:56
And it seems like a real cultural thing that that is going to require quite a fight to overcome. >> Yeah. Yeah. Yeah.
28:04
And that's that is something that I I started to see the cultural connection over time after the novelty search algorithm.
28:11
So if you can imagine like at first because like the way I saw myself in my job is just like to do AI.
28:18
Uh I wasn't thinking about a cultural angle on this.
28:21
I was just thinking like this is a very interesting algorithmic insight for the field of AI.
28:27
Um that you don't always have to have objectives and actually sometimes it's better not to.
28:30
And I was giving a lot of talks about that because it is contrarian.
28:34
So I think um people started inviting me a lot to give talks cuz views are kind of entertaining.
28:39
Um and so I was giving a lot of these talks and I I did gradually start to appreciate that interestingly this is not just about AI.
28:49
Um, you know, and people would ask questions like, well, I have objectives at my company.
28:53
objectives at my company. Um, like, you know, does this should I think about that differently, which is a really strange question like at at an at an AI conference, you know, which is usually like about the algorithms and and and
29:04
eventually I was actually invited to speak to um the Rhode Island School of Design to a bunch of artists um because someone who was at that school attended a computer science conference and said, "Hey, I think these artists would find us interesting." And it was there that I
29:16
And it was there that I really came to understand that the the cultural connection is actually pretty important.
29:23
And uh we should do something about that.
29:26
Uh because like we understand the message.
29:28
I mean that we're basically the messenger um myself and Joel.
29:31
And so that's what led to thinking we should write a book because I didn't know any other way to um to get the word out that like this is worth discussing.
29:42
um that like the the very objective rigid objective culture that we live inside um has enormous downsides and flaws.
29:50
It's not to say we should never have an objective.
29:51
Like that would be a perversion of the point.
29:53
Um but just that like we need to understand that sometimes it's not the best way to do things and some resources should go to non-objective pursuits.
30:01
Um and so I could I could appreciate that it's very hard to make this argument, you know, so stories like when you say it's like hard to convince people.
30:10
Um, like this is kind of philosophically like a way I want to approach the business at least in part.
30:17
Um, but everybody thinks I'm crazy.
30:19
Like that story come goes comes over and over and over again.
30:21
Um, people constantly feel like they're living inside of a culture where even if they think, well, I think that way, but no one else can accept this.
30:30
Um, and so nothing ever happens to change things.
30:34
And so I think the the book was an attempt to try to empower people to at least have a conversation about this.
30:40
Um and it was it was a very difficult uh thing to attempt I think for myself and Joel having only ever written to academic audiences in a narrow niche.
30:49
Um but we said okay let's at least just try this.
30:53
So and it's been somewhat successful since here I am on your show.
30:56
So I'm getting a chance to talk about this.
30:58
Um, and I think that the the the the cultural part of it is I mean it's a very unusual story in the field of AI for there to be an algorithmic insight which leads to social critique.
31:09
Like I I don't actually know of other stories like this.
31:13
Um it could be the only I don't want to claim it's the only maybe I'm wrong but it's very unusual.
31:16
Um but it's been a great pleasure like to be able to have some kind of cultural impact also because it's not something I was expecting but that that sort of like goes with the meta story. it wasn't my goal.
31:27
And here I am now talking in in kind of a social critic form.
31:29
And so now I've had many years to to discuss this with people.
31:34
And I think I just wanted to key in on one other point you made when you talked about the difficulty of communicating this, which is the word random that constantly comes up when people worry about this point.
31:43
Like that's sort of a initial reaction is like you're telling me that I should just act randomly and this is actually going to do something better than like planning it out and doing things because I have a goal.
31:53
And and I just want to make the point that like a very deep insight here is that it is not random to pursue things because they're interesting.
32:01
Like when if you think that I'm saying because you pursue novelty and interestingness that you're acting randomly, then you don't understand what randomness is.
32:10
Um because what is interesting, which is ultimately what I'm arguing to pursue, just do something because it's interesting.
32:16
Like we did pickreeder because we thought it was interesting.
32:19
We don't have an exact hypothesis or objective, but we thought something really interesting would come out of this.
32:24
It's like it opens up a new playground of possibilities and low hanging fruit.
32:29
And so, so the point though is that when I make that that that um identification of something being interesting, I'm using my entire life experience, all of my intellect to make that designation of interesting.
32:43
It's not like, oh, I just spun a wheel and it just pointed to one thing and that's what I decided to do that day.
32:48
It's like everything I know goes into it.
32:51
In fact, if you think about it, there's more information informing those kinds of decisions than inform objective decisions.
32:59
Because when you're talking about in a far-off objective, you know very little about that situation that's in the future in a fog.
33:05
Like, I don't know the context, how we got there, what are the stepping stones that lead there.
33:11
But I do know everything about the past.
33:13
I know everything that I've ever experienced very well and how it helps to tune my interestingness compass towards pointing in a certain direction.
33:21
And so it is the complete opposite of random.
33:24
It's taking advantage of the intuitive understanding of the world that people have built up over decades.
33:30
Um which is something that we deny ourselves in our society.
33:32
Like every time somebody say if if somebody goes to their boss and says this would be really interesting. Let's try this.
33:38
They're going to get shot down.
33:39
who's going to fund or you know put resources behind that?
33:42
It's like what's the goal?
33:42
How do we measure progress? Like it's too risky.
33:46
Like that sounds random to me.
33:46
So we are denying ourselves the opportunity I think to use the most valuable resource that we've developed.
33:53
We invested 20 30 years of education into a person um and and and created the the unique intuitions that that person has about what's interesting in the world.
34:07
And it's anything but random.
34:10
That is such an important point and uh one of the things in my own career that I noticed and again so quant which means rulebased which means algorithmic but one of the things that I developed over my career was what I call imbued or saturated intuition.
34:29
It is exactly the point you are making.
34:33
What happened was this lifetime of experience of learning of seeing all the things that worked and didn't work that does feed you in your intuition.
34:45
And so I found myself in the in the somewhat uncomfortable position of saying to clients, well, I have an intuition about this.
34:55
And they would look at me like like you know, you're a quant you can't have intuitions. [laughter] Right. Right.
35:06
>> But but but that is where like all of kind of the gold >> ended up coming from. Right.
35:12
Like huh wonder I wonder if we do like for example I started a company in 1999 which was meant to be the first online investment advisor where where it was called Netfolio.
35:26
And what you could do is you could take a little quiz, you could uh uh tell it your likes and dislikes and then it would give you a quantitative portfolio, but one that was malible that like say you don't like uh smoking.
35:40
My mom died of emphyma, so I'm very antismoking.
35:43
Uh and Philip Morris was the top of the list, you just kick it off and then another stock fills it up.
35:51
>> [clears throat] >> But at the same time, the push back that I got was, well, wait a minute, that doesn't that negate all of your uh back tests on your algorithmic models?
36:00
And I was like, well, not really, because directionally it's still going to be the same underlying criteria.
36:08
And don't get caught don't get too caught up on the fact that, you know, Philip Morris is being swapped for something else.
36:16
They they have the same underlying characteristics.
36:19
And that was the other problem that I found myself facing a lot.
36:24
People get so stuck on labels, they get so stuck on names that it got to the point where even my own staff uh was like, "Oh, we can't buy that."
36:34
And and I always noticed it was because they were focused on the name or the label.
36:38
And I seriously considered taking all the names and ticker symbols out of our system and just giving them numbers because it would allow people to not have that built-in bias.
36:49
And we all have built-in biases and and often find what we're looking for because we're biased toward finding that.
36:58
and the but the thing that I just find absolutely fascinating about your work and and like a lot of a lot of what we're doing at my current firm which Ventures actually fits in really nice with your work.
37:12
I mentioned to you before we started recording that because we have the on-prem AI and it knows everything about me.
37:21
me. I asked it right before we got on was hey what what do Ken and I have in in common and you know the number of pages to it was pretty extraordinary and one of them was and I I this is the large language model saying this to me
37:38
so I'm not making this claim about myself because it sound a little grandiose uh but it said you institutionalize serendipity and and you do that through your fellowships where you're looking for those bohemian bees and you're like, "Well, this is kind of interesting." Again, your term, we give those to what
37:59
Again, your term, we give those to what our team finds interesting.
38:06
And when other people who don't think like that look at our our grants, they're like, "I'm not seeing a lot of rhyme or reason here."
38:17
And my answer always is that's kind of the point.
38:20
Um, and the other thing that drives me, I'm sure you're a fan of Claude Shannon and information theory and like how it's the novel. I mean, right?
38:36
His his whole argument was information equals being novel.
38:40
And and I remember when I was reading a book about him by now my now colleague Jimmy Sony who is the editor and chief of our infinite books division.
38:49
Um and and there was this great quote from Shannon saying basically like a poem can be incredibly informationri a political speech has zero information. >> Yeah.
39:07
um you know it's it's um that that's that's that word serendipity is is just um you know completely aligned with the the themes in the book.
39:16
I mean it's really about serendipity.
39:18
Um but it's kind of about I would say the idea that you actually can mechanize serendipity.
39:26
Um which is super counterintuitive to a lot of people.
39:28
Um seems almost like wrong like that can't be true.
39:33
Um but you know like because everybody also again associates the word random with serendipity.
39:38
It's like random good things happening.
39:40
Um but the thing is that you have to one thing that that might lead to to some caution about that is if you look at the uh Wikipedia page on serendipity and if you actually look at the list of serendipitous discoveries because they have this long list.
39:56
They're always made by people who have a really good track record which is very strange if it's random.
40:01
Like what what help would it be to have a good track record if serendipity is random?
40:06
Like if it's random, the best serendipitous discoverer would be somebody like some lunatic on the street just jumping and hitting walls or something like that.
40:14
Um and something cool might happen, but that's not the kind of people you see in this list.
40:19
Um and it shows that like there's a lot more to serendipity than just randomness.
40:25
Um and it's it's actually it's a form of opportunism, but it's also a form of collecting opportunities.
40:31
I usually call it stepping stones.
40:31
Um, and so the more stepping stones you collect, the more possible jumping off points you have when the opportunity presents itself, but you don't yet know what opportunity you're looking for, but it allows you to be opportunistic.
40:42
Um, and so like novelty search is in a way it's it's algorithmatized serendipity.
40:50
Um, and and of course it doesn't guarantee a particular result.
40:52
So it's more like the point is that there's a way to organize things such that something interesting will happen.
40:58
I just can't say what it is.
40:59
Um, and but it's going to be much more interesting than if you only insist that everything you do has to be driven by objectives.
41:07
You're probably going to be finding much less interesting things.
41:09
If you want objectives that are guaranteed to be safe, I guess nothing's guaranteed, but as close as you can get, you won't do anything interesting.
41:18
I mean, that's the trade-off.
41:20
You might be safe, though.
41:22
Um, and I'm, by the way, I'm not trying to make any kind of like moral judgment.
41:25
Like, I'm not saying that being safe is morally wrong.
41:27
Everybody has a right to to take the risks they want to take.
41:29
And in some sense, I think we're talking about risk here.
41:32
Like that obviously in order to maximize the opportunity for serendipity.
41:36
You need to take some risks and you're not obligated to take risks, you can play it safe.
41:41
But the one thing that I can guarantee is that if you don't pursue things that interest you, you won't do anything interesting.
41:45
Um that's just how it works.
41:47
Now, it's just that also you might not succeed.
41:49
So, you know, there are different risk tolerances and like I think people um you know have depending on their tolerance might build up a a cushion behind themselves before taking their risks but other people are willing to take risks right away and so it just depends on the person.
42:06
Um but ultimately um like serendipity is something that you can increase the probability of happening but you just have to recognize you don't know where you're going. >> Yeah.
42:18
And I think your uh idea about it being riskbased is spoton.
42:23
Um I think that uh it's always really fascinated me that you know we have the distributions that we have.
42:34
It seems to me that a fairly large majority of we humans are risk averse, right?
42:40
and rule followers and doing everything to maximize safety.
42:47
And it's a bit like AI, right?
42:50
If you maximize the objective function for safety, well, you're probably not going to find anything interesting, right? You you might be safe.
43:00
I don't know about that, by the way.
43:02
I think that's you can argue that you can argue that maximizing the objective function for safety does not make you safe.
43:08
make you safe. Uh but the like I have a a young friend George Mack is his name and he's got a thesis which he calls increasing your surface area for look and it and it fits beautifully with your way of looking at the world and and he
43:26
is able to articulate it in a fashion where people who haven't thought really deeply about this go oh yeah and and so his his way of doing it is to set up a dichotomy and he says if you're if you're if you have two opportunities for this evening, right? Um go with the one
43:41
Um go with the one that maximizes your surface area of luck.
43:47
And so then he gives an example.
43:49
Uh you could lie on the couch and watch Netflix or hey, you got invited to this art event uh in Manhattan.
43:55
And which one are you going to choose?
44:00
Well, if you're trying to increase the surface area of luck, you're always going to choose going to the art event over watching Netflix.
44:07
I I teased him and said, "Yeah, but well, what if like you're watching something on Netflix and you just suddenly think,"I never thought of that."
44:15
And he he laughed, of course, >> but it's kind of something that we've been trying to create an operating system for.
44:24
>> Basically, never turn search off, maximize for novelty, um, pay for novelty, gate it with kind of minimal criteria, remember the good forks.
44:35
You've got to always that's your stepping stones.
44:39
You've always got to bank the stepping stones so that you have a plethora of jumping off points.
44:45
It's not your point is very well taken.
44:48
We're not >> like giving people fellowships to go throw themselves against walls in the city.
44:54
Wait, but we are we do have a relative minimum amount of criteria that we will then be willing to pay for novelty and and uh when we find it, we keep it and then we sort of from a business point of view commit after a quorum seems to be like, okay, this seems to really be working and then we lean into that and and the challenge challenge goes back to what we started our conversation.
45:29
It seems to me that part of our human operating system, this this this desire for what is ultimately, in my opinion, an illusion of certainty really really mucks up the works >> when you're trying to get people to to operate this way.
45:49
And so another thing that we did was say, "Okay, look, you what I'd like at my company, I'm selling this to my teammates.
45:57
What I'd like you to do is, okay, if you just budget budget 30% of your time or just following what really really interests you and I've and I've had some good success with that, but overcoming this sort of inertial because it almost I had one fellow We didn't end up hiring him. Very good guy.
46:22
But he he was like he had come out of the military and like he looked at me like I am coming out of a command and control structure that is going to make it very difficult for me.
46:31
And then I made the argument to him well what do you think destroyed all the old communist societies.
46:37
It was the command economies the four-year plan the five-year plan. And guess what? They didn't work.
46:45
And it's one of the reasons why I believe free markets are much more modeled on this idea, right?
46:53
Like a command economy is never going to build an iPhone in my opinion.
46:55
A command economy is never going to come up with a Rubik's cube or a pet rock or, you know, fill in the blank.
47:04
and tell me a little bit more about what what you're working on right now because I'm find that also very very fascinating with the idea that you know you can you can really scale the AI uh to create some pretty interesting innovative things. >> Uh sure. Yeah.
47:27
I mean man you just you just hit on so many interesting topics there.
47:31
I'm just thinking there's a lot of things that could be said.
47:32
I I um I just want to touch on a couple of those cuz cuz they brought some couple things to mind like um you know this the issue of um I in our book we called it a like a security blanket like wanting to feel like everything is certain.
47:48
Um so but it's actually not. It's the problem.
47:53
Like you you can create objectives and they make you feel like you know where you're going and and also make you feel like you're going to get there because you have a measure that tells you you're moving in the right direction.
47:59
Um, but the world usually doesn't work that way unless it's a very simple problem.
48:04
Um, and so like the the particular issue that makes the world not work that way is is deception.
48:11
And deception means that it looks like you're going in the right direction, but you're actually not.
48:16
Or it looks like you're going in the wrong direction, but you're actually going in the right direction.
48:19
Um, and deception is just completely rife throughout complex spaces.
48:23
Like that's the way that reality is structured.
48:25
Um and this is so like to give a concrete example like which shows that like we don't know where stepping stones lead like if you think about vacuum tubes leading to computers like the first computers were built on vacuum tubes but the question is were the people who worked on vacuum tubes before that thinking about building a computer? Um and the answer is no.
48:47
And actually like it's an interesting thought experiment like if you went back to the year 1850 or something and and and got a bunch of brilliant scientists working on vacuum tubes in a room and told them like ditch this boring stuff just build a computer.
49:01
Uh that would be much more interesting than this like tube.
49:03
Like who even needs this?
49:05
Like well we would have no computers or vacuum tubes is what's interesting about that.
49:09
And the reason is because the world is deceptive.
49:11
It's not clear that a vacuum tube is important for computation.
49:14
So the only really way to to to get the vacuum tube registered into the world is to not care about computers.
49:21
Um which is totally counterintuitive.
49:23
Um which is why like it it does make sense um like inside of a company or or like like the way that you described your own organization um to allow people to follow interest because they need to be able to find those deceptive stepping stones.
49:37
Um and and sometimes they're really deceptive like you said like in the case where it could be that the things that lead to safety may appear unsafe and so and vice versa like things that are appearing to be safe like actually might make the world less safe.
49:54
This is like this is the decept the paradox of deception or I some I sometimes call it the objective paradox.
50:00
Um which is which is which is the reason that like these kinds of uh endeavors like how to make the world safe in some totally airtight way that goes on for perpetuity um is almost impossible like you can't actually do that.
50:15
And so it is actually dangerous to pursue that as an objective which which is just countered and unsatisfying and strips off the security blanket because the whole thing is we want the security blanket to feel like what we're doing is going to get us what we want which is just eternal safety for example from AI.
50:31
Um but if the world is deceptive we could be actually hurting ourselves.
50:33
Um, and it's grappling with that complexity that's like just unfortunately like the way that we have to operate like which means there does need to be some room to do things that aren't on the face of it clearly safe.
50:46
Um, and and so you know part of the reason like to understand that is that like things evolve over time like society evolves and so like checks and balances that once kept things in check like could eventually turn into something that actually hurts you.
51:01
Um, and so you can't just have any static view of like what does it mean to be in a safe world.
51:06
Um, everything has to evolve.
51:08
That's why institutions evolve and the government evolves and things like that as society evolves.
51:11
And in some ways like the most safe institution is probably one that's able to evolve because we need to change as as everything in the world changes.
51:18
Um, but it all goes back to deception.
51:20
And so I totally agree with this kind of philosophical uh view of things.
51:25
Um, that like it we can't just constantly try to feel safe. um it won't protect us.
51:32
Um we just have to deal with the reality that the world is deceptive and and this is why non-objective thinking is really important.
51:37
Um and so then that that doesn't necessarily segue that naturally to the question, what am I doing now?
51:43
But I I'll try to go to that because that's that's really the question that you're asking.
51:45
Um I just wanted to acknowledge that part of it. >> Yeah.
51:49
But but before you go there, uh there's a great example from finance, right?
51:54
And that is in the short term, United States Treasury bills are the safest investment you can make.
51:59
In the long term, they are the riskiest.
52:04
>> If you look at the returns to treasury bills versus bonds versus stocks over very long periods of time, treasuries in the very short term are very safe. That is correct.
52:17
Over the long term, a dollar invested back in the n late 1920s inflation adjusted grows to about a buck 80 in treasury bills grows to about 300 and odd dollars in stocks.
52:31
and odd dollars in stocks. So yeah, the the higher variance investment >> which is riskier of course it is but it's I always use that as an example because you are what you what you just said is absolutely true trying to and like I
52:52
don't know if you're a fan of David Deutsch but his book um the beginning of infinity he makes this case beautifully he builds a beautiful scaffolding for if you try to be at, you know, through precautionary principle and just let lock everything down. That is the least
53:08
That is the least safe thing you can do. Life is a verb. It's not a noun. >> Yeah. Yeah. Yeah.
53:16
And people always bring up David Deutsch and I I'm really embarrassed to say I still haven't read the book.
53:21
I have to because like everybody says you you must have read David Deutsch. So I should do that. >> You should.
53:26
I think I think you really and I would uh recommend starting with the beginning of infinity because what's beautiful about it is he really builds a very robust intellectual framework for what we're talking about right now.
53:39
what we're talking about right now. Um he he gives it he gives it reasons like he has a great quote in there what goes some I'm gonna butcher it but it's like hey what were uh uh people talk what were the smartest people talking about it in 1900 what were their opinions on quantum physics and the internet and his answer was they didn't have opinions on
54:02
quantum physics and the internet because it hadn't been invented >> right right yeah I mean it's uh I mean this this security blanket issue is a very serious problem in society, I think, um is that that everything is saturated with this this need to make everything mechanized so that we can guarantee that we know where we're going, which is all just security blankets thrown on top of everything. Um
54:23
Um and it's just so naive and unrealistic and and it's it's thwarting innovation and actually making the world more dangerous.
54:30
Um, and you know, and I look at it like I I was because I was a professor for a long time.
54:34
The point the pain point where it often bothered me was in applying for for scientific funding.
54:41
Um, because always like the National Science Foundation or other agencies that can fund research, they want to know what your deliverables are going to be.
54:48
Um, they ask for like the the impacts and like what's going to happen.
54:52
It's like I just want to be like I don't know what the impacts are yet.
54:55
like you by forcing me to tell you and know what's going to happen, I can't tell you the most interesting ideas I have and I know they're not going to get funded because I don't know where they're going to lead.
55:04
Um, and yet like the entire like funding industry is built on this premise that you can just tell the people where you're going.
55:10
Um, and it just seems to me it's really irritating and frustrating to me.
55:14
It's like I'm okay with like some amount of grants doing that.
55:19
Okay, let's have some safe areas where people just do things that are modest that we know we're going to be able to get something out of. But I mean, come on.
55:25
This is supposed to be about innovation.
55:27
Like ultimately, some amount of resources should be should be devoted to things where we don't know where we're going. That has to be the case.
55:34
But it's just this security blanket culture that just completely saturates everything.
55:38
Um and so so yeah, I'm hoping the book does a little bit to nudge the pendulum in the other direction.
55:45
>> Well, it you know, it certainly did in our case.
55:47
I mean like uh one of the reasons behind the fellowship and grant program was because we wanted to do exactly that.
55:56
I could not more vehemently agree with you about the funding uh issue is particularly in the sciences.
56:04
uh to the point where one of our objectives as we build out our own uh uh AI system here at OSV is I I I want uh a portion of it to devoted to just developing and publishing automatically to a uh accessible database for everyone to uh have access to null hypothesis, right?
56:26
Because it's like Sherlock Holmes, right?
56:30
The the hounds of Baskerville.
56:32
The only reason that he figured out that they knew the intruder was the dog didn't bark.
56:36
And via negativia, it just doesn't seem to be in our in our cognitive firmware.
56:42
And it's just like it there's so much richness there and and and we just avoid it.
56:51
And your point about the the safe funding, I think it is significantly advancements not only in science but across the board.
57:00
It's gotten institutionalized.
57:05
It's gotten, you know, group think.
57:07
There's there's an omniculture that that covers that.
57:10
And you look around and you think, "Oh my god, look at all the look at all the advancements and innovation that we made like 50 years ago and specifically in like physics and and and the deep deeper sciences like the last 50 years." Bupkus. >> Yeah. Yeah. It's true.
57:29
But you know, I think there's an interesting uh an interesting thing to think about in what you were saying because you mentioned the firmware.
57:35
Um and so because this this question is is it kind of deep in our firmware that we kind of have a desire and and need to live this way?
57:42
Um or is it just a cultural artifact?
57:45
And you know sometimes I think about children cuz I think you don't see it in their firmware.
57:50
Like you bring a little toddler to the playground like they're not going to say what are my goals for today?
57:57
Like I'm going to learn how to climb that monkey bar.
57:58
that's what I like and then I'll be better at Bucky Bars next time.
58:01
Like they don't think objectively at all.
58:03
It's just like they see a playground, it's just endless opportunities and that's like really fun.
58:08
Um and so maybe it seems like to me that it's been beaten out of us culturally.
58:13
Um because like as soon as you start like the first grade, then you're basically saturated with goals.
58:21
Like this is what you should do.
58:21
This is what you need to accomplish.
58:22
Um and and you're taught through reinforcement that like if you do the thing that you were told to do because it was your goal, you will be rewarded.
58:30
Um and reward after reward after reward over decades.
58:33
I mean, you just learn that that's how the world works, you know, cuz I remember in high school like I remember thinking I had a lot of I thought I thought at least to myself, I have a lot of ideas.
58:45
Like I would write down ideas when I would have them.
58:46
But I actually thought that that it's not that useful to have ideas.
58:50
Like I explicitly thought that.
58:50
I was like, you know, it doesn't really help you to have ideas because like what how would that help me like get get A's in school or nobody really cares. It's more like for fun.
59:00
Um, but it was an absurd thought like that you could grow up thinking that is absolutely crazy.
59:04
It occurred to me later like somewhere in grad school that it actually might be useful to have ideas now because like you know I'm supposed to have a PhD thesis and that really made me feel good.
59:16
I was like wow like finally it might actually be good for me to have an idea.
59:19
But the fact that it could take like 24 years before somebody thinks, "Oh, actually ideas might actually be good just shows there there is some kind of social pathology like that that's really hurting things."
59:30
Like people should not lose that playground mentality completely over the course of life.
59:36
And and the fact we beat it out of them is it's the opposite of what education should be doing. >> Totally agree.
59:42
Robert Anton Wilson, who was way ahead of his time, called it uh the installation of the correct answer machine in young minds.
59:50
And I have six grandchildren and they range from, you know, a little over one to 11.
59:57
And watching the younger ones, I it's just so obvious to me that you your point is well taken.
1:00:09
It is not an innate part of our firmware. It's cultural.
1:00:12
We, you know, it we liter this I don't want to get on the soap box because I feel very passionately about this, but like literally we beat the creativity, the discovery, all of that which children naturally have.
1:00:30
That's the original firmware.
1:00:33
And then we spend all of our time beating it out of them.
1:00:36
of our time beating it out of them. And of co and and it's it's one of the reasons why I think that the current I I have a thesis called the great reshuffle which is basically that I started I don't know 2015 and with the advent of machine learning with the advent of connectivity etc that everything was
1:00:57
going to get reshuffled and one of the it's we're in the middle of it now and what and one of the reasons that I think that you see the established elites of today squawking as loudly as they are squawking is because the these new ideas and these all of that is coming from a much smaller part of society, but guess what? They're They're going to topple.
1:01:23
It's like the whole accreditation thing.
1:01:25
I I have deep respect for people who have PhDs. Absolutely.
1:01:31
And and and so I I hate this binary, oh, you don't have to go to school and all that.
1:01:36
Well, yeah, you do for a lot of very serious things, but but what happens is we tip over into being just slaves to credentialism.
1:01:46
And I think that's a mistake, right?
1:01:49
It's like I mentioned to you um our CTO was very young.
1:01:55
He doesn't have a PhD, but his proof of work is absolutely extraordinary.
1:02:01
you can if if you're a little more open-minded, you you can find these things that don't have to go up that traditional ladder.
1:02:09
I think that's a uh we're we're breaking with that past, but the culture culture takes a while to catch up. It's much slower.
1:02:20
Cumulative cultural evolution is a lot slower than I think both you and I would like it to be. >> Yeah. Yeah.
1:02:27
I mean it's it's almost like I mean I think risk mitigation won out over innovation or something like risk mitigation seems to be so popular even though it's not talked about as if it's popular but it's like it just overwhelms everything.
1:02:40
Um and it's I mean people are taught implicitly through school that like you do things to mitigate risks ultimately make sure that you turn out fine and you have you can protect your family and things.
1:02:49
Obviously these aren't bad things but like there's this other aspect of the world that like it's just not really being fostered.
1:02:54
um which is less objective.
1:02:57
You know, it goes back to like the original insights in in Picker like ultimately this has to do with sometimes doing things without having an objective and and by the way objective is connected to the word metric, you know, because like once you have an objective, then you can measure something.
1:03:10
So now you have a metric and so often objectives are expressed through metrics and these are like the grades and the credentials and all those things.
1:03:16
It's an attempt to make everything objective so that we just know what's going to happen, where we're going, who's the best person.
1:03:22
Everything is being measured objectively.
1:03:23
But the problem is objectives don't get you everywhere, you know, especially in innovative systems.
1:03:28
So it's true like selecting a really great CTO with extremely innovative mindset like you can't just look to the credential or some objective measure.
1:03:36
It's actually it's very subtle and implicit like what what actually would make like a real innovator.
1:03:41
Um and so um so so I think it's it is it is these are these are really important social challenges uh to to overcome um just the way culture has evolved.
1:03:52
And actually I just want to give one anecdotal story because it it just reminds me of this like >> the this really came up when um a long time ago uh when I was trying to start a company um and uh and I was and I was a professor at the time and the person who from the university side whose job it was to shepherd the deal because they want to make sure about compliance and things like that was a compliance officer.
1:04:18
Um, and this made things unbelievably hard because the only incentive there is just that it complies with everything.
1:04:24
Um, and so there's no like, you know, enthusiasm for just how amazing this could be.
1:04:28
It's all just compliance, compliance, compliance.
1:04:32
And it made almost impossible to get this to actually happen even though the whole thing had so much opportunity um, for everybody involved.
1:04:39
Um, and I just like think, you know, it's it's this is like the whole world is like a one compliance officer facing you in front of everything before you can actually get to doing anything interesting just to mitigate risk.
1:04:49
It's like that's the only thing we're worried about is risk.
1:04:55
And of course, we have suffocated a lot of innovation just through the uh explosion of regulations and and rules and laws and everything else.
1:05:06
Again, when you say compliance to a guy who made his bones in finance markets, it is the most regulated of many or not the most, but one of the most regulated of all industries.
1:05:20
And you know, I I was lucky because I had my own company.
1:05:23
Uh I could uh yes, of course, we had a compliance officer.
1:05:29
Uh but um it was not you're not running things. His name was Ray.
1:05:36
Ray, you're not running things.
1:05:36
You You're here to make sure that no one does anything illegal. Absolutely.
1:05:45
But the the culture of compliance is Oh man, it I was briefly at Beer Sterns and they were pretty good about it, but you know, when you saw them coming, it's like, okay, here's where I went to their offices.
1:06:00
I would always joke as I was entering, I'm entering where ideas go to die. Yeah. Yeah.
1:06:07
I mean, and I'm not against complying with rules.
1:06:09
I mean, I I'm pretty much I'm a rule following type person, but but I mean, it's kind of like the it's just trying to reduce every single little piece of risk that could exist in the world.
1:06:19
That is just going crazy >> and it leads to stasis, which is death. >> Yeah. Yeah.
1:06:24
I mean, so something bad might happen like sometimes like that's okay.
1:06:29
It's not not always okay, but like every possible failure and misuse of a dollar, it's like sometimes it's just okay.
1:06:36
Like that's that's the cost of like taking risks.
1:06:37
Um, so we shouldn't we shouldn't be so completely averse to doing things that are interesting.
1:06:42
Um, so anyway, yeah, it's I guess we share the the the the the discouragement with that stuff.
1:06:49
Um I don't know if we >> Yeah, >> I would love to get I Yeah, I would love to get on to fractured entanglement uh representation hypothesis. >> Ah okay. Yeah.
1:07:00
Um that that does follow well from all of this because it it's it's the second lesson kind of um which is much less known except for that paper that we very recently put out about this fractured entangled representation hypothesis.
1:07:12
Um it it's a lesserk known thing but it it I think it's equally deep and profound um which is that we observed uh early on in in pickreeder this other really fascinating thing which is that the underlying representations of the images because remember it was evolved images that were produced by neural networks that the representation of those images was quite amazing.
1:07:36
Um and if you're not familiar with neural networks you might might not be clear what does it mean to have an amazing representation.
1:07:41
We're talking about the the way that the image is actually built inside the network node by node, neuron by neuron, all the way from the bottom to the top.
1:07:51
There's a way it's done and actually we have ways of visualizing because of the nature of these images, how it's being done.
1:07:59
So what are the intermediate uh images or you could call them like masks that are used to build up the image?
1:08:06
It's very easy for us to see those.
1:08:06
And by looking at and visualizing those, we saw that it was incredible.
1:08:10
It was like things like and what I mean by incredible is things like it had separated out the mouth of the skull in such a way that there was a single weight inside the network means one parameter basically that if you changed the number on that parameter you could open and close the mouth.
1:08:25
Um and so it's not just a picture of a skull.
1:08:28
It looks like a skull on the surface but it's actually the underlying mechanics of what it means to be a skull.
1:08:34
It's like it's deeper than just a a picture.
1:08:35
it actually understands what a skull is at some level or at least some of the components are are represented in this way.
1:08:43
And in some way it's like astonishing because like you know it seems like you'd have to be an engineer to think of like how would I organize the representation of a skull so that there's a single knob I can pull that would like move the mouth open and closed.
1:08:55
It seems like a human must be involved in this. But it isn't like that.
1:08:59
Of course humans are involved in picking the intermediate stepping stones but they didn't design the underlying representation. Nobody designed it.
1:09:07
That's what's so strange about it.
1:09:07
It's just emergent and yet it has this really organized feel which is as if it was designed.
1:09:14
And so that was just an interesting observation.
1:09:16
And one thing that occurred to us was that it may be that actually we had noticed that like if you run uh neat which is also what's in pigreeder we talked about neat at the beginning to solve a problem um say like to get a robot to go through a maze.
1:09:33
um if it if it does solve it and then if you also run novelty search which remember is kind of modeled on the non-objective way peepreeder works and it solves it.
1:09:43
So we have a case here where both algorithms solve it like the novelty version of need and the objective driven version of need.
1:09:49
We kept finding that like the the version that solves it with novelty would be three times more compact.
1:09:55
Um which is a very interesting observation.
1:09:58
We're not sure exactly why would this be, but like when you try to solve the problem when it's not done objectively.
1:10:04
So I guess I shouldn't say you're trying to solve the problem when you're not trying to solve it, but as a side effect of just exploring, you actually do accidentally solve it.
1:10:12
Your representation is way more compact um than if you actually are trying to solve the problem.
1:10:17
And then if you couple that with the observation um in PIP reader of these beautiful representations, it's intriguing.
1:10:22
So then we should ask ourselves like well much of machine learning as a field is about setting an objective and then using gradient descent or back propagation on a neural network which means moving towards an objective imp on purpose deliberately.
1:10:38
Um and much of the field tries to do it that way.
1:10:40
Um like stochastic gradient descent.
1:10:43
Um maybe uh those resultant underlying representations are actually pretty bad.
1:10:50
Um and so we thought to let's see what happens if we try to get the usual conventional way of learning the SGD or stochastic gradient descent um to to to output pipreer images and just see what it looks like inside.
1:11:04
And indeed it was it was terrible.
1:11:07
It's like really terrible.
1:11:09
It's it's just like a giant mishmash of meaningless blobs which somehow all come together to make the skull for example.
1:11:15
But like it doesn't look like how you would decompose a skull.
1:11:19
Um, and that's what we called fractured and tangled representation. It's fractured.
1:11:24
The the words have a certain meaning like fractured in the sense that things that should be unified together like the left and right side of a face are actually broken apart inside.
1:11:35
So that they're not connected.
1:11:35
So it means like you could mess up the left side and the right side would do nothing.
1:11:39
And and and so they're not connected the way you would expect them to be.
1:11:42
Um because that's what symmetry is is there's a connection there.
1:11:44
Um, and so in in in the FER, fractured entangled representation, we'd have this kind of fracture that we could see it explicitly.
1:11:51
And it's also entangled, which means that things that shouldn't be involved with each other are.
1:11:55
So it's like if you change the size of the eye, like the background changes or something like that, like these things shouldn't be entangled at all, but they are.
1:12:03
So there's fracture and entanglement all throughout the representation.
1:12:06
And then in neat uh in the pipreer version um which wasn't objectively discovered it's the opposite beautiful decomposition like you could actually understand how it built this thing just by eyeballing it like you don't need to be an expert in the field it's just obvious that it was decomposed into meaningful parts.
1:12:24
Um and so this this paper and we called that uh unified factored representation sort of the opposite opposite of fractured and entangled.
1:12:32
And so so this paper just shows this like it shows the contrast visually and says what does it mean like what does it mean that the way you got to the solution changes the underlying nature of the solution.
1:12:48
See this is about what this is about is deceptive appearances.
1:12:51
It's like both look just as good.
1:12:53
Like both produce a beautiful skull, the same skull, but like underneath the hood one is kind of an impostor.
1:13:00
It's like a complete piece of junk and the other one is beautiful.
1:13:04
Um, and so what does that mean?
1:13:04
Like there's many possible implications like like one possible implication is that what you could do with that skull going forward is very different, you know.
1:13:13
So like to imagine new skulls or new faces or things like that.
1:13:18
It's much easier if the representation is unified and factored than if it's fractured and entangled.
1:13:22
And we showed this in the paper that like you can you can um do what we call weight sweeps and see that like it's easy to get new kinds of images that mean something um if you have the the UFR version, the good version.
1:13:35
Um it's the opposite with the other one.
1:13:37
Um and so so so that's one one really uh deep point is that like the the continued use of these things uh is is impacted which is really important in the world of LLMs now or large models because what does it say about their potential creativity if they're characterized by FE or fracture and entanglement? We don't know.
1:13:57
It's not as easy to look inside these big neural networks and know but this is a hint it might be like that.
1:14:04
And so it sort of raises the question.
1:14:05
And the second big question is um why is the representation so much better when you find [snorts] something open-endedly without an objective versus when you find it with an objective?
1:14:17
What's the reason for that?
1:14:19
Because if we understood the reason for that um then we could algorithmically exploit that to make sure that the representation is good.
1:14:25
Um and so I think everybody who's reading that paper is thinking about this now like what what what is this like what how do we abstract this lesson in a way that we can actually formalize?
1:14:34
But actually I wanted to mention a third point because it's the cultural point which is which is funny that this comes up again that we can have make a cultural point but if you think about it um and this is something I've really been very recently thinking about this idea that the way that you got to where you are can matter more than where you are.
1:14:55
are. It's not just about neural networks like that's also about life you know because it's like who is the person you became even though you're successful as a result of the path you took like that might matter a lot for what happens in
1:15:10
the future and it's not even just about people like it's even like companies like the path you took to become a successful company um to have high profit whatever it is that you're looking for uh to be a unicorn might mean that the way you are inside is still VR. So, it's still fractured and
1:15:24
So, it's still fractured and entangled.
1:15:26
And what that means basically is that you aren't flexible and adaptable to change in the world because you came to the solution in a very objective way.
1:15:35
And even though you succeeded, you're paying a cost that's invisible right now.
1:15:38
invisible right now. Um, and I think that's really intriguing, you know, to think about this like in in the broader uh social human context than just in uh in terms of algorithmic optimization that it may have further implications uh there that the path you take actually
1:15:55
matters a lot um to to to who you became um and and who who you actually want to be like who are you on the inside um and so uh so that's something that I could I could imagine it could be another book but I'm not exactly planning to write a book right now. Um, but it has it has
1:16:10
Um, but it has it has kind of the feel of the original book.
1:16:14
Like there there's a lesson there that's really interesting.
1:16:18
>> I am absolutely fascinated by that.
1:16:18
I have thought about I I'm a journal keeper and I've kept journals for since I was 18 and I'm 65.
1:16:25
So I have the ability to look back and I I was looking for a journal but it's it's down there buried under some stuff.
1:16:33
But what I was going to show you was a graphic representation that I wrote when I was 23.
1:16:39
Basically what you just said and it was me at that point and I and I put all of these tree paths out.
1:16:47
And one of the notes that I made under it was these these matter.
1:16:54
These will go back and fall back in on themselves.
1:16:58
And the path that you end up taking is is going to matter.
1:17:04
It's going to matter if you took it like the traditional way.
1:17:08
I I was oriented much more towards the non-traditional way of doing things.
1:17:12
Uh like again when I founded my first company when I was 28, it was an asset management company and I was telling a friend about it and he goes, "Oh, cool. Who's your backer?"
1:17:24
And I went, "I I I don't have a backer."
1:17:26
And he he just looked at me and he goes, "Um, Jimmy, you're gonna fail."
1:17:33
[laughter] And I was like, "Well, I think that I've discovered some pretty cool things.
1:17:36
Um, I'm going to write a book about it and and I think that that book is going to be my calling card."
1:17:42
And he just like he was about 20 years older than me.
1:17:45
He just shook his his head and was like, "Oh, you poor naive young man.
1:17:49
There's there's there's no way that that is going to work."
1:17:54
But even the book, the first book that I wrote called In Best Like the Best, which basically showed that you could back to the idea of people getting obsessed on labels and names.
1:18:05
Um, when I first got exposed to the stock market, everyone was talking about the CEO or, you know, the company.
1:18:14
They were they they they were just talking about that particular thing.
1:18:17
and and and I was like a teenager and I was like I think you should like look at the underlying structure of these companies and see what unites them there.
1:18:27
And that was what my first book was about how you could clone any of your favorite money managers by putting the stocks that they own in their portfolios over time on a big database sucking out what the v what the variables were. Right?
1:18:42
So this manager loves companies that have high forecasted growth and low pees, whatever the combination and to abstract that knowledge and make it an algorithm and but then that gave me the idea, oh hell's bells.
1:19:00
I like there's no Bay Decker uh guide to, you know, like everyone says I like stocks with low PE.
1:19:09
All right, let show me how they perform.
1:19:12
And I went looking and there was [clears throat] some academic work by French and Bama but only on price the book and that gave me the book that actually did make my career what works on Wall Street.
1:19:21
And so the I I love that notion.
1:19:26
I really think you should turn it into a book because I think it I think a it also has a much broader cultural impact as you yourself were implying. >> Yeah. Yeah. This is a great story. Yeah. I It's true. True.
1:19:40
I mean, it's um I I hadn't thought about it as much over my life, but it it is it is intriguing and it's very culturally relevant like that how much the path can matter even if you're successful.
1:19:52
Um you know, which just means you you met the objective criteria that you were you were hoping to meet.
1:19:57
Um so it's just another downside of objective thinking.
1:20:00
downside of objective thinking. um you know and and I was I was talking to another I was talking to another uh podcast interviewer um about this and you know he was he often advises um founders is what kind of his his work his normal work is and he was saying you
1:20:17
know like a lot of founders they they get to their milestone that they do what they were hoping to do but they're getting divorced and they're depressed and all their employees hate them and it's like there's all this other stuff that comes um which is like part of just being objective. It's like if you're
1:20:30
It's like if you're just only objective and you pursue things in the most deliberate determined way, you have to get to this point and that's the only point you care about like there's all these casualties in the underlying well in the neural net we call it the representation but here it's like the the underlying fabric of your life.
1:20:46
Um just how it's interconnected that's just really I think it's just very thoughtprovoking you know that there's a way to to get to a point that's more that's that's more virtuous than another.
1:20:57
Um and it's not just about moral things.
1:20:59
moral things. I mean it relates to things like creativity, you know, because if you think about um like the difference between somebody who became who who got like a great score on some college test but got there by always following the textbook, always acing all
1:21:13
the tests on the road versus somebody who got there also did well on that test but got there through a completely different path of inventing things for themsel, exploring for themsel, maybe having contrarian opinions that they they looked into on their own for their own purposes. like that person is coming
1:21:25
like that person is coming in with a completely different representation of reality.
1:21:29
And so the question is like given these two people who both acceed the test, which one is going to have a more interesting future career?
1:21:37
Um, and it's it's it's probably path dependent.
1:21:41
I mean, obviously I think it's probably the more creative path, but even if it wasn't, it still shows path dependency.
1:21:46
It's like how you got there is going to affect what you're going to do with that success in the future, what kind of person you're going to be.
1:21:53
Um, and there's just endless things to think about there, you know, because I think it it can kind of diffuse a little bit of objective obsession and the feeling that the only thing that matters is like the final metric of where you've gotten.
1:22:04
Like it sort of helps to show why you don't have to feel like that cuz I think that's a very stressful way of looking at the world.
1:22:11
It's like this is the only measure of my success to understand that like you're you're not just successful just because you met the metric.
1:22:16
Um, there's a lot more to it than that.
1:22:18
And actually some of that other stuff is more important and even if you don't meet the metric the other stuff could still be good.
1:22:24
Um and so like life is much more complicated than that. >> Yeah.
1:22:29
We have a an informal heristic that we are far more interested in the second of your two candidates.
1:22:36
Um the the one who got there um in a circuitous or in a very nontraditional path.
1:22:43
Now, it doesn't always work, like nothing always works, but we have found that it's worked pretty well for us when we're working with people.
1:22:55
The the the the ladder climber is what I call them.
1:23:02
The the ladder climber, great. Good for you. God bless.
1:23:04
Uh we're much more interested in the one that is like, "Huh, I wonder if there's another way for me to get there." That ladder.
1:23:13
Yeah, but I bet that there I bet I could do a very clever thing.
1:23:19
It's like the old Prussian general who um who was asked, you know, how do you determine where you're going to send a soldier? And he had a matrix.
1:23:29
Um and and the the part of the matrix was lazy and clever was his best thing [laughter] because he he he said promote to the highest part of command.
1:23:42
and and foolish and and lazy, they they were cannon fodder.
1:23:48
But the the clever and lazy because they'll they they will almost by definition >> figure out a way to do it more efficiently, more elegantly, whatever you have.
1:24:02
I am I cannot believe that you and I have been talking for nearly 90 minutes.
1:24:07
My producer is just texting me.
1:24:10
Um this has been absolutely fascinating.
1:24:13
I hope you enjoyed it as well. Uh I love your work.
1:24:16
Uh everyone listening, buy his book, read his paper.
1:24:19
This is really fascinating stuff.
1:24:22
And you you you you need to understand it because the world we're going into, I think, is going to be uh you're going to be a lot more successful whatever you want to be doing if if you understand uh the way Ken looks at the world.
1:24:37
Um, as as if you've ever listened to the podcast, you you know that uh our our last question here is always kind of a fun one uh because it's very different uh from from your perspective, but that is we're going to make you emperor of the world.
1:24:55
You are the emperor of the world and you can not put anyone in a re-education camp. You cannot kill anyone.
1:25:05
But what you can do is we're going to give you a magic microphone.
1:25:09
You can say two things into it that will incept the entire population of the world.
1:25:15
The next morning, whenever their next morning is, they're going to wake up and think, you know, that's really interesting, these two ideas I've just had.
1:25:26
I'm going to pursue both of them.
1:25:28
What are you going to incept into the world, Ken?
1:25:35
And and you said two words or is that >> No, no, two ideas.
1:25:37
They can they can two ideas. >> Yeah, two ideas. >> Okay. Okay. Um and Okay. Yeah.
1:25:42
So I um so yeah, first I just want to say that this this also was a really fast 90 minutes. I really enjoyed it. Uh thank you for this.
1:25:50
Thank you for this opportunity.
1:25:51
Um and so what do I want to incept into the world?
1:25:54
Um well first just that um try to imagine what you would do if you had a day where you had no objectives and what would you do?
1:26:03
Um and um secondly, um sort of going back to the to the the second main insight that we talked about, um I think it would be interesting.
1:26:21
Well, actually, but I'm supposed to be commanding the world in some way. Is that right?
1:26:25
So, I need to >> Well, think of you think of yourself as more of uh you you are the uh world whisperer. >> Okay. Here's what I'll do. Here's what I'll do.
1:26:36
Um, all the universities and colleges, um, you can no longer consider grades in admissions and there's no more grant applications.
1:26:47
You're not allowed to make people apply for funding.
1:26:48
Um, figure out how to operate and how to get the right people into the university. >> I love both of those.
1:26:55
[laughter] Man, if that one actually happened, just think of the the Cambrianlike explosion of innovation and interesting ideas.
1:27:07
Ken, this has been so much fun. Thank you for your time. Thank you for the book.
1:27:12
I recommend it highly and uh really delighted to finally be able to have a chat with you. >> Likewise. Yeah, thank you. This was great. I wish we had longer. It was a lot of fun.