Ep. 68 — Kevin Zatloukal: Machine Learning And Its Applications

0:02

Hi, I'm Jim O'Shaughnessy and welcome to  Infinite Loops.

0:02

Sometimes we get caught up in what feel like infinite loops  when trying to figure things out.

0:11

Markets go up and down, research is presented and  then refuted, and we find ourselves right back where we started.

0:17

The goal of this podcast  is to learn how we can reset our thinking on issues that hopefully leaves us with a better  understanding as to why we think the way we think and how we might be able to change that to  avoid going in infinite loops of thought.

0:36

We hope to offer our listeners a fresh perspective  on a variety of issues and look at them through a multifaceted lens — including history, philosophy,  art, science, linguistics, and yes, also through quantitative analysis.

0:50

And through these  discussions help you not only become a better investor, but also become a more nuanced thinker.

0:57

With each episode we hope to bring you along with us as we learn together.

1:04

Thanks for joining us,  now please enjoy this episode of Infinite Loops.

1:11

Disclaimer: Jim O'Shaughnessy is chairman and  Co-Chief Investment Officer of O'Shaughnessy Asset Management, where Jamie Catherwood is  an associate.

1:17

All opinions expressed by Jim, Jamie and podcast guests are solely their  own opinions and do not reflect the opinions of O'Shaughnessy Asset Management.

1:26

This  podcast is for informational purposes only, and should not be relied upon as  a basis for investment decisions.

1:33

Clients of O'Shaughnessy Asset Management  may maintain positions in the securities discussed in this podcast.

1:37

Jim O'Shaughnessy: Well, hello everyone.

1:41

It's Jim O'Shaughnessy with  my colleague Jamie Catherwood for yet another Infinite Loops.

1:48

I am delighted with today's guest,  Kevin Zatloukal, who is a data scientist and an OSAM Research Partner. Kevin, welcome.

1:56

Kevin Zatloukal: Thank you. Glad to be here.

1:59

Jim O'Shaughnessy: I was very excited about this because you are a  machine learning, artificial intelligence expert.

2:08

You came out to Connecticut  and gave what I thought was like one of the most illuminating  talks on what it is and what it is not.

2:18

And I still remember one thing you said to me  afterwards, you said, "Listen, if you heard anything about what AI can do from a marketing  person, that's wrong."

2:23

If you wouldn't mind, given the fact that we have the new mRNA vaccine ...

2:30

I guess it's really not a vaccine though because it doesn't actually have any of the COVID in it.

2:37

And that sprung from AI's ability to fold protein.

2:44

If you could just give us ...

2:44

Because I thought  I knew something about machine learning, and then I was there for your talk and I knew  I basically didn't know anything about machine learning.

2:54

My guess is that a lot of our listeners  are going to be in the same boat.

2:54

So if you could just kind of at a high level explain both  the process and where you see it going.

3:05

Kevin Zatloukal: Let me start out by saying, I also feel often like I have no idea what's going  on in this field because it's changing so fast.

3:12

And one of the things that's happened for  me is it's been kind of this process of unlearning things I learned when I  first learned the subject 20 years ago, because the new developments are just teaching  us a whole bunch of things we thought we knew were wrong.

3:24

All the deep learning stuff in  particular is so opposite from what we were taught that I'm having to erase things from my  brain as fast as I'm adding things to my brain.

3:35

So it's hard to feel like an expert,  I'll say that about any of these topics.

3:38

But yeah, the basic idea with machine learning is  to ...

3:38

Actually, my preferred form is the one that Pedro Domingos likes to use, where he says that,  "Normally when you're writing a computer program, you're producing this program that takes inputs  and produces outputs.

3:48

In machine learning, you take inputs and outputs and it produces a program  that can go from those inputs to those outputs."

3:57

And so the idea is to give a bunch  of examples with the correct outputs, we call them labels usually, and then ask the  machine learning algorithms, try to figure out a mapping from those inputs to those outlets.

4:06

And in particular, the whole idea of the subject is that it needs to work well when you give it  examples it's never seen before. That's the whole trick.

4:16

It's very easy to make something that takes  the inputs you have seen to those outlets.

4:16

You can just memorize them, right? I mean, have a table.

4:20

But to go, we need to come up with a mapping that when you give it something you've never seen  before, it does a really good job of actually correctly guessing what the output label is going  to be.

4:27

And that of course is hard, but it turns out quite possible.

4:31

And lots of techniques  that do this are getting better and better and better as you've I'm sure seen.

4:35

Jim O'Shaughnessy: Definitely.

4:37

And I was talking with somebody  else earlier and they were talking about the, I guess it's ...

4:41

What's Google's deep  ...

4:41

What do they call their thing? DeepMind?

4:45

Kevin Zatloukal: DeepMind is a, yeah, separate organization, I think, or a separate part of Google. Yeah.

4:47

Jim O'Shaughnessy: But the guy I was talking to is a chess champion,  and he was talking about how for the first time all they did was load in ...

4:55

They didn't load in  any of the games, they just loaded in the rules.

5:00

And the program quickly figured out how  to beat and it just started beating all of the grand masters. Kevin Zatloukal: Yeah.

5:05

This is AlphaZero and, yeah, it's  changed chess so completely.

5:05

I have to admit, I've gotten into chess a lot over the last few  months, and partly because my kids enjoy watching chess videos and chest streamers, and my life's  all about spending time with my kids before both of the bigger ones are out the door.

5:18

But I started  watching these videos originally to see AlphaZero play, and it's incredible.

5:22

I mean, it just  appends ...

5:22

It's a wonderful example of how ...

5:26

You think of machine learning as we're just trying  to automate some process, but then you can do it so well in some cases that it can append our  understanding of the topic.

5:30

And actually the protein folding you mentioned is another case  where it's doing things we couldn't do.

5:33

The chess programs are playing chess better than humans  ever could and they're teaching us about chess.

5:42

And the protein folders are  folding proteins that we couldn't have figured out before and advancing science.

5:45

We have this just incredible acceleration of what machine learning is doing that it just feels like,  as I said, it's hard to feel like an expert when everything's changing.

5:53

The capabilities are just,  they're just washing over all of us right now and it's hard to see what's going to happen next.

5:58

Jim O'Shaughnessy: One of the things that I had asked you  about in a separate conversation was the abundance of data issue.

6:04

And I think I  put the question to you like, if we take me, and I'm 61 years old, if we use pictures of me as  an example, the pictures of me in 1960 would there be one.

6:19

And then when I was 18, maybe there'd  be 100, and now there'd be like 500,000, right?

6:26

Because of all the data that we now have captured  and have the ability to feed into the programs.

6:32

And my question to you was, "Is this creating  some form of bias that we're not even aware of?"

6:38

By that I mean, is there a recency bias, right?

6:38

This data set that we've just experienced, somewhat unusual, right?

6:44

And my question goes  to, is there a way to correct that?

6:44

Can you load in just false data?

6:49

Kevin Zatloukal: Yeah, I'm sure that it does introduce a bias. You're absolutely right.

6:51

And machine learning researchers study other kinds of biases, right?

6:55

I'm sure you've seen some of these where they'll train a machine learning system to do something  instead of humans, and the humans would normally do themselves.

7:03

And then they discover that because  they get examples, let's say, included racism or sexism or whatever, the machine just learned to  be a racist or sexist, right, from the examples.

7:14

So that sort of thing does happen, and so people  do study it and have to ...

7:14

They require human beings to look through and figure out what's going  on.

7:17

The machine can't tell, right?

7:17

It doesn't know what racism or sexism are, it's just looking at  the examples.

7:21

It requires a human being to study it, find the biases and then come up with ways  to fix them.

7:25

And so I don't know that there's general techniques that I've seen, it's case  by case.

7:30

It's not, "When we identify this, how do we fix it?"

7:33

usually it's by adding ...

7:33

The simplest thing would be just add a lot more data from the ones that are under-sampled so  that the machine will learn properly.

7:36

But then you try that and then you look at how a human  being looks at the output and sees, "Is the bias gone?"

7:44

And then if not, they try  something else.

7:44

I mean, one thing that you quickly learn about machine learning,  if you read the reports from the media, they typically describe machine learning as, "We  took this computer and we set it out in the rain, and it was struck by lightning and now it's  become conscious."

7:56

And that's not what happens.

8:01

It's like then you read the report and it's  like these 12 PhDs worked on this project for three years and now the machine can do this, and  it's like, "Well, what were the 12 PhDs doing for three years?"

8:10

It's actually really hard to get  these things to work and also a lot of human intelligence to keep trying to tweak this thing to  force it to do the right stuff.

8:13

Machine learning is very hard.

8:18

It's very hard and requires a lot of  human work to make it happen.

8:18

But once it's done, it's a huge ...

8:23

You've automated this process or  maybe done some process you couldn't have done yourself, like in the case of protein folding.

8:27

Jim O'Shaughnessy: But that leads me to the question, when we're  auditing it, for example, we all have biases, we all have priors.

8:35

And I wonder whether are  we actually reducing the ethicacy of a machine learning program if we tell it, "No, you can't  think that.

8:42

You can't make that connection," or do you think that, that comes out in the wash?

8:48

Kevin Zatloukal: Well, I mean, you can measure the efficacy also,  and so you can watch the two and try to be careful about that.

8:55

But sometimes people will decide.

8:55

There's issues of fairness that come into play that are more important than the accuracy.

8:59

I mean,  if just to pick a silly example, if it's like it's extremely accurate if we just don't  give loans to people in this zip code, it's just not fair to people who were born  in that zip code, or maybe let's say it's not where you live now, it's where you were  born.

9:10

Literally, you just can't change it.

9:14

We just say it's not fair.

9:14

It doesn't matter if  it's accurate or not.

9:14

People do study this sort of thing and they take it seriously.

9:18

And they're  working hard at trying to identify them and then fix them.

9:22

So far I haven't seen any that can't be  fixed, but step one is you got to keep your eyes open and actually check for these kinds of things.

9:26

Jim O'Shaughnessy: Another thing that I was chatting with my cousin's  daughter's husband ... We'll leave it there.

9:28

He has an MD and he has a degree, a PhD in machine  learning.

9:35

And so he was telling me about being retained by a big pharmaceutical company.

9:41

They'd  spent a lot of money on a drug, and when the trials, the traditional trials came back, it said,  "No good. No efficacy. It's not going to work."

9:56

And somebody at the company had been out with him  [inaudible] explaining, "You might be wrong about that.

10:02

If you run it through some of these machine  learning programs, there might be a good use for this drug."

10:08

Long story short, they retained  Grant's firm.

10:08

They did the runs on the data and they found indeed that the drug was incredibly  efficacious for a very specific sub-segment.

10:24

So post-menopausal women who were slightly  overweight, not obese, but slightly overweight.

10:30

And so a drug that would have been a billion  dollar hole, actually became, went into circulation, and apparently the results that it's  achieving for this sub-group are extraordinary.

10:42

I guess the question that I have for you,  and maybe I'm a bit Panglossia in here, but are we moving into an era where we're going  to see the emergence of sort of personalized medical treatments after the machine learning  figures out how it's going to be good for people like me or you, and then we get our own genome  typed, or is that still like science fiction?

11:07

Kevin Zatloukal: I mean, I think there are public companies that are doing things along, that are  certainly doing personalized medicine.

11:08

I'm trying to remember which ones.

11:11

I'm certainly not a health  care or a biopharma expert or anything like that, but I'm trying to think.

11:16

I'm pretty sure there are  public companies that actually work on this right now.

11:20

And I don't know that they all use AI, but  AI has been very effective in other ...

11:20

For drug companies, for example, they have AI programs  that will try to figure out which of a whole bunch of drug candidates are likely to work.

11:29

And then they can filter down to a set that seem most likely and then start testing  those.

11:34

And those have been a huge speed up in trying to identify drugs.

11:38

I think the answer  to both of those is, yes, I guess I just don't know whether those two are tied together,  whether the AI will play a role specifically in doing the personalized part of it.

11:47

But definitely  personalized, here's where things are in the works now, and AI is playing a huge role in healthcare  now.

11:52

So you would expect that those two will intersect at some at some point.

11:57

Jim O'Shaughnessy: Another thing, I am very bullish on this space  and so I always try to check my priors.

11:58

Because I talk to a lot of people who are frightened.

12:04

They're frightened about ...

12:04

And I'm not talking about dummies here, I'm talking about ...

12:07

And  I've read some essays, Nick Bostrom, I don't know whether you read his stuff, but he was the  guy who came up with the simulation hypothesis, the serious version of it.

12:16

And he writes about,  essentially he's at Oxbridge and he writes about existential risks.

12:21

And one of his things on that  list is emergent aware AI, and how that AI might have a very different view of humanity then than  we have of ourselves.

12:33

Any worries from where you sit about that kind of thing happening?

12:39

Kevin Zatloukal: No, I mean, not personally.

12:42

I'm not  worried about that sort of thing at all.

12:45

First, because, again, machine learning is  really hard, so somehow we're going to make some super intelligent being.

12:50

I mean, I've seen  multiple movies and TV shows about this idea, so I think the is certainly well ahead of ...

12:55

I mean, Person of Interest, right, had this with the AI and then of course the Terminator and  ...

12:59

This is very popular in entertainment land.

13:05

But I mean, we have no idea how to do anything  like that.

13:05

I mean, we're not even ...

13:05

I mean, replicating the general intelligence of the human  brain, I've never heard any plausible idea about how we can do that other than let's simulate the  physics of the brain directly.

13:14

People will say, "We'll eventually will be able to do it because  we'll just simulate all the physics of your brain, and then we can match the human brain."

13:22

And I guess in principle, maybe that seems reasonable.

13:25

It doesn't seem like it'd be very  efficient at least with the computing power we have now, but I don't have any counter  argument to that.

13:28

But again, what will be the point of just simulating a human brain?

13:31

If  you're trying to say, "Well, it's come up with some super intelligence.

13:35

It's more intelligent  than human beings," we don't ... Yeah.

13:35

There's no mechanism in my mind that we could  get there, so I guess that reason I'm not afraid of it. Jim O'Shaughnessy: Yeah.

13:41

And one of my reasons behind it is  we still have a very poor understanding of consciousness itself. Kevin Zatloukal: Yeah.

13:48

We really have no idea, right?

13:48

It's like this fundamental question about human beings that just we don't even know how to  tackle it.

13:51

It's one of those questions that's so hard that there's a 99% chance if you feel like  you made progress on it, you just fooled yourself.

14:00

You convinced yourself of something  stupid, that's the most likely explanation.

14:03

Jim O'Shaughnessy: And another one of my little hobby horses that I've talked to you about and hope to talk to  you about even more is the use of machine learning in stock market and other investing situations.

14:12

I put it to you that I sort of look at the stock market as a complex adaptive system with feedback  loops that works pretty well in general because of heterogeneity in opinions, right?

14:27

So you might  be buying Apple because you've got young kids.

14:32

I might be selling Apple because I want to  give a gift to my grandson.

14:32

And they're both legitimate and good reasons, right?

14:38

But part of  my theory also is that when something happens and it has a bunch of different names, Black Swan,  whatever, that a information cascade starts.

14:53

And people's opinions move from being a  heterogeneous to being homogeneous.

14:53

In other words, everybody's thinking the same thing.

14:58

And one of my questions, and I'm not afraid to talk about it because it's going to be  like you say, machine learning is hard, is there a path forward using machine learning  or AI to not predict an event, but to confirm an event?

15:20

In other words, let's take the  oil plunge.

15:20

Remember when it briefly went negative.

15:27

Do you think that is there a way that  machine learning experts could build programs to look at the data that would allow us to infer  anything at all, or am I just being crazy here?

15:40

Kevin Zatloukal: Well, you're asking incredibly good questions.

15:42

Let me start with saying I don't know.

15:42

It's a  great question to think about.

15:42

I think it sounds very plausible.

15:46

I mean, the difficulty is going to  be we don't have that many examples, right?

15:46

And so it's obviously harder the fewer examples you have.

15:50

It's harder for anybody, a machine or a computer.

15:54

I like to point that out too when people sometimes  will say about machine learning, "Oh, well, what happens if the future is not like the past?"

15:59

it's  like, "Well, then you can fire all of us, because what are any of us going to do in that case?"

16:03

So yeah, it needs examples to learn from that the future has to be like those.

16:06

But yeah, it seems  plausible that if there's things you can look at under the surface that can show you that it  is one of these information cascades going on, that maybe now we do have the data to track  it.

16:16

We're watching Twitter, or what have you, picking up clues and how people are  talking that it's really the same idea, and it's spreading and becoming universal.

16:23

It's possible that you could pick that up.

16:27

I'm not sure that you'd even need machine learning  for it, but probably natural language processing you'd probably need, but which is very AI heavy  these days.

16:31

But yeah, it seems very plausible that you could see that, and then after the fact,  from those things, you could confirm that, "Okay, when it gets ...

16:39

That's what's going on  here," and you could come up with a marker that was at least on the historical examples perfectly  accurate predicting when it's happening.

16:42

That doesn't mean you won't see one where it's wrong  in the future, but it seems very plausible to me.

16:50

Jim O'Shaughnessy: of course.

16:50

As a quant, we look for things that are directional.

16:52

And we buy  a group of investment stocks in our case, where we go into the purchase knowing that X percent  of them were going to be wrong, because of the tests that we've done in the past.

17:03

We're always  looking at the aggregate.

17:03

And I just think that complex adaptive systems, emergence comes from the  bottom, not the top, I think.

17:09

But just to further drill on that question a little bit, what ...

17:15

We have Jamie on who's a financial historian and could probably populate God knows how many stories  that actually did happen from financial history, but what about alternate histories where we  just make it up?

17:27

What do you think about that?

17:31

Kevin Zatloukal: Can you say more about what you mean there? Jim O'Shaughnessy: Sure.

17:33

In 1987, rather than recovering quickly, the crash of 1987 ushered in the Second Great  Depression.

17:39

And we could speculate and build a pretty robust model in my opinion of what  that would mean in terms of unemployment, in terms of stock prices, in terms of national  mood, psyche, et cetera, because we have a lot of depressions that we can sample from.

18:02

That  would be one example.

18:02

Another example would be the great financial crisis, the fed decided,  "No, we're just going to let them all fail."

18:14

And we had a cascade effect, because by  the way, that's what would have happened if the fed had not decided to intervene.

18:20

And  so we could pause it, a fed that says, "No, we believe that there has to be a skin in the  game and that moral hazard is a real thing, and you guys up.

18:34

And guess what, all your banks  are going out of business."

18:34

And by the way, that would have included banks like Goldman Sachs,  that would have included ...

18:39

I mean, the list would have been heavy.

18:43

So none of that happened,  right?

18:43

Kevin, what do you think about that?

18:43

Do you think that counterfactuals, alternative history,  do you think that, that would help or hinder?

18:54

Because after all, it's not real.

18:54

We didn't have  another great depression, the fed didn't let those banks fail other than Bear Stearns and Lehman  Brothers.

18:59

Do you think that, that would be helpful or have no effect? What's your thoughts?

19:04

Kevin Zatloukal: Well, let me go with I don't know again.

19:07

But  I guess I want to return to my point about machine learning being very hard, because it's  a very interesting idea, high level.

19:11

But then computers are incredibly stupid, right?

19:16

That's why  programming is hard.

19:16

All they know how to do is multiply and add numbers, divide them and compare  them, right? That's it.

19:20

Turning anything you want to do into that is a really difficult task.

19:25

That's  why programmers get paid the big bucks, right?

19:30

To translate complicated things like you  recognizing faces and to just adding numbers and so on.

19:34

So you have to turn it into ...

19:34

Before you can unleash the computer on it, you've really got to nail it down in terms of  exactly what you want.

19:37

What date is it going to look at and what exactly is it predicting?

19:42

And so I'm not sure what that would be for the counterfactuals, because for the cases we have  now, you could say, "I'm going to try to learn an information cascade from the Twitter data."

19:51

If the counterfactual, we wouldn't have the Twitter data.

19:55

We could imagine a price history  and we could say, "Can we figure it out from our price history if it's an information cascade" And  so we could make a bunch of those and it would be able to learn from them. That's certainly true.

20:03

We  would be able to build a function that says this price history is ...

20:07

But it will be just based on  the examples you gave it.

20:07

So it would really be learning the theory that was in your head that  caused you to come up with that price history.

20:16

I think, I guess for me it feels more like a  human task, because it really involves theory.

20:22

It involves using your general, your understanding  of not just markets, things outside markets, this really general intelligence that humans  have rather than this very, very specific things that computers can do.

20:32

I think ultimately if  you gave enough examples the machine could try to learn what you told it, but it's really  just learning.

20:36

The real work was producing the examples, right?

20:39

And the machines, I think, not  really giving anything clever to the process, if that makes any sense. Jim O'Shaughnessy: Yeah. No, it actually does.

20:45

And I will  pepper you offline with some other questions that maybe I don't want to give away the  whole thing.

20:50

Let's turn to a love of yours, and that is your use of machine learning  and AI in sports. Tell us about that. Kevin Zatloukal: Yeah.

21:01

I'm a computer scientist.

21:01

I took machine learning as an undergrad and a grad  student.

21:03

It's one of many subjects I studied, but I didn't really start doing it a lot  until I started playing Fantasy Sports.

21:12

I guess I can go back a little further in this  story, if you want.

21:12

It's kind of a sad story, so I'm not sure [crosstalk] Jim O'Shaughnessy: Well, sad trombone. We want to hear it. Kevin Zatloukal: Okay.

21:21

I don't know, some number of years  ago, I guess eight years ago my dad had terminal cancer.

21:26

And we were trying to spend  as much time with them as I can before he died, and so I'm going three days a week, four days a  week.

21:31

And you just run out of things to talk about at some point, right?

21:35

But you just, you want  to spend time together.

21:35

One thing we did is we actually watched a ton of Man v. Food.

21:38

So  if I ever meet Adam Richman, I want to thank him for all the hours I got to spend with my dad  and that gave us something to do together.

21:43

Even though it's lighthearted, it really helped us.

21:47

But the other thing was my dad's big passion is football, American football.

21:51

And so I thought,  "I don't know that much about football."

21:51

I mean, I've gone to some games.

21:56

But what I can do is  get this fantasy side of it.

21:56

This Fantasy Sports, it's all statistics and everything.

22:01

It seems  like it'd be fun for me.

22:01

And in fact, I'd already started playing a little bit with some friends  Fantasy Premier League, which Jamie plays also.

22:10

And so I've been doing a bit of that and I  thought, "I'll go ahead and study this."

22:10

And so it turned out once I started doing it, I  got really into it.

22:13

And eventually, and it helped there at the time because it wasn't long  before I knew the names of all the players, and all the starting players and all the  teams, at least the fantasy relevant ones.

22:23

And so my dad and I had a lot to talk  about.

22:23

But then after he passed away, I'm still really into it.

22:26

I'm still into football  because it was my dad's favorite sport.

22:26

But the fantasy part I just gotten poured more and more  into it.

22:30

So it's machine learning and every new advanced machine learning technique, I'm trying it  out.

22:35

And I'm certainly not the only one, there's been really success with deep learning applied  to various Fantasy Football problems of late.

22:44

They have this thing called the big data bowl  every year where they hand out a bunch of data related to football and let people compete over  coming up with a machine learning system that can predict this or that.

22:53

And the last two years,  I think, the winners have been deep learning type systems and actually people who didn't know a  lot about football.

22:58

They've been people with machine learning expertise have come in and  crushed the people were football knowledge.

23:04

And so these techniques have been really useful  at predicting, trying to understand what's going on.

23:08

Anyway, I've been just looking for ways  to beat my friends at Fantasy Football, and that's gone well.

23:11

Although like anything, like  you said, a lot of these things are, these sort of markets, if you will, are complex adaptive. They  change over time.

23:14

And so I had an advantage at one point, for example, in Fantasy Premier League that  I was doing really well and that got ebbed away.

23:24

Eventually the information I had ...

23:24

I was  scraping ESPN data.

23:24

I'm not sure if I should admit this on the stream, but anyway, scraping  ESPN data, finding the exact shot locations of where every shot took place in a soccer match,  and then using that information to predict who is going to have better games in the future.

23:37

And  then eventually other people started doing the same thing.

23:41

And now you can just go to a website  and the projections are right there for everybody.

23:44

Jim O'Shaughnessy: Is this like xG stuff?

23:45

Kevin Zatloukal: Yeah, this is xG and xA and so on. Yeah.

23:47

I was doing that number of years ago when  nobody had it and it was a huge advantage, but now everyone has it so it's gone.

23:52

And I don't have the  edge at Fantasy Premier League that I once did, so I need to put some time in.

23:57

In Fantasy NFL, I  still have edges, continue to do a really good job of predicting which rookies are going to succeed  in the NFL.

24:02

And that gives me a nice advantage.

24:07

There's some other arbitragers I do using  a model on this data to predict that data, and then look for mispricings basically on the  other data and then buy and sell and so on. That's all working.

24:16

So because it's working, I don't  have to spend that much time out.

24:16

I just collect the money at the end of the year from the people  I play with.

24:19

I mean, I don't win every league, but I have good seasons and win more than I  lose.

24:23

So yeah, it's going great and it's been a great playground for machine learning.

24:27

And that's really what I said got me into machine learning, again, after grad school and, yeah.

24:31

Now, of course I do it all the time, as you know.

24:36

It's just a constant learning about new types of  machine learning techniques.

24:36

And first thing I do, of course, is financial data now, but the second  thing I do is go and figure out if I can use it to win a little more Fantasy Football.

24:44

Jim O'Shaughnessy: Well, betting markets in general, I think, would  be very interesting from a machine learning standpoint, right?

24:53

I mean- Kevin Zatloukal: Yeah, absolutely. Jim O'Shaughnessy: ...

24:55

That leads me to a question, as you pointed  out, about things getting armed away.

24:55

I'm sure most of our listeners are familiar with the book  Moneyball and how Billy Beane built that team using basically insights that were foreign to most  baseball scouts.

25:08

And I wonder, as we go forward, how quickly do you see this stuff getting ebbed  away?

25:17

And is there a potential for a so-called arms race, if you will.

25:24

And if so, is the alpha  portion of your wins going to shrink over time or can you stay far enough ahead you or anyone  else building these machine learning programs where they can get kind of good consistent alpha?

25:40

Kevin Zatloukal: Well, let me think of different answers for  Fantasy Sports versus the market, so let's say the stock market.

25:47

Fantasy Sports, I'm still feeling  good about my edges.

25:47

And the reason is there's so little data in Fantasy Football it's really easy  for people to find excuses for losing every year.

25:58

They can explain away this example and that  example, and that example.

25:58

It doesn't take long before they think, "I just got unlucky."

26:01

And  they never look back at the last five years to see how much of their money I have now to really ...

26:06

So there's this behavioral edge that they don't want to confront that, that I think is keeping  it there now.

26:10

I'm not as concerned about that getting ebbed.

26:14

And also, there's not a ton of  money on the line in any case.

26:14

The stock market, I think it will eventually get there.

26:18

I think  I often look at sort of where chess is now, that we'll be in a similar place eventually  with stock market, where if you watch any chess like I do, everybody wants to know ...

26:28

They'll put grandmasters on, and if you listen to the grandmasters then it's like, "Yeah, yeah.

26:32

But what does the engine say?"

26:32

They want to know the engine, because the engine is better than the  grandmasters, and the engine can tell you is this a good position or not.

26:39

And so I think we will  end up in that position where we have engines for ... If you go to Yahoo!

26:44

Finance or  whatever, or Bloomberg, and you just see, "Here's the engine's evaluation on this stock, and  it's better than the humans," and it doesn't mean that humans can't beat it, because  actually the grand masters can tell you when the computer is wrong and they'll  know why it's wrong and why can't see it.

27:00

But most of the time that's actually the default,  is what the machine says.

27:00

And I think we'll end up there.

27:05

But I don't think it's going to take ...

27:05

I don't think it's going to be fast, I think it's going to be slow, and Moneyball shows you why.

27:07

There's this culture class in Moneyball.

27:07

There's books about that as much as anything.

27:12

It's easy to  imagine it with stock pickers, right, that they're not going to want to trust the machines.

27:18

And then  even within machine learning, I mentioned at the beginning, I have to do this unlearning process  to move on to new techniques, because things that I was taught 20 years ago, they're not true now.

27:26

And it's hard to take something that worked for you and you were told was true and erase it from  your brain.

27:31

I started with traditional statistics type techniques, and then eventually you see  erasing statistics part of it makes it better.

27:36

And it's hard to do that when you took classes in it  and you were told this is the right way to do it.

27:46

And then you throw away that and you go to  these more machine learning type techniques, but even there, one of the basic ideas is that  you're trying to prevent over-fitting by keeping the models simpler, right? Not too complicated.

27:54

And then deep learning comes along and they say, "No, we're going to make the model really  complicated, really complicated.

27:58

We're going to over-parametrize it."

28:01

And it actually wins  and you're like, "This is the exact opposite of what I was taught."

28:05

It's hard mentally  for any person to go through these changes, and I think for those reasons it's going  to be a slow process. That's my sense. Jim O'Shaughnessy: Yeah.

28:15

And I agree with you on that, because I think another thing, a question that  I put to you was, and I think I phrased it, "You're normal human being given the way we  are designed."

28:21

And our human OS that we all run is going to have a very hard time,  because if there was a program, for example, that could tell you what  and when, but couldn't tell you why, that people would just have a really difficult  time accepting that because we are such a narrative-based creature.

28:43

And we do that  for a lot of reasons, illusion of control.

28:49

That's a whole different topic.

28:49

But I would have  no problem with that.

28:49

In fact, I would say, "Yeah, bring it on. I love it."

28:54

Kevin Zatloukal: I agree with you completely. Jim O'Shaughnessy: Yeah.

28:57

I mean- Kevin Zatloukal: Let me, if I might.

28:58

You know more about this,  so let me ask you.

28:58

I feel that people have this really strong psychological need to know why.

29:01

When I turn on like CNBC or Bloomberg, what I see is all day long people saying, "Why is this  happening? This went up today. That went ... Why? Why?"

29:11

And then they bring in people and people  say, "Oh, I think it's this. I think it's this."

29:15

And the humans watching it, it seems to me like  they're given this sort of buffet of options, and then they find one, "Oh,  that one resonates with me."

29:22

And they don't actually really care if that's the  right explanation.

29:22

No one's going back after the fact of trying to figure out what was the right  explanation.

29:25

It's just find a why that works for you and that makes sense to you, and then you can  accept it and you feel good, or it gives you this positive feeling.

29:34

You feel more comfortable and  you move on.

29:34

It doesn't really matter whether it was true or not. What do you think? Jim O'Shaughnessy: ...

29:39

That's absolutely correct, and it's been  the focus of my work for most of my career.

29:43

I believe that it's rooted mostly in the  need for an illusion of control and for an abhorrence for assigning anything a random reason  for it happening. People hate random.

29:50

And I'm always banging on about we are deterministic  thinkers living in a probabilistic universe, hilarity or tragedy often ensue.

30:04

And  thinking probabilistically is something that you really have to train yourself  to do. It's hard. It's really hard.

30:15

Because of your insight, people's desire for  a explanation, as long as it resonates with you, right, then you're okay.

30:23

And it's kind of  like, "Well, okay.

30:23

Yeah, the markets went up."

30:27

And I always make fun of it, right?

30:27

The  markets went up because we have no idea why.

30:33

Gold fell for probably one of several different  reasons, and we don't know which one it is.

30:40

People, and this is why the  prognosticator business, the guru business is such a good business, because forecast  early, forecast wrong, but forecast often.

30:53

And my idea behind it always was we should make  them put money on it.

30:53

And because my friend, Danny Duke says, "You really can change somebody real  quickly."

30:59

If you say, "Hey, who do you think is going to win, whatever, the Super Bowl?"

31:04

and, "Oh,  the, whatever, The Vikings," my old hometown team are going to win.

31:12

I would bet against that.

31:12

But  anyway, then you changed the question and you say, "Okay, Kevin, you pick the Vikings?

31:18

How much are  you willing to bet?"

31:18

"Well, now wait a minute."

31:25

And so I think like a great app would be an app  where you got to put money down.

31:25

And it could be not just for markets, it could be for everything.

31:32

The people like COVID making forecasts, okay, put some money on it, and you'll bring them in.

31:38

Kevin Zatloukal: If you want to a story of this, Fantasy Football  is nice because it has the property that people put money on it. Right?

31:45

Even in the cases where  there's no money in the line.

31:45

So I'm wearing my Scott Fish Bowl shirt today, it's a charity  competition, but it's highly prestigious, right?

31:54

It's maybe the most prestigious.

31:54

Everybody  wants to win even though there's no money.

31:58

It's something real, tangible is on the line.

31:58

I  guess, intangible is on line.

31:58

Something important is on the line.

32:01

And yeah, it changes people.

32:01

Yeah, I don't know if you've ever watched ESPN and Fox Sports like hot takes shows,  right?

32:06

So like Skip Bayless and Stephen A.

32:10

Smith and so on, and they give their  hot takes, and people enjoy watching this.

32:14

And that's one side of the entertainment  side, but when you switch to the fantasy side and there's you're going to be scoring right  and wrong, the behavior changes.

32:18

I've actually seen people switch from one to the other.

32:22

There was this, maybe I shouldn't say his name, but there was a kid who used to write  fantasy articles that were full of hot takes.

32:31

And I remember one of my first strong opinions  about this was I thought this kid's going to own this town, because he knows how to spit these hot  takes. People love this stuff.

32:36

And then he went out and started playing sort of professionally,  like full-time job playing Fantasy Sports, and it destroyed his ability to hot take. He can't do it  anymore.

32:46

And his podcasts became totally boring.

32:52

And he had to think probabilistically,  he no longer had any strong opinions, and he just, he couldn't do it.

32:56

And I thought he ruined himself because he had a really bright future with this  hot take business.

32:59

I mean, I'm sure Stephen A.

33:03

Smith and Skip Bayless make millions of dollars a  year.

33:03

And you can go on and do ...

33:03

You can do this kind of hot taking in the financial world too. People love it.

33:08

They love really strong takes, even though they're totally wrong.

33:12

But when if you  want to make money, you've got to switch to this probabilistic view and you're much less certain.

33:16

Anyway, it was fun watching a person try to shift from one to the other.

33:20

Jim O'Shaughnessy: I love that story actually, because it's a great  illustration, because you're right.

33:21

It depends on what your objective is, right?

33:25

If you want  to make money, you better be probabilistic, you better do your homework, you better have pretty  robust thesis and hypothesis about why you're doing what you're doing.

33:34

And you have to be very  willing to say, "Oh, I'm wrong," and correct it immediately, which is not natural to human beings. And I love it.

33:40

The hot take business, if we just change the frame a little bit and say, "That's  entertainment," I have no problem with it. Right?

33:52

But I think that's why people who look at the  world the way we do at OSAM are going to be able to maintain an edge, because we're arbitraging  human behavior. That's all we're doing.

34:00

And unless human behavior changes dramatically, I think we're  going to keep our day jobs, which is a good thing.

34:11

Kevin Zatloukal: If I can add a little more there.

34:13

Jim O'Shaughnessy: Please.

34:13

Kevin Zatloukal: You see this really directly in the Fantasy Sports world.

34:15

As I said, it's really ...

34:15

I mean, the sports world, the hot take people and the stat people are just worlds apart.

34:20

But I've  been involved in a bunch of these websites that publish Fantasy Football information, and you're  looking for clicks, right?

34:24

You don't have to learn machine learning to figure out what gets clicks. And it's hot takes.

34:29

People love their hot takes.

34:34

And so you got to decide, if you want to be  successful as a personality, you got a hot take.

34:39

That's what the people want.

34:39

You got to give  the people what they want.

34:39

And telling them, "This one is 53% and that's 47%," they don't  want to hear that even if it's the truth.

34:47

There's this tension that's really visible in the  fantasy and sports world, where to make money, you got to go that way, and I think it's  really obvious there.

34:52

I wanted to add, when I- Jamie Catherwood: A hot taker on Twitter is Joe Weisenthal, because he takes the hot take to another level of  like you don't even know if he's doing a fake hot take, just like a ironic hot take or [crosstalk] Kevin Zatloukal: ...

35:08

I thought his was he was always trolling.

35:08

I  always interpret every tweet he does as a troll. He's an artist, right?

35:14

Jim O'Shaughnessy: Yes, he is an artist.

35:16

Jamie Catherwood: I know.

35:17

It's like you're always are guessing,  "Does he actually believe this or is he trolling, or is this a genuine hot take?"

35:20

I think he does  have some actual hot takes and then others are just trolling, and then he's seeing the people in  the comments some of whom are like, "Great work," and others are like, "How can you believe this?"

35:30

Kevin Zatloukal: I have maintained ...

35:34

I don't know if you guys  have ever seen Skip Bayless, but I always like to make this joke about the best actor I know  is the actor that plays Skip Bayless.

35:38

Part of me thinks he's ...

35:42

It's like an Andy Kaufman-esque  role, because it's so over the top.

35:42

No human being could actually be that pigheaded or whatever.

35:49

It's like it can't be true. He's trolling, right?

35:55

And I think it's an actual person, but I think my  gut says 60-40 on the evidence it's trolling, but the rest of me says it's an actual person.

36:01

If I  could go back for a second, Jim, because I wanted to connect to something you were saying before. Jim O'Shaughnessy: Yeah.

36:08

Kevin Zatloukal: About the people not knowing why.

36:08

I mostly I  want to do this way so give me an opportunity to compliment you.

36:12

But one place where you do  see this a bit is, in the space of strategies that are out there now, is the trend following  strategies.

36:17

They don't have a why, and you can watch it drive people crazy. And it's fun.

36:22

I  use this [inaudible] strategy from time to time, and there's nothing more fun than when a stock is  going up and everybody is screaming, "This doesn't make any sense.

36:34

It can't do this," and you're  getting the money and you're laughing at everyone else.

36:38

It's just the best feeling. Yeah.

36:38

I'll say- Jim O'Shaughnessy: It really is. Kevin Zatloukal: ...

36:42

I wanted to mention that also because my  favorite quant strategy still is sort of loving feelings toward, I guess, is to have the  strongest love is your trending value strategy.

36:49

And I know it's funny to me that I feel  like if I had invented that strategy, I'd have T-shirts made out that said trending value.

36:54

I'd  been running down the hall high-fiving everybody, we'd be depending value people, and you never  even talk about it.

36:59

That's such a good strategy.

37:03

And your book shows, I mean, it works well in  every sector and every time period.

37:03

I just- Jim O'Shaughnessy: Thank you for the compliment. It is a great strategy.

37:10

I love your comments  about the trending stuff, because it does drive people crazy.

37:16

And I often say that narrative  follows price.

37:16

And just that statement alone, if I'm in a room with somebody and I say  that, you'd see one of these triggered, because everyone is prime to believe that it's the  other way around. Right? And that it's not.

37:30

And so a great illustration of that is ...

37:36

And again, why you have to avoid premature certainty, why you have to be  open-minded about things, because some things are going to work and continue to work, just  keep working without making a lot of sense.

37:45

And so we had one period in the early 2000s where  a strategy very much like trending value was picking up tiny steel companies.

37:59

And so I'm looking at the computer, and normally I don't really care and I was like, "Okay, that's  the model picks it. That's great."

38:05

But I'm looking at it and I get intrigued, I'm sort of like,  "Okay, this is a growth strategy.

38:11

It's picking up these tiny steel companies.

38:17

What the hell is going  on?"

38:17

And so I do the news search, I do everything, nothing.

38:24

There's nothing in the news.

38:24

And so I  ask the team to come in and poll them, "Why do you think this is happening?"

38:30

And basically  I get just a collective shrug, "No idea."

38:35

Well, six months later these stocks have doubled,  some have gone up even more than double, and every news magazine, when we still had magazines,  was writing stories about how the fact that China was building the equivalent of one Boston  every month. Aha!

38:49

The narrative came six months later.

38:57

No one noticed it until some  reporter who was touring around China was like, "God, how many of these cities are you building?"

39:04

And then of course the narrative machine starts.

39:09

And often what you'll find is that's exactly what  our models are saying, "Sell. Get out of it."

39:15

Kevin Zatloukal: You had a prior guest who said this too. I think is it Naufal?

39:17

Am  I remembering correctly there, or I'm mixing up the name here?

39:21

Jim O'Shaughnessy: Yeah, I think you- Jamie Catherwood: No, no, Naufal. We had him on. Jim O'Shaughnessy: ... Oh, yeah, yeah.

39:23

Jamie Catherwood: Naufal Sanaullah.

39:24

Jim O'Shaughnessy: Oh, yes, yes, yes.

39:25

Jamie Catherwood: I don't know if that's his  last name, but [crosstalk] Kevin Zatloukal: He said exactly this point.

39:28

He said, "Once the narrative is figured out, it's  time to get out. The best part's over."

39:35

Jim O'Shaughnessy: Exactly.

39:36

Kevin Zatloukal: And I think sometimes when this thing is happening, it might be that  nobody who's even doing the buying and selling knows what's going on. Right?

39:41

They have their  own individual reasons for buying and selling, they haven't picked it up.

39:45

Once finally  the narrative is out there and people know, it's over, it's time to get out.

39:49

You can only  make the money when it doesn't make sense. Yeah.

39:52

Jim O'Shaughnessy: Exactly.

39:52

And the other interesting thing with this whole mean stock thing, I like it as entertainment  as much as the next guy. Right?

39:55

I mean, it's kind of fun.

40:00

But to watch the narrative,  try to convince people that these little Davids have wiped out the Goliaths, it's just not  true.

40:08

And people get really angry at me when I talk about this, because it's like, "You're just  saying a little guy can't do it," I'm saying "No, no, no, that's not true."

40:20

I absolutely  support individuals taking up investing, doing it responsibly. I think it's fantastic. Yes.

40:26

But if you don't think that those chat rooms are covered by people who work at those hedge  funds with crazy names, right? Big Bob rat man, right?

40:41

It's called painting the tape. It says old as ...

40:41

Jamie would know a lot about it.

40:46

It's as old as Wall Street.

40:46

You paint the tape  and you feed ...

40:46

It's like chumming the waters.

40:51

And then these narratives get constructed that  guess what, news people want to write about it because it sounds exciting, right?

40:57

Oh, David  just took out Goliath.

40:57

And I always think of the Damon Runyon quote, which is, "The race is not  always to the swift nor the battle to the strong, but that's the way to bet."

41:08

Jamie, you  want to chime in there, or am I wrong?

41:15

Jamie Catherwood: No, you're right.

41:15

One of the favorite examples is actually from the Dutch Stock Market, Amsterdam  Exchange from that book we did a podcast on, Confusion of Confusions, like the first finance  book, first definitely behavioral finance book written in 1688, and it's a guy explaining how  the stock market works.

41:31

It's in kind of weird format.

41:37

It's like a conversation  dialogue between a shareholder, a philosopher and the business person, and the  shareholders explaining how the market works.

41:46

But he's talking about this game or kind of trick  that speculators will use where they will drop.

41:53

They'll make it look like an accident.

41:53

They drop  a slip of paper saying that they're about to buy a massive position in the East India company,  that they're going to really buy up some shares.

42:02

And then they leave it in a spot where they know  some kind of ignorant trader will come pick it up and think they found like a gold mine, and then go  tell all their trader friends about it and like, "We've got to trade on this.

42:12

Look, they're about  to really move the price up."

42:12

And then obviously, the original speculator does the total opposite  and leaves them kind of holding the bag.

42:17

But it's just you think you have something, but it's- Jim O'Shaughnessy: I used to- Jamie Catherwood: ...

42:25

someone just really misleading you. Jim O'Shaughnessy: ...

42:27

When I was a teenager, I was a  professional magician.

42:27

And misdirection, man, I mean, that is- Jamie Catherwood: Wait? What? Jim O'Shaughnessy: ...

42:34

You didn't know that, did you?

42:34

Jamie Catherwood: You just flossed over that.

42:35

Jim O'Shaughnessy: We've worked together for almost two years now  and you didn't know that. That's interesting.

42:42

So misdirection, and it's the soul of good magic,  it's the soul of things like this.

42:42

It reminds me of two stories that also illustrate it in a book  called The Plungers & the Peacocks.

42:48

It's talking about Harriman, the railroad baron who is also a  stock speculator.

42:54

And a pool came to him and they said, "Hey, could you bull the price of stock  X, Y, Z up?"

43:00

Let's say it was trading at 60.

43:05

"Could you move it up to 80?"

43:05

And he's like,  "No," he goes, "I can move it up to 160 and let you sell it down to 80."

43:12

He goes, "But if I  moved it $20, that's not going to excite anyone.

43:18

That's not going to get their competitive juices  flowing."

43:18

And then you had Rothschild who had carrier pigeons at the Battle of Waterloo, and so  he knew ahead of time that the British won.

43:24

And so everyone is watching him, of course, because  everyone thinks he's going to know something.

43:39

And he walks in, he has all of his agents sell  British counsels, everybody panics, they say, "He's selling that means Napoleon I."

43:46

Everybody  dumps them, and of course he thought his other group of people buying from them.

43:51

I mean, this  is as old as markets in Jamie's 1688 book.

43:51

This is as old as markets.

43:55

And so it feeds into that  whole we're easy to fool in many ways.

43:55

Let's- Jamie Catherwood: Daniel Drew did the exact same trick 200 years later where he, he called it  the handkerchief trick, where he would have a note accidentally fall out of his handkerchief as he  was on the way out of a bar near the Wall Street Stock Exchange. Same exact thing. Jim O'Shaughnessy: ... I love it. I love it.

44:17

And it's just like  we never learn. We never learn.

44:17

All right, so- Kevin Zatloukal: Those are fantastic stories. Jim O'Shaughnessy: ... Well, thanks.

44:24

Kevin Zatloukal: And I can't compete with them, but I did want to return to your prior point to that  about the David versus Goliath.

44:26

I know one thing I've noticed my ears are very finely tuned to is  anyone making a stock market prediction, that's a morality tale.

44:35

And whenever I pick that up, I  instantly want to take the other side.

44:35

Because the stock market is not here to give out distributed  justice or whatever.

44:41

And anyone who's betting that way is doing it for emotional reasons.

44:45

That's  free money to me, so I'll take the opposite side.

44:50

Jim O'Shaughnessy: That is an excellent rule to live by.

44:50

So that's a good heuristic for you, because I totally  agree with you. Who was it, Jamie? You might know.

44:59

No one is so innocently as engaged as a  merchant trying to make money.

44:59

Do you know who's [crosstalk]- Jamie Catherwood: No.

45:05

It sounds familiar, but ... Jim O'Shaughnessy: ...

45:06

Maybe Mill or somebody like that.

45:06

Well,  I want to get back to machine learning with Kevin and then ...

45:10

Wow, we've been having too  much fun and we're already 10 minutes over.

45:16

I guess my last question, Kevin, would be for  speculation on your part.

45:16

I'm very bullish on what I'm calling the great reshuffle, which is  all these things happening at the same time, sort of geography collapsing, space time also kind  of collapsing.

45:28

We can interact with one another all around the world.

45:35

If you're a knowledge  worker, the world's literally becoming your oyster.

45:40

What do you see as like machine learning  10 years hence, if you care to speculate?

45:47

Kevin Zatloukal: Well, I will start again with I don't know, but I expect it will be ubiquitous.

45:49

The  other thing, I guess I hinted at this before, that what we're doing with machine learning is  changing.

45:54

And as it changes, tools become more and more powerful.

46:00

And they're so powerful  they break out of the boxes we had them in.

46:04

And so when I look at some of these deep  learning packages, PyTorch, TensorFlow and so on, they're so ...

46:09

To me, as a computer  scientist, it's very obvious that these will ...

46:12

And of course I say it feels obvious. I may be  totally wrong.

46:12

But these are going to be just included in every programmer's toolkit soon.

46:17

Let  me see if I can describe it this way.

46:17

Because the tools they provide are much more useful than  just for doing vision tasks or something, or even what looks like machine learning, they  give you a toolkit where ...

46:27

Normally, say you're programming.

46:31

So you're trying to explain to this  dumb computer that can just add and multiply numbers how to do something complicated, right?

46:35

Well, it gives you this new ability.

46:35

So I'm sitting down and I say, "Okay, I need to compute  this.

46:39

Okay, so I need to take the input and I want to multiply it by something.

46:44

I'm not sure what it  is I'll just leave that blank.

46:44

And then I got to add something else, I'm not sure. Ah, I'll leave  a blank.

46:48

I'll compare it to this other thing and take the bigger one, and then maybe subtract out  some amount. How much?

46:52

I don't know, we'll put a blank there."

46:56

And then when you're done, you go,  "Okay, well, I know it should look like that.

46:56

I just don't know what these blanks are. Here's some  examples.

46:59

Computer go fill in the blanks for me."

47:04

And that's just an incredibly new,  powerful tool you can add to programming.

47:07

Just leave some blanks and let it fill them  in for you from data.

47:07

That's going to change programming.

47:11

And the people who weren't doing  that, I don't think anyone would not be doing that at some point.

47:13

It's just way too useful.

47:13

It's such  a huge productivity improvement for programmers.

47:18

The world's already short of programmers or  people who can program effectively, so I expect that it will be, machine learning would just  be in everybody's toolbox programming wise.

47:26

And as far as the problems we're going to solve,  I mean, I have no idea, but it's moving so fast right now.

47:31

It's doing all kinds of incredible  things.

47:31

There's just new things ...

47:31

It's coming so fast, it feels foolish to even predict  anything other than that there's going to be a ton of stuff we don't see coming.

47:38

And it's going to be one after another for quite a while now.

47:41

Jim O'Shaughnessy: Yeah, my sentiments exactly.

47:42

I am very, very  bullish.

47:42

These kinds of current chaotic times are tough on people, and I understand that.

47:49

And you  see people reaching their Shannon limits and wanting a simpler explanation.

47:56

Another reason why  people like those simple explanations, because they feel comfortable.

48:01

But I think it was Lord  Whitehead who said something along the lines of every great age was either proceeded by or was  a chaotic age.

48:07

And so I think that unbalance the stuff that's going to come out of this,  like you point out with the new toolkits, and I just agree.

48:19

I think that what a time to  be alive.

48:19

I think I'm certainly happy I am.

48:26

Kevin Zatloukal: It's unfortunate that it comes at the same time as sort of this political turmoil.

48:28

And  you'd like it if the world around us was changing, but we were all united politically, let's say,  and we all had this view of it. "Okay.

48:33

We don't know whose jobs are going to get replaced,  but we're all in this together," sort of mentality, that we're going to figure it out.

48:40

But that's not where we're at.

48:40

We're at this heightened divisiveness and feel like you can't  get anything done governmentally, at the same time that it feels like we have all these concerns. So that's a problem.

48:48

Although my faith in America certainly would be very difficult to shake.

48:52

Jim O'Shaughnessy: As is mine.

48:53

I will belong American until the day  I die.

48:53

What we do at the end of each of these is ...

49:00

It's been kind of fun because the answers  have been really interesting.

49:00

The game we're going to play is we're going to wave a wand and  we're going to make you the emperor of the world for one day. You can't kill anyone.

49:09

You can't  put anyone in a reeducation camp. But what you can ...

49:15

Have you seen the movie Inception? Kevin Zatloukal: Yeah. Jim O'Shaughnessy: Okay.

49:17

What you can do is incept.

49:17

And this  is the whole world, not just America.

49:23

You can incept people, they're going to wake up  the next day and they're going to think that they had two ideas. What you got for me? Kevin Zatloukal: Okay.

49:31

Well, one comes to mind right away.

49:31

I'll  see if I get a second one.

49:31

First thing that comes to mind is ...

49:34

Since you're on Twitter, you  know this, right?

49:34

You've seen how people react.

49:39

Whenever anyone puts out an idea, and I'm just  saying Twitter because it's where we get to see everyone's behavior.

49:43

That's what's great about  it.

49:43

Someone puts out an idea, the comments that come back are a mix of this.

49:46

There's a bunch  of group of people that will instantly respond, "Boo! That's bad. Bad.

49:51

And you're  bad for thinking it's good."

49:56

And then the other people will be like,  "That's exactly what I always thought. Absolutely. 100%. Yay!"

50:00

So there's this some part  of people's brains they just want to go around and label everything in the world with good or  bad.

50:06

It's like they walk along and started smelling the flowers, they put a little sign on  this one, "This is good and this one's bad."

50:10

So rule number one, I mean, I want to wipe away that.

50:14

Whatever that says that instantly I have to slap a label on it, I can't just see what it is and try  to ...

50:19

I guess I'll turn this into part one is I don't need to instantly label everything as good  or bad.

50:23

I can just see it as it is.

50:23

And then the second would be maybe try to understand it from  the other side.

50:27

Usually there are reasonable ideas in there.

50:32

What can I get out of turning  it around and looking why are they saying this?

50:36

Once you've washed away the idea  of, "I might instantly label it," then you can try to see from the other side and  see what's valuable.

50:40

Maybe even though this flower smells bad, it has some useful properties.

50:44

Let's  just perceive it, and try to understand it and get something useful out of it.

50:48

That will  be a better world, I think, in my opinion. Jim O'Shaughnessy: Amen.

50:51

You are preaching to the choir.

50:53

I completely agree with you.

50:53

Labeling and  these instant ...

50:53

When people ask you for advice, they're really not asking you for advice,  they're asking you to tell them they're right.

51:04

And lots of times they're not.

51:04

I'm often wrong, so  don't ask me because I'll be wrong when I tell you that you're right.

51:11

Listen, Kevin, this has been  really fun.

51:11

Thank you for giving us the time, and thanks for being the same research  partner. We really appreciate it.

51:20

Kevin Zatloukal: Oh, thank you. Thank you for having me.

51:20

It's been fantastic for me.

51:22

And also, thank you for chatting  with me.

51:22

Every time we chat, I have so much fun.

51:25

Jim O'Shaughnessy: As do I.