0:01
Casinos are almost perfect examples of a highly designed system that is intentionally taking advantage of innumeracy.
Search this video
0:11
Most people's vocabulary when you're talking about probability is limited to one in a million, 50/50 or sure thing.
0:19
I bought a single lottery ticket and after getting it I tore it up and people act like I kicked a puppy or something.
0:29
Our human OS is actually more driven by the emotional centers of the brain than the rational centers of the brain.
0:36
Most people don't really have a visceral feel for the difference between millions, billions and trillions.
0:42
The words rhyme but they're vastly different. Well, hello everyone.
0:51
It's Jim O'Shaughnessy with another episode of Infinite Loops.
0:56
Today I have on a man who actually changed my mind so much that I bought a dozen copies of his first book that I read at least called Innumeracy.
1:11
John Allen Paulos is has a PhD in mathematics from the University of Wisconsin.
1:16
He is the author of Innumeracy, A Mathematician Plays the Stock Market, A Mathematician Reads the Newspaper and he had a really profound effect on me, Professor Paulos. First off, welcome.
1:33
Um, uh, it's good to be here and I'm flattered that the book had as you said profound effect. I hope it was positive.
1:42
Very positive, very positive.
1:42
I bought a dozen copies and gave it to friends who specifically some friends who when I would start talking about math or well, you really need to understand statistics don't really work that way.
1:55
They would just put their hands up and they'd say I hate math.
2:01
They thought I was So so so I guess that's my first question.
2:05
Why do so many people feel almost proud to say, oh, I hate math. That's a good question.
2:13
I think part of the reason is the way it's taught.
2:14
It's generally taught as a bunch of formulas uh, and you're supposed to kind of blindly follow the algorithm and get a get an answer.
2:25
Uh, and that's, you know, part of it.
2:29
But if if analogy sometimes used, if imagine you're in English class in middle school, high school and all you ever did was diagram sentences.
2:40
Uh, you won't have a real keen appreciation for English literature when you got to college.
2:45
So the these, you know, taking, you know, finding the neutral quadratic equation, the 100 different equations or the derivatives of polynomials, 100 different polynomials.
2:58
I mean, that doesn't get to the actual in a sense analog of the literature of mathematics.
3:07
It's, you know, it's kind of playing uh chords on a piano.
3:11
You if you never play a song, you're not going to be real fond of uh, your piano lessons.
3:19
That's a great way of putting it.
3:22
Um, and so I know that you've advocated uh, for trying to teach math in a different way.
3:30
Uh, if you wouldn't mind, give us some examples of ways that you think we could better get everyone from children all the way up to adults more excited about learning about math.
3:41
I think is to kind of embed it in one's, you know, life's activity.
3:44
In elementary school, I mean, play games, board games.
3:50
Uh, all board games are a little passe now.
3:52
People only play, kids only play digital games.
3:56
But when I was a kid played Monopoly, they played Clue.
3:58
But in any case, in elementary school, you know, talk about anything kids are interested in.
4:06
Some things on TikTok, how many people do you think saw it? What percentage is that?
4:11
What are percentages and recipes, you know, same thing.
4:13
How do we uh how do we double that recipe?
4:20
And when you get older, I mean, comparable problems regarding money if you're talking about the stock market or even just personal finance.
4:33
And you know, some appreciation for common common numbers with respect to which you can, you know, place the number that that is replaced before you.
4:42
I mean, most people, kind of a tried example but I give it all the time, don't really have a visceral feel for a difference between millions, billions and trillions.
4:51
And a million's 11 and 1/2 seconds approximately, a billion's about 32 years and a trillion's about 32,000 years.
5:01
So the numbers rhyme, the words rhyme but they're vastly different and unless you're aware of uh, some fear, you know, visceral feel for that difference.
5:13
I mean, you're going to be any little hot button issue that all we're going to spend $56 million on that. That's horrible.
5:22
I disagree with that and uh, but we're going to do, you know, increase expenditures by two trillion or cut taxes by three trillion.
5:30
Whatever the merits of that you should realize that's a vastly different number than millions or billions.
5:40
And you know, some feeling for for probability with most people's vocabulary when you're talking about probability is limited to um one in a million, 50/50 or sure thing.
5:54
And there's not too much you can do with that very sparse vocabulary.
5:59
And and then talk about, you know, puzzles.
6:01
I think puzzles are undervalued in mathematical puzzles, uh puzzles from science uh, get away from the the rote.
6:11
I mean, you need to the you know, the master the rote basics but you want to induce people to to think. Does that make sense?
6:21
And uh and with regard to math, I mean, that's very important and but if people have this very restricted view of what mathematics is, of course they're going to put their hands up and say, oh, I hate math.
6:32
I Now, on a plane if somebody asks me what I do, I often you know, say something else or change the subject because it's tiresome to hear that they hate math.
6:47
Well, persistence through continual rejection is a way that path towards success many times, not always.
6:54
Um I I I like all of those examples and when I was trying to teach my kids math uh, after having read your books one of the things that I tried to do was make it a fun story.
7:08
Um, so for example, uh, compounding, exponential functions, all those like imagine a king is uh, needs something done for him and the winning bidder says all I want is one grain of rice and then that had doubled on a chessboard, right?
7:27
And I still remember my son was chess player and we had some downtime and I gave him that example.
7:34
And it it made it delighted me because his eyes got so wide when when I said, how many grains of sand do you think are going to be on that last square on the chessboard? Right, right.
7:50
And what I found was really cool was it animated him. It got him excited.
7:53
Are there other stories that we could be working with with our children all the way up to adults that would get it I I like your idea about game playing.
8:04
Like is there a a process that that you think would be vastly better if we're trying to get people to both appreciate, understand the importance of math and how not having a firm grip on it can really be detrimental to many of your outcomes in life. Right.
8:26
Uh, it's not something that one should do, educators should do but something in addition that they should do.
8:32
And that is as I say, related to things.
8:34
I mean, the puzzles for I mean, imagine there's a a bag of potatoes which weighs a 100 lbs and it's 99% water.
8:48
And you leave it out overnight and the water some of the water evaporate evaporates and now the bag of potatoes is 98% water.
8:59
Now, it originally weighed 100 lbs.
8:59
How much does it when it was 99% water, how much does it weigh when it's only 98% water?
9:07
And it forces you to think through it and the answer is counterintuitive.
9:11
The answer is only 50 lbs.
9:15
And but yet problems like that where uh the answer is either counterintuitive or just shocking.
9:24
I mean, the the the number of um of ways of ordering a deck of cards you know, it's is 52 factorial.
9:31
The first card in the deck has could be any of the 52.
9:36
That could be combined with any of the remaining 51 cards to be the second card, combined with any of the remaining 50.
9:45
It's 52 factorial which is 10 to the approximately 10 to the 68th power.
9:51
And that's you know, a number we can't grasp.
9:53
I mean, it's probably the case that uh no one who shuffled a deck of cards ever got that particular order in in the whole history of card playing and for the next 10,000 years of card playing.
10:08
But um again, a feel for the vastness of some numbers, the you know, the tininess of numbers, nano nano everything is getting much more common.
10:22
And so even just a basic idea of the size of numbers.
10:25
If you're going to You don't have to talk about algebraic topology or different sorts of um geometries.
10:33
I don't just If people understood numbers and how to deal with them, as you said, the exponential growth is a very uh important topic and most people don't have any idea what it is.
10:47
And they they they use the word exponential kind of loosely like, "Oh, it's growing exponentially."
10:51
It just means it's getting bigger.
10:53
It doesn't necessarily mean it's exponential.
10:57
Sometimes if people really want to impress other people, they say, "Oh, this is growing logarithmically."
11:04
But they don't know that means it's growing more and more slowly.
11:12
The law of large numbers is something that I have often tried to through just fun quizzes get people understand.
11:24
And yet the number of like really intelligent people in general, right?
11:28
Like for example, I would ask uh okay, so uh the the the norm for births in a hospital uh is about five boys and five girls every night.
11:43
Uh if there were two hospitals, one was tiny and one was the biggest uh hospital in New York City.
11:50
And I told you that there were nine boys and only one girl born, which hospital do you think it's more likely that they were born at?
12:01
What do you think the vast majority of people I asked that question guess?
12:06
Yeah, well, I would guess that they would guess incorrectly.
12:12
Yeah, no, I mean, that's why often uh you know, the best uh results for whatever you're trying to measure often occur in in small towns.
12:24
Whereas well as the worst results.
12:27
And it's nothing that's uh particularly the the case of but their smallness other than their just their smallness.
12:36
And uh yeah, no, the the central limit theorem isn't uh well understood.
12:43
Uh just statistics in general.
12:43
I mean, there there's it's not I mean, there's a mistake to go too far and say everything is normally distributed, which is not.
12:51
I mean, there's you know, as many people have pointed out, there are long tails and uh but for most everyday kind of quantities, the normal distribution is a good place to start.
13:06
Yeah, Mandelbrot uh turned me on to the chaotic distribution pattern in his uh The Misbehavior of Markets.
13:14
Um and uh that it's interesting cuz I got so used to uh the uh the normal distribution that uh starting to think about different distributions leads to different things.
13:28
But back to the law of large numbers for a second, the thing that I would move on to then would be this has profound implications in a variety of uh disciplines, government, medicine, etc.
13:43
And how would you go about helping say a newspaper uh uh journalists uh better express uh the the essence of a statement um where they can easily fall into these traps of uh you know, the monkeying with statistics and or not understanding that uh most unusual results come from the smaller sample size versus the large sample size.
14:14
It it is there a way that they could get better at that and communicate clearly the difference?
14:21
Maybe use examples not necessarily from business or finance or medicine, but more everyday examples.
14:31
I mean, let me think if I can How about popcorn?
14:35
It starts popping, there are a few few pops and then more and more and then there's a crescendo, stays there for a while and then fewer and fewer and then just the the last few pops.
14:47
And okay, maybe that's not the greatest example, but even the the the rate of uh uh popcorn popping follows a normal distribution.
14:55
Or you you know, look at a stair stairway, let's say a wooden stairway and um to your house and or any any building.
15:05
And people go down the down the flight of stairs, generally they stick towards the middle or uh and so if you look at the way the stairs wear themselves are are worn, it's kind of an upside down normal distribution.
15:20
It's it's more worn in the middle and less so on either side and very minimally so unless you know, or some people kind of maybe want to hug the hug the wall, I don't know.
15:31
But that would be everyday example of a normal distribution or just you know, the the common examples that are often given there, the circumference of babies' heads or the lengths of leaves of a certain kind of tree um would follow a normal distribution.
15:55
And um I mean, it it takes some getting used to cuz people aren't used to thinking um statistically.
16:03
There wasn't any real reason to uh for that particular cognitive uh uh attributes to be developed.
16:10
I mean, that and you know, I'm speaking kind of very simplistically, but uh you know, and long ago, if you heard a you heard a rustle in the bushes, uh it paid to run like hell and not plug into base there and just say it's very unlikely to be a tiger.
16:32
But I often use that uh to joke that we we are all uh descendants of the people who ran away.
16:45
That's your for cowardice. Exactly.
16:49
And and and I find that cultural lag really fascinating because we we I I often say we live in a probabilistic world and yet we remain deterministic thinkers.
17:03
Uh tragedy or comedy often ensue.
17:04
And like like like literally, there are things like for example, the lottery, right?
17:13
Uh I to me, I am I'm opposed to lotteries just because it looks to me like governments and and others running them are literally taking advantage of innumeracy. There.
17:27
And you know, I guess the cutest example that I often give is a quote from Fran Lebowitz who says that she never bothers to buy a lottery ticket because she feels her chances of winning are the same or not. Yeah.
17:43
But but despite are these things impervious to education?
17:48
I don't think so, but they they do require it.
17:51
I mean, they're talking about uh uh humorous takes on them.
17:56
Voltaire and said through it have remarked that the lotteries are tax on stupidity. Quite Christian.
18:05
Actually, your friends who didn't see a difference between your chances of winning reminds me of an incident.
18:12
I was on a television show once and it was focused on This was a long time ago when innumeracy was on the best sellers.
18:19
So I went into a bodega with a television crew behind me.
18:25
And I bought uh a single lottery ticket.
18:27
And it was kind of look saying care where investigating you know, how people react and how many tickets they buy.
18:35
I bought a single one and the focus was on me, the cameraman was there.
18:41
And after getting it, I tore it up.
18:44
And people act like I kicked a a puppy or something.
18:51
There's no no discernible difference between my chances of winning, but I it was worth the look on many of their faces just to do that.
19:02
Yeah, and that's one of the things that I that fascinates me.
19:07
Uh you know, that's a visceral reaction based on, you know, erroneous understanding of how lotteries work.
19:13
And and I just wonder sometimes how much of that is just so in It's like pattern recognition in a way.
19:26
Uh listen, humans are great pattern recognition machines, but when taken to an extreme, we impose patterns on random on random data and that can often lead us very far astray.
19:43
I I just wonder like I What techniques would you advocate?
19:46
Um I like to tearing up a lottery ticket, by the way.
19:51
What techniques these more I'm most interested in trying to help people get to just basic understanding of concepts that are going to really add to their success in outcomes, right?
20:02
Because you you For example, casinos.
20:05
Um I I used to give a lot of talks when I was still running an asset management company, which was quantitative and empirically uh driven.
20:18
And uh I used to give a lot of talks in Las Vegas.
20:21
And one of the first thing that I did was mention to the entire audience that I always stay in the only hotel, which at the time was the Four Seasons.
20:31
I don't know what it is now.
20:32
That didn't have a casino.
20:32
And like literally the gasps from the audience were audible.
20:38
And and then like after the talks people would come up to me and almost always the number one question was what what why gambling is fun.
20:50
I You'd think somebody like you knows, you know, the odds or the numbers would would really love it.
20:54
And then of course they bring up um you know, the Kelly criterion and all that sort of stuff. Yeah.
21:00
It seems to me that casinos are almost perfect examples of a highly designed system that is intentionally taking advantage of innumeracy. I I think so. I think so.
21:16
I mean now apophenia is a word I like.
21:18
It's seeing as you kind of uh uh referred to just recently uh seeing patterns where they're not there.
21:26
I mean COVID is a good example.
21:28
Oh, if you take the vaccine, you're going to get a heart attack or blood clot.
21:33
But you look at how frequently heart attacks, blood clots, and and other things occur naturally and and uh there is not a higher rate with vaccines.
21:45
But you it's very hard to convince somebody who just got the vaccine and now has a blood clot uh serious blood clots that, you know, some people get blood clots or period.
21:57
Why I mean the randomness in general is difficult to to uh to recognize.
21:59
Uh uh there's uh you know, if you let's say you've got two people flipping a coin flipping a coin.
22:08
Call one Harry and the other uh Tom for heads and tails.
22:12
Uh and you flip a coin uh single coin and and Harry bets on heads and Tom bets on tails.
22:20
Uh You do it uh 10,000 times.
22:24
One of them at the beginning after a few hundred is going to be ahead.
22:29
Every there's going to be more heads than tails up to that point or more tails than heads.
22:35
But it's very likely that whoever's ahead uh near the beginning is going to be ahead at the end.
22:40
And it doesn't mean that, you know, Harry's uh let's say Harry's a little bit ahead in the beginning uh that he's or you know, clear winner.
22:50
But uh it just means that uh you know, the changing leads are kind of uh un happening frequently.
23:00
And um And you know, it's not a big difference.
23:03
It could be a fairly small difference.
23:05
After a thousand it could be 20 head uh 20 heads ahead of the tails.
23:08
And you know, at the end it might be, you know, 15 or something.
23:15
But uh or even just uh runs.
23:18
Runs are much more common than you think.
23:21
You flip a coin a thousand times, oh, never going to get uh six heads in a row or seven tails in a row. Uh but that's not true.
23:29
It's fairly fairly likely.
23:32
And um And and the the the the number of runs, their lengths, uh this uh resistance to changing the lead, and a lot of other uh characteristics of randomness uh seem decidedly unrandom.
23:50
So it's it's difficult to recognize randomness and um every time we make a mistake I mean a lot of mistakes in public policy and so on are because uh people have no idea of what a random sample is.
24:04
I mean they talk to their 23 of their friends.
24:06
Oh, and most of them they Biden or something.
24:09
Yeah, but who were your friends?
24:12
I mean uh And uh actually my my wife does that a lot.
24:18
She always says everybody says this. And who's everybody?
24:21
Oh, yeah, those those people. Uh people anyway.
24:26
Uh So yeah, randomness is uh I mean there are lots of characterizations of it.
24:32
I mean the one way of you want to characterize a sequence of zeros and ones uh if you if it's random, there's no way to really describe that sequence.
24:42
But if it's not, you can develop a formula of which you know, would shorten let's say there's 10,000 zeros and ones in in uh your and uh your sequence.
24:54
Uh but if you can shorten that, if there's a formula that that has some info I mean some uh significance uh instead of just saying copy these 10,000 numbers in this order, maybe your your program only needs uh 8,000 or 5,000 because there there's some uh order in there that's uh not apparent.
25:15
So if uh if the sequence is not compressible, then it's random. But that's hard to show.
25:21
And back to the example you gave with COVID and comorbidities, it is there a way because you're absolutely right.
25:31
I people don't disambiguate the idea that lots of people get heart attacks all the time.
25:38
Lots of people have blood clots.
25:38
And and I I just wonder about the ability uh someone trying to communicate that as you do quite well in your numerous books.
25:53
Like it it seems to me that there's got to be a better way to avoid all of that.
26:02
If it would would there be some sort of I don't know.
26:06
Like program that would say, well, you've got to to really understand if these vaccines are having any uh horrible effects, uh you've got to normalize for the things over here.
26:19
I Just like And and if there is, why aren't news organizations doing it? I'm not sure there is.
26:28
Uh But there's a a lot of I mean even with regard to COVID.
26:34
I mean it might be the case if most people are vaccinated and people and and anti-vaxers will say, "Look, half the people in hospitals are ones who have been vaccinated."
26:47
Yeah, but most people have been vaccinated.
26:51
And and let's say you're in a in a area where 90% of people, 95% of people are vaccinated.
26:59
And the anti-vaxers will say, "Go to the hospital.
27:02
Half the people there have been vaccinated."
27:05
Yeah, but they come from 95% of the population, not from 5%.
27:10
And um yeah, it's hard to to get across.
27:14
And you're right about lotteries among other things.
27:16
I mean almost designed to induce innumeracy.
27:21
And they and it's it Now you talk about what's built in.
27:24
I mean we're uh as has often been said, we're pattern-seeking animals.
27:29
So apophenia is kind of uh a foible that we're all vulnerable to.
27:36
We we look for patterns and um well, often they're not there.
27:40
We we want them to be there.
27:42
And um they're they're not. So we'll make them up.
27:48
Yeah, I've often said that uh our human OS is actually more driven by the emotional centers of the brain than the rational centers of the brain.
27:57
And and that that causes a lot of these problems.
28:02
I wonder what do you think about uh books and movies like Moneyball or The Big Short?
28:07
Do you think that they did a uh service to people understanding that better or did they do a disservice to people understanding that better?
28:18
I think overall a kind of a service.
28:21
It will show that, you know, okay, it's not what you seem what seems to be the case.
28:28
It looks uh or let's say GameStop or whatever.
28:33
Uh it's I mean at anytime you have examples of you know, these cognitive foibles uh uh holding sway uh to expose those examples, uh I think that's useful.
28:47
Let's say you look at uh Kahneman's book uh Thinking Fast Thinking Slow.
28:50
You know, the the standard cognitive foibles, the anchoring effect, and uh uh halo effect, and uh various other uh weird effects.
29:03
I mean the anchoring effect I think it is is very relevant to the market.
29:09
I mean people are anchored to whatever number they're they're given whether it makes sense or not.
29:16
And actually I I did uh in my classes, I did I did the following experiment because I had uh one semester I took my colleague's course and he instead of teaching two, I taught three cuz the old lady wanted cuz she had an operation or something.
29:29
Anyway, I asked the three different courses uh to estimate the population of Turkey.
29:38
You know, country about which most people don't know anything.
29:42
Uh but the first class I say is it more or less than 7 million people?
29:47
Most people and I said write it on a piece of paper, don't consult anything.
29:51
First answer the question and then give me your estimate.
29:53
So, is it more or less than 7 million people?
29:55
And most gives an advice that it's more than 7 million.
30:00
Their estimates varied all over the place.
30:02
More or less normally actually.
30:04
Uh with the average being uh 50 and 20 million people.
30:09
Different group of people, same thing.
30:10
Is the population of Turkey more or less than 70 million people?
30:15
Uh maybe it was 80 million. I said that.
30:18
It's And again, tell me whether it's more or less, write it on a piece of paper and then give me your your estimate.
30:24
And again, that the number that that I gave, 80 million after them, they they thought it was less, but maybe it's only around 60 million people.
30:35
And then the third group of people, I did the same thing, but I had a little dial, a little randomizing device with the number 7 million, 80 million, 120 million, whatever.
30:45
So, in front of the class, I I spun the dial and wherever it landed, I said tell me whether you think the population of Turkey is more or less than that number and then give me your estimate.
30:58
And oddly enough, the same effect happened.
31:00
I mean, in the first two cases, it might be rational to think, well, this guy doesn't know anything about uh uh I don't know anything about Turkey, but this guy probably does.
31:09
It's probably pretty close to the number he threw out there.
31:14
But certainly the dial doesn't know anything about Turkey and yet the same thing happened.
31:18
If the dial landed on a big number, people guessed a big number for the population.
31:22
If they landed on a small number, they guessed a small number.
31:26
So, uh you know, if if they can attribute, you know, intelligence to a random dial, um they're clearly going to be anchored to uh whatever number you give them and not just the population of Turkey, but in general.
31:43
That's kind of an amazing phenomenon.
31:46
Yeah, and I I've seen that example uh used as, you know, uh when was Genghis Khan born, for example. Same same idea as yours.
31:57
They they would give a number and people would anchor to it, then they would give no number, and then they would randomize the number. Same effect. Same effect.
32:09
Um and and that also kind of brings us into like when people are trying to make estimates.
32:15
I often ask a question uh just to to I'm interested in how people are going to respond.
32:24
Um you know, and and this is I guess what you'd call a confidence interval.
32:29
Most people Most people when I ask, how many people do you think uh are in the United States that have are worth a million dollars or more?
32:36
The number of people who give a specific number always amazes me. Yeah, yeah.
32:43
The ones that I'm far more impressed by are the ones who give me quite a range.
32:51
What they're demons- What they're demonstrating, as you know, is that, huh, I I don't know.
32:55
Uh let me see if I can put it between two very broad uh hats.
33:02
Now, I I find that when I ask uh technologists this, for the most part programmers, they give the they give the range.
33:11
Uh but but layman not trained in computer science often give a very specific number.
33:18
What and that and that has often caused me just great consternation.
33:24
Uh a while back, there was a uh very popular thing on Wall Street called value at risk.
33:31
And it's called VAR for short. Okay.
33:36
I I campaigned against value at risk everywhere I was because what it did was it it maintained that it could give you a very specific number. You you have a 13. 7% chance of uh right? I I hate that.
33:57
People are always giving very specific numbers.
33:59
I mean, any anytime you look at commercial or anything. I mean, it's absurd. You really think that?
34:06
And yet there a lot of people do it.
34:06
And it is a good way to distinguish people with a technical background from those who don't cuz if if you give something and you have this box and you give its volume and its weight and if you give its density by dividing one by the other to seven decimal points, uh most people say, oh, this guy really knows his stuff.
34:29
And if you know, if you were to give it to someone with a technical background, they'd say, how could you get seven decimal points for that?
34:35
You gave us the volume and the weight of the rod and you know, not very precisely and then you get this precise a number out of it, but that happens all the time.
34:46
Just watch television for half an hour and I guess that people think that it it is more confidence on this and really know your stuff.
34:56
It's uh you know, 8,420 people 23 people signed up for that auction.
35:04
I think that that also is part of just the way we're constructed.
35:10
Um people prefer certainty over uncertainty. Right.
35:13
And you know, one of the things that I'm always banging on about is listen, like most of the things you are going to confront in your life are highly uncertain.
35:27
Just by their very, very nature.
35:27
And yet if you look at who wins the election, who wins the debate, who wins all of those things and and I've studied a lot of this and almost always it's the person who's has that false sense of precision, right?
35:44
Like you can't tell me because I know that 22.
35:48
3% of the affected population is this and then the other guy is kind of like, oh, wow, you really knows his numbers.
35:57
Is there Is there Is Is Is there a way around that?
36:02
Like if we were debating and and I pulled that trick and I said, well, Professor, you don't clearly understand Wall Street because I can tell you that 93% of all trades held for more than 5 years always lead to a positive outcome. Yeah.
36:25
What What would you smash me with?
36:29
Uh one thing I I sometimes uh have done is I imagine flipping a coin a thousand times.
36:37
Uh the most likely outcome is you get 500 heads.
36:42
How many of you want to bet that you're going to get 500 heads?
36:43
So, let me flip the coin a thousand times.
36:45
And then if they have money on the line or even just virtually, uh they they might recognize, okay, that's the number you'll likely get, but that likely is not the same as most likely.
36:58
It's not It's not most likely.
37:02
And uh I mean, that's one way I one quote I have in one of my books that gets cited a lot on Twitter and elsewhere is that the only certainty uncer- I'm sorry.
37:13
Uncertainty is the only certainty.
37:17
And the only security is living learning how to live with insecurity.
37:21
And yeah, most people, you're right.
37:23
They they need certainty or they think they need it.
37:26
And uh you know, it's it's scary if you just say, I don't know.
37:31
It's not that you don't know.
37:31
I mean, you do know things, but your generally a confidence interval is uh you know, you have to give me more confidence in saying uh encompass the right number, whatever it is.
37:44
I always try to insert uh when I'm asking questions uh and hoping that I'm going to get a a broader confidence interval answer.
37:53
I I will add uh rather than asking for a specific uh numbers, I'll say, which way do you think directionally this might go?
38:06
And I found that when I do that, people sort of automatically move modes, right?
38:13
They rather than trying to assert some single number uh and or uh the frequency of occurrence or what have you.
38:23
When you when you ask them directionally, what do you think about this?
38:27
Well, I'm not sure I understand.
38:27
What What do you mean by directionally?
38:31
By directionally, I'll say, um we've just invested in this company and uh here here are the here are the uh things that have happened since we invested that are good outcomes, like, you know, uh earnings where we thought that they should be, number of customers that we thought that they should be able to and then I would introduce and and here are the things that kind of went badly.
38:57
They didn't go the way that we were expecting them to go.
38:58
And you take all of this uh uh in uh and and combine it and kind of say to me directional directionally which way do you which way do you think that this company What What would be a reasonable uh expectation on our part as investors to to directionally say this company should continue to do well or this company should maybe uh will maybe will stumble more.
39:28
So, it's a little bit like a confidence interval, but just a one-sided confidence interval. Exactly. Yeah, that's nice. I like that.
39:37
And I found that certain words can unlock the the sign sort of deterministic thinking I've got to give one number.
39:45
I've got to give a yes no, right? Response to this.
39:52
Because I I am always amazed by the way you can I I guess baffle them with right?
40:01
Like if you start throwing a bunch of numbers that people I've actually seen it affect people like really affect them.
40:10
They they even their body language changes.
40:12
And and that to me again just reinforces this idea that I would love to come up with a way educationally and and maybe maybe it's maybe it's just using all real world examples.
40:33
That could be one that is helpful.
40:37
But also on on simple things like you know the old the classic um uh question about when when you add something the chances of it declining, right?
40:50
So Mary is a librarian and is very active in the women's movement and and then you say then you say there are so many librarians and there are so many women who are very active in the women's movement.
41:15
Anyway, I'm doing this wrong cuz I'm doing it from memory, but the idea that so many people will answer when they given that information at the start will say, "No, I'm certain that Mary is a librarian, yes, but she's also active in the women's movement." Right, yeah.
41:31
Which of which of course doesn't work mathematically, right? No, right.
41:35
I mean yeah, cuz the probability of two conditions being mapped in they're smaller than the probability of one of them.
41:45
I mean yeah, I mean I think like I think one way Mary is um a cashier at a local supermarket.
41:54
And two, Mary got a PhD in physics.
41:59
Which would be more extreme.
41:59
What's more likely she's a cashier at the supermarket or she's a cashier at the supermarket and well maybe that the and is active in the women's movement. Forget the physics.
42:10
And anyway, you can make it so that I mean there's always a trade-off between probability and plausibility.
42:18
Often things that to make things more plausible you add details often and the details make the outcome less probable but more plausible.
42:29
And people like stories and if you fill it out a little bit they a lot of different conditions the probability of all those conditions holding is going to be smaller than the probability of some prosaic condition that you start out with.
42:43
And I think most people don't realize this trade-off between plausibility and and probability.
42:49
One of the things that I put in one of my books on this subject was the idea and I can't remember whose study this was, but it really did grab my attention.
43:02
And that was this, if you randomly select kids in undergraduate class on math and and you say to the to the group I'm going to tell you a fact.
43:17
The fact is there's a town of 100,000 people.
43:20
And within that town there are 70,000 lawyers and there are 30,000 engineers. Okay.
43:29
Then the first group you randomly select names out of a bag and it's just a name. Just a name. Jim, Paul, Mark, Mary.
43:41
When they have when they have just the name they use the and their object objective is to get the most right, right?
43:49
Most will say, "Well, to get the most right I'm going to guess that they're all lawyers because there's 70% lawyers, 30% engineers.
43:55
I'm going to say they're all lawyers.
43:59
I'll get seven out of 10 right." Right.
44:00
And the subjects actually do behave that way.
44:03
Okay, that's that's right.
44:07
when you add meaningless descriptive information to the name Tom is 38 years old and likes to guard.
44:18
People begin to abandon the probabilities that they know, right?
44:21
70% lawyers, 30% engineers. Yeah.
44:24
And their correct answers goes down. Yeah.
44:30
And then the clincher is they add stereotypical information. Yeah, right.
44:36
Frank is 42 years old, likes mathematical puzzles and is an introvert.
44:43
People will literally argue with the professor saying, "I don't care if it's it's got to be an engineer." Right?
44:48
And and so that got me very interested in is there a way that we can use these default assumptions on the part of people to better get them to understand taking the next step in thinking oh, that really doesn't matter.
45:10
That those superfluous details don't really matter if I know the base rate, right? Yeah.
45:16
Uh yeah, I I mean one way is just to expose students to these kinds of things over and over again.
45:23
I mean if you're if how many times is this going to happen?
45:28
You ask somebody and then gives you a point estimate.
45:30
Just ask somebody else in class and then somebody else and then somebody else and presumably they'll differ and you say why would they differ?
45:38
Man, maybe why did not be better to say it's it's not 72 but 70 to 80 or 65 to 85.
45:44
And I yeah, I mean it is amazing.
45:49
Well, one marginally relevant thing comes to mind.
45:56
If I is a question I ask in an elementary probability course.
45:58
Uh there's two people and you're flipping I I'm fond of as many probable lists or fond of coins.
46:09
But there's two people and you flip a coin 100 times and before each flip you ask the person to to tell you what they think it's going to be.
46:19
And the first person gets uh uh 50 52 correct guesses out of 100. He anticipated the coin.
46:27
He said heads 52 times it was heads.
46:30
And the other one predicts uh 28 times he was correct 28 times.
46:38
And I said which one of these people has any claim to being a clairvoyant?
46:48
And where which result is most in need of an explanation?
46:53
And um Anyway, they often students often get that wrong.
46:58
I mean 28 correct answers is amazingly improbable.
47:04
But they don't really it doesn't register that they're so improbable when one person got 52 correct guesses and the other 28.
47:09
And it never it's it takes a while.
47:13
I mean after enough badgering and other examples for them to say, "Oh yeah, it's very unlikely."
47:18
I mean you don't have we can talk about standard deviation but how likely it is and so on.
47:23
But even just intuitively, you know, 28 I mean you must be either clairvoyant which is probably absurd or uh just ignorant of statistics. Standard deviation.
47:40
That was another one of my little hobby horses while I was running my company O'Shaughnessy Asset Management was the idea that the standard deviation of return.
47:51
That became a term of art for asset managers, right?
47:57
And so people would say, "What's the standard deviation of return?
47:59
What's the standard deviation of return?" And I would tell them.
48:02
And then I would say, "But are you interested in that or are you asking that because you're worried about losing money?"
48:13
And they would say, "Well, I'm worried about losing money."
48:15
And then I was like "Wouldn't you want the highest standard deviation of return for something that was going like a rocket ship up?
48:26
What you're really worried about is the semi standard deviation below zero.
48:29
In other words when things are losing money."
48:36
I thought that that was relatively straightforward and like the the looks that I got were like just I for our listeners who aren't watching the video, I am I am looking quite confused.
48:52
So the other thing that I know you've written about and I've always been fascinated with is the prisoner's dilemma. Right.
49:01
You know, books and books and books have been written about it.
49:03
What what do you I'm I'm sure you're familiar with Douglas Hofstadter's tit-for-tat solution to prisoner's dilemma. Right.
49:14
Why do you think so many people misplay the prisoner's dilemma, number one.
49:18
And and is it is it something that that like tit-for-tat by the way for our listeners and viewers is basically they used to have competitions back in the '70s where people would write code to try to beat a continuous prisoner's dilemma game, right? Right.
49:37
And and and the number of codes that were submitted that were pages and pages and pages was quite high.
49:43
And the one that won was was called Tit-for-Tat.
49:47
In other words, begin by cooperating.
49:53
Um and then when you're when you the other prisoner rats you out, retaliate, but go back to cooperation. Right.
50:01
And and that that continued to win.
50:01
Um and and people just It seems to me when I talk about that with people, they seem to really dislike a simple solution.
50:11
Is is that part of the math anxiety? I don't know.
50:16
I mean, it's more a function of the I mean, more relevant to one's psychology.
50:21
I mean, maybe people enjoy retribution. Or insist on it.
50:26
They mean, well, he they screwed me.
50:29
I'm going to get him good.
50:32
And uh so, I mean, again, it's uh maybe a a failure to think abstractly about a situation and instead you know, view it from a very personal viewpoint.
50:41
We I think that a feeling for abstraction is important.
50:47
It doesn't It doesn't mean you can, you know, do all kinds of things in partial differential equations, but just kind of abstract the situation and see if it um can be clarified.
50:58
I mean, even something as uh um Well, that's not a good example, but The researcher Flynn uh who got the uh the title the Flynn effect on rising IQs Right.
51:13
uh observed that he he felt in what he wrote and in speeches I heard him give, he felt that it was our ability as human beings to become better at abstraction, which was the reason why IQs generally went up over that period of time that he was looking at.
51:36
And he made his case by looking at answers to quizzes given I think in the year 1900.
51:42
And all of those answers were very concrete, no abstraction at all.
51:51
Um and and then with the rise of technology, etc.
51:54
, uh children, adults became more comfortable, at least somewhat more comfortable with the idea of Yeah. Yeah.
52:04
But I mean, you got to it doesn't mean that people easily uh go to the abstract level to better appreciate the the logical skeleton of whatever they're dealing with.
52:14
What it does mean that people have gotten have improved.
52:18
I mean, they're they're playing games online.
52:20
There there's much more of an emphasis on uh abstract rules.
52:26
I mean, they don't people don't call them that, but than was the case.
52:30
And I I think uh if you get if that's going to be a measure of IQ, I mean, that's kind of explain the the Flynn effect.
52:40
Which uh hasn't gone far enough, however. Indeed.
52:47
Actually, even that what I was going to mention, maybe it's worth mentioning.
52:51
Even simple puzzles, I mean, can benefit the if you just look at them or kind of subtract the human context.
52:58
I mean, like the Monty Hall problem.
53:00
Like that one behind one door there's a car, behind the other two there's nothing.
53:07
And uh you pick a door and then the the host asks you if you want to switch or not. And you should switch.
53:15
Uh and you can make that more plausible if there's 10 doors behind one of which is a car.
53:22
You pick uh you know, uh you pick a a door.
53:26
The host says you can switch, but first he opens uh eight other doors.
53:32
And then it's very clear that you should switch.
53:34
But the abstraction comes in and what if you wanted to avoid the car?
53:37
Instead of a car, there was a toxic blast of gas that came out if you picked that door.
53:45
And then if you pick a door and the host asks you if you want to switch, you should not switch.
53:50
You don't want to increase your chances of getting that toxic blast.
53:54
So, I mean, again, it's perhaps a trivial example, but everyday puzzles and and games often contain the kernel of more human situations that are the contain the logical skeleton of more human situations.
54:10
So, if we gave you kind of carte blanche to design uh introduction to mathematics for for everyone.
54:19
Let's not just limit it to children.
54:22
Um what what does that course look like? Well, it's hard to say.
54:26
I mean, and I don't want to venture into a very contentious realm called the math wars.
54:35
But uh you know, it would have some emphasis on computation.
54:37
That that shouldn't be uh left out, but the anything that in troll people's um uh thinking about problems.
54:46
I I again, want them to a higher level.
54:48
If they're puzzles were an example, that would be good.
54:54
If we're talking about uh analytic geometry, immediately explore uh shapes that correspond to certain equations.
55:04
And you can do that in a in a way if without um you know, being too uh rigid.
55:11
I mean, you know, Y = X squared, then what does Y = X squared + 3 look like? That's very simple.
55:16
You just look the parabola up 3 yards.
55:18
So, I mean, I And also stories.
55:21
I mean, I'm I realize I'm being very nebulous here, but uh uh stories, vignettes are if if possible, and often they are, a better way to impart mathematical ideas than just the equations and formulas.
55:35
I mean, it's one thing I've done in in most of my Well, not all my books, but some of them have other things.
55:43
Dealt with other things, but um you know, you To the extent you can tell a story and give a little vignette, to give a little aphorism, sometimes even that'll do.
55:53
You can get a mathematical idea across without uh uh raising people's um uh unnatural fear of mathematics.
56:02
I mean, this isn't mathematics.
56:05
This is a story about somebody who collects these things.
56:09
And then for every two he gets, he gives away three three, and so on.
56:14
Uh so, vignettes, stories, whatever.
56:14
Uh again, it's uh a tall subject and and again, it's very contentious.
56:23
I mean, in California, uh they're doing doing something like that, but they they're running into trouble cuz sometimes they go on too far.
56:36
And at at the other extreme, there are people who just want to focus on uh algorithms and computation without regard to anything other than the main algorithm and many different examples of it.
56:53
I think one of the things that I've encountered um is just the the term algorithm. Right?
57:00
Uh people seem to me to to get worked up about that um in an unusual way uh because I I asked one person, a friend, who is one of those people.
57:13
Um and I said, you know, what is it about that term that so gets you so excited against it?
57:22
And and he goes, I I don't want, you know, my my actions, which uh are up to me, I don't want the I don't want some abstract algorithmic thing up here determining what I'm going to do next.
57:37
And and I'm kind of looked at him and I'm like, I don't think it's determining what you're going to do next.
57:43
I think what it's trying to do is predict what you're going to do next.
57:48
And yet, like they remained flustered.
57:51
And and again, I like your version of telling stories.
57:55
I used to I finally gave in uh because basically the way that we invested was highly algorithmic was highly based on quantitative methodologies.
58:07
And and yet when I was giving talks, I would often finally begin the talk with, I'm going to tell you a series of stories about why you shouldn't pay attention to stories when selecting investments.
58:23
You said And and and and and this It worked. It worked.
58:26
I got people to understand things like base rates, things like you know, various distributions historically, what do they imply for the future, all of that.
58:38
But I only really seem to be able to break through to a general audience.
58:43
Now, I'm not talking about talking to other uh asset managers, but a general audience. Yeah.
58:50
And and and I like it worked so well, I never stopped.
58:54
I just kept saying, okay, let me tell you a story about that.
58:58
And like the the uh study I mentioned earlier was one of those stories about the people with the lawyers and engineers in the town. Right.
59:08
People seem to really get that and understand, oh, that's why base rates are important.
59:12
That's why knowing those numbers are important.
59:15
numbers are important. And and it I I would actually have people come up to me and say, well, like you know, I here I thought you were going to just be throwing a bunch of math at me and like I'm just so delighted that uh you didn't do that, which brings us
59:33
kind of back to um you know, math anxiety and and the challenge that we face is, you know, there are some life and death death decisions being made where innumeracy can lead the person unfortunately, to make the wrong choice and and end up dying. Yeah, look at look
59:55
Yeah, look at look at Goldman. Goldman's an example.
59:58
So is global warming that people poo-poo cuz they don't look at the numbers at all.
1:00:05
But yeah, I mean, every big issue.
1:00:05
I mean, but yeah, I think the uh cognitive play balls is a good way to uh I mean, should be introduced in math class.
1:00:16
I mean, just give There's lots I mean, look at the Kahneman's book, Thinking, Fast Thinking, Slow.
1:00:20
There's lots of examples whose uh conclusions are very counterintuitive and but yet very common.
1:00:30
And I I think uh it is goes some way towards getting people uh not to believe everything they think.
1:00:39
But you've also um another big uh uh tool in the toolkit of behavioral economists uh is this idea of nudging. Yeah.
1:00:50
You you from what I've read uh that that you've written, you have some problems with nudging.
1:00:55
Do you mind sharing those with us?
1:01:00
Uh well, it can go too far.
1:01:00
It can be very patronizing, very limiting.
1:01:02
But uh I mean, the general idea to to make the in some sense the right decision, assuming you know it, uh easier to decide upon, I have no problem with, but uh uh often people do that through decisions uh um uh in furtherance of salvation, which is not the right decision, always uniformly the right decision.
1:01:30
And it can be uh you know, kind of big brotherish, but overall, it's okay.
1:01:34
Yeah, the big brother aspect has always been uh one that made me a little reticent uh because for example, um uh a classic nudge that seems to work and leads to what I think is generally, in most circumstances, a a better outcome is on uh when a new employee joins a firm.
1:01:59
Uh the simple changing of having them opt into the 401k versus a nudge which changes it to they must opt out of the 401k. Yeah.
1:02:08
Well, you can see what happens there.
1:02:12
The vast majority end up in the 401k because they don't want to opt out.
1:02:17
But when they're opting in, it's quite a bit lower.
1:02:19
So those kinds of nudges, I think, you know, obviously can be quite helpful.
1:02:28
But then the big brotherish type, you know, it does become just a question of, I guess, extremes.
1:02:33
How much you you want to do that.
1:02:36
People always further this idea of nudging with a non-controversial example, and that being, I'm sure you all know the result, uh at the bottom of the urinal, you put a little uh of spider or something that looks like a spider or butterfly, and people want to pee on it.
1:02:55
So there's less less pee on the floor.
1:02:59
But uh but you know, that's uncontroversial, but uh lots of uh not some other things are are are not.
1:03:07
Yeah, and and and that's the challenge.
1:03:10
Another thing that I've found is that, you know, if you look and study at some of the greatest financial fiascos in history, some pretty simple math was at the center of causing and I I just sometimes get I've had conversations with people on leverage, for example.
1:03:32
Um and how deadly leverage can be, and once you use an certain level of leverage, you're pretty much guaranteed to end up going broke uh if you're using it incorrectly. Yeah.
1:03:48
And even like professional investment people that I will give these examples to like dismiss me out of hand.
1:04:01
And uh you know, I had uh talks with some uh folks who were involved in a very big financial fiasco back in the '90s.
1:04:10
And and even afterwards, they just didn't they couldn't grasp that using 40 to 1 leverage on a relatively illiquid asset generally leads to ruin.
1:04:25
And so I was lucky enough to meet Danny Kahneman several times and have these conversations with him.
1:04:31
Of course, he he just he just died. I know. I know. I a a great loss.
1:04:38
He was a lovely, wonderful man. Yeah, he was.
1:04:42
And so I wonder if there would be some sort of like the whole value at risk thing we talked about earlier.
1:04:50
I Do you see any way where what the category that I generally put this in is people who should know better? Right.
1:04:57
Uh may may making that like an investment advisor or a portfolio manager, etc.
1:05:03
Do you Do you think that there would be if if if you were going to give them one or two classes that they should attend to really get a better handle on things like we've been discussing, which which classes would you point them at?
1:05:23
Uh I guess uh some sort of math class that used uh innumeracy and a mathematician plays the stock market or mathematician reads newspaper, but there's some kind of statistics class that that wasn't uh the kind of here's a bunch of formulas and do this and use a T-test and you get this and find the P value, which doesn't really help much, I don't think.
1:05:48
I mean, it it's too much I Oh, that's math. I hate it.
1:05:52
Well, I mean, which is not to say you shouldn't it's worthwhile discussing those things, but they're built right.
1:05:57
What I mean through a context and through a story where it makes makes sense. And that's hard to do.
1:06:04
I mean, there's a book on statistics by somebody named Friedman, Pisano, and somebody else who does a good job with the technical aspects of probability. I would use that book.
1:06:17
And uh you know, there there are it's it's it's hard to to say.
1:06:19
I mean, even simple things like uh I mean, the Black-Scholes Not that that's simple, but uh I mean, part of what went wrong there is they assumed one should events were independent when they're not.
1:06:35
I mean, I mean, forget Black-Scholes.
1:06:37
I mean, what's the probability that uh you know, 3,000 people or whatever are going to be dying in New York tomorrow? That's absurd. That's minuscule.
1:06:46
Unless they're all in the same building on 9/11. Right.
1:06:54
And so I mean, with that, you know, you can never worry about independence, and often it's not clear when when events are independent or not.
1:07:01
I mean, people want to assume they are cuz it makes things easier, and you generally just multiply.
1:07:09
But um and but that's not a mathematical um uh decision. That that's above this.
1:07:17
Meta-mathematics in a different sense of meta-mathematics where you got to know the context that that makes sense of it.
1:07:24
And um and that's not taught a lot.
1:07:27
Instead, the formal procedures are emphasized.
1:07:31
And they're not just emphasized, but almost to the exclusion of anything else.
1:07:36
It not strictly mathematically uh relevant, but I always thought that uh people like uh Professor Feynman, um who had a certain degree of the He was very theatrical.
1:07:51
And if you remember when the Challenger crashed, the O-ring. Exactly.
1:07:53
And and but but the brilliance of him asking quite specifically, Could you bring me a glass filled with ice?
1:08:04
And then he takes the material from the O-ring, drops it in the glass of ice, and it ultimately disintegrates.
1:08:09
I I wonder I wonder if that could also be a useful way to get uh mathematical concepts across like in in in a form in a form of entertainment, really. Yeah, of course, yeah. I think so.
1:08:26
Yeah, I mean, math class doesn't have to be uh dull and boring.
1:08:34
And uh Even even tricks, even whatever and uh tricks on people I think would arouse many people's interest, especially young people.
1:08:47
I mean, the so-called magic tricks and uh scam people out of a bunch of money. Why? Why does it work? Exactly.
1:08:57
Well, you you have certainly done that in in your many books.
1:08:59
Uh do you have anything on the horizon?
1:09:02
I know you just had one published uh a couple of years ago.
1:09:07
Yeah, who who's counting?
1:09:07
I had a lot of puzzles in it, but uh I I'm not yet.
1:09:09
I mean, what I generally do is I put a bunch of ideas into a file and let them stay there and until they begin to marinate, if they ever do, and and marinate into something coherent.
1:09:25
So I'm in the uh waiting for marination stage.
1:09:35
Well, listen, this has been absolutely phenomenal. Um I'm a huge fan.
1:09:38
I will I I think uh uh one of my associates who's here saw this book with cuz I pulled it off the shelf and he's like, "Ooh, can I borrow that book?"
1:09:52
So, it may maybe maybe just literally leaving your books around uh where people can actually see them as as opposed to in the bookshelf might be the right thing.
1:10:03
Uh at the end of this podcast, we play a little game, uh which is Wait wait, before we do that, let me say I I really enjoyed talking to you, Jim.
1:10:11
I It was a very good interview. Oh, well, thank you.
1:10:13
I really enjoyed talking to you, as well.
1:10:15
Uh I I passionate about trying to help people understand these concepts better as and as are you, and you're you're much better at it than I am.
1:10:27
Um so, but at the end here, what we do is we we do a little bit of magical thinking, and we say that we're going to wave a wand, and we're going to make you emperor of the world for 1 day. You can't kill anyone.
1:10:40
You can't put anyone in a reeducation camp.
1:10:42
But what you can do is, since this is all magic anyway, we're going to give you a magical microphone, and you can say two things into that microphone.
1:10:54
And all of the world's population, whenever their tomorrow is, is going to wake up, and they're going to think, "I've just had two of the best ideas I've ever had, and unlike all those other times, I'm going to write them down, and after writing them down, I'm going to actually start acting against them."
1:11:14
What two things are you going to insept in the world's population?
1:11:16
Well, I think uh anything that induces a certain weariness, I mean, measured weariness.
1:11:23
You don't want people and uh you don't want people to believe everything they think.
1:11:30
You want people to entertain the possibility that they're wrong.
1:11:35
And they should that should be they should ask themselves uh that daily.
1:11:38
Uh yeah, I say this, I'm pretty sure it's right, and probably is, but let me just think about it a a little more, and not be be a little bit tentative.
1:11:51
Oh, but not empty-headed, not entertaining everything, but um So, skepticism, weariness, uh and an appreciation of their uh inherent uncertainty of life are um are good traits to develop.
1:12:11
And they're uh unfortunately, they're I think in short supply. Yeah, I I agree.
1:12:17
Uh the author, Robert Anton Wilson, had a lovely quip that was uh anytime you stop and think to yourself, "Maybe I'm just a cosmic schmuck," he says, "You make yourself for a little bit of time a little less of a cosmic schmuck." That's all right.
1:12:38
That's That's a nice two words to end this interview on.
1:12:44
Well, thank you so much for joining me.
1:12:47
Uh really really enjoyed it, and uh let's see if we can chip away at innumeracy. I think we can.