Seth Stephens-Davidowitz — Who Makes the NBA? | Episode 250

0:00

writing a book is just such a pain in the ass I'm just like I don't know if I can do this again then I came across code interpreter I'm just like oh my God things that were taking me four months are all of a sudden taking me four hours a lot of things get overhyped but on the things that you've just mentioned it is

0:18

very difficult to overhype what that will be able to do across every industry the geek shall truly inherit the earth there's been an idea for a while that NBA players come from more troubled backgrounds and that's never been true if you look at the data both African-American and Caucasian males

0:40

much more likely to reach the NBA if they come from a two parent home if they come from a middle class background rather than a four backgrounds wealth is a huge advantage to reaching the NBA sometimes the story of someone who overcame the odds is so compelling that we forget the reason that it's so

0:57

compelling is because it's not normal it's hard it's difficult well hello everybody it's jimo shasy with yet another infinite Loops I am so excited by today's guest Seth Stevens David a witz did I get that right close enough yeah close enough in the ballar for a guy who's oh what to

1:21

see I get that all the time so so I'm used to it I like you are one of my favorite people for so many reasons but I I think one of the main ones is we both love data data data data more data can unlock so many things that you do not intuitively get to listen you you've

1:44

got a ba in Philosophy from Sanford you were 58 to Kappa PhD in Harvard and economics wow you are quite the overachiever my friend data scientist best-selling author sought after keynote speaker all so this is the part that I love a graduate of the American Comedy Institute yeah so that is true yeah what

2:09

a what a talent stack welcome SE well it was more just I was so burnt out I was doing a PhD in economics and I was just so burnt out one summer I'm like I can't read another economics paper like I need to do something create more creative or different so I Googled comedy standup classes and I found American Comedy

2:28

Institute and uh yeah it was really fun it ended with a show in New York City I I kind of rigged the audience I had all my friends and family there so everyone like just cheered even when my jokes sucked everyone just cheered anyways but yeah yes I always try to rig things as best I

2:45

can as well so we we uh have we're sympatico there as well I I have a funny story I have a useless as it turns out degree just a be yeah which I barely got frankly only because my mother wanted me to uh in economics but uh I was taking a graduate level course in economics and

3:05

the the guy was going on and on about you know the model does this the model does that and it was you know typical Kan I'm 64 so it was when kanes was still kind of semi uh important uh back then it was just the dawn of the rational expectations guys uh but anyway I I raised my hand in the class as he's

3:26

droning on and on about the model this the model that and and I said uh I'm sorry professor and I don't mean to be rude but have you ever a looked at what actual humans do he goes maybe you better try comedy instead of Economics Mr well what I love I love all your books I recommend them to people who

3:52

makes the NBA don't trust your gut everybody lies they are just packed full with really unintuitive nonintuitive truths that I hope we're going to get a chance to talk about all of them today but but I want to start with a comment that you made when you when you discovered code interpreter you were

4:16

like holy [ __ ] this stuff rocks and then you made yourself a promise I'm gonna write a book in 30 days using Ai and the code interpreter and you did it and it's a great book so let's start there what was it kind of like just this Satori or Eureka moment when you found her yeah so

4:39

I was so I'd written these two books everybody lies and don't trust your G and I was really happy you know proud of them and they led to all these great opportunities and you know they're bestsellers and traveling the world talking and Consulting and everything so I'm like this is great but writing a

4:55

book is just such a pain in the ass and like uh I'm just like I don't know if I can do this again you know like the you know for for me I'm just a perfectionist so you know like there you know I got to disappear for years in writing these books and you know I I do a lot of

5:13

regional research data analysis and then I came across code interpreter and I'm just like oh my God like things that were taking me four months or all of a sudden taking me four hours it's crazy and all the things I don't like doing it was doing so like I don't like you know cleaning a data set merging a data set

5:32

like you know all these annoying kind of wrote tasks that are just so big a part of like a data scientist life code interpreters just doing it in seconds and I'm just like wow that just leaves me to do the things I actually really like which is coming up with ideas testing them uh writing kind of writing

5:49

them out explaining them so it was the 30-day thing was just yeah like I I it just gave me kind of motivation and was kind of like a fun way to do it and uh and yeah it was it was like it was kind of I kind of say it was the best 30 days of my life which I was explain to a

6:08

friends that recently and I just like said that like that was a normal thing and they're like set that's kind of odd like most people when they describe the best 30 days of their life it's like their honeymoon like when when when their child was born like second semester senior year like not I was

6:24

working on a book for 30 days but that's probably says a lot about me oh you must be just but here's the thing Seth you are a blast to talk with and had a cocktail party and all of those things so it it you know what some would see I always I've been saying a lot recently the geek shall the geek shall

6:47

truly inherit the Earth because of the power that we now have in our hands with uh things like code interpreter AI models in general Etc um and having written four books yeah I completely agree with you but man I have been dreaming of AI and that type of stuff I keep journals and like the early

7:10

I didn't call it AI but one of the earliest references I had was like in 1982 wouldn't it be [ __ ] great if we had a machine that could do all this and and that's the part about it that really excites me your book about the NBA which we're going to get into in a minute is really fun because it's about the NBA

7:33

right what AI is really really good at is getting rid of all of that laborious boring just timeconsuming stuff I I had like one of my big things when I was at osam which we subsequently sold to Frank devton was we came out of beer Sterns into the great financial crisis and I kind of sat down with my team and said

7:57

well we're probably not going to sell another long only portfolio for the next three years so here's what I want you to do and the thing that they rebelled most against was cleaning the data yeah because it is having done it for my book what works on Wall Street oh my god when

8:16

you start dreaming of the numbers going down a screen and you see the wrong one in your dream you know that things are pretty dire but you know the data is dirty a lot of people don't get that right like especially financial data and to get them a little more revved up I kind of I took I think it was Apple uh

8:38

and put it up on the screen in the conference room and I said what's the PE of apple and like everyone sort of the the Traders shouted it out and some of the data guys said a different number and then I pulled up all the various sources fact set one PE uh Reuters another PE Financial timeses third PE

8:58

none of the PE is batched I said guys apple is one of the biggest companies in the world and none of it none of this is cohesive at all yeah you start yeah and I think AI is gonna really fix a lot of this revolutionize a lot of it and uh you know that that get the right numbers

9:19

clean the the numbers fix the numbers explain when the numbers are off why they might be off and uh you know yeah quantitative Finance is definitely an area where uh AI is you know coming big time and going to improve things big time so uh but but there're just all kinds of areas like you know code

9:38

interpreter was just kind of yeah my Eureka moment where I'm just it it was like oh my God this changes everything you know I thought about AI up until that point as like you know people are talking about it I kind of thought it was cool I used it a couple times you

9:53

know I I wrote some poems for my girlfriend using chat gbt and they were okay poems but then code interpreter was the first time I'm like holy cow this is just going to change absolutely everything in the areas I know I'm like I can just guarantee this is like a total revolution in you know the the

10:14

creative process and I think you know a lot of things get overhyped I honestly think even with all now some much of the hype is [ __ ] in my opinion because well that would be a long conversation but on the things that you've just mentioned it is very difficult to overhype what that will be

10:37

able to do across every industry now certain industries are going to be affected more than others you make a point that if you really want to clean up find an industry that is really bad at collecting data using data and if you glance at obb's website you're going to see a lot of verticals there that are

10:59

those Industries right and and obviously with my Quant Finance background uh it is ever present in my mind but let's talk let's talk about the NBA book because I it was such a joy you talk to a group of osers and guests and like people talked about your talk endlessly after it because what you

11:23

found in the book was really many times counterintuitive I'm not a sports guy uh but I many of the people who work for me are and like they were some of them were like really reacting kind of like I can't believe the thing about the names for example we'll get the names in a minute but let's start with the obvious

11:44

one height yeah yeah tell us about what you found about the correlations of height and the NBA yeah so I mean I think you'd have to be blind not to notice that tall people are advantaged in basketball uh like obviously you know Height plays a role but kind of when you do the math do the data analysis I

12:04

didn't realize just quite how it plays out in that each inch roughly doubles your chances of becoming an NBA player which is just wild like throughout the height distribution so if you're 511 you have twice the chance of making the NBA compared to someone who's 5'10 Al the

12:22

way and if you're 65 you have twice the chances to make the NBA compared to someone 6'4 all the way out as far as we can measure to you're 7 feet tall you have twice the chance compared to someone 611 it's like a perfect uh linear log relationship basically and what that means is if you're under six

12:39

foot tall you have less than one in a million chance of making the NBA and if you're seven feet or above you have a one in seven chance of making the NBA which is just wild like there's no other trait that gives you such a great odds of becoming you know a multi multi

12:59

billionaire just with that genetic trait uh I don't think there's anything comparable and then another thing I kind of started you know I found that then I talked to my friend and he's like you know one of the things that means is if you're really really tall you don't have to be a great athlete to

13:20

make the NBA uh because if you have a one in seven chance like if one in seven seven Footers are in the NBA like that's you don't have to be that good you just have to be one in seven athletic Talent so one of the things you also see in the data is that the tallest NBA players are just not

13:37

particularly good athletes you know they jump about the average level of you know a high school person they run the about the average speed of maybe a high school track an average high school track athlete uh they're te you know I could shoot free throws better than many of the Hall of Fame NBA tall NBA players

13:57

just because height is such an advantage everything else just kind of isn't that important uh if you're if you're so tall so yeah that that's that was kind of really cool to see in the data and it led you to develop a very cool uh new stat you called it the mugsies uh stat talk to us a little bit

14:15

about that yeah so Mugsy stat is just how good every player would be uh if they were the same height and there's a whole bunch of math backing it up which I won't bore people with but uh one of the things I like about it is I so mugsies the number one the player ranked

14:32

number one on mugsies was Mugsy Bogues who was 5 foot three and a player in the NBA for 14 Seasons it's kind of an insane accomplishment to be 5 foot three and a you know a competent NBA player for 14 years so he's number one so I'm like you know uh so I called the stat bugies after him and I'm like you know

14:54

it' be really interesting uh I should like that should be an acronym for something mugy should stand for something you know but I couldn't think what should Mugsy stand for so I asked chat GPT you know what is Mugsy stand for and it came up with a metric for understanding game given sporting

15:14

individuals Effectiveness and size uh which is just so so good and I think also shows the power of these AI tools where I don't think I ever would have come up with an acronym uh like that and uh and chat GPT was able to do it you know AI was able to do that so it it does kind of uh you know these things

15:37

that we wouldn't be able to do otherwise we are able to do thanks to AI you know it just brings up so many questions like is basketball ready for its money ball moment uh through data analytics and then a side question that doesn't have a lot to do with basketball but is there is there a height

16:01

equivalent uh for data scientists like in other words 7 foot tall one and seven chance to get into the NBA right what about a guy not being height but is there a similar metric for finding that fabulous data scientist that's a good question I I don't think so I don't think one in s of

16:23

being you know a multi multi-millionaire I think having a you know being reasonably good at math you probably have a one in you know seven chance of having a pretty good job I would say but I wouldn't say a one in seven chance of you know being uh you know a worth uh TW you know a $10 million a

16:47

year salary and all the other perks of of NBA stardom so I think you know yeah what what makes it and I don't think one in seven chance of reaching the very very top I don't think other fields are so dependent on on one thing in the same way that NBA basketball is there are so many skills that tend to you know

17:08

contribute to someone being good even just other sports like you know what makes a great soccer player I don't think there's something like height that is so important relative to everything else I think there are lots of traits that come into play on that note do you think then for

17:25

the first part of the question is is there a way you can m ball basketball yeah I think there definitely are it's a little bit more complicated I actually have been talking to people I I'm trying to figure out kind of other things I want to do with my life and I'm like maybe I should work in basketball

17:44

because I love the game so much and I'm kind of like you know you know I you know what would what would that be like and so I talked to a whole bunch of teams trying to figure out if there's a way to uh you know get into the the the field and it's a little complicated uh I

18:02

I found out some interesting things one of the things that I uh found out is that kind of the stats people so much of picking the players just have to do with Arcane and very formulaic salary cap rules so like they're not trying to find necessarily the you know in massive inefficiencies in the market uh they're

18:24

trying to find out you know that yeah just like who fits in due to the weird salary cap we have so it's it's not as Money Ball in some ways because of that uh which is kind of interesting I think and another thing I learned is I talked to an assistant general manager of the

18:41

Denver Nuggets and they drafted if you're not a sports fan you might I don't know but Nicola jokic like one of the greatest players they drafted him in the second rounds even I know even you know yeah so they draft him in the second round and it's very rare to get a great player in the second round so I'm

18:57

like you know how did you you know come up with this idea to draft jic and he's like you know it was coming to our pick and we had six players ranked you know the six players we were looking at and we had them ranked this is our favorite this is our second favorite this is our

19:12

third play favorite this is our fourth favorite this is our fifth favorite sixth favorite the five right before us were drafted and that left us with six who is jokic and we picked him so it's entirely luck like there was no uh great great genius that led to that pick you know if another team hadn't have picked

19:31

uh the player that they had ranked higher they would have just picked that player so there's a lot of luck in in you know in and even the great picks in basketball yeah and and outside of sports there are many other industries that kind of fall into the same uh sunt cost fallacy uh and by that I mean like

19:52

no this is the way we've always done it and we love it this way and there's a lot of tradition around it and you're and it becomes a social Norm right and social norms are much more restrictive in my opinion than many people believe they are like like the data you're finding here I

20:12

think one of the reasons why so many of my colleagues were really excited by your talk was because they just were scratching their head right and and that kind of leads to another thing you found which is the names of players specifically black players in the NBA which are the majority um and and the

20:33

the common um wisdom is that they have unusual strikingly unusual in many uh uh instances names but you found that was not true well so there's an idea there's been an idea in a while for a while that NBA players come from more troubled backgrounds uh and you know there are examples so LeBron LeBron James you know

21:00

one of the greatest NBA players of all time the greatest player of the last 20 years uh he was born in poverty akan Ohio single mother and uh there are lots of examples of players NBA players who come from similarly troubled back difficult backgrounds and there's an idea that that gave them the drive that

21:23

you know compare to compare that to you know a kid who grew up to know was raised by a doctor and a lawyer in the suburbs and they have so many options are they going to spend every afternoon at the basketball court you know someone from a difficult background their only

21:39

Escape is through basketball so they're going to do whatever it takes they're just hungrier than everyone else and that's never been true uh if you look at the data both African-American and Caucasian males much more likely to reach the NBA if they come from a two parent home if they come from a middle

21:57

class background rather than a poor background wealth is a huge advantage to reaching the NBA and the way you see it most striking is in the first names of black NBA players uh so the name the title for the chapter was why are so many NBA players named Chris and it turns out

22:19

that uh NBA players uh black NBA players are much more likely to have common names names that are given to you know many to many people uh so names like Chris and Paul and Kevin uh those names are much more frequently uh given to NBA players than unique names names that aren't given to anybody else uh names

22:44

like Shena you know things like that they're much less likely so compared to the average uh African-American NBA black players are about twice as likely to have uh common names which is and and what and what what that the reason for that is that your name can give away your socio economics that wealthier

23:06

African-Americans upper middle class African-Americans are much more likely to have common names to be given names like Chris and Paul so the fact that when you listen to an NBA game you hear that Chris is passing the ball to Paul or uh Kevin just blocks the shot of uh James that's a clue of the

23:24

socioeconomics of the guys on the court which is much more likely to uh mid middle class upper middle class even wealthy and I think one of the reasons why people were struck by that um is because elsewhere you you write about you know there's a huge percentage of stories that fit into six structures

23:47

right and and one of them is the rags to riches uh and we just love those stories right actually our infinite books division is writing a book about uh David Rooney who is uh a surgeon came from a very disadvantaged background um and and got a scholarship to Annapolis because of

24:12

basketball um and was driven in all of those things and sometimes the our imaginations right David's story which is amazing and that's why we're writing a book about it right like what he overcame is he's an extraordinary guy uh but the the I think one of the reasons it appeals is because

24:35

of what he had to overcome to go on to be so successful and so you know that makes a lot of sense when you think about it kind of logically well yeah of course but we we're so drawn by that archetype of that story you know Rags to Riches or overcoming obstacles and adversity Etc for a more interesting

25:00

story for people to get involved with emotionally but you also point out that location where where the player grows up very very important the the father super important to the player as well and and these things are are not intuitive to many people right yeah they in some ways

25:23

they should be intuitive there's like I think you're right the sometimes the story of someone who overcame the odds is so compelling that we forget the reason that it's so compelling is because it's not normal it's hard it's difficult I think also people sometimes make the wrong career decisions based on

25:43

this you know they learn about a story that is so compelling that they forget that it's not normal and they end up you know thinking you know making decisions that there's a line in business uh use your unfair Advantage uh right so like you don't want to you know do things that are

26:04

necessarily really hard for you or really you know lean into your disadvantage even if it would make a better story you want to pick something where you have an un where you have an unfair Advantage uh where you have you know whether it's capital or your network or the university you went to or

26:21

your expertise uh you know I in one of my books I talk about what makes a successful entrepreneur and they tend to be older uh the average successful entrepreneur is 45 years old and the uh chances of creating a successful business increase until the age of 60 and they tend to be uh insiders like really they know their

26:44

business well and they tend to have already been good employees they were good at their job before they left out of their own to be an entrepreneur and it makes perfect sense but it goes against some of these kind of wacky sto stories that we hear we love the story of someone who you know came out of

27:01

totally nowhere they were disgruntled they hated their job nothing worked and they're just like well I'm gonna you know create some wild business I talk about some woman who created POI uh a product that uh basically gets rid of the smell of poop despite having no backgrounds in you

27:21

know chemistry or anything in that area and everyone just like loves this story they're like that what an amazing story uh I want to do that too but if you look at the data you know it's not a smart gamble to try to start a business in something you know nothing about uh compared to something you've been

27:40

studying for 10 15 20 years uh that that's going to you're much more likely to have success in something you're an Insider at well yeah and that that that might be consistent with how we evolv too right like what I I think Evolution made us really cognizant of Novel things right especially dangers

28:02

we pay really close attention to novel dangers dangers that have been around for a long long time we're like he yeah we we we know about that and as I was listening to you of course intuitively one would get at least if you ask me right who's could probably have a higher probability chance of

28:25

doing well some individual who grew up in a two parent household that was relatively well off well educated Etc or somebody who had a very dysfunctional childhood single parent maybe even foster care um I'm gonna always pick this one over here right I'm gonna say you know on a

28:47

big probability basis this this individual is gonna probably go on to do better and I think that's part of the appeal right like that's not novel that's kind of table Stakes over here we love the novelty of the fact that good Lord look at what they overcame or look at you know they they invented something

29:09

they had no expertise in those it's the vividness of those stories like one of my favorite sports uh movies is Rudy now I'm I like Notre Dame and I two kids graduate from there but again it just it your 5 foot nothing you weigh a buck nothing and Rudy didn't go on to be a

29:32

big sports star but the the whole story was just about the fact that he gave so much to the Notre Dame team that they put him on the field so he could be included in the picture of the uh of that particular class of football players and and and I think that it makes sense that the the Vivid unexpected

29:56

unusual is is going to grabb our attention much more than the much more you know pedestrian mundane data but you're probably better off right if if you're not like Rudy or not one of those people as you say m may maybe don't look at some inspiring story a one in a million type guy and say I'm gonna be

30:20

the second guy yeah I mean and it's dangerous you know I I talk in one of my books don't trust your about you know after The Social Network movie came out where they talked about Mark Mark Zuckerberg creating Facebook like there was just a huge rise in 18-year-old entrepreneurs people dropping out of

30:39

college to start their business and you know the the failure rate of an 18-year-old entrepreneur is just insane like through the roof you know it's very very hard to start a business at the age of 18 and you know Mark Zuckerberg is that one in you know he's he's a one in a billion outcome and

30:58

you know I think it is dangerous to build your life around these one in a billion outcomes rather than you know studying the people who had a success that was higher probability that there are many examples of them and again use your unfair Advantage you know if you're

31:16

seven feet tall go into basketball if you've been studying a field for 10 15 years use your network to start a business there like you know there there are all these areas where you can really have a great probability of a big success success but they may not be as exciting as kind of like gambling it all

31:32

on some you know oneoff thing they're not going to make a movie about you basically if you do that necessarily right right yeah and and that's the part that intrigues me to no end right the the movies are entertainment right and what's entertaining what's entertaining is not

31:51

watching a guy go build a car dealership in akan Ohio yeah and you know but it also focuses Societies or our our hive mind in ways that as you point out can lead to really bad decisions down the road right be because if you if you are um going the root of I gonna take this incredibly rare individual who I have

32:20

great admiration for and and I'm going to do that you are stacking the deck against yourself not the other way around right like when when I was early on in Factor investing which is Quant investing like the the the stories that I would tell are you know would you go to a doctor who says he's gonna wing

32:40

your treatment like you walk in and you're like oh my god I've got this horrible pain here in my right side of my belly and he looks at you and says you know uh farmaceutical rep just came in and handed me these little yellow pills and he said they were great why don't you try them you're going to run

32:58

like hell you're going to want to go to a doctor that says here's what you have we've done massive meta studies on the medications that work in this particular case therefore you know this one is going to be what's right for you and but like even when I would tell people that

33:16

I used to say I tell stories to explain to you why you shouldn't pay attention to stories when making your investment decisions but we we it seems to me that we humans are OS runs on stories right and so one of the things that I love about you is that you are really good at telling a fun and compelling story but

33:40

basing those fun and compelling stories on data yeah that that's right that's something I try to do and you know it's it is a challenge it's it's always a challenge you know how do you explain to people that you know the average successful entrepreneur owns a average

33:58

Distribution Company like you know yeah it's not as fun as the stories about the woman who cured poop or Zuckerberg you know being an entrepreneur at 25 years old so it's a challenge but I think people do stories do stick in people's minds so you can't just you know necessarily just show the charts and the

34:19

data you know you have to add the story and add you know I I talk about someone I talk to who runs a beverage Distribution Company and kind of explained what his life is like and try to make it more compelling to people and I think you know for me even just charts and graphs and data there is something

34:37

inherently like I see the stories in that like you know in some ways they're better stories because there are millions of people behind that there are millions of stories you know that are leading to that incredible clear pattern in the data uh so I I think in some ways

34:53

data can be better for storytelling than other methods of telling stories cuz you know they are the stories of the masses they're this the stories of how Society Works in some like fundamental way and that is something I do put a lot of work into like how do you show the data show the numbers allow people to really

35:13

understand how the world works but also make it compelling to people yeah I think uh that was something I learned the hard way and and thus change the way I presented what I was doing to people uh because the first time around I was a bit geeky about it to be honest and I showed a lot of

35:31

numbers and I showed a lot of graphs and I I was very disappointed after like one of my first real big presentations people were just kind of like but and then I kind of realized well that's because you didn't tell any stories yeah yes and no because one of my favorite pieces of data I think this

35:55

one was also don't trust your gut is in dating they like what who who are the most successful daters who get the most matches on online dating sites and like the the most successful daters are kind of the obvious people you'd expect so they're you know just incredibly good-looking they look like Natalie pman

36:14

or Brad Pit and everyone wants to date them okay you know we know that I'm not in that group of people most people aren't in those group of people so that's not going to work for you so who else is successful in dating and they found that the most successful dating ERS are really

36:29

polarizing so they have like it's women who shaved their head or dyed their hair blue or just they get a lot of really terrible ratings and a lot of really great ratings but they have an audience who loves them and I think there's something similar in business sometimes where even if you are like a total geek

36:50

like maybe you just need to sell the total Geeks right if you came and gave the presentation it was all numbers and graphs and I money to invest I'd be like okay Jim's my guy right so it's not totally you don't always need to it depends if you need to appeal to how many people you need to appeal to and

37:08

how strongly you need to appeal to them right sometimes what you want to do is it's it's okay if a bunch of people in the room are falling asleep or hate you or don't understand you as long as a few people in the room really love you and think you're unique and special so yeah

37:24

um and not surprisingly the group grp that uh I did extremely well with were engineers in the Bay Area yeah yeah they were just like just the numbers man just show me the numbers yeah I think it's you know in dating in business and life a lot of it is also finding your audience so leaning into who you are and

37:47

then finding your audience so I think some people make a mistake they give a presentation and nobody likes it and they think the problem is the presentation the problem may have been the audience in that you haven't found your crowd yet uh and yeah you know similar you're

38:03

getting rejected over and over in dating it doesn't necessarily mean that you're doing something wrong you may just have not found the right people to go on dates with so yeah and that also brings up your observation that people often uh with regularity choose the wrong metric

38:22

right like Beauty career are they Rich are they beautiful those are pretty shitty metrics actually if you really want to go on a date and you and you point out that the predictive power belongs to looking at psychological traits uh and the one you just mentioned you know they're somewhat

38:42

polarizing yeah so if you do studies of what leads to successful marriage and they've studied you know tens of thousands of couples and how happy are they theyve used machine you know speaking of AI they've used machine learning to build models like of every possible trait on these two couples what

38:58

predicts that they're going to get along lead to be in a happy lasting relationship and uh the overwhelming evidence is that the predictive power the partner uh that's going to most likely make you happy in the long term the things that don't matter are all the superficial things like you know are

39:18

they beautiful uh what career are they in uh what how tall are they uh and the things that do make you happy are various psychological traits you know growth mindset conscientiousness something called a secure attachment style satisfaction with life basically a happy person kind of a happy well adjusted nice

39:41

person uh you know conscientious like that's what really matters in the long run uh in according to the data so that's kind of another area where uh shininess kind of tricks us right so if someone shows up to the date like one of them is just stunning but they don't really have their stuff

40:01

together and the other one is a little plain looking but gets the job done you know kind of yeah a lot of us kind of also evolutionarily we're going to be drawn to that stunning face right and stunning body but it's not necessarily what leads to long-term happiness yeah and it also uh one of the

40:19

reasons why it intrigues me so much is it seems to be Universal across whatever you're looking at right so stocks obviously ly right glamour stocks they even called them glamour stocks uh and uh generally speaking at least historically now we've had a a long period where this has not been true

40:39

so we'll we'll have to wait and see as data comes in but um glamor stocks historically not great performers at all uh we we often would do a study where we took something intuitive like for example what if we just bought the stocks with the greatest percentage increase in revenues over the

41:01

previous one three and 5e period right sounds like something oh that they're doing something right like their revenues are doing really really well and then I did I found a little sub period where that particular single line strategy by the 50 stocks with the highest revenue gain kind of like

41:21

tripled the S&P 500 over a 5year period right and I would put that chart up and I tell the story right like does this look like a really cool way to invest really simple you go to any stock screener there most of them are free you could find this what do you think and people were like yeah that's a great

41:39

idea that you blah blah blah then I would show them here's the results of starting with all of the data that we have and in this instance it went back to the mid 1950s and that strategy underperformed teils and and so trying to get them to focus on the larger picture as opposed to the smaller was was very helpful but

42:05

I think that we also we can't do that everywhere it's it's like the famous story about the Brits trying to get rid of rats in uh Colonial India and and they said yeah we'll give you a bounty for a rat's tail and of course what happened was people started breeding rats so they could cut off the taale I

42:24

are there other metrics elsewhere uh that that you've seen and what God everybody looks at this and it has no predictive power at all uh yeah I mean definitely there are lots of examples of that I'm trying to think what would be even the the the best one but uh I think you know definitely definitely one

42:52

of the things that stands out is kind of the danger of Glamour uh it I think you're right that it's more Universal so I also did a study of you know you can predict how long a business is gonna St like last how long how quickly it's going to fold and one of the biggest

43:08

predictors is basically just how sexy the business is so the worst businesses are things like record stores the single worst historically has been a record store uh Andy store Toy Store beauty store all these things that sound really cool and fun uh they they might have movies about them they're you know uh

43:29

massively uh on they they're massive uh shot they have a massive shot of going out of business very quickly so I think glamour in general I found tends to be a negative predictor we think it's a positive predictor and if anything uh over time over the long term it seems to be negative predictors so that's kind of

43:49

a universal pattern i' I've seen in in many areas yeah one of the things that is also difficult and I wonder if you've run into this situation as well sometimes when a pattern or a again I'm thinking in kind of terms of stock selection models many times when it really goes against

44:14

intuition even if it works beautifully both in back test and real time right so you've got 50 years of showing it's been maxed you all of that sort of stuff you've got 10 years of showing it doing really well in real time people still would be like ah it just I I it really bothers me that you've got this

44:39

particular variable in there I remember one time I got very far along and a huge assignment for a small to midcap growth uh uh placement from a group in London and and then we got on the uh Final Call and the guy was very polite and he's like we love your stuff we we've really tried to break it as many ways

45:04

which we can but we're we're GNA pass I'm like okay why and he said it's just too simple yeah do you hear that of yeah I definitely I think there is I mean that that was a problem I had in Academia as well like I never really loved Academia because I think they do glorify

45:26

complexity for complexity's sake and you know you need a really wild math model and you need a simulate you know crazy simulations and and it's kind of like sometimes the best ideas I found are just very very simple and very very clear and you could explain them to your you know your mom your grandma uh your

45:46

siblings that's definitely thing another thing I've also you know thinking about things that don't predict outcomes you have to be a little careful about that so Google did this study famous ly where they found out that uh GPA of applicants has no correlation with how the employees

46:06

perform so you know so if someone has a 4.0 someone has a 2.0 they're just as likely to be good Google employees but there's a little danger of going from that to we're not going to care about GPA at all because a lot of people with the 2.0 GPA they had something else that made Google want to hire them despite

46:25

the 2.0 GPA PA right so they had you know maybe they were you know won a coding competition or they you know have amazing reference or there are lots of other things that led to the hiring without the the great GPA so going forward you have to be a little careful sometimes the reason for the the

46:44

correlation or the lack of correlation there could be something else that's driving that that you have to be cautious with in using you know using the the correlation the past to make decisions in the future yeah uh obviously but uh I would argue back that um at the very least you

47:04

wouldn't want to ignore GPA entirely but you wouldn't want to make it your centerpiece you wouldn't want to have a model that says everyone with a GPA below 3.8 is excluded right because then you're going to miss all of those extra special somethings right for sure and I try to be like I'm a big supporter of

47:26

bay rule which is you're kind of you're constantly updating your understanding of the world based on information so anytime you have a correlation that's surprising you're going to make slight adjustments right so if you were expecting that there would be a strong positive relationship between GPA and

47:42

how an employee performed and you find there's no correlation you should be using GPA Less in your analysis than you might have otherwise you know because you now have new information that surprise you that changes your model of how the world works so kind of always adjusting my

47:58

understanding of the world Based on data I'm coming across and my personal experience here is sort of interesting because I look at the world same way as you do and uh another thing that was found to actually not add anything and in certain studies subtract is the standard interview uh you know where do you see

48:21

yourself in five years what is your greatest weakness I just care too much right and yet I work too hard sometimes I work too hard I just I care too much about the company I'm working for that's a real it it really kills me you know I just care too much but but the reason I

48:41

bring it up is because I didn't I stopped doing traditional interviews and luckily for most of my career I've had my own company and so I could do what I wanted to do for for a while I worked for Bear Sterns R running their uh quantitative investing investing group and the push back that I got from people

49:06

about abandoning the traditional interview process was really extreme and it goes back to what I mentioned earlier I was violating a well-established cultural norm and people really don't like it when you do that and and so you know I'm tell you funny story so they let me once and you'll learn why it was

49:32

only once they let me interview candidates for an internship at be Sterns and literally I only recommended one person um who by the way got recommendations from every other big Investment Bank and chose to go I think with Goldman uh sacks but what I did in the interview process was I wanted to

49:55

see how they thought so the the question that I gave all of them was the S&P 500 is a total return index in other words it reinvests dividends and gives you the total return had you reinvested dividends over this period from 1926 27 through here that itself has some things that isn't quite right about it because

50:19

it isn't the s&p500 it's a proxy for it Etc and then I would say the Dow Jones Industrial Average is only 30 stocks and it doesn't include uh the reinvestment of dividend and it's not cap weighted like the S&P 500 is and then I said if the Dow was cap weighted and reinvested

50:42

dividends what would it be at today and like literally I knew they weren't going to come up with the right answer I didn't come up with the right answer when I tried to do it what I was looking for how do they do things how do they go about solving problem right and and that led to like this

51:01

sorting mechanism which got this guy but again they were mad at me they were like really mad at me um and and I think one of the reasons were uh like a math major I asked when they sat down I was like oh math major cool what's 177% of 72 and like just because I was not I

51:24

don't care what I want was the reaction how do they react to that um do you see that as kind of a pre-existing kind of cultural reticence to to be too oriented toward like data analytics Etc or am I did I just stumble into the wrong interview process no I think one of the frustrations I have with like the world

51:54

and definitely business is just people don't collect enough data so like it's really frustrating to me that they've been doing all these interviews and they haven't you know correlated how did they rate people on interviews and how does that lead how happy are they with the employee down

52:07

the road like nobody does that and like they just do it you know people just do things because they did them and I think even if you know you have a different idea like I recommend people collect data on your new idea cuz maybe you're just so pissed at this old idea that you

52:24

come up with a new idea that actually doesn't work work but then you have your own biases in that direction and maybe you know so I I don't know I think there's just not enough data on these things like these are answerable questions in my opinion like do interviews predict long-term performance

52:40

uh are answering questions that are more about how you think than the answer does that you know lead to better performance I think all these questions do have an answer but I think for whatever reason people just aren't in the habit of you know testing their hypothesis here and you know or they you know kind of

52:58

speaking to the idea that uh one example shiny example kind of biases us if there's one time someone answered a question really well at an interview and they let were a great performer everyone's going to be like that's the question we need to ask when it's really only one example that

53:18

led to that so uh I think we sometimes do you know that that we're not we're not very systematic in some of these decisions and like really understanding uh we all have our biases and our ideas and our anger about the way our old boss did it or the way we were passed over that lead to you know theories that we

53:38

that aren't necessarily data driven by the way the they didn't call my group at be Stern's quantitative Equity they called it systematic Equity right just used say yeah right and one of the things that um I have always believed is I'm probably wrong and so when you have that as one of your mental

54:01

models that leads you to seek tons of data right what you you don't I I learned early on I trained myself to be interested in those kind of one in a million stories because they are interesting but I also trained myself to say they're not likely and you know they're possible

54:22

they're not probable and and one of the things that I found and and you know I developed some pretty good stories honestly using data and compelling people to pay more attention to it but I did notice and continue to notice a a Renaissance to go to there's a old Sports thing where the guy says let's go

54:44

to the videotape and and I used to say that all the time and that people would get that but I also find like the whole idea of AB testing I you think think it's one of the most important things in the world to do right I who knows whether this idea that I think is great really is great or

55:06

really is going to attract the right type of audience that I'm interested in and one of the things you call it doubleganger search but nearest neighbors like that's how Facebook makes all of its money people people like well how is Facebook still surviving with you know the at the average age of the user

55:26

there is like 60 Minutes these days how how are they doing it well they're doing it by you know doing a lot of AB testing in silico and talk about that a little bit because that's another love that we share yeah so I mean basically uh what tech companies realize you know the gold standard of seeing whether something

55:50

works we know this in medicine is randomized controll trial so you want to know if a pill works you give uh two groups controll group gets uh placebo group Placebo medicine treatment group gets the medicine then you measure their outcomes and see you know whether the people in treatment

56:09

group less likely to have the disease in however many months and it's kind of a you know standard for doing things standard for you know increasingly the rest other areas of social science have used uh randomized control trials but randomized control Trials take forever they're uh that they're expensive you

56:29

have to raise all this money you have to get people you have ethical issues you have to get them through an IRB board and uh what Silicon Valley companies realize they have all this amazing data you can do a create a randomized control trial with a couple lines of code so you

56:49

can show different groups of uh your users different versions of the website and see which one leads to more clicks more advertising spending more time on your site whatever metric you're interested in and I think Google was one of the first ones that really used this and you know initially they're doing a

57:09

few experiments and they realized you know we could do way more expens we do thousands of experiments I think I you know Facebook does more experiments in a day than the FDA does approves in a in a year it's insane how many experiments these companies use and uh you know all

57:26

the top companies have Ed this you know Netflix has brought data to entertainment which usually was something where people kind of winged it reli on their intuition and uh it it it's really powerful tool for understanding what makes people want to stay at your website you know okay so if

57:46

the button's red versus green you know the red button may lead to an extra 10 minutes a month on your website so you're going to use the red button you know all these different examples it's a little creepy you know one of the reasons that people are so addicted to these sites is that the data analysis is

58:03

so powerful that AB testing is so powerful they've really figured out exactly what maximizes the chances that you do what they want you to do which is spend more time on the site and click on more ads yeah um it but also uh to just make the opposite argument like in all the

58:24

verticals that we have books entertainment podcasts Etc we also look at it as finding the right fit with the audience that wants that kind of content right I hate that word by the way but I've got to figure out a new word for Content because I don't like it uh but like uh with books

58:45

for example um Amazon uh there there's a reason why they're recommending you books that you end up reading and say God I really love this book because they looked at thousands and thousands and thousands of people who read similar books to yours that you liked right and then they recommended to you the ones

59:08

that you haven't read yet and and I definitely think I mean dating for example I'm seeing a lot of new things we're getting pitched on on dating apps that are are only matching People based on their literary interests or their other kind of like Hobbies Etc and the data especially on the literary side uh

59:31

is pretty compelling uh that that people like who love the same authors and love the same genres tend to get along with each other a lot better than than you know like putting together I only read non-fiction about quantum physics with I love romance right might not go together

59:51

well yeah so you know that's another example of scale so one of the examples of scale scale in these big tech companies is they can just do a lot of experiments CU it's very easy to get a group of a thousand people in you know for a treatment control group to do an experiment and another advantage of

1:00:09

scale is they can find someone just like you what I call a doppelganger and you know Amazon or Facebook or Google we think we're so unique but you know they're probably a thousand people who have pretty similar taste to you on many dimensions you know similar political views similar taste in

1:00:28

books similar taste in music and you know they know that everybody with these taste has really liked uh you know this new show then they can inform you about this new show and that is really really powerful and I think is is largely you know I think AP testing I'm not sure it it feels a

1:00:50

little creepy to me and like they may be uh you know getting you to addicted to things and getting you to spend too much time on these websites so I'm the ethical issues around AB testing I'm I I think are real but I think there aren't really necessarily ethical issues about doppelganger searches I think that's

1:01:08

just a cool thing to find you know exactly you know the the the people who are like you and what they like and then you know informing you about that uh and that that is a really powerful tool that all these firms Amazon Netflix Facebook Google they're all using them YouTube I

1:01:25

think that I I agree by the way uh about um doing AB testing just to addict you to a particular type of behavior or time on a site or anything like that but I think that um the idea of nearest neighbors the idea of Doppel gangers right like it's going to unlock so much

1:01:48

knowledge in my opinion in medicine right how cool will it be to have me in silicone and then do all sorts of horrible things to my silicon uh doble ganger that you could not do ethically to me as a human and and find yeah for this particular individual or group of individuals this this composition of the

1:02:13

meds works best Etc I have a um uh a relative who like you highly degreed he's also got an MD but he's got I think he's got a PhD in data science as well and that's what he does he's brought in by big pharmaceutical companies to interrogate the data uh with drugs that failed for whatever reason uh for what

1:02:39

the pharmaceutical company hoped they would be able to be efficacious at and then Grant will go in and say oh well you you've got a drug here that well you wouldn't want to prescribe it broadly its efficacy in post just just postmenopausal women who are slightly overweight not obese is like much much

1:03:03

better do you see those kinds of things getting unlocked across various disciplines and industries for sure but you have to be careful I don't want to get too technical but obviously overfitting is a huge danger here that group is so small yeah using the story to illustrate the idea of the use I

1:03:25

agree yeah yeah you don't want to over you don't want to overfit and find you know the you know thank you founds you have to be very careful with these tiny groups that you know is that a real effect or just noise in the data uh but uh you see so I I say that the first time we really saw the power of

1:03:42

doppelganger search was actually in sports and baseball where one of the ways they figured out is a player still going to be good or are they shot is just compare all the other players who had a similar profile and see what happen happened to them and uh definitely then it kind of came to

1:04:01

recommending uh content with the big companies and I think it is coming more and more to health you know one of the huge frustrations in health is just the data is all over the place there's so much data but you know it it seems clear that you could cure all kinds of diseases at this point uh if someone

1:04:21

just had access to all the different data siloed in different companies and different nonprofits and in file drawers you know doctor notes uh and it's it's it's hard to kind of get a hold of it and get you know to get the the huge data sets that really you know these tools you know AB testing doppelganger

1:04:41

searches they really rely on enormous data sets and uh you know you can't do it with a small sample of you know surveying a thousand patients or anything you're not going to find the middle-aged overweight but not obese post metapa usal woman uh but you are you're going to need you know an

1:05:01

enormous data set to find that yeah and um the obviously the quality of the data as you point out and the availability of the data are critical um and one of the reasons why uh I've invested in a lot of various AI companies and one of the first things that we found was that that issue right

1:05:27

companies were absolutely terrified of letting an large language model behind their firewall because they did not want their data escaping into the wild and and the uh the this seems to be a really praci problem do do you have any ideas for like how that could be solved uh I I don't totally know

1:05:57

it it just strikes me that people are so I there's this example I talk about everybody lies my first book it was all about all the things we could learned people about people from search data and there's this study where they followed people over time who had got diagnosed

1:06:14

with pancreatic cancer their searches over time so that you know and pancreatic cancer that's a disease the earlier you find out about it the more likely you are to survive like it's a not good disease it's very low survival rate but you can double or triple the survival rate if you learn early enough

1:06:31

from you know 5 to 15% and they basically followed people on the internet all their searches what symptoms did they search before they they got diagnosed with panart cancer and they found these really subtle patterns so like indigestion followed by abdominal pain is a risk factor for

1:06:51

eventually getting diagnosed with pancreatic cancer whereas abdominal pain followed by indigestion or abdominal pain by itself or indigestion by itself are not risk factors so it's like the precise order indigestion first then abdominal pain and that's just not known by the medical community nobody has data

1:07:06

sets that big you know with the time series of symptoms and I it's it's kind of like but you know so Microsoft did this study but I don't think Google's done a study like this and it kind of angered me I'm just like we could do this study like I don't see any like I don't see the risk here like you you're not you're

1:07:30

you're not talking about the any individual patient you're just talking about patterns writ large and like I there's a question was this study ethical like is you know was it e is ethical to do this study I think it's unethical not to do the study yeah I'm like it pisses me off that all these

1:07:52

websites have all this information about you know People's Health and that could potentially save lives and they're not doing studies on it so I think one of the biggest things is just you know changing the idea of what's ethical and what's not ethical you know and I think there is a a bit of a

1:08:16

paranoia I I feel around some of these issues you know why why do we care that a big company is mining data people's symptoms to see what predicts disease like isn't that unambiguously a good thing like isn't that what we want them to be doing but you know when they do a something

1:08:33

like that there is kind of a question well you know this is such personal information so I think there's yeah I don't know maybe we're not like explaining the the positive case enough like just how powerful these tools can be and how much good they could do that's my main thought because

1:08:54

I think the data is all there like you know the Google has data on all people's search symptoms and their diseases and you know the you know and they could bu work with other companies who also have data so the data is there it's just matter of explaining you know why we want to analyze and understand some of

1:09:13

this data yeah I I think a a big factor here is cultural leg in other words the time it takes a culture to change and adopt some of the new tools that it it can be uh if you're an entrepreneur it can be a great advantage to you if you do non-consensus things before other people

1:09:37

um but the the go ahead yeah well just one thing I want to say is I don't want to minimize that there are issues with companies knowing so much about us so you know you could imagine course a company knowing that you're at risk of pancreatic cancer well they are they going to charge you higher insurance

1:09:55

rates or you know there are all kinds of you know I understand why we have fears of companies having you know that much knowledge on us sure uh so yeah I I don't want to say it's unambiguously good to for companies to to do this or we don't have to have you know regulations in other areas but it just

1:10:16

does I I sure that there are diseases that could be cured and that and there are you know L many many lives that could be saved sa with existing data that we should be able to study without you know harming anybody's privacy yeah and and I think that that caveat is absolutely correct and true

1:10:39

and I agree with you entirely um I think that it's sometimes used not for that reason though it's sometimes used to because again culturally we feel really weird about uh the things that these these models can unlock and you know what they know about us and everything else so I I get it but

1:11:03

I think bringing out the idea does who who raise your hand if you think it's a bad idea to take all of this data and then possibly come up with a protocol that saves a ton of lives by diagnosing pancreatic Ser uh cancer earlier because you know it leads that way it's it obviously complex issue but

1:11:26

I I think that the but it links to my my next question for you um so I was a Google fanatic right because I'm a research juny and I just love to do that stuff but I have pretty much entirely switched my searches to perplexity and some other large language models do you think that

1:11:48

that is going to I I you know Google was a gravity well right like every everything went in there do you think that it becoming more fragmented uh with less people using Google for all their searches and using large language models like perplexity how does that play out yeah I defin if I were Google I'd be pretty

1:12:16

paranoid uh because I definitely am seeing you know I'm definitely using perplexity large language models uh you know yeah even just chat gbt if I'm looking I was saying I'm going to St Barts and I would used to say you know i' Google things to do in St Barts or and you know now I just go to chat gbt

1:12:37

you know things to do in St BS and I could say well I have seven days and we be good to do on a weekend and plan a whole itinerary and do all these things that you couldn't do with uh Google so I definitely uh you know I'd be worried if I were Google because they can't seem to

1:12:54

be they can't seem to keep up with you know some of these smaller AI companies in the AI products they're offering and uh it does seem like there's a there's a change in in Behavior I think you know generally these changes start with kind of you and I probably are I'm guessing based on everything you

1:13:16

know about me and everything I know about myself that we're the type that is very excited by new ideas and we don't want to do just because we've been using gole Google for 10 years you know I know I guess 25 years at this point we're not like oh well now I I don't want to learn

1:13:33

how to use perplexity or you know as soon as we see perplexity as soon as we see chatu our first instinct is oh this can change everything how can I replace all my you know ways of doing things with these new tools so I don't think it's led to like large scale changes yet where it's really hurting Google's

1:13:50

business but I would imagine it could down the road for sure yeah it's something I'm I've been thinking a lot about recently because of what we've been discussing right Google was kind of the ideal data source for so many things because so many people used it let's use that as a segue uh to talk

1:14:11

about some of the fun stuff a lot of the fun stuff in everybody lies um if you if you could for our listeners and viewers point out and this probably if I'm getting guess right about what what examples you're going to give of what surprise too many people but like this whole idea between uh stated

1:14:36

preferences and revealed preferences right uh often there's a Chasm between your stated pres uh preferences and your revealed preferences and how better to reveal preferences that to look what people search for and and you have a very amusing and and fun look at that the difference between what women search

1:14:59

for and what men search for if you could give our listeners viewers some examples of those oh yeah well there there are lots of different examples so one of my favorite examples I compare uh social media and search so social media is the ultimate stated preference right so when

1:15:17

you're on social media all your friends see it it's very public and search is kind of you know revealed preference nobody's seeing it so I compared how people describe their husbands on the different sources so on social media when they're posting my husband is is my husband is the best the greatest so

1:15:36

acute adorable and then on search when it's private my husband is a jerk cheating on me you know I can't stand my husband he's so annoying and I think that's a great example of the difference you know based on the incentives you give people what kind of data you get and also the difference

1:15:56

in uh you know you what's really going on I think a lot of depression these days and you know particularly among teenagers comes from social media makes it seem like everybody else has such an amazing life and they're always on a great vacation their family life is perfect their relationship is perfect

1:16:17

and I think it's good to keep in mind that that's the stated uh ideas and it's this you know it's not there necessarily the the full picture uh and you see more of the full picture definitely in people's Google searches uh I have long said social media is the ultimate poent good

1:16:38

Village yeah yeah I definitely agree with that other you know sometimes there's a Darkness to what you see in the search data so I think I start the book talking about racism and that's an area where you ask people in a survey are you racist everybody says no no of course but I was shocked by how many people on

1:16:57

Google make really really explicit racist searches and the areas in the country where people are make those racist searches you know African-Americans had worse outcomes they're more likely to be stopped by police worse Health outcomes lower wages uh so you know it's some some of the

1:17:13

things you learn are just are amusing and funny and cute and some of the things can be really dark uh because there is a you know Darkness inside you know in the underbelly of society that isn't always talked about that can be dangerous too you know can lead to riots and uh Wars and all kinds of

1:17:35

problems yeah and uh should not be ignored um and it brings us back to that point right like I I I agree with you on the sensitivity of the data that you have to how do you ethically do that but by the same token if we could use or have an early warning system right on dark things like racism

1:18:05

likelihood of teen suicide uh all those kinds of things like wow that would that would be really hard for me to argue against having that available to people right and and potentially it changes their mind yeah I I think it was a public so a lot of the data I analyze and everybody lies is

1:18:27

Google Treads data and they make all this Anonymous Area data available and I think part of it is PR you know Google's in the news lot although Google doesn't necessarily need PR everyone knows who Google is I think part of it was Google's just such a research you know it's it's an

1:18:45

organization with tons of scientists researchers and I think they did feel like this was somewhat of a public service and could be of use to the research community and you know that there there have been examples of researchers using it to predict flu out breaks or various disease outbreaks uh

1:19:03

and there are so many more things that can be done with this type of data to help impr improve Society in various ways the other thing that we've touched on uh throughout our conversation but we haven't explicitly called out and and that is the the seeming preference of many people for small data sets where

1:19:22

they they themselves have you know like a doctor right I I'm I'm of the 100 patients that I saw you seem to fit into this particular thing and small samples always have the craziest distribution pattern larger samples the classic example is the question you ask people if I told you that uh there were two

1:19:47

hospitals one which was in a city of 8 million people and one was a town of 880,000 and that there nine boys and one girl born in the hospital which hospital do you think it is the big city one or the the one in in the small town most people guess it's in the big city uh

1:20:08

hospital for I think somewhat obvious reason like well there's so much more and blah blah blah but that's wrong of course the the small one is far more likely to have the unusual outcomes what what are some of the other aspects of big day data that make it really unique for doing these kinds of research

1:20:29

studies and and learning so many new things that that people should consider yeah I think a lot of it is slicing and dicing the data set so you know you the example of you know postmenopausal overweight but not obese women you know no doctor who's seen 100 patients is going to be able to pick up

1:20:50

a pattern like that you know a pattern like that needs an enormous data set to be uncovered uh and you know just all kinds of things seeing what's happening in one Town versus another town you know like uh there there there you know uh if you do a survey of 2,000 people you're G to have like 10 people from all of

1:21:13

Connecticut right whereas if you have a enormous data set you're going to have millions of people from Connecticut and you can really see how Connecticut people differ from New Jersey people differ from be and California and you know really seeing zooming in on various patterns I think is one of the biggest

1:21:28

powers of you know end normous data sets yeah I agree are you working on a new book right now uh not I don't know I maybe I'm going to the same Parts trip and if I get inspired I might start one I kind of initially when I wrote who makes the NBA I'm like man I all I want

1:21:47

to do this is the best month my life I just want to sit back and write books like this whole time and you know the problem is I'm just and maybe you can relate to this I'm just such an obsessive person that like when I'm working on a project like I don't know how to work like nine to like six like I work like I

1:22:11

like you know I just I get all consumed and it's 18 hour days and it's just like uh yeah it's just I'm just so obsessed with work so you know I have other things going on in my life you know I have Consulting projects I have a relationship I have I I'm trying to spend more time with my parents in for

1:22:32

various reasons I'm like I really I'm I'm a little wary to just like you know go deep in another project right now but I I I probably will pretty soon so I I have a whole bunch of ideas cooking I definitely hear you on that obsessive uh pull um I very much like you in in that respect

1:22:54

I guess we could say uh gotta always make [ __ ] for infinite books maybe you should spend even more time in St Barts because and then do the work there because just think about the beautiful environment you be able to do do all that and then uh we'll give you a lot of help over at infinite folks if you don't

1:23:13

want to self-publish or we've been talking about maybe working together on something which could be could be fun uh you know we would love to do that because we are very very sympatic the way that we look at the world yeah um well this has been absolutely fascinating Seth um I always ask um my

1:23:33

guest uh at the end of our conversation we're gonna make you the emperor of the world you can't kill anyone you can't put anyone in a re-education camp but what you can do is we're going to hand you a magical microphone and you can speak two things into it that are going

1:23:52

to incept the entire population of the world they're going to wake up whenever their next morning is and say you know what I just had two of the greatest ideas and unlike all the other times I'm gonna actually act on these two ideas starting today what what are you going to incept in the world's

1:24:15

population wait so are they just going to hear the idea decide whether to act or they're going to hear the idea just they're going to think they came up with the idea th Inception okay and then they're gonna act on it okay uh well I guess because it's me it's to like record your data to just get more data

1:24:39

on everything about you so I'm big into the Quantified Self movement and just like stop winging it in your life uh you know read get record all your data record when you feel good when you have high energy uh when you are happy and notice patterns in it you know detect patterns in it what led to that I think

1:25:03

that could be I think a lot of people this is maybe more than one sentence sorry but a lot of a lot of people are basically I think a lot of people just have like very obvious things holding them back that if they just saw the data they wouldn't like you know they they're

1:25:23

they're sleep they're they have a drink before sleeping and it ruins their sleep every time they do that and then they're in a cranky mood and then they're not getting work done and like they've been doing that for three years you know for 30 years and they'd be way better at

1:25:39

functioning on every level if they just didn't do that so really I think recording your data is would be what I'd insist upon okay that just counts as one you got another one another one would be sry to do the obvious things that make you happy that make people happy so you know I I and don't trust your got I

1:26:10

talk about all the research on happiness and the things they find make people happy being with friends being there a beautiful like Lake uh you know Hunting Fishing gardening things that we've kind of done for you know since our hunter gather days and I think a lot of people just don't do

1:26:32

those things and you know you live in a city you work 80 hour weeks and you know you're not seeing your friends very much and you know so I definitely encourage people to read the research on happiness and then if you're not happy just how much of your week are you spending doing the very obvious things that tend to

1:26:53

make people happy like do that ask yourself that question uh every Sunday or something and then reevaluate your next week I would say I love both of those uh data uh can definitely set you free if you use it the right way and ask the right questions it can also enslave you as we talked about uh when when people know so

1:27:15

much about keeping you glued to Doom scrolling or staying on that particular website um that that's not such a great outcome but I I think we shouldn't let it um besmer the absolute power uh Big Data married to AI I just think is going to cause such a revolution and Discovery and bunch of

1:27:43

Innovations and obviously a bunch of bad stuff too so we have to be very careful about that and be always aware of it and not be Pang glossy about our attit ude uh oh this is just going to work out perfectly totally totally agree where can people find you we'll have it in the

1:28:02

show notes but I guess I'm I'm on X although I don't post as much as I probably should Seth sore D on X that's probably the best place Perfect all right my friend this has been great I really appreciate you joining me and uh can't wait to get together and hear about that maybe next book thanks so much Jim