0:00
[Music] we don't need super intelligent AI for work to get disrupted we have a general purpose technology the first in a generation nothing has been adopted this quick Fleet nothing is quite this alien nothing's been this out of control it's
Search this video
0:13
going to take a decade or more to just absorb the impact even without having a single day more advanced technology and more advanced technology is probably covering I anticipate that uh jewels of AI will in fact be used for ill we are very much wanting to have a constructive
0:29
and practical conversation so I think the message I want to get across is agency this is not something being done to us we get to make decisions about what this is and we should actively make those decisions we shouldn't passively step back the only way to prepare for
0:42
the future is to embrace it there is a lot of opportunity now to access resources you couldn't access before no one's guaranteed to you success but there's never been a better time to do that this is a great time to kind of shake off the explorers of habit and
0:56
look around and say what do I want to do differently [Music] well hello everybody it's Jim O'Shaughnessy with yet another infinite Loops my guest today is someone who I absolutely love the work he is doing is Ethan Mullick the associate professor at
1:18
the Wharton School of the University of Pennsylvania he is in charge of researching Innovation and Entrepreneurship all things that I love he also leads the Wharton interactive uh system to democratize education using games simulations and Ai and wait there's more you're also the author of
1:41
several books uh you took your ba from Harvard and your PhD and MBA from the Sloan School of Management at MIT and you co-founded a startup I mean I don't even know where to begin welcome Ethan I'm so happy to be here thanks for having me
1:59
so I love your work primarily because you were one of the first to come out saying you know what I started as an AI skeptic and after I saw it I just thought oh my God this is this is going to change everything um can you tell us that can you give us the origin story
2:22
um of how that happened and and uh sure I've been Ai and Jason for a very long time so I was I was at MIT I worked to work with the media lab um and with um with Marvel Minsky's AI group there um so I've hung around the AI people but I'm not a computer scientist
2:40
um but I just putting a lot of time trying to think about how we democratize access to education so I'd be building games to do that I've been using all the latest tools um and when gpd3 came out in sort of these large language models the initial
2:52
early ones I've had my students play with them to sort of see what the future looked like and had to cheat with gpt3 um and we're kind of in the middle of that assignment when chat came out and it was a pretty fast transition because it was like you know I knew what the
3:06
other systems did and there was such a huge leap in quality between gbt3 and gbt 3.5 that was immediately a parent but by the next class I taught on Tuesday like four days later I introduced it to the class that day just with my own experiments right there's no
3:20
book or anything else by the end of the class one of my students and this is entrepreneurship class for undergrads had demoed a created the demo for their product using Code using libraries they never used before I posted on Twitter that night too BC Scouts contacted him
3:33
by the next day by two days later 60 of my students had used chat for various reasons in their class so like it was immediately apparent as somebody who thinks about sort of Entrepreneurship or teaching they're like we hit something that was kind of light lightning here um
3:46
and then not everyone knew about that uh early on I think people started out but they didn't early on yeah one of the things you've said uh that really resonates with me is the only way to prepare for the future is to embrace it and I love to see you doing
4:02
that in in your class um and one of the other things that you've said that I'd like you to comment a little bit more on is even if AI stops making any progress from this point forward it's still going to account for massive changes talk a little bit about
4:20
that so I think when we think about Ai and where we are in the future people tend to think about either things staying static which I think is impressive enough we'll talk about that or they had to think about sort of this third case of AI becomes exponentially better every
4:38
day and um then we have you know our real general intelligence and we all get murdered raps right this Extinction event and I think that's worth worrying about right obviously right but it's also a distraction for what I think the most likely case is for AI which is it
4:53
keeps getting better where it is now right we don't need super intelligent AI for work to get disrupted we don't need Super diligent AI for entrepreneurship to get disrupted like the tools we have now are so profoundly transformative
5:05
it's going to take a decade or more just absorb the impact even without a single day more advanced technology and more advanced technology is probably coming yeah and you touch on something there that it's like near and dear to my heart which is the the goal posts have been
5:20
set so wide I think in my opinion on the one hand we have the the people writing like what when I first saw like yudowski's thing in time um you know calling for the bombing of GPU plants I thought okay so wow but then there's also the other side right
5:42
which is like oh AGI we're gonna have that next month and I think that both of those those Polar Opposites are the least likely possibilities I agree with you that we absolutely have to have a discussion about this powerful new technology because it it really is
6:02
powerful but it's a it's a tool and tools can be used for good or ill I anticipate that uh the tools of AI will in fact be used for ill so those of us involved in in the industry I'm uh an investor in and chairman of stability AI which is an open source AI company
6:23
um and and we are uh very much wanting to have a constructive and practical conversation uh with with every which which everyone we hopefully will be able to take part in we're thinking of like a moderated Wiki for example um and so I very much uh uh am in favor of
6:46
that what do you think about people staking out like these really kind of extreme claims on both sides I'm not just talking about the the doomers I'm talking about the you know AI Utopias too so I mean look I I have been AI for a while you know AI related for a while
7:04
including you know remember Minsky frustrating person one of the founding fathers of the field that's what we'll answer the key AI people and you know it it is very clear that like you know like this is unexpected that we're getting this kind of gain right so certainly I
7:19
used to listen to people like Ray kurzwell and be like this seems a little over the top given what I know certainly my priors have changed as of November right it no longer seems impossible that we have AGI but I have no insight about whether that's possible or not and I do
7:31
worry about living and absolutely when I talk to people the government I say look I want you to have a plan for what happens if this is the case I think we absolutely need to be worried about this I think we should think about regulation
7:41
but I also worry that ends up being a distraction both the Doomer and the you know and the Optimus view from what is it about to happen we have a general purpose technology the first in a generation more than that like you know computers were the general purpose
7:54
technology maybe the internet was a general purpose technology uh they were talking about electricity in Steam but nothing has been adopted this quickly nothing is quite this alien nothing's been this out of control of organizations and companies and in the
8:07
hands of individuals before it turns out everything we do and as you said it weighs good and Ill if and you know so I think we need to worry about this future and I'm really glad there's people worrying about this writing letters and
8:17
AI possess I don't I'm not an expert now I don't have that much to add to that debate other than I'm glad we should be concerned but even if there's a complete halt tomorrow to building a I don't think anyone thinks GPT 4 is what they're thinking about halting right or
8:29
like that's some future Ai and the truth is unless we went backwards you know six months or a year we're already in for disruption and it's going to be different in every field and you know is it everything from you know to fundamentally changing jobs to
8:44
fundamentally changing the kind of information we can trust uh there's a lot of a lot of change right general purpose Technologies touch everything and there's no outside expert who can tell you how it's going to affect your industry at this stage you have to work
8:56
you know it's there's nobody to do it I think that also is baffling to people they sort of think like there's an instruction manual there's not an instruction manual you can't go open AI doesn't have document the documentation is their API is document kind of there's
9:07
no other piece there right nobody can tell you what stability is going to be used for right what Stables feel like we're going to use this all the time like you know nobody knows the answer because the general purpose technology so I think that we have to kind of
9:19
grapple with that piece much more urgently than we are yeah I I agree and so I support for example one thing that we're trying to put together at so the AI would be a a big prize for a deep fake hackathon uh where the winning software would uh identify deep fakes at least
9:42
in existence then but as you know for the next month right exactly because the the generative adversarial uh AIS will go back and forth once that's out they'll start uh you know gaming it but I think that just showing the intentionality and and continually
10:01
updating that that software and and our goal of course would be if we do come up with something that works at least for the prior cases right we don't have any Illusions um but that would at least be something that we would give away for free
10:18
um we would make sure that everybody could download it Etc and then we would work on keeping that updated but sort of shift gears you've also pointed out that you know the there's going to be an explosion of minimally viable products right with experimentation available
10:36
using AI very cheap very easy and and tell me a bit about some of the use cases that you've seen from your classroom and and others that if you're like wow I never thought about that I mean so I am seeing use cases ever I mean certainly if they use it as a tool
10:55
to help you do stuff right so you can interview the AI and it gives you reasonably good answers as a starting place for good customer interviews right it actually answers survey questions accurately According to some of the most recent research that came out like
11:05
there's a whole bunch of tool uses like my students now I used to have them for their assignment right for you know and we have a very successful class at work and I teach it along with other professors I'd love to give myself a full credit for but it's it's a good
11:17
course with Incredible students helping many people but you know people have raised probably 2 billion in VC ahead of that class right you know report came out of that class at about which is by Google came out of class Bureau came out
11:27
like there's a lot of startups that come out of their class but you know it was a reasonably you know you put together an outline at the end you have a pitch deck now I'm requiring students to do like infinitely more work I actually sell them they have to do at least one
11:39
impossible thing thanks to AI so they can't code I need working software if they don't know how to do design work I need a website that's fully operational every one of the projects they turn in has their criticism by at least four famous entrepreneurs
11:52
um that they have to invoke and have them criticize their business plan piece so I think one piece is the like co-founder side note kind of p is a huge deal right helps you overcome inertia and deal with them and then on the idea to generation you know we have this
12:04
interesting problem which is AI is pretty lousy software right now like it doesn't work well with software it doesn't do things software does you get like I'm building a bunch of simulation software using Ai and it is just wild because sometimes I refuse to
12:18
answer sometimes it gets mad at you sometimes it gets offended sometimes it's really insightful sometimes it isn't like that does not how software works right software doesn't sometimes insults your customers and then sometimes solve their tax problems AI
12:30
could do that so thinking about like software is a problem too it's best to almost think about like people it's not a person but think of as a person somebody you bring in so unless I'm seeing lots of startup ideas around here I wonder if anything are going to matter
12:42
because it's ultimately going to be a problem that you're gonna give one of the foundational models or what you know and make things happen that way so I'm viewing the people side is more important than the and that's that's what's really enable it right and also
12:53
by the way now your competition just increased everybody in you know in um you know uh in picker you know pick a state you know Congo or whatever is now speaking um fluid full English of you know American English could code like there's
13:09
179 countries have access to Bing or gpd4 you can't get a better model outside of being inside one of the companies that's building these things that's insane yeah and you had a great quote which I loved which is kind of think about a AI as uh this like infinitely talented uh
13:28
uh graduate student who likes to lie sometimes but really wants to keep you happy I thought that was such a great quote because it's really true um it's not and you had another observation that I'd like you to to elaborate on a little bit AI is not a
13:46
search a bot right it don't use it at to search use it as a generative tool use it for a variety of reasons but if you think you're gonna like go to chat GPT or gpt4 and say you know uh you know what's the capital of fill in Black it'll give it to you but that's the
14:06
wrong way to use it right yeah and I think the chatbot sort of breaks the model a bit which is quite interesting I just got access to Google's um you know generative tool that lets you read document this coming to our office and every something else too
14:19
and you can just see the difference who is going to start from a blank page anymore and write down you know they're like they're not going to do that people are going to instead be you know start with a request to the AI to make something happen that's going to change
14:33
how all kinds of communications happen right so I think this idea that you're you you know moving away the search engine piece it's bad AI demo is very badly because the three things or the four things people do is the first thing you do is do a search it doesn't do a
14:47
good search or maybe it lies the second thing they do is they almost always ask it a personality basic question for the future of AI what do you think the future of AI is going to be and if there's one thing that every AI company hasn't thought about is how do I put
14:59
that on enough guard rails so it just answers the blame as possible that humans are also very important like it's not it's just you're bored out your mind right then you might ask it for you know a poem and you're like this is kind of
15:11
neat and most people end up poetry then sometimes you ask it tell me about myself and it lies to you about things and you're like this is stupid and you walk away right so people walk away all the time or else they actually do start to engage with it and then they
15:22
encounter the inevitable existential Terror that comes from realizing this thing is real and then you put it down I've talked to a lot of really smart people they're like it's just starting to freak me out so I walked away so it's a really hard system because you have to
15:33
be avoid being you know being fully into existential angst but also get past that additional it's hostile to use kind of experience to get to see what's in the middle yeah and that is so true and I remember the first time I was playing around with uh the earlier models of AI
15:50
and I'm like I this is this this is really interesting but it doesn't seem to work the way I think it should work I think that we'll probably end up spending a lot of time on as people develop uh uh more stable systems more stable use cases
16:10
that that will you know as you have said it gives us a larger context window into human knowledge which is one of the things that really excites me the most um I I uh follow Brian romelli who's doing these incredible long prompts I don't know whether you know his work
16:29
yeah absolutely yeah yeah yeah and and you've done a lot of work on it as well um do you think that that's kind of a temporary situation do you think that the large language yeah I agree I mean prompt crafting is I keep getting closer
16:47
but like teach me programs like you know it feels like it's like almost an old school lesson just use it that's 80 of the way there I mean look there's property techniques that could do stuff but also they're inconsistent right we are you know um I work with my wife who
17:01
is a doctor in education we both do you know a lot of our papers together she's very good at creating these educational prompts that are general purpose but they still don't always work right and the model updates sometimes and you can do these very elaborate and there's
17:12
there are techniques right you can do shade of thought prob they can multi-step product but you have to embrace the fact that this is weird it's like working with a person so you give the very long instruction set sometimes they'll remember the whole thing
17:22
sometimes they won't sometimes it'll be let us straight it's very hard to use like that as a technique and the truth is is that hey open AI has mentioned this and I've seen the same thing by the way mid-journey and you know and increasingly I'm sure with stability as
17:35
well but like it isn't that hard anymore it used to be to get a good image out of mid-journey I had to write a paragraph you know micro Lids far outrage you know throwing red in words AK HD our fungal art station like it was a magic spell
17:48
and now I'm like like show me a picture of you know of an otter on an airplane and it just does it right and it's the same way with gpp4 like I want to write a book okay to help tell me how I should be getting help from you to do that it starts doing it so prop to crafting is a
18:02
very temporary stage of Affairs to the extent that it ends up mattering it's gonna be you know as we start to discover fundamentally how to do better prompts those will be built into the systems ultimately and not ultimately like in the short term so I think this
18:15
idea that uh crafting is the next thing that you should pay millions of dollars for you know a hundred thousand dollars for someone to do you know to get you proper crafting or you know half a million crafting exercise you know for uh someone to hire that feels weird to
18:27
me there's obviously a set of skills in working with you and kind of whispering to it right now but it's not prompt crafting it's honestly 10 hours of use like that's my my threshold I tell people use it for 10 hours you'll figure it out yeah I agree and uh
18:41
experimentation uh with it seems to be the the best way to get the best results uh you and I I think are aligned on the idea that I I personally think that the cenotar model uh that was introduced by Gary Kasparov when he lost to Deep Blue
18:57
uh and then he decided well why don't we put together a team uh of humans plus AI uh I I certainly think that that's kind of at least for the foreseeable future let's say next five months I I I think I think I I mean I think you're both talking about exactly the
19:17
right thing and exactly the worry right which is like if I mean in some ways if AI stop development right now we're trying to be okay because it's sort of in the 50th to 90th percentile on most tasks so everybody certainly has at least some
19:29
set of skills they're better than the AI act right and it definitely needs Direction and there's all kinds of issues right so in this time there's no doubt and by the way when you use AI more like you're saying experimentation when I write using AI I am never having
19:41
the AI write entire paragraphs what I'm doing is on that said did site get stuck on I'm like give me 20 versions of this I'm like I can't quite make this work make this better I want to throw in stuff about this five difficult you just summarize it and then I'll work from
19:52
that piece it's all my voice right but I'm using the AI sort of collectively to do work and some things you delegate more to the AI right it's like right this you know Grant form which is like oh you know just finish this thing No One's Gonna read it and do this material
20:06
so I think that that centroid model is clearly the key in the short term that makes AI incredibly idiosyncratically incredibly individual though right it means that like my version of AI is different than someone else's it's tied to the person and I think you're also
20:19
right we kind of worried like okay this is the five month model right and you know hopefully Edward you know what happens with GPT 4.5 what happens to gb5 what happens these models get better fine-tuned I think those are bigger questions
20:33
yeah I agree and I I think one of the uh other things that you highlight often is your work is something that I have found and and that is I look at it kind of as a co-creator as a collaborator um and and the idea of I'm writing something and I get stuck right and I
20:54
love your idea just give me give me 20 or 30 sentences here and then you find one that that works for you I personally think the use case of like having having multimodal Ai and for example feeding your favorite universes into it so I I'm
21:12
a big toll team fan and so I see it in Lord of the Rings and I love Dune and I feed that in and then I ask the AI I don't want a universe or a world built in the style of Tolkien or toon or whatever but I want you to find all the way they agree find all the way that
21:31
they disagree and then kind of give me a new world that I can iterate off of do you see that as like one of the use cases yeah I mean I think creativity in general is kind of underrated and by the way I mean what AIS is a connection machine and the people I know who are
21:45
the scariest users of the system and I know a few people like this don't even bother dropping them back into our real world right so what they would say is compare the vector space of Tolkien to the vector space of a raucous and then find three points of agreement four
21:59
three points of disagreement then let's manipulate those then finally tell me what this means in in human token language rather than uh you know in in and so people are doing this have been searching with all sorts of things so I think that there is this sort of
22:11
secondary model but I absolutely agree with you I think part of this is about connections between things creativity spitting off ideas I mean one of my favorite little demos is pick a random patent number a you know a uh like a favorite food and a obscure you know
22:26
sociological Theory from the 1800s and asked the AI find business ideas based on all three and it does a pretty good job of that I love that I'm gonna steal that from you so thank you I I haven't done that one yet um one of the other things that I think
22:45
that I'm really excited about and that you are obviously putting into practice already and that's this idea of the the AI and education Revolution um I I really resonate with your message which is a one-to-one tutoring is going to be great um we're going to be able to upscale a
23:04
lot of people but that does not mean that the classroom environment is going to be replaced I think rather that it means that it's going to be wonderfully augmented and that you and brilliant people like you are going to be able to use it to create a different dynamic in
23:22
the classroom where it's collaborative where it's you're synthesizing knowledge how how have your students been responding because you're like you're on the front lines there I mean you know I'd love to give myself a lot of credit I mean people are just
23:35
interested in using the AI they feel empowered right getting huge CG scores everything but I think that you're right also just about the model like education was the first Model threatened by AI right because we I mean look we rely on
23:47
essays for lots of things just to pick the most simple example right essays are a chance for people to think to demonstrate their knowledge to integrate knowledge was it great at all these things no but it was good at these things it AI just completely slashed the
23:59
heart out of essays right in in a lot of different ways I mean obviously cheating is one way but that was always happening but also you know what does it mean to collaborate with the AI where you know how are we what are we smarter and Dumber about all those questions become
24:10
really real but education will also adapt because there is Need for classroom you know there is Need for application even the best one-on-one tutors there's always this really one-on-one tutor to replace everything it won't AI is still going to kind of
24:21
mess that up so but it does let us do something we knew for a long time batters and we teach in entrepreneurship classes all the time but anyway which is the idea of the flipped classroom right the idea that we are um that we are going to do the boring lecture based
24:35
stuff outside of class and in class is all about applications and the evidence shows that this is better if the in-class materials is highly active and if the outlet Active Learning of the outside of class material is high quality those were both problems before
24:49
high quality outside of class material you watch a video of me talking or something that's not great but interacting with you know you know come Mingo or any of these other you know AI tutors you start to realize really quickly oh this is actually really
25:01
powerful and this has potential to be that sort of loom to Sigma T but you still want to apply something Cloud so AI can actually help you come up with in-class exercises and activities to do with students to apply knowledge so I think classrooms will change I think
25:15
it's going to be like and of course they'll still be English composition where you're going to be expected to in class use a computer not connect to the internet and learn how to write like we solve this problem in math it's not going to break everything right I have a
25:26
big question what we're training people for post AI but I'm not that worried the classes will not get better and adjust as a result yeah I I I'm 63 so I came of age when calculators were a big deal and like I I wanted a calculator more than anything
25:43
so I had a like I think it was my 12th birthday or whatever and my parents were like what do you want I was like I really want this is 1972. I really want a calculator and I remember my mom and dad actually talked to my teachers at in
25:58
grade school and they're like oh we don't know we don't I mean that that's cheating right and and so I think that you're bang on with your observation that that this isn't that's a great example right where that didn't stop people in fact it did the opposite it it
26:18
freed them uh or being able to better understand math in many cases uh and and I see kind of AI doing the same thing yeah you've written extensively about this so let me ask you to kind of play futurist for a minute and and say like and and I won't put a timer because
26:38
we'll be woefully wrong but like what is what is a kind of fully enabled classroom with the AI look like to you so I mean I I think that first of all it's going to vary depending on location I think AI is going to work a lot like sort of cell phones did right so there
26:55
are all these countries throughout the world that couldn't invest in landline phones and it was causing a huge problem but then cell phone service came along and then you know Kenya and the Philippines all have better sell have better cell phone service in the US for
27:06
a very long time right and leave frog an entire generation of technology so I think in much of the world we're going to see a real transformation of teaching where teaching is not high quality right now and we're going to see a leap into another model but in in the U.S in
27:19
Europe um probably in China other you know Japan other places with with uh that already a fairly highly developed Educational Systems uh I think the future looks like one where um you're going to still go to school as you did before during the same class
27:34
Prairie times but you're going to be basically the sort of easiest version to think about is you're gonna be doing a much more engaging version of homework in school uh project-based learning experiences building stuff trying to create things trying to apply things and
27:48
outside of School your goal your homework assignments are going to be monitored by Ai and you'll be working with an AI tutor that will generate lessons for you and teach you Concepts and make sure you're on track it will let the teacher know the person's
28:02
struggling at this despite this and will help the teacher set up teams so that you don't have this portable team assignments and track grades so I think that there's a model that we already know works like we still give lectures it's like 5 000 years old to give me
28:14
lectures and we know they don't work but we keep doing that right so it's going to force us to shake ourselves out of a bunch of stuff that doesn't work well and I'm sure there'll still be some classes that look like classes right I think English composition will still
28:25
look like English composition I think math is still going to look a lot like math but I think a lot of your other projects will be more and I also think there's going to be more disparity in you know possibly more individualness in how we get you know rated in advance
28:37
right so if you're good at something AI will not arbitrarily stop you this is going to be an issue with work too right like we have a whole bunch of systems designed to keep people in sync what happens if we aren't being sick anymore so you know your kid is amazing in
28:49
history and they go deeper than anyone else are they going to be getting the same you know you have a possibility that being dynamically assigned to work with groups that are not all in the same grade level or the same school so that's
29:01
that difference between you know Finding people's giftedness and being able to let them run as far as they want with it rather than arbitrarily you need to wait for the lecture is a really exciting option too so I think how much we Embrace this is going to matter and I
29:13
think we're going to see a lot of differences up here pretty quickly yeah and another point you make that I uh agree with entirely uh is that you know the my great excitement about AI is the ability to unlock creativity um and uh I know that you've written
29:33
about that a lot in fact you you mentioned something uh in one of your pieces that really struck me and it was like a third of Americans had ideas for startups right and and they just didn't do anything with it because they didn't kind of know where to start do you see
29:50
the uh startup activity Etc accelerating because of AI absolutely I mean one of the things I'm most excited about is I mean there's a couple things here one is you know um opportunity is much less evenly is distributed across the world than Talent
30:04
right and we need to look no further than you know Venture Capital as an example right so um you know my city of Philly uh has raised more money last year in BC than all of Japan did right Penn Brads grads from University of Pennsylvania raised
30:18
more money last year and then everybody in France and Germany put together and it's challenges my students are and our alumni are and they are and um they I think those two Nations have a pretty good track record of coming up with animations right so that's just an
30:31
example of you know and similarly little bits of Entrepreneurship education make a big difference even a three-week course in a Patrol study Uganda found huge effects on you know on individual visual long-term outcomes by just learning entrepreneurship a little bit
30:44
so there's a there's sort of an educational and opportunity but there's also this ability to have a co-founder right so people get stuck all kinds of places along the way and you know I have some research suggesting that co-founders actually don't help company
30:56
survival because you know they fight to their co-founder the reason why people always push for co-founders is you have a partner there who's always with you who could support you who could fill gaps that you don't have who keeps pushing you forward
31:07
I wonder how much AI could go some of those roles right like I don't want to write this letter write this letter for me you know to him like the stuff that people little things hold people back all the time right I don't even know how to start you know doing market research
31:20
AI how do I do market research I've assessed a lot of that from an entrepreneurship perspective Urban entrepreneur myself kind of perspective and like when I say it's as great as the best advice we give at top classrooms no when I say it's better than anything you
31:33
could probably read in a book and 80 of the way they're invited to the way definitely if you follow that advice it would help you on the right path so I do think as an enabler for lowering friction for Innovation it's huge yeah I agree uh and to come back to your
31:49
point about the leap frogging you know I use that example about the us having a great landline telephone system which is why everyone leapfrogged Us in in cellular technology and and we're already starting to see um some like for example Japan's most
32:06
recent announcement about copyright and the that at least in Japan uh you're gonna be able to train Ai and and not go follow of copyrights do you see kind of this jockeying for position at the kind of nation state level do you think we're going to see a lot more of that and are
32:28
we going to see like emerging economy say hell yeah we're gonna go all in on AI I mean so I think so there's there's like that's a really complex question right because it's a whole bunch of Dynamics like one Dynamic like it is definitely a competitive booster so if
32:43
you can figure out how to use AI in your organization and you know and where you you're in the world you're just gaining competitive Advantage I keep trying to tell CEOs of U.S companies like you're this this did a lot to competition the
32:53
systems that were great to get you here aren't as useful anymore in preventing competition right your startup competitions they have a much more widespread we were depending on language barriers coding barriers knowledge barriers it's things that were stopping
33:04
people those won't have the same effects right so I do think Nations will start doing breakfast this for but it's a general purpose technology embrace it for what so things you could obviously do is streamline some aspects of you know permitting and others and talk to
33:16
people thinking about those issues you could obviously do things where you are going to approach um you know you're going to think about using this to streamline company operations the increase coding speed education we talked about so I think we're going to
33:29
see some of that I you know I think that there's a broader worry about AI jockeying too which is nobody thought large language models were going to be that interesting until November when shot GPD came out right and I think even in opening hours got to take it by
33:40
surprise and I've been playing with you know a lot of the the sort of uh AI image tools they were very cool but still kind of primitive right it was it was the last six months have really seen things changed dramatically so if there aren't at least a dozen nation states
33:53
that are spitting up their own are on guard rails large language models right now or national language models I would be shocked right so the world's about to change it again in the next few months as those things start to come online
34:03
either secretly or not um and so the national competition is not always going to be benign yeah and and that's the challenge right because uh that's underlined it's not always going to be benign and so um you know I I have thought you know
34:21
just you mentioning that just you Penn graduates I mean that is an astounding statistic that they were able to raise more than countries and like it seems to me that one of our advantages here rule of law obviously uh freedom of speech Etc do you see like as AI takes
34:43
um a more important role in emerging economies do you see that like maybe affecting the idea that they wow we really do need a rule of law we really do need this or or no and is that a bridge too far I don't know I mean I I think that the you know the
35:01
destabilization dangers are so great that I don't know which which way things are going to push right like I mean you know I keep trying to tell people people and here's these fun posts where I try and you know show pick it was like we shouldn't believe anything we see online
35:13
anymore like this is it's it's you know and I appreciate what is trying to do with cardboard but like it doesn't matter it's whether it's it's clear that you don't need a giant team for good image generation right because that as opposed to a foundational model would
35:25
cost you a billion dollars there's 20 teams coming up with really good stuff right um and you know and like you know I put my own voice I put my own image like and this is well within the capability of a small team of people doing stuff and then we know that large language models
35:40
can do phishing attacks at scale right and they like so I mean I think that we're gonna have these countervailing factors I I mean I actually think from 100 years from now history books assuming we make it that long which I'm very optimistic we will
35:52
um I think the period from like basically 2008 until whenever 2032 we'll look at one period of transition right we're starting with the iPhone internet and sort of social media released through large like we have we have not coped with the world we're creating and
36:08
I think there's counter growing forces of authoritarianism and liberalism that both swirl around in the world we're creating I don't know where we end up with that yeah um I as a big fan of the rule of law and freedom of speech I'm I'm hoping
36:23
I I am too I I mean I think so I think that just our tools are inadequate to our approach I mean I we absolutely need you know rule of law is absolutely necessary right you know as his freedom of speech but it starts to become a really big question like do AIS have the
36:36
freedom of speech do you have the ability to have you know if if the freedom of speech always depends on a Marketplace of ideas or where ideas are competing with each other when you know when you have high level manipulation you know nothing you should reduce the
36:47
freedom of speech but then we have to think differently about how we you know what how do we privilege sort of can I just feature over others or what kinds of conversations they're having there's just a lot of open questions and our pools were all really good until about
36:58
2008 for these things like we had a very clear model for how you built a successful you know Democratic growing regime and I think that's an open question now like manipulation at scale and I think you know maybe the centralized platforms solve it I mean
37:12
look China's solving its own way there's a whole nice economics paper about you know is very easy like these tools become very easy to shift over to government tools um I don't know I think we're really in for a period of change yeah I that that is the one constant as
37:31
good old Eric clitus they preach Socratic uh but it was slow then you had I kind of wait a few hundred years for bronze like it was that was like that was easy right like you know I mean change really I mean if you look at and like we have been in a singularity since
37:46
like 1890 right like that's when everything sort of accelerated and you know you're trying to keep it up but I think that the pace just went up faster and I also think that like look even if they're not sentient AI is an alien entity it's not doesn't it's neither
38:02
things nor does it not think right like our categories don't like that's a pretty fundamental challenge it's both creative and not creative we both know exactly how it operates and can't understand how it works that's a pretty new category for a thing that everyone
38:14
has access to yeah and I I completely agree in terms of you know we've got brand new questions that we hadn't even contemplated and we don't have the structures yet or or uh infrastructure right Mark you've mentioned Marketplace of ideas we that's vital and so I I
38:35
couldn't agree more in terms of we're facing a bunch of new questions that we we really are going to have to begin to to start grappling with another thing you've pointed out that I I really uh enjoy uh is this idea that people with a
38:50
broad knowledge of maybe an Arcane field like they they might have huge opportunities at least for now uh with uh interacting and iterating with uh AI give me an example uh of like what you would see as like a Specialized or Arcane field where somebody with deep
39:11
domain knowledge now uh with the AI is going to be able to really come up with some really cool stuff I mean let's start with image generation sort of the space that you're in right so I mean the ai's absorbed all of this art from all of history and what do people end up
39:26
doing they know like the same four artists all the time right like do a Monet do it like but if you know about batik painting and you know about you know German expressionism and you know about you know the 14th century Flemish art schools you can produce results that
39:40
nobody else sees right that don't look samey in the same way if you're writing essays and you know the style of like let's do this in the style of you know uh you know uh you know John McPhee or whatever like you're gonna get a very different result because it's absorbed
39:52
everything and what you're trying to do is move move the context that the AI is looking at the right part of sort of related space to get a unique answer and the best way to do that is to know the space that I can explore notice about
40:02
human history as possible like for example you know like not only does um you know I tried to get the AI we're talking about token earlier and you know how to translate from Sumerian to one of the eldrich languages and if you know the various Elvis languages you get
40:15
different and you know it has trouble with uh with speech and Mordor with black speech because it says it's not there's not enough written of it but I'll extrapolate for you as best I can um so I mean the more you know about that the more you can move the window to
40:27
something interesting also the more expertise you have to be able to call BS when the thing hallucinates so you know there is expertise really doesn't matter more at least in the short term yeah and uh I I'm I've collected art for a long time and love art and uh when I
40:43
was playing with Dolly too um like people would say why did you do that right and yeah the reason I'm thinking about it is I used German expressionism uh and and you know in the style of Monk and and whatnot and everyone's like how are you getting all
40:59
of these results why doesn't that look like exactly why does it look like the 16th because again most people are doing very soon you think about this and this is where I I push back you know people talk about like the AI producers or the
41:11
average you know of what people would do and it's sort of true but sort of not true just the other default side right the average writing is pretty bad and so there's still a lot of open question and I've been posing and never gonna fully satisfying answer there's theories about
41:25
it but like why is AI writing so good when the average of writing on the internet that I read is pretty lousy right and so that's pretty insane right it doesn't write read like a Wikipedia article it doesn't read like a forum post but in the same way it gets even
41:39
less like an or like that if you push into the set of space that you want it's why one of the few pieces of prompting advice that I definitely get people is give this a context tell yeah who it is you're not doing that to make the AI into that person for real it doesn't
41:52
turn into one buff it but it does put it into the space where it's knows what the audience is and is speaking in its style that you don't get elsewhere yeah and one of the things that I'm really excited about is uh the the ability to look into liminal
42:09
spaces that we simply don't have as humans right the ability to kind of like analyze quickly hundreds of millions of vectors and for example one of our projects is we're going to feed every Britain uh story that we have access to uh all the way back to
42:28
um you know Sanskrit Etc and and and say okay tell us about ourselves what other kind of projects like that are are you thinking about in terms of what we can do going forward I mean so that's actually something I've been playing with a bit uh as well there are
42:46
these well-known indices of fairy tales that have been kind of created based on various arcs right so 3 to 43 of the I don't know something Index right 343a is like you know uh the princess eaten by a bear with the teacups or whatever and you know crosses the culture there's a
43:02
really nice econ story showing that the kind of Stories We Tell shape the kind ends of uh you know how we approach entrepreneurship stories where tricksters are punished with exercise where that's common there's less entrepreneurship so I think fundamental
43:15
really interesting insights are Humanity providing connections is amazing right another thing I did just for fun was I fed into um CPT with code interpreter there's a famous passage from The Iliad which is the catalog of ships which is just this list of all the locations and I said
43:32
just map these for me right and you know there's been some very big project UVA and stuff to do this and well you know obviously you know it actually was able to create an interactive map with me for the first you know 50 or 60 ship locations and find modern equivalents
43:47
I'm sure there were errors in it but the idea that as a scholar I could get that kind of progress I mean I you know I fed up my own books and I've been asking questions of it like you this idea of working with the sort of mass knowledge
43:58
of humanity and discovering new things I think running for a golden era of sort of you know literary social science studies and the other thing is really cool of this is it takes some of the like there has been a long problem you
44:10
know some people probably advantage in Academia where I'm an academic where everything's become Quantified right I do quantitative research myself but like everything has become very quantitative which you know there's a lot of people who don't do project research
44:22
qualitative research who have been kind of left out of this like well we need precise effects and we can argue whether economics actually does what it's supposed to do with the 10 or whatever else but what's cool about this is you don't need to be technical anymore you
44:32
can literally ask questions with this like give me you know I was shocked I was able to ask the AI tell me show me five metaphors for my book because metaphor something is really hard to nail down right because it doesn't use Leica you know as there's no clear
44:45
indicators not a simile it was able to find metaphors like there's something very deep about its ability to understand human language that will let us do some pretty extraordinary things yeah um and I I am doing a similar project I uh was a m a journal keeper I've kept
45:03
journals for 42 years and like we're going to transcribe them all because they're all hand written and uh we'll feed him in and my joke is I I want the AI to explain myself to me absolutely I mean I I've spoken to somebody who's already doing this who is
45:22
um you know and and who you know deep into AI stuff who basically they send you know they're sent they have to be you know GBP 3.5 train up or they train up their own system and uh great team bettings and their own work and they literally send the models to each other
45:36
to look for points of agreement and disagreement in in that sort of you know Vector space so I think there's really interesting things we're going to learn about people as a result you know I think that there's you know there's this
45:46
sort of transhumanist Utopia dream of like oh we'll live together and through the a you know forever through a you know AI uploading our brains you know I certainly don't have a strong thought that that is absolutely true but there is something very interesting about
45:57
being able to query your own knowledge or somebody else's knowledge in a deeper way than we ever able to before yeah and I I also love the idea of uh I've also written four books and and I would love to beat all of those in either all Market oriented uh investing
46:15
books uh but uh and then contrast and compare right like what what did I get wrong uh what can I learn from you know people who are non-quots I think there's a ton of information there that you know would have just seemed too arduous uh in a non-ai environment that now is just
46:36
like it's like I feel like a kid in a candy store and and you must as well given the amount of research done yeah and the transition I mean it's fascinating right like you start to relate to books very differently and source of knowledge very differently
46:48
right and you know and also the way people can relate to my books different like I I've now been doing things like pasting in a chapter in my book and asking some questions to turn them into exercises you know read this chapter on how to generate ideas and then
47:01
um walk someone through the process of doing it I send people a link now to my idea you know they could just use my book to generate ideas um it's pretty cool yeah which I and that kind of leads me to another thing you've observed that I agree with in terms of early adopters
47:19
are going to gain some pretty radical advantages to to people who you know the the the later adopter tends to want something that's shrink wrapped easy to use you know kind of like the iPhone I guess right you can take these out of the box and you can use them immediately
47:37
and pretty much you could when they launched it too um well what do you see happening with those later adopters do you think that um you know we're gonna see a series of I guess what we call products um or or or no so I mean I think that we're gonna see
47:58
that we're speed running adoption curves right partially because you know look you think about the fastest adopting technologies that are always you know they're like you know one of the categories has always been letting students cheat on tests is always like
48:12
if there's a way of doing that students will figure out a way to do it right um and so you know I think in the same way the productivity gains are so potentially large that potentially the LeapFrogs are normal problems right so and again because this is being
48:26
integrated into word and Google Docs and you're gonna have that little blanking cursor saying what do you want to write about I think we're going to see a speed run that we don't usually see in these cases that doesn't mean people will be
48:37
using their full capabilities right mostly with satisfied saying doing just enough with it but um you know I think we're gonna see it like integrated into products in that fairly deep way uh it also makes sense that like you know look I can't wait to have a fully working you
48:51
know true AI assistant there's nothing more frustrating now than going back to Siri after using Bing right where it's like you know I could say to being like hey I want tacos tonight I don't want to go I'm going to be tired so I want to go
49:02
more than a couple hours it can't be that crowded and I want to make sure that there's vegetarian options available and making something that's kind of fun you know a fun environment maybe and like tell me you know and it will find something right if I didn't
49:13
tell that to Siri it'll be like you said cheese and you know like it's it's useless right so I do think that there's going to be this sort of ubiquitous assistant thing that's going to be really interesting too so I don't know like because this was chat first
49:29
like when I'm trying to create applications with the API I kind of end up feeling like I should just take this away go back to chat because it feels like the much more you know or at the very least or the API playground right like what am I bothering with this other
49:41
methods because just drink right from the fire hose yeah um and and that kind of leads to the idea about you know curiosity I keep coming back to the idea that you know it's really curious people are the ones who really are just all in kind of on AI
50:00
like you and and I just wonder can you I can you teach curiosity so it so there's a specific part of concept actually called a specific curiosity that is that idea that like it's always been tied with actually coming up with ideas right so support curiosity is not always the same
50:19
creativity but it ends up being creative because the active searching for answers right if you're the kind of person who can't let you know can't let something go by without Googling it right and like can't you know want to know once in a while something works then you're always
50:31
doing a hypothesis in your head and testing them and that helps you come up with ideas some people are you know people are curious about different things it's very rare for people not to be curious about anything but they might be curious about
50:40
their hobby or about their you know a sport they follow or they might be curious about an activity they do right and so I think that that's where this opens things up as people start to view this I think that as an early adopter it's really less curiosity than you know
50:54
I have a lot of ideas and every time I've had to implement ideas before now I've had to do something really annoying like start a whole company to do it or raise grant money or start a research project and then I've done that a lot and it all it's so much more like
51:06
there's you got a team and you've got like I love my teams they're amazing and they're much smarter than me and they do great work but like it's a lot to make the world change now I'm like huh I wanted to say I could do this and the answer is usually yes like I'm like I
51:17
wonder what would happen if you know if we if I took the bail tapestry it has to create realistic photos let's do that it takes two seconds these would have been passing and thoughts before and that's where I want more people to be is like
51:27
you know like I I think about like our handyman who helped us do some Renovations in the house it's just an absolute genius at you know and Woodworking and you know Master Carpenter and I just think about the stuff he would do with a multimodal AI
51:40
where you take a picture of stuff and be like hey how would this look this way like what other Beamer should I use like I think this is there's everyone there's there's tons of depth of expertise and I think we need to broaden access to that because General Curiosities are and I
51:54
don't even know if it's that valuable right I'm an academic but it is but specific curiosity can be very powerful yeah I agree I was talking with uh our uh interior designer and I said and she she was she's kind of like uh oh I don't I don't want to know about this Ai and
52:12
I'm like maybe you do because wouldn't you like if you could just dictate this is the couch that I want to see and then you could just throw it right onto the uh in 3D uh and Mont and she's like that could happen I went oh absolutely that
52:29
could happen and suddenly she was like okay now I'm very very interested so I love your idea that you know there's so many different people who have very specific and deep domain knowledge on things that might not naturally like like say oh I wonder if I could do this
52:45
but the minute they hear it they're like oh wow I could definitely definitely do that absolutely and the whole idea is to feel like it's so funny is the tool diffuse faster than how to use it right so like you know um there's just categories of stuff
53:00
where just clearly dominates where normal people would yeah where you normally do other kinds of work I was thinking about you know my dad had uh he has you know hearing aids that connect to his iPhone and we wanted it so it muted the iPhone you know it it only
53:12
took in voice calls but not in music as a result you know I did the usual thing to do this search I did Google search and I'm hoping someone in Reddit had the exact same problem but they didn't write it then you had 400 product forums that
53:23
are not clear and but I just asked Bing like how do I solve this problem and even though there was no direct pants from the internet I was able to extrapolate from all of the materials the market material and gave you the exact right feature to solve the problem
53:34
I could find a button and told me where I was right like there's categories of stuff that when you start to use you're like oh this is just Superior and it still reminds me of another thing which is trying to emphasize to people like this is already here right I think too
53:47
many people are thinking about this is like it's blockchain it's you know nft is like four years from the Nerds are all into it and it'll probably it'll probably die out but if it doesn't I'll worry about it in five years and the thing I keep trying to tell Executives
53:59
is like no no it's here now 10 of your employees are doing 90 of the work with AI and they're just hanging out and you figure this out or you don't this is not a long-term weight yeah and you've made the point that we're certainly seeing uh which is this is
54:16
unusual in that individuals who are like embracing this Corp at the corporate level we're not seeing that quite yet now there's a lot of reasons for that and you know for example one of our uh use cases for open source AI is that ultimately corporations that are going
54:35
to want to use it are going to have to have access to every line of code that their cyber security people can sign off on and they're not going to be willing to use a black box right uh whereas individuals like you saw what happened with Samsung where they uh were using it
54:52
and then suddenly their code got out a while but it didn't it didn't though right like that ended up being like the problem is like this is all being done by rumors but the actual Samsung story was that they got worried because we were typing Samsung secrets into AI it
55:07
never put reduce the content as far as I mean I spend a bunch of time trying to track this down it's all rumors right like people use tools all the time they don't have the code for what's going on here like people like I said except the
55:17
AWS constantly to have AWS do stuff that they don't quite know what it is like the idea that you're not going to do this to Azure or to open it like it's not true like this is what happens when you when things run a rumor right and
55:30
like you know opening is finally has a Security Site up you go to read all the certificates now like it is 100 rumor like AIC you know something that your company secrets and like that's not actually that big a problem versus the extent that it is it's happening in the
55:44
reinforcement learning thing there's a whole bunch of privacy options there'll be even more in a month lawyers are creating all kinds of problems here for stopping stuff because they don't it'd be okay if they stop it knowing what the problem was but and then the other
55:55
problem is it departments where they get their hands on this end up you know sort of viewing as a bad idea solution it's not those things and so I I actually pushed back on that like the Samsung thing it's amazing or that lawyer you know this story all over the news a
56:08
couple days ago but this lawyer like that's also like obviously this lawyer was being lazy and dumb like it's not even like defending the idea like this is not something you would do in legal service but obviously if you're a good
56:20
lawyer you can get a lot of help and I talk to lawyers all the time we're using AI for all kinds of purposes you just need to know when to use it yeah and that that's a really interesting observation uh why why do you think that is it is it fear of the new why do you
56:34
think that there's so many of these rumors especially at the corporate level um in in terms of just a fear of of the unknown what do you think is going on there so I I think that partially this is because this lost so suddenly and there's no there's no everyone is used
56:53
to I.T rolling out from the position of like hey I have to make a decision about which Erp solution I want to buy and there's a bunch of material to buy there's people to talk to there's well-defined apis you know and it cost them and no competitor could get me
57:06
because I I'm adopting in the next three years and we have to retrain everybody and let's get off the 8 million dollars or higher we'll hire you know Accenture to do it there's a PO there is a way the world has worked for Building Technology that is the exact opposite of how this
57:20
thing is working so people are just their analogies are messed up they're thinking about business like there's no one to call to adopt this I mean look Microsoft would be happy to give you Azure servers or whatever to you know but like there's no way to talk to you
57:32
can't call McKenzie and they're not going to be able to help you like they'll try they're smart people but why would their skills be better than the skills of your you know and so one of the things I keep trying to emphasize to Executives is the people who know how to
57:44
use this are already working at your company who will make your company successful so your question to them is are they going to tell you and if they're worried that your first education is I have 30 performance Improvement we're gonna fire 30 of
57:55
workers then No One's Gonna ever tell you anything so you have to build the system around the idea that you people need to feel for like that this is going to only help them when they tell you that they're specifically doing 90 of their work through AI
58:09
yeah and you know that that is like uh I mean that could be an entire podcast in itself the the this idea of the the old models are collapsing and there is not yet sort of the uh as you point out you can't go to McKinsey and say okay fix this for us right because that they
58:30
don't know either and I completely align on this idea that you've got to create a different environment because you're right I know lots of people in big companies who I talk to quite a bit and they are killing it with AI and I'm like have you have you uh like socialized
58:50
that and and like some of them I I would look at me and be like are you [ __ ] kidding me no right exactly why why would you what a terrible idea exactly all the advantage goes away instantaneously yeah when you socialize it and yeah you know I think that that's
59:12
kind of a fascinating to watch it unfold uh I think you're right though that you know we've got to get uh an environment where we do get those people say okay yeah I will step up I will show you how this could be helpful for us as a as a
59:28
company right and then required a c-level Executives to unders to make a commitment in advance to the idea that they're using this to advance the company rather than to do the things sea level Executives had to do when they get a new technology which is cost cutting
59:39
right and that isn't easy but I think that you know again golden time for startups right I just got three or four free employees yeah and uh uh absolutely golden time for startups I think it's an absolutely fascinating time well I am very mindful that you
59:57
have been very generous with uh starting early for me I know you have an engagement uh to go to afterwards you're a very busy uh guy uh this has been absolutely fascinating I would love to have you back because I only touched on like half of the questions after reading
1:00:14
all of your stuff that I wanted to ask you about but one of the things that we do here is um at the end we we make you the emperor of the world but just for a day um you can't put anyone in a re-education camp and you can't kill anybody what's the point but but but you
1:00:34
but you can incept we're going to give you a magic microphone and you can speak two things into it and incept all eight billion people on the planet they're gonna wake up whenever their morning is and they're gonna think hey you know I
1:00:47
just had these two great ideas and I'm going to act on both of them what what are you going to incept Ethan all right so so a general piece in a specific piece so I think and they're both related so I think the message I want to get across is agency we have decision
1:01:02
making and choice we have decision making choices AI World about what this means what does it mean to have this in you know human-ish intelligence involved in our lives and we get to decide what that is right this is not something being done to us we get to make
1:01:17
decisions about what this is and we should actively make those decisions we shouldn't passively step back and that relates to the second point which is you have agency over your life to a large degree stuff happens to everybody there's a lot of luck involved but there
1:01:29
is a lot of opportunity now to especially with the world of AI to access resources you couldn't access before to try and make a dream come true you know get that education you wanted to get right building something you couldn't build before no one's
1:01:43
guaranteeing your success but there's never been a better time to do that and I would kind of urge people to like this is a great time you've had to shake off the the sports of habit and look around and say what do I want to do differently because I have more tools to do that now
1:01:56
than I ever did before oh amen I love both of those agency is a is a wonderful thing well listen thank you so much for coming on infinite Loops uh I really appreciate your time I love your work and keep going thank you this was a really fascinating conversation