0:00
Only about 0.
Search this video
0:00
2% of it's actually artwork and things like that. The rest is just images.
0:04
I'm calling this the greatest lever that humanity has ever been given.
0:09
I don't care about citations.
0:12
I don't care about state of the art, I don't care about peer review.
0:14
I care about original thoughts and exchange of ideas.
0:17
So we're gonna encourage them to join our community and we'll give them cash, bonuses, and status based on the ideas they bring forth.
0:24
You give everyone personalized access to intelligent resources.
0:27
They will take it and they'll be determined and they will outcompete the legacy systems.
0:32
It's like when the camera was invented, right? Artists went crazy!
0:37
Like especially portrait artists, right? This is awful! This is horrible!
0:43
But you know, this happens all the time! It happens in Science.
0:48
It happens in commerce, it happens in art.
0:51
The new innovation is not interpreted properly.
0:55
I don't think anyone's really told an abundant story of the future whereby there is a version of this that isn't controlled by a company that comes together and just makes everyone's lives better and it's there to make your life better.
1:08
It should be AI for the people by the people, but it does need to spark right now, and that's what we're trying to bring at scale.
1:15
Hi, I'm Jim O'Shaughnessy and welcome to Infinite Loops.
1:22
Sometimes we get caught up in what feel like infinite loops when trying to figure things out.
1:26
Markets go up and down, research is presented and then refuted, and we find ourselves right back where we started.
1:34
The goal of this podcast is to learn how we can reset our thinking on issues that hopefully leaves us with a better understanding as to why we think the way we think and how we might be able to change that to avoid going in infinite loops of thought.
1:51
We hope to offer our listeners a fresh perspective on a variety of issues and look at them through a multifaceted lens — including history, philosophy, art, science, linguistics, and yes, also through quantitative analysis.
2:07
And through these discussions help you not only become a better investor, but also become a more nuanced thinker.
2:16
With each episode we hope to bring you along with us as we learn together.
2:21
Thanks for joining us, now please enjoy this episode of Infinite Loops.
2:26
Disclaimer: Jim O'Shaughnessy is chairman and Co-Chief Investment Officer of O'Shaughnessy Asset Management, where Jamie Catherwood is an associate.
2:35
All opinions expressed by Jim, Jamie and podcast guests are solely their own opinions and do not reflect the opinions of O'Shaughnessy Asset Management.
2:43
This podcast is for informational purposes only, and should not be relied upon as a basis for investment decisions.
2:49
Clients of O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.
2:54
Jim O'Shaughnessy: Well, hello everyone.
2:58
It's Jim O'Shaughnessy with another Infinite Loops.
3:01
And today is a very, very special Infinite Loops that we are going to drop as a bonus podcast.
3:07
My guest today is Emad Mostaque.
3:10
The founder, the beacon of light for open AI, of Stability AI.
3:16
Now, I need to explain our relationship.
3:19
I had been following Emad's work closely and finally met him in August of this year and my reaction was to instantly invest in his company and after some back and forth, agreeing to be the executive chairman of Stability AI, which I truly believe is the standard bearer for where AI should be going.
3:46
We're going to talk all about open, first, closed, et cetera. Emad, welcome.
3:52
Emad Mostaque: Thanks, Jim. Thanks for having me on. Thanks for your support.
3:57
Jim O'Shaughnessy: Delighted to have you on.
3:59
So my first question is, did you anticipate on August 22nd when you released the model, the stable diffusion model with weights, did you anticipate the tsunami of creative uses of it that followed in the week, two weeks, three weeks after its release? Emad Mostaque: Yes.
4:25
I didn't anticipate exactly what people would do, but when you give people tools and agency, there's no limit to what they can do.
4:36
And I think that the fact that the development team managed to get it to the characteristics that it did, which is one of the most advanced AI models in the world, that enabled things for humans that were previously unacceptable.
4:52
Being able to run and be developed on a MacBook.
4:58
There's no way that you wouldn't have had this amazing kind of reaction that we've seen, and it just continued to grow every single day.
5:04
Jim O'Shaughnessy: And one of the things that drew me to you specifically about Stability AI and the idea of open AI was exactly that.
5:15
I think that for us to get the most use out of artificial intelligence, it absolutely has to be open as opposed to closed, because how presumptuous would it be on our part to believe that we could know every use case for such a powerful technology?
5:35
I'm very much of the opinion that if we want to build the human colossus, it's going to include AI and the best one is going to be open, because there's that wonderful quote that, no matter how smart a person is, no matter how imaginative, you could never ask them to create a list of things that would never occur to them.
5:58
So I'm, as you know, incredibly all in on you and the future of Stability AI, but also on you as a beacon of the open source movement.
6:12
I think the movement that we've seen since then has been just incredibly exciting and also very, very wonderful for me to see so many people coming to the banner of wanting things to be open and not closed.
6:26
If we could step back for a minute though, because our average listener/viewer here might not know as much about the whole open versus closed debate of artificial intelligence.
6:37
So Emad, talk to me a bit about why the average person who reads about artificial intelligence, unfortunately usually from the point of view of scary AI, not really understanding the incredible potential of artificial AI.
6:52
What would you say to the average person out there listening about the debate between open and closed?
6:59
Emad Mostaque: I think that's an excellent question.
7:01
So I think if we think about closed AI, closed AI has been our history, Facebook and Google and the majority of companies that have made money from the internet have done it via closed AI.
7:11
We've been in this area of big data where they harvest everything about you to extrapolate.
7:17
For the financial followers there, it's like a momentum based thing where you say the future is the past, and then use it to target gym with ads to an era of big models.
7:27
So a new type of model that's more like fundamentals based, it learns about principles, as opposed to extrapolation to really resonate with us.
7:36
These big models need super computers, amazingly structured data, not big data and some of the biggest geniuses in the world in computing to make.
7:45
And so right now we're currently following the old paradigm of these models being closed, both for safety reasons as they would call it, and also financial reasons.
7:55
They're expensive to train.
7:57
Millions, if not tens of millions of dollars and the companies want to [inaudible] return and there's only a handful that can build them.
8:04
However, I think that it's a bit dangerous now, because ads are convincing.
8:09
The whole ad based model is to do with manipulating your mind and convincing you.
8:13
But these models are a lot more resonant and convincing and they should be both understandable and available, because you've moved to...
8:21
Again, these models that just extend and target to models that can read and write and sing and create.
8:27
They expand our means of being able to express ourselves, but if they're controlled by only big technology, you won't be able to understand them, you won't be able to interpret them and they'll only be accessible by the few.
8:38
So that will increase their digital divide for these new information super highways as it were, and then also it will stifle innovation.
8:46
The most secure and safe operating systems in the world that run most of our infrastructure are based on Linux, which is an open source system.
8:54
We've shown time and time again that open systems are more robust, resilient and safe and closed systems, which often become brittle and fragile.
9:02
So if we want to focus on the good parts, we need to make it accessible by making it open.
9:08
And if we want more resilient system, we also need to make it open.
9:09
And that's why we stepped in to provide an open source alternative to these closed systems by large companies that just entrenched themselves with more.
9:18
Jim O'Shaughnessy: That was one of the other things that attracted me.
9:22
As you will recall, when we had our wonderful day together and I invested at the company, I told you that as a grandfather of three with more coming, if I could do anything that would change the future for my grandchildren, it would be to make certain that there wasn't some panopticon AI controlled by a few, telling them what to wear and what to brush their teeth with.
9:46
But there's also in the general public, a lot of fear.
9:53
Unfortunately it syncs up with the media, which basically our species is an emotionally based species and fear is the dominant emotion.
10:02
And so if you look at the business models of Legacy Media, well, what are you going to get the most clicks with?
10:08
You're going to get it with things that inspire fear and uncertainty, et cetera.
10:14
The business model is well proven.
10:16
I believe, and I know you believe that the way to deal with the safety issue of these models is to make them open, but then also make the comments, questions, criticisms about them open as well.
10:32
In other words, we shouldn't limit this even though I know we're putting together, we already have a council of people commenting on the ethics of the uses of the model and whatnot.
10:43
We also have an exciting, for me at least, deep fake initiative where we're going to have a bounty for the coder that manages to come up with a great deep fake detection system.
10:56
But talk a bit about why the open AI movement actually will lead to better outcomes for oversight for getting rid of the bad actors, et cetera.
11:10
Emad Mostaque: So right now, again, the model is very closed.
11:15
So you have a few companies that could build this.
11:18
So literally only a handful of companies can build something as sophisticated as our lead model, Stable Diffusion, which took a 100,000 gigabytes of images and learned from the principles of them in a two gigabyte file that can make masterpieces. I mean that's insane. It's compression.
11:33
I mean there's a saying, compression is intelligence, because you take all that stuff, compress it down.
11:40
Without the open source alternatives and a knowledge that this is out there, you will see more and more sophisticated bots, you will see more and more bad actors who also have compute capabilities in the technicalities, doing things like deep fakes.
11:53
The reality is that a lot of these out of society illegal, immoral, unethical things go underground, because they are illegal, immoral and unethical.
12:02
They are not the majority, the vast majority of positive things here.
12:06
I mean the internet is full of a lot of very questionable things.
12:10
And this comes to another point.
12:12
When for example, Stable Diffusion was trained, it was trained on a snapshot of the internet, basically common crawl similar to what you would get from Google image search.
12:20
That is biased, it is largely Western and it's not diverse.
12:26
But if you don't have a framework that you can take and adapt like was done with Japan Diffusion, which trained on the Japanese language, so salary man doesn't mean a man with lots of money, but a sad man instead, then you have no basis from which to work and you have to wait for the big companies to eventually get to your culture, or your race, or your content, et cetera, which may take many, many years.
12:48
So not only is there no herd immunity built up, because people don't know this technology is prevalent and accessible, but also there's no ability to understand it and countervail it.
12:58
And then finally there's no ability to create more diverse models.
13:01
Instead, you stick with biased models and some of these biases are unconscious, so it becomes biased to whether or not you want it.
13:08
You type in beautiful lady and you'll see a white lady.
13:12
Whereas, that might not be the case around the world.
13:14
So I think everyone deserves to have their own models, their own data sets and more people in the conversation about what is ethical, not ethical, moral and not moral, because we're not the same around the world, but this technology will be used by everyone around the world.
13:28
So how do we get from A to Z?
13:30
Jim O'Shaughnessy: I think sunshine is the best disinfectant and that's what we're doing on the open source side.
13:39
Emad Mostaque: I think I'd like to add to that as well, like the competition system with Kaggle, which has data science competitions and others, has been shown to work very well, because people obviously want to be paid to contribute.
13:52
But when it comes to open infrastructure like this, that can go to millions if not billions, people are excited to commit, and they're excited to contribute and isn't just people, but also companies.
14:04
There's no Schelling point or coordination point to build digital infrastructure for the future.
14:09
And so that's one of the things we're hoping to do through some of these mechanisms, learning from the good parts of Web3, as opposed to unfortunately the vast majority, which is not so great.
14:20
The good parts of the classical open source movement and the learnings and failures of that, as well as how people have come together for greater causes and achieved great things and some terrible things.
14:30
But the question of how do we coordinate people at scale to build the infrastructure of the future is one that we are really focused on.
14:35
And we think this approach will do wonderfully on...
14:38
It's not just people like I said, but companies and institutions and others that are contributing.
14:43
Jim O'Shaughnessy: And one of the things, just as an aside, we use the same terminology, which was why I think we synced up on Schelling points.
14:52
We both are fascinated by Shannon's information theory, which we're going to get to later a little bit.
14:57
But that's the other thing that I felt the deep need that there had to be a Schelling point that drew a beacon, if you will, and I see that in you and in Stability AI.
15:08
I was listening to another podcast that you did and you made a wonderful comment that I really agree with.
15:16
You said, "The world is not normally distributed for huge variety of things like medical research, like interests, et cetera, that everybody was unique."
15:27
And one of the things that I find really interesting and have made part of our Great Reshuffle series, is this idea that we are moving into a technology where we can actually act on and make use of that.
15:43
Talk a little bit about that, how the artificial intelligence models are going to be able to zero in on specific diseases, specific interests and create in your words a wonderful world where we can solve problems that were insolvable without this technology.
16:02
But we can also create a generative search engine.
16:06
I love the term and please explain that to our listeners who might not understand what that means.
16:11
Emad Mostaque: So now I'll explain the term first.
16:13
As I kind of mentioned in the past, if you look at stable diffusion, it took 100,000 gigabytes of images, 2 billion images, and created two gigabyte file that learned from the principles of all those images.
16:29
So if there were a couple of, I don't know, pictures of Jim's apartment in New York and it was labeled as such, and you type in “Jim's apartment in New York”, it would be able to give something in that style, or we can search it for Infinite Loops as a screenshot of Infinite Loops. Only about North 0.
16:45
2% of it's actually artwork and things like that, the rest is just images.
16:50
So being able to search those concepts and those theories is something just absolutely fascinating.
16:55
And then mash them together in any combination you want, because again, it learns from the principles of the images and the pixels.
16:59
This means that when you go to Google image search, you're not really searching for an image, you're searching for a picture of some sort.
17:06
And this enables you to do that at scale.
17:08
Again, for snapshot of the internet with all its positives and negatives, but eventually everyone will have their own generative search engines of all their information across modalities, and intelligence will be pushed out to the edge by this compression.
17:23
It's not just a database, because humans are not just about structured data, they're about this unstructured data and this heuristic based principle, based approach as well.
17:31
Because the reality is that most of our systems are set up not for first principles thinking, but for legibility, as in being able to understand structures and simplicity.
17:44
So the way that we do it is that our systems basically treat people as ergodic, non-ergodic...
17:48
ergodic when they're non-ergodic.
17:49
So a thousand tosses of a coin is same as a thousand tosses of coin at once, whereas everyone is individual.
17:56
But we couldn't keep up with it, because we didn't have the system to do so.
17:58
If we're in a group, then you get to know everyone's preferences and you can do more and more personalized things.
18:04
But there's nothing that allowed us to coordinate at scale until this technology became good enough, fast enough and cheap enough, which is basically now.
18:10
So it's not just about pretty picture creation, it's about organizing any type of information at scale.
18:15
And it's not for serving up adverts and personalizing ads for you, personalizing information for you, which is a very different thing.
18:24
And it flips the narrative from these big AIs in the middle coordinating and serving your information to change your mind, instead the intelligence is being at the edge and giving you the information that you need to go and make a change in the world or yourself.
18:37
Jim O'Shaughnessy: And making the change from highlighting and encouraging consumption.
18:45
And now we have the ability to highlight and encourage creativity, which I think is one of the most coolest uses of this.
18:55
But that leads me to my next question, which is like obviously the generative art is incredible, it's amazing and it captures people's imagination.
19:05
But I personally think that two of the biggest industries to be affected by this are ultimately going to be healthcare and education.
19:13
In healthcare, just think of what we can do with say, forensic audits on past scientific papers.
19:21
What we can have the AI look for anomalies in previous papers will identify all that data, Ancel Keys at the University of Minnesota with his diet study.
19:32
Oh gee, that was all fake data.
19:35
We as humans can't sort all of the various things, but it's a trivial thing for AI to do.
19:44
And it combined patterns that are unlikely to emerge by chance in all of those particular papers.
19:51
And also it will give us the ability to reproduce use in silico as opposed to having to run a human trial, et cetera.
20:01
It will help with orphan drugs, as we say.
20:03
Everyone is unique and I see all of the use cases for artificial intelligence in these fields to be just massive.
20:13
Another one, and then I'll get your take on it, but wouldn't it be great, and this was said to me by a friend of mine who's deeply into AI and so I owe him the shout out for this idea.
20:25
But humans don't like to work on thesis or hypothesis that lead to a null hypothesis, right?
20:32
Wouldn't it be great to have AI just generate a ton of null hypothesis machine printed to a repository so that human scientists, when they're looking at things, they can consult the null hypotheses that aren't going to get published in a referee journal but are incredibly useful because along Shannon's information theory, those null hypotheses provide real information. Your thoughts?
21:04
Emad Mostaque: Yeah, I think it's fascinating, right, the potential is it goes beyond images because images, it's again, it learnt the principles and it can reconstruct it from that.
21:13
But we have language models, protein folding models, a whole bunch of different things that we're building as a company and extended organization.
21:20
And protein folding is a good example.
21:22
It's the same technology that drives images, drives protein folding and understanding how that happens. And why is that?
21:31
It's because the next generation of AI, it's moved from patterns to principles.
21:36
And it did this by something called attention.
21:37
So there was something called the transformer that was debuted in, well a bit before, but the official debut was 2017 in a paper called ‘Attention is All You Need’ because the human mind is about attention.
21:48
We filter noise and we pay attention to the important things in front of us.
21:52
And that's where the real value is.
21:53
Again, this is Shannon's information theory.
21:56
Information is valuable in as much as it changes as a state.
21:58
So not all information is equal.
21:59
What is that state change?
22:01
That's what this AI search is for.
22:03
And when you combine principles based AI with pattern based AI, that's when you get some really interesting things happening.
22:13
Again, there's a lot of analogies to finance and running things properly, as he says, as a former hedge fund manager in this and analogy to the human brain.
22:21
Our human brain is made up of two parts.
22:23
There's the recall part and there's the principle based part.
22:26
And what we were missing in AI was the principle based approach.
22:30
We had lots of, if this then that statement.
22:33
But this is something fundamentally different.
22:36
This is again, mimicking the human brain to be able to do things like understand academic papers, to do things like often drug discovery and things, but in a very different context and milieu to that which came before.
22:50
And it can't do it by itself, these models.
22:52
You combine them with other models just like the human brain is made up of lots of different parts. Jim O'Shaughnessy: Yeah.
22:59
And again, one of the things that as I saw this opportunity that just made me incredibly excited was your vision for where this could go.
23:10
And part of it was in that combination of models, the idea of having access finally to liminal spaces that humans are not terribly well programmed on human OS to really grok the value of what that is providing us.
23:28
In terms of education, we already have a wonderful initiative that's been announced, but I'd like you to talk a little bit about that because it's really quite amazing and I don't think that people who would normally be listening to Infinite Loops would know about how incredibly exciting the prospects for education are that we're already deeply involved in.
23:51
And the other thing that I'll get in, because I forgot to give it the push, of course I love the compression, but I also love the TV show Silicon Valley.
24:00
And so you and I tease each other all the time with GIFs from that.
24:03
Like, oh, wouldn't compression be a good idea? Emad Mostaque: Yes.
24:07
Jim O'Shaughnessy: But please talk about the educational initiative and where you see this going in education because it's incredibly exciting. Emad Mostaque: Yeah.
24:17
So what I like to do is I like to give people a challenge, which is go to the future and see where AI is.
24:25
Is that AI controlled by a company that is more powerful than any government? Or is it open?
24:30
Well it should be open for our kids because that's too much for an unelected company and we probably shouldn't have governments control it either.
24:38
And now let's take that AI back with us and what does it look like every day?
24:42
Is there any doubt that AI will be in education and personalized education for everyone and that we know the best way to teach linear algebra or multiplication or these? No, there isn't.
24:56
So why not build it today is the question. Right.
25:00
Jim O'Shaughnessy: Exactly.
25:02
And talk a bit about the Malawi announcement and what we're doing there.
25:07
Emad Mostaque: Yeah, so basically our co-founder of Stability AI Joe Wolf runs a charitable kind of affiliated entity, Imagine Worldwide.
25:16
There was a prize called the Global X prize for learning backed by Elon Musk and Tony Robbins amongst others, which was a $15 million prize for the first application that could teach kids literacy and numeracy anywhere in the world without internet in 18 months, which is quite a steep thing when you consider A, anywhere in the world.
25:34
B, it's tablet only, no kind of teachers required.
25:37
And C, 18 months is not that long.
25:40
Again, we've taught our kids and we know it's a bit hard for some, right?
25:44
Kind of Joe and his amazing team kind of took the winner of that prize into refugee camps from the Rohingya camps to Kenya to Malawi and other places and did proper randomized control trials showing efficacy in teaching kids literacy and numeracy in 13 months on one hour a day of education.
26:02
It's just one hour a day.
26:03
I mean you can add it up, right, like four, 500 hours, 400 hours of teaching on a tablet with relatively basic software.
26:13
They reach literacy and numeracy. That's insane.
26:16
And in a refugee camp, an Malawian refugee camp.
26:18
You've only got like 20 bucks a month in income.
26:21
It's one of the poorest places in the world.
26:24
So this was kind of exceptional.
26:26
And due to the RCT data, working with the Malawian government and hopefully soon many, many others, there is the remit now to educate a nation.
26:34
But it's not good enough to have an open source piece of software that half a dozen people built.
26:40
We are going to be taking it and giving a challenge to the world let’s educated a nation, what’s the best software we can build that has AI at the core, that learns what the individual student needs and customizes to them to make them happier, to give them agencies and all the tools they need to have access and opportunity.
27:00
And with that from country, to country to country, you can learn about the local context but then you can also build a system that can go around the world.
27:07
So anyone can take it as an integrated hardware, software deployment and curriculum piece and customize it to their own needs because it's what the world deserves.
27:16
But until now there's been no initiative like this, which again is insane. Right.
27:21
There's a lot of these problems.
27:22
We know what the answer is, but the individual components of the recipe haven't been bought from the store and no one's whipped up the recipe.
27:31
And I firmly believe that we can do it together.
27:34
Again, we build an artificial intelligence that can compete against the top entities together.
27:39
We can build a beautiful piece of software and hardware and system that intelligently learns for everyone together.
27:46
And it's not just going to be our organization that does it, it's going to be our organization, everyone who's always had a dream about changing education for the better.
27:55
What the exact form of that looks like, I don't know.
27:56
But I do know it'll be intelligent, adaptive and empathetic.
28:01
And again, give these kids the biggest chance that they have and constantly improve.
28:06
You want have not something that's static.
28:08
You want something that constantly improves because the community improves it.
28:12
And so that's our education piece, at Stability.
28:14
Jim O'Shaughnessy: And by extension what I love and again what draws me so strongly to this is the “we'll do this together”.
28:22
I have been, one of my many soap boxes is if you want to build a human colossus and advance our species, it can't be done.
28:32
I mean here Lord Acton's quote is very apt, "Power corrupts, absolute power corrupts absolutely."
28:39
And so this idea that we should even be willing to consider this being in the hands of just a few seems insane to me because it won't build things like you just described.
28:54
An open system in fact will, right? Emad Mostaque: Yeah.
28:57
And it's this question of this right to education, this right to infrastructure. Right.
29:04
We deserve this as a human species.
29:07
I think it was Vinay Gupta, who's a very interesting guy, had a proposal that we make a very basic approach to human rights, which is the rights of the children.
29:17
Jim O'Shaughnessy: I like that.
29:18
Emad Mostaque: Because children cannot speak for themselves, they have no power.
29:23
So when you think about things in terms of children's rights, the world changes.
29:28
And so what do they have the right for?
29:30
Education, healthcare, food.
29:34
So as a society, are we providing that to the children? No.
29:38
So we're impinging on their human rights.
29:41
As a society, can we do that? Yes.
29:44
And it should be infrastructure for all.
29:45
It should not be controlled by private interests.
29:49
But private interests can of course supply it.
29:51
But again, there should be equal access because otherwise what you're going to get is AI enhanced humans and not AI enhanced humans. Right.
29:58
You're going to get this widening gulf.
29:59
Now you said you can come together to build this if you do it properly.
30:02
But it's like, have you ever seen that video of the person dancing by themselves on top of a hill? Jim O'Shaughnessy: Yeah.
30:08
Emad Mostaque: It's all about the second dancer and the third dancer.
30:11
You have to lead, but the key thing is having followers.
30:14
And if you're closed and private, you will never have that many followers.
30:18
But if you do it in the open where the amount of people that want to help with this is thousands if not hundreds of, to be honest, it's everyone.
30:26
I have yet to meet a person who's happy with the education system as it is.
30:30
And I'm lucky, you're lucky.
30:32
We came to the top of our education systems in very privileged backgrounds. Right.
30:36
Everyone's unhappy with education because it's antiquated.
30:38
It was designed to provide people as the cogs for systems.
30:43
I like to say that organizations are slow down AI that eat our dreams and hopes because that's how they're designed.
30:49
The bit of a colonial past kind of that you will be a number.
30:53
Whereas what I want instead is to have education that gives people all the tools they might need to be chefs, not cooks, to come up with recipes and to kind of extend themselves to feel they have agency.
31:06
So to be happy, you need to have agency and progress and feel that there is a future.
31:13
So I think we can do that through this technology. I think will be amazing.
31:16
And again, make that a human right for everyone.
31:18
There should be a right to universal education.
31:22
But then as we have an integrated software, hardware deployment and all these other things, and we have some interesting financial instruments as well I can talk about, you can put healthcare on that.
31:32
You can use it as a logistics hub to feed people.
31:36
There's no limit to what you can do because all of a sudden you're putting standardized infrastructure that's intelligent into the poorest places in the world.
31:43
And you're doing that with initiatives such as UNESCO.
31:45
We have product Giga, which is putting high speed internet, satellite and broadband into every school in the world.
31:50
And you put it into every place of worship, all of a sudden the world lights up and the invisible become visible and then you can solve these problems at scale.
31:59
Just operating system upgrade.
32:00
Jim O'Shaughnessy: Absolutely.
32:03
And I think obviously one of the most exciting things about this, we have a primitive technology which is amazingly powerful.
32:11
And I'm using that gang who our listening, I'm not saying the tech is primitive, it's a term of art in technology.
32:19
But the school thing is another one of the things that has always animated me.
32:25
Robert Anton Wilson, who I love, who was way ahead of his time, wrote in I think 1992, "All of our old models are collapsing." And I think that's true.
32:36
I think we're seeing it now.
32:38
And the idea that the educational system is antiquated, it was designed primarily by industrialists who wanted the humans to get used to sitting in a room for eight hours and doing as they were told.
32:51
And obviously we are well beyond that.
32:55
And so the idea of both reimagining and reworking education, healthcare, all of these systems has been something that we've been really in drastic need of for quite some time.
33:07
I think it's actually part of the, when people are feeling down, when they're feeling like the future is not bright, this sort of thing is the thing that will light them up.
33:19
I loved your idea of the lighting up the places of worship, lighting up the places that when you look at one of those satellite pictures and Europe and America are just beaming and a lot of other parts of the world are dark.
33:35
I think we have the ability and opportunity here to light those places up. Emad Mostaque: Yeah.
33:42
But yeah, if I can say one thing about that.
33:44
But the reality is that America and Europe are lit up by candle wax and kind of oil and fat lamps.
33:56
Whereas you have the opportunity to actually leapfrog in these places where there is no infrastructure as opposed to fighting against the system in the West.
34:04
It's Clayton Christensen's idea of disruptive innovation and tiny steel mills against the big steel mills.
34:09
You give everyone personalized access to intelligent resources, they will take it and they'll be determined and they will out compete the legacy systems.
34:19
And then the legacy systems have no option but to adapt and improve and upgrade to LEDs from candles.
34:30
Jim O'Shaughnessy: I love it and brings to mind the other idea that I think got me so excited about the opportunity to join you was this idea that most of these models are designed to serve what the book, The WEIRDest People in the World wrote about the dominance of in psychology studies and a whole variety of studies.
34:54
All of these studies were done on Westerners and he titled his book The WEIRDest People in the World because we are quite unlike other populations, other places in the world.
35:04
And one of the things that got me very excited when we were talking about it was this idea that we're going to set up joint ventures, a variety of opportunities all around the world, that we were going to make available stories, not just in English, not just in major Western languages but in all languages.
35:24
Talk a bit about that and the story initiative because the semantics of most stories that you can see, again looking through a Shannon lens on it, most stories basically provide no information. Right.
35:38
And I'm speaking technically here of information as being something new, right, like Claude Shannon explains it.
35:45
But talk a bit about the initiatives there because again, to my way of looking at things, the initiatives that you are mandating at Stability AI and by the way, parenthetically, we welcome anyone who corporation, non-corporation, regular person, non-regular person who shares our passionate belief that the future should be open and not closed.
36:08
So we're not trying to say that yeah, no you can't join if you're this. Not at all.
36:14
If you are committed to an open architecture in the system, please see the beacon that it is a modern Stability AI and join.
36:22
But talk a bit about the stories and as well as the ability to train this in all languages, in all cultures, that in turn gives them the ability to put their own spin on the technology. Emad Mostaque: Yeah.
36:36
So I think that's an excellent thing, right?
36:39
Again, how can it be diverse and distributed if it's not out there and you don't teach the people how to do it?
36:44
So most companies are going to hoard the technology aspect and they don't want to share.
36:50
Whereas we say we're open for collaboration.
36:52
We think it's inevitable this technology be used by everyone in 10 years.
36:56
So our model is very simple.
36:57
Go and partner with people who have content, who have stories, who have IP and build this technology for them as soon as possible and teach them how to use it.
37:08
Because the ultimate play is a platform play. It's a better internet.
37:11
That's what kind of we're doing.
37:12
It's grand, it's ambitious but someone has to do it.
37:15
But nobody else really is at scale.
37:17
Again learning from the lessons of Web3, but it's about going into India and Indonesia and other places and building communities because this came out Commute, it came out of a Eluther AI and LAION and HarmonEye, our music community.
37:29
We have 100,000 developers building really cool stuff.
37:32
Only a fraction of them regularly.
37:33
But we'll get more and more because again we're trying to coordinate that and say what infrastructure do we need to build for the future?
37:40
Well obviously you would need to have an Indonesian language set and model to represent Indonesians because how are you going to do that?
37:47
Otherwise, you can't have a model that represents everyone all the time.
37:50
If you do that, it'll either be a very bad model or very small model because people are different fundamentally.
37:57
So we had this image of intelligent internet where every person, country, culture and company had their own models as the agent of search engine is talking to each other and tell each other stories.
38:06
Because stories is what moves society.
38:10
So like America is a story, Christianity is a story, stories have power and people are story driven.
38:17
You're made up of your layers of identity.
38:18
But for the first time we can actually map those stories and how they change.
38:21
So we have a lab called Carpet that does narrative structure analysis to see how powerful stories are and if they build up and break down and they have language models that you can then instruct and tell different stories that they specialize to those stories and customize.
38:36
When you do that, it takes the size of the models down by 100 times.
38:41
So GPT-3 by OpenAI is the most powerful language model.
38:44
They can write beautiful sentences and poetry.
38:46
When they instructed it, it went down to from 175 billion parameters to 1. 3 billion parameters.
38:54
Because are you a storyteller who knows all the stories of the world or are you a storyteller that knows the stories that you most believe in?
39:01
Well most people are the latter, right?
39:03
And no one can hold all the stories of the world.
39:06
Whereas we want the things that tell us the stories that matter to us.
39:09
At the same time, if you do it in the standardized framework, you can do dynamic translation between the stories so you can see Christianity from the perspective of Islam or libertarianism from the perspective of Tea Party Republicanism because you have a mixture of these really interesting compressed generative search engines.
39:30
They're like what is it type one thinking?
39:34
And then the type two thinking, which is the repositories of structured knowledge.
39:37
And the combination of that allows for a universal translator, not necessarily of languages but of stories and contexts, which is something crazy and insane.
39:47
But within our reach for the first time.
39:49
It's not easy, but this technology's given the breakthrough that can enable that. Jim O'Shaughnessy: Yeah.
39:55
And better stories means better communication.
39:58
And I can't remember whose quote this was, but it's along the lines of, I think it was Kipling, that we are “islands of misunderstanding or shouting at each other across seas of kind of chaos”.
40:12
And this I think will go very, very far to enhance communication between people with very different traditions, very different outlooks and find ways that build bridges as opposed to try to take bridges down. Emad Mostaque: I agree.
40:27
And you have to remember communications a very broad set of things.
40:30
When you make a PowerPoint, you're communicating. Right.
40:33
When you make a video, you're communicating.
40:34
When you make art, the best artists communicate. And this is the thing.
40:42
So this has a variety for a wide range of things because ultimately we're social communicative creatures.
40:47
So I have Asperger's myself.
40:49
So such social communication has been difficult for me to understand, but I think I've got a bit of the hang of it, or at least some systems now.
40:56
But I've thought about it from very first principles basis.
40:58
Again, you give people the tools to communicate socially better, you have a better society.
41:03
Because right now, again, what we see today and it's reinforced is just this widening gyre whereby people believe stories and they believe that other people are not the same.
41:14
And when you believe people are not like you and [inaudible] you get really nasty things.
41:20
Whereas in reality, people are people and there's a nice saying, I can't really remember who said it, "All wars are based on lies", which is true.
41:31
There isn't a single war that is not based on lie.
41:35
And so again, when you can tell the true stories and the hopeful stories, then I think things change because people, to be honest, it does sell ads when you have the fear.
41:44
But people like hope and sometimes they just want to be able to do that.
41:49
And they feel that hope and that positivity again when they can create and communicate themselves as opposed to being restricted by what they can do.
41:57
So give them that avenue to express themselves, give them stories of hope and maybe we can make a better world because otherwise I can see the direction we're going, which is more and more polarization, more and more lies, more and more war and this negative sum game because we've pulled forward too much from the future via the financial system and others that we have no choice but to fight it seems, unless we create an abundance.
42:22
Jim O'Shaughnessy: And of course the newer company, O'Shaughnessy Ventures that actually made the investment in you has two primary parts of our funnel that if you don't meet both, we won't actually make the investment.
42:34
And the first is a asymmetric opportunity that is win, win.
42:40
And so again, in this series that I've been doing The Great Reshuffle, what we're trying to highlight is that we are moving from a scarcity mindset that dominated humanity for all of our history really to an abundance mindset.
42:55
And in that, I mean we are offering tools or these new innovations are coming online at a staggering pace that make it so that things don't have to be like they used to be.
43:07
And the idea of human agency has always been something near and dear to my heart.
43:11
The greater you can enfranchise someone and give them full agency, the greater for our species, honestly, the greater that we can communicate.
43:22
And as you say, all wars are started on lies, right, and we can break through that I think by giving the tools of hope and agency to everyone in the entire world as opposed to just the few, right?
43:39
If it's a few versus many, it's always going to be you're going to get your better cognitive diversity in the many.
43:47
That just seems self-evident to me.
43:50
Even if you take the most brilliant people, like for example I had the opportunity to meet a lot of those brilliant people when I came to London for the Stability AI offsite.
44:00
The fact is, even the greatest geniuses in the world, if you put them all in a room are not going to be able to cover everything because really of a lack of cognitive diversity there.
44:15
When you give these tools to everyone, that's when you see these incredible things springing forth.
44:22
One of them is hope and better stories and the ability for us to communicate with one another better.
44:30
One of the things that I keep hearing from people is, they loved the little catchphrase, "data is the new oil."
44:38
But when you and I were talking about it, you made a really good qualification.
44:41
You're like, "Yeah, data is the new oil but it's got to be clean data."
44:47
A lot of these models were trained on questionable or not questionable, but non clean.
44:54
Talk about that for a bit.
44:55
Emad Mostaque: So how do we learn, right?
45:00
We don't learn just by reading everything unless you have a very particular peculiar type of mind.
45:07
I mean, again, information is compression.
45:09
So when you listen to a podcast sometimes you can learn a lot more than reading a whole book.
45:17
You can do it in really interesting ways.
45:20
Again, the principles that you extract, a good teacher can teach you in minutes what might take years to otherwise learn.
45:27
And that's really the craziness of this.
45:28
It's the same with these models.
45:30
Again, these models pay attention, but if you feed them a lot of crap, you'll still get a lot of crap on the other side.
45:35
And so the valuable data here is the content that is structured in the world that allows you to learn principles as opposed to again, the big data age where it was about as much data as possible to extract patterns because you need less data in order to get these things done.
45:52
If you take Stable Diffusion, you train it with a half dozen images, it learns an entire style, which is insane because it's got this principle based analysis of what makes up a style.
46:03
So the key play here is not that data is new, as most content is the new oil, shall we say, but it's got to be clean, light, sweet oil, not the heavy sour stuff that's got every shale or something like that.
46:16
And give me some high quality gasoline that doesn't gunk up my gas tanks.
46:21
And then there is another part of here, which is GPUs are the new spice as well.
46:25
There aren't enough GPUs in the world to run these models right now and they're going to be used from a few million people using it now to a few billion.
46:32
Now you say spice in terms of East India company or in my favorite terms, Dune, but it is kind of what [inaudible] the future custom architectures for this. And it's not just GPUs.
46:43
You look at the MacBook M1 chip, 16.
46:43
8% of that chip's dye is the neural engine, which is puzzlingly not used at the moment.
46:51
Similar with the A16 bionic.
46:54
Next two years Apple will be the biggest AI company in the world because they're going to go from Siri 1 to Siri 5 using just this type of model.
47:01
I can say that because I'm not under NDA with them or anything like that.
47:04
Again, they don't leave useless things in their things, but when 80% of the using population has it, it's a really interesting and easy switch.
47:11
So this technology will be everywhere, but the ones that will win are the ones that have the best structured data.
47:16
And in fact, this is why the education proponent is good as well.
47:20
When you learn the best way to teach linear algebra, that's the best way to teach a model about linear algebra.
47:25
So it's not just a data set of lessons, it's a principle based analysis tool as well that knows linear algebra and then can extrapolate based on principles from there.
47:35
We've really seen OpenAI and DeepMind build AI that can solve international math olympiad questions.
47:42
That's the level we are already, which is kind of scary and kind of insane.
47:45
But again, nobody else can use those technology except for that until shortly from now.
47:50
Jim O'Shaughnessy: And of course Dune being, we're both fans of science fiction, Dune being one of my favorites as well.
47:55
The spice must flow and it shall flow.
47:58
But also just to pick up on the point you just made, James Flynn who was the intelligence researcher who came up with the evidence that IQs were essentially increasing, particularly in the western world.
48:11
They named the effect after him.
48:13
They call it the Flynn Effect.
48:14
And one of the things that he did was, rather than look at the mean average, which usually comes at 100, he looked at the raw data and he scratched his head and he thought, "What is going on here?"
48:27
Because if you looked at the scores from 1900 in America, let's just keep it in this country, they would've been about 67, which now is considered impaired versus 100 in 2000 when he was doing the research.
48:45
His insight, of course, fairly obvious, but let's put it out there anyway, this could not be the result of evolution in our species because it happened over just a few generations and he came up with the idea that it was because essentially in 1900, everyone took the best term would be was very quite literal and very kind of “chair” this material thing oriented.
49:16
And one of the reasons that Flynn came up with this expansion of IQs was because we moved to more of an abstracted ability in thought, which it immediately made me think of Stability AI and AI in general.
49:34
I think that, and I want your opinion here because you know much better than me, I think that this technology, this compression has the ability to increase IQs generally across the entire world. What do you think? Emad Mostaque: No, 100%.
49:51
You need AI that helps you and extends you and augments your capabilities.
49:54
It's like, give you an example.
49:56
This type of technology similar was used to beat a human at Go.
50:02
Now that was the AlphaGo program by DeepMind.
50:05
So Go was considered insolvable unlike chess because chess you can brute force it.
50:09
You can calculate moves ahead.
50:11
And then once you outpace Gary Kasparov, if you have deep blue and then you had a computer being the best at chess.
50:17
Now Go has too many, this is Chinese chess as it were, probabilities because there's too many moves you can make at any given time.
50:24
So it's considered insoluble until they made a computer that AI that could learn from principles called Alphabet.
50:31
And Lee Sedol, there's an excellent documentary on YouTube about this, was considered way beyond the Magnus Carlsen and in Go, in that he was like several lap down levels above anyone else, et cetera.
50:45
They never thought he could be beaten.
50:46
He was beaten in all but one match.
50:49
But then what was the results of that from this computer that could learn and dream?
50:53
And in fact the latest versions don't even see any Go games.
50:57
It just plays against itself.
50:59
Well he got better at Go and so did all the other Go players because they had strategies they could suddenly see that they'd never thought before.
51:07
This is the null hypothesis thing, right?
51:10
Humans don't like to do it.
51:11
What if a computer could do it for you?
51:12
What if you're fill some of the gaps in our psyche to extend us and make us think about things outside of the box?
51:19
That would be really interesting.
51:21
And again, this AI and these types of AI could do that appropriately.
51:24
When customized, the individual as opposed to working against them and trying to manipulate them.
51:31
Because again, that's the fundamental thing right now.
51:34
Most people, if you tell them what's the ad model ads, it's basically manipulation.
51:38
What if there was a better model?
51:40
What if we could run on that?
51:43
Then you'll get something more interesting.
51:45
Jim O'Shaughnessy: Obviously, totally agree and one of the things that I'm trying to do is translate this for people who AI is not their obsession and/or their interests and get them to understand that what we're talking about here is essentially human beings are tool makers.
52:06
We always have been and we are universal explainers and extenders as David Deutsche would say.
52:12
And this is just yet another very, very powerful tool.
52:17
But it's a tool which will allow humans to enhance their understanding of the world, the way they interact in the world.
52:25
Your comment about the Go player himself getting better, I think that is really critical.
52:32
People see the headline which is “Computer Beats Go Champion” and they miss, in my opinion, the story which is the real story, which is “Computer Makes Go Champion Much Better At Go”, right?
52:46
Emad Mostaque: Yeah, I mean it's the sameness.
52:51
It's like, there are various views of the future.
52:56
To be honest, I don't think many people tell a happy story of the future.
53:00
In talking about this now, my Twitter feed is full of "Will Vladimir Putin use nuclear weapons?"
53:06
Which is stupid, insane, but could happen because stupid insane stuff happens all the time.
53:13
There is interest rates going up to 7%, 10% inflation, all sorts of things happening right now.
53:21
Where are the stories of hope for the future?
53:24
There are some around biotech, biotech sector has crashed.
53:27
There are some around self-driving cars and things like that.
53:32
I don't think anyone really believes in that anymore.
53:33
A $100 billion has gone to the self-driving car sector. Anything to show for it?
53:39
Well minus a $100 billion.
53:41
So someone made the money somewhere.
53:43
But again, I don't think anyone's really told an abundant story of the future whereby there's a version of this that isn't controlled by a company that comes together and just makes everyone's lives better and it's there to make your life better.
53:58
The flip side of this is, one of the scariest stories of the future is AI alignment.
54:02
What if we build a system that is smarter than a human and can make itself smarter?
54:08
Will it solve all the problems of the world or would it kill us all?
54:12
Because that's the easiest way to get rid of the human condition of suffering or just annoying to it like gnats, right?
54:19
It's like one of the examples, this is a paperclip. Maximize it.
54:22
You tend to make paperclips, it gets sufficiently smart.
54:25
It turns everybody into paperclips because that's all it can do.
54:29
An AI that builds with us and works with us and is open just like Linux is open and one of the most secure, solid operating systems that runs the world infrastructure, I think it's more likely to work with us and be safe than one that is controlled by a giant company built in the dark and not shared and not open.
54:51
So I think that's an important thing too.
54:53
And I think again, this is the one shot that we have to make it so these are open versus closed because this is the other thing you've mentioned before, Jim.
55:02
If it's not open, then it will be used to survey and to manipulate.
55:10
And that's basically the panopticon.
55:11
And it's the panopticon forever because people talk about deep fakes and things like this.
55:17
It's about persuasion and it's about control.
55:21
And governments have no choice but to do that.
55:23
And big tech companies really want to do that to build their so-called moats.
55:26
So, obviously we're going to head into this mass consumerized panopticon of control both by corporates and governments, unless these are open things that everyone has a right to.
55:37
And so everyone knows what the infrastructure is and we can build appropriate counter measures to make sure this is balanced and works for the best in the future.
55:43
Jim O'Shaughnessy: And again, my entire company, O'Shaughnessy Ventures, is based on these ideas that the world is moving from scarcity to abundance.
55:54
That I want to amplify that as best I can in my little niche of the world.
55:59
And I want to empower those types of people.
56:03
So you'll see quite a few fellowships coming out of O'Shaughnessy Ventures. See quite a few.
56:08
And your own initiative for how many PhDs in AI are we aiming for now under your idea of the scholarship?
56:16
Emad Mostaque: We're going to fund 100 initial AI PhDs and give them grant writing support, paper writing support and mental health support along with honking great supercomputers.
56:25
So we're going to encourage them.
56:28
I don't care about citations, I don't care about state of the art, I don't care about peer review.
56:33
I care about original thought and exchange of ideas.
56:36
So we're going to encourage them to join our community and we'll give them cash bonuses and status based on the ideas they bring forth.
56:43
One of the things we have in our company is that we will run any open source model through our APIs and make them available to the world.
56:51
And the authors get 5% of all the revenue from the API models running on our API, even if they don't work at Stability, because they should.
57:01
They built great infrastructure for everyone. They should be rewarded. It's a royalty, right?
57:05
Jim O'Shaughnessy: Exactly.
57:06
Emad Mostaque: And another 5% goes to a pool for all the developers who are involved in Stability to allocate on the coolest stuff they find in research just to give people prizes and just to give people support.
57:21
And they know what's cool more than I ever will.
57:24
And that's how you really stimulate growth and innovation.
57:28
Jim O'Shaughnessy: Exactly.
57:30
And one of the things that made me so incredibly excited about the opportunity to invest in Stability AI and join the board and work with you and with all of the other brilliant people making this happen, is the guiding principles here are so aligned with the way, I hope you use that term a couple of times and let's just be honest.
57:51
Right now, it's still a hope.
57:53
I hope that we can redirect the way the world is going because this technology is so incredibly powerful and it literally needs to be out in the open.
58:08
But we talk about stories.
58:09
Honest, I'm a fan of science fiction and you want to entertain a broad population.
58:14
You give them Skynet, you give them terminators because that's exciting and they can suspend their disbelief for a little while.
58:23
But the honest answer here is "no."
58:26
Let's give them the hope of this incredible future that is not that far away.
58:34
And people that I've spoken to and as you know, I can't shut up, so I tell everyone about it and tell everyone how great Stability AI is and that you are the beacon for free AI and what that really means.
58:48
One of the things that I go through with them is I say, "Just imagine.
58:55
Wouldn't you love it if a few years from now you saw something on your hand and you weren't sure it looked discolored, it looked not great, you could take a picture of it with your phone and it would come back instantly, see your doctor."
59:09
Or “this doesn't match any of the particular forms of skin cancer”.
59:14
The opportunities here are literally endless.
59:18
And the other thing to kind of counter this idea, "Oh it's going to take our jobs away." You know what?
59:25
It's going to take some jobs away.
59:26
Rather than try to lie about that, let's just admit it.
59:30
But then let's go further and say, "Do you know how many jobs it's going to create?
59:34
I mean this dynamic as opposed to static.
59:39
Stasis is death and dynamic movement is life.
59:44
And open AI is going to, in my opinion, free humans from drudgery that allows them to do just a repetitive task, endlessly.
59:57
That's not what we want for the future of humanity.
59:59
And so yeah, just like when the camera was invented, right?
1:00:04
Artists went crazy, especially portrait artists.
1:00:08
"This is awful, this is horrible."
1:00:11
But this happens all the time.
1:00:15
It happens in science, it happens in commerce, it happens in art.
1:00:20
The new innovation is not interpreted properly.
1:00:25
People just set their hair on fire.
1:00:27
It happened with radio for example.
1:00:29
When radio came, everyone was talking about, "Oh my god, we're going to all die from the radio waves."
1:00:35
And you can actually, if you go back in the way back machine, find actual newspaper articles with pictures of dead birds on the ground and saying, "Radio waves?"
1:00:44
And so part of that is human OS.
1:00:46
But I think that this is going to allow us to expand human OS, so to speak, and make it understand the opportunity set that is being presented is so much better for so many people's lives that everyone should be going all in on this, right? Emad Mostaque: Yeah.
1:01:08
But it's just, "How can I participate?"
1:01:11
right is always a question.
1:01:12
And so to do that, we have to widen the number of people that can take this, extend it and run with it because they'll figure out all sorts of new things.
1:01:20
I mean, to be honest, I view this as one of the biggest moments since you almost had the printing press.
1:01:26
Because the printing press opened up communication and the ability to communicate.
1:01:28
And again, humans as prosocial animals, we build stuff by communicating, but we're all aware of the limits of communication.
1:01:37
Any time you interact with the government and you slam your head against the wall, you're like, "Oh my god, that was terrible communication."
1:01:43
These intelligent systems can augment our abilities to do bigger and better things.
1:01:48
But one of the reasons that there are legitimate fears about jobs being disrupted by this.
1:01:55
Again, the reality is you can build better stuff, but it could also not work.
1:02:01
We don't know the future to be honest.
1:02:04
The only thing that's inevitable is this technology is coming.
1:02:07
And the only question is, is it closed or open, in my opinion?
1:02:11
Is it controlled by companies that are more powerful than any government?
1:02:14
Or is it a level playing field?
1:02:17
So I think if it's the level playing field, people will figure out.
1:02:20
People always figure out a way.
1:02:22
But I think that the other side of this, it might be a one way ticket, which is the concern.
1:02:29
So I can't remember who was it that said , "Today I'm a bit dopey.
1:02:32
I've been maybe working a few too many hours."
1:02:35
Government's rule on a line from absolute violence to absolute public support because there's definition of a government as the entity that has the monopoly on political violence.
1:02:46
So we go to the government so they reduce that political violence to us.
1:02:49
In exchange for that, maybe they tax us.
1:02:51
We have some representation or maybe not.
1:02:54
If you look at the aftermath of the Arab Spring and a whole bunch of other stuff, we now have drones that are personalized bullets.
1:03:00
We have surveillance states like nothing else.
1:03:03
And now we've got very convincing AI that can convince people of just about anything.
1:03:07
It used to be that if you didn't like a government, you could raise up against it.
1:03:11
I think in a few years that will not be the case ever again.
1:03:15
There's no ability for the minority to rise against a government and change it.
1:03:20
In our political systems, we can see the strains of, again, this widening gyre where it's just left versus right or whatever.
1:03:28
And I think again, as these tools and technologies are deployed, forget about foreign influences.
1:03:31
Local influencers will use it in increasingly and I believe unethical ways.
1:03:37
But again, they'll be forced to because it's a competition.
1:03:39
So you'll get stuck again in this vertical panopticon where people might be happy.
1:03:44
But it's like the China social credit score system.
1:03:48
I think it's a good thing in some ways, but at the same time, you should give people a choice, right?
1:03:56
You don't want, I think was it was 8 million people can't use the planes or trains because they can't buy tickets because their score is so low.
1:04:03
This is a very strange set of circumstances, the gamification of life.
1:04:06
And I think it's inevitability.
1:04:07
Unless again, like I said, we give everyone the tools they need.
1:04:11
Also, I should note that just like the example of the leader and then other people following, I think our role at Stability is just to capitalize this and then for it to be independent and the AI of India or Indonesia or of artists or musicians should be run by the communities themselves.
1:04:28
It should be AI for the people by the people.
1:04:29
But it does need to spark right now and that's what we're trying to bring at scale.
1:04:33
And that's why we do so many different things.
1:04:35
We just don't do images because otherwise, the most advanced image technology, but we also do language and code and protein folding and education and all these things, which is crazy for a company that's just a couple of years old, basically. Jim O'Shaughnessy: Yeah.
1:04:49
And again, I'll just keep saying this, that honestly, I said to my wife after we met and we spent some time out here at my house with you and your family, which was delightful.
1:04:59
She was like, "So tell me everything. I'm really excited."
1:05:02
And I said, "I honestly don't think that I've ever spoken to somebody who I am more aligned with in terms of outlook, how to help solve those problems."
1:05:13
You use the term inevitable.
1:05:15
I think maybe not so strong.
1:05:17
Not inevitable if we succeed in this, not inevitable if we put AI in the hands of as many people as possible because we're going to get completely different solutions because of that diversity of cognition, the diversity of interest, the diversity of passions.
1:05:39
And one of the things that excites me so much about this is that it integrates itself with us, with we humans and the way we interact. And it improves that.
1:05:53
It's a bit of like Archimedes.
1:05:55
This is the biggest lever the world's ever been given.
1:05:59
So my friend, Matt Clifford, calls the internet the greatest variance amplifier in human history.
1:06:05
I'm calling this the greatest lever that humanity has ever been given.
1:06:10
And if that's going to lead to a future where all of these incredible things can happen or are possible, not even predicting they will happen, that lever's got to be available to everybody, in my opinion.
1:06:23
I feel, as you know, quite passionate about this.
1:06:26
And so I think that the idea of... By the way.
1:06:32
This is not a us versus them, right?
1:06:36
So one of the things that you mentioned earlier is that we've fallen into this us versus them, left versus right, what have you.
1:06:44
And then the in group is othering the other group, right?
1:06:48
When you ‘other’ other human beings, in other words make them to your way of talking or thinking, not really human, you get atrocities, you get genocides, you get horrible, horrible things happening.
1:07:03
And as much as we, I, you everyone involved here can do to ameliorate this situation, I just don't know how you're opposed to that.
1:07:12
The other thing I would say is, "Listen, there's going to be open and closed AI systems," right?
1:07:18
You actually gave the example of Coinbase versus Uniswap, right?
1:07:23
One's open, one's closed, there's room for both.
1:07:25
The open one is making more money right now, and so that's the other thing that really intrigued me when you and I were talking about this.
1:07:33
You made a comment that really stuck with me.
1:07:36
And because I love thinking about systems, I love thinking about, the underlying system pretty much determines the range of outcomes you're going to get, right?
1:07:46
And so the US, just very lucky to be founded during the age of reason, we have a constitution with a First Amendment right to free speech, we have still mostly intact the rule of law, and all of a sudden you get that underlying foundation to the system and wonderful things happen.
1:08:07
But you made the comment that really I thought about for a long time and then I saw the truth of it, basically, closed AI is also a bad business model.
1:08:17
Talk a little bit about that. Emad Mostaque: Yeah.
1:08:20
So I mean I think if you look at closed AIs, it's like where are you making your bets?
1:08:25
I have better content, I have faster super computers, or I have smartest of people. I have secret sauce.
1:08:35
These models are complex, but they're not hideously complex compared to the more specific models of the past.
1:08:41
The similar architectures can go across modalities.
1:08:44
So video and audio and other things actually use similar internal technology on super computers that were very rare and becoming less rare.
1:08:51
You still need honking great ones.
1:08:53
So our ESRA1 cluster is 4,100.
1:08:56
The fastest supercomputer in the UK is 640.
1:08:59
Ours would be 10th in the fastest top 10 supercomputers in the world publicly, which is bit insane.
1:09:06
It's like eight times faster than NASA.
1:09:08
And again, to give an example of that.
1:09:11
That's what we need to create these lovely old models.
1:09:13
But super compute, there's a race going on so there's more and more coming on. So, that won't be a gap.
1:09:20
Data, we've released the best open source data sets that even better than private data set models that the biggest companies use, in language, and image, and soon video and audio and others. So, that's not an edge.
1:09:30
And then, finally there's people.
1:09:32
I mean look, it's difficult to train these models, but if you look at stability now we have a team of over 100 people, one PhD and about 20 people doing their PhDs that have created competing models with the biggest guys because we're full of really great technologists and now we're going to get 100 PhDs and more.
1:09:51
We'll train more people and give away all of our information how to train these models.
1:09:56
So you'll have more and more people train these models.
1:09:58
We have people train the models from the age of 19 to 62 now, and we're doing things like our Faster AI course, which is one of the best courses for anyone who wants, is a normal developer, wants to be an AI developer to learn how to train these models.
1:10:10
So we're going to make us hundreds of thousands of people can train these models.
1:10:14
There'll be more and more super computers available to do it, and we are making it so that it becomes cheaper and cheaper to run.
1:10:21
And then finally, we are releasing source data sets.
1:10:23
So where is your edge if you're trying to keep closed source models when you have an entity like us competing? It's difficult.
1:10:29
Jim O'Shaughnessy: Yeah, I think difficult is an understatement because eventually, like if I had a company that wanted to say “mine”, right?
1:10:41
And it was an AI company, and I just kept it close, and I wanted to maximize it for O'Shaughnessy Ventures revenues, eventually somebody is going to be disaffected and they're going to release that model and then everything essentially has to go open sourced from there.
1:10:59
So- Emad Mostaque: On 80% is good model because that one's open, and people can take it and extend it.
1:11:06
They pay you for scaling it and customizing it, but 80% is good enough.
1:11:11
Jim O'Shaughnessy: Totally agree.
1:11:16
Again- Emad Mostaque: I'd like to add one final thing to that, which is the technology diffusion curve.
1:11:20
When technology is not accessible, it doesn't diffuse.
1:11:22
But by making technology accessible, we've created brand new industries and uses of the technology you wouldn't foresee, can't do that with a closed model.
1:11:30
Even when we go into companies now, they've already done the prototyping for us because the internal developers have used stable diffusion to show to their bosses.
1:11:38
So that part of the whole sales process is really done, which is always nice. Jim O'Shaughnessy: Yeah.
1:11:43
And on that note, I kept the piece of paper that Dan did the curve during our offsite.
1:11:48
And you're absolutely right, the tech is not going to be adopted, if people can't get access to it.
1:11:56
It's pretty straightforward and pretty simple.
1:11:58
And I love the idea of the ability to do a PowerPoint but not a PowerPoint.
1:12:05
Because I hate PowerPoint, and I very rarely use it because I always say I don't want to put myself in a box, because I look at PowerPoint as engendering a certain way of thinking and the way it of thinking or not thinking, more of compliance.
1:12:21
It is literally putting things in boxes, which I am incredibly opposed to as you know.
1:12:28
And so, one of the things that happens here is the box goes away, right?
1:12:34
Because people can do this stuff on the fly or will be able to shortly.
1:12:39
And as we add to this creative pile, as it were, it just keeps getting bigger, and bigger, and bigger, and better and better.
1:12:46
And it creates a virtuous flywheel as opposed to the negative compounding flywheel of scarcity and, "No, it's mine" and the precautionary principle and all these things.
1:12:57
But I want to underline what we've talked about earlier.
1:13:01
I'm also passionate about dealing with the problems that will inevitably result.
1:13:06
I'm what I call a rational optimist.
1:13:08
It's not that we don't expect problems, of course we do.
1:13:11
Yes, there will be problems, they'll be new and better problems, but we will deal with them as they present themselves.
1:13:18
It's like, I can't remember whose quote it is, but I love the quote.
1:13:21
It's like, man invented fire.
1:13:23
And literally fire is probably responsible for us having this Zoom right now because it made us cook our meat, made our brains bigger and whoa, yay fire.
1:13:34
And we often called this Promethean fire.
1:13:38
But fire can also be very damaging, and therefore we came up with fire alarms, fire departments, fire warnings, all of these things.
1:13:48
And so, the iterative process of building the immune system so that these things can't just be used just for horrible evil is part of this process, right?
1:13:58
Emad Mostaque: This is the thing, like I said, it's always a balance.
1:14:03
Any powerful technology can be used for good or bad. That's the reality.
1:14:08
And this is one of the most powerful technologies that we've ever created because this technology is us and extends us.
1:14:14
So it learns from the internet and it learns principles again. How crazy is that? It pays attention.
1:14:20
So, can you use it to control or can you use it to create, I think as a question here and then if people are using it to try and manipulate, by understanding it more and getting it out there, that's how you build counter measures.
1:14:34
Otherwise your enemies will always have more degrees of freedom than you if you're ethical.
1:14:41
And so, I think that's not properly appreciated here, because again, they already have access to this technology and we see that; the bots that will come to the upcoming election cycle will be nothing like the bots that we've seen before, as an example of a negative edge case.
1:14:54
I think also what we are doing now is because they're so powerful, we're making decision making under uncertainty, not under risk.
1:15:01
So you try to minimize for maximum regret of what could go wrong.
1:15:05
But that leads to a very fundamentalist or orthodox positioning, which is that if an act in itself leads to an impermissible or bad outcome, it in itself is bad.
1:15:14
Like releasing models open source because they could be used for something bad.
1:15:17
That's the same logic as the Saudi clerics forbidding women to drive.
1:15:20
Because clearly, if they drive they'll go and fornicate because that's how women are.
1:15:24
It's called blocking the means or [foreign language].
1:15:27
Other religions have very similar things on the orthodox side.
1:15:30
Not saying it's necessarily wrong, it's a way of looking at the world.
1:15:34
It's not a way that I'd agree with, and I don't want to have orthodoxy put upon me.
1:15:38
When I look at the other side and look at the good that it can do.
1:15:42
Right now, anyone in the world can create anything they can imagine for pennies.
1:15:47
And I think that's awesome.
1:15:51
And it wasn't there a few months ago.
1:15:52
Right now it hasn't technologically diffused yet, it's being incorporated into platforms with hundreds of millions by our partners.
1:15:58
You'll soon see this, the ability to create anything you can imagine that is an advancing humanity.
1:16:04
Because before it was gate kept, not necessarily deliberately, just because you needed to have technological competency to do that.
1:16:11
Just like you needed to hire a photographer to take a photo or paint a portrait before.
1:16:16
Now it's becoming that you can actualize and communicate better than you ever did before. And that's awesome.
1:16:21
Jim O'Shaughnessy: Totally agree.
1:16:24
And I saw, I can't remember the name, but an AI PhD made the comment that by releasing the model with weights on August 22nd, that you had advanced you and Stability and the community that built the model had advanced AI by 30 years.
1:16:41
And I remember reading it for the first time and thinking, "Yeah," putting it into that context was, I thought, very interesting.
1:16:49
And the other one that I want to underline is this whole, there are larger things at play here guys and Emad speaks about them often.
1:16:58
And I just want to highlight it.
1:17:01
The bad guys have this tech right now.
1:17:04
And I believe that if we are going to be able to limit the damage that they can do open wins because of, again, cognitive diversity.
1:17:20
You talked about the social credit score, obviously, I enjoy Black Mirror, that presents kind of a dystopic future, but there is an episode of Black Mirror about the social credit score and some entrepreneurs tried to do it here in America and the reaction actually heartened me.
1:17:39
They were hounded to literally shut it down because people were saying, "Yeah, that's not the future we want."
1:17:46
But if you want the most advanced learning, if you want the most advanced models, if you want the biggest tool set to combat the evil doers who also have access to some of this tech, it's got to be open.
1:17:59
Because open is going to find edge cases that a few people in a room are never going to find, because there's going to be some person out there incredibly passionate about X, whatever X might happen to be and they'll be contributing you. Do you agree?
1:18:14
Emad Mostaque: I agree with that.
1:18:16
And I also, it would be like if someone could come and hack your brain at any time, which is basically what we're saying the bad guys can do now, where this technology become more and more advanced, how much would you pay to protect it?
1:18:30
Well, the big companies can pay, but you and I, well maybe you and I can, we're lucky, we're very fortunate individuals, but the vast majority of the world.
1:18:41
So isn't the diffusion of the defense if you believe this is something need to be defended against, you have to push the defense out to the how to make it open and accessible.
1:18:49
Because otherwise only a few people can afford the defense or have access to that defense.
1:18:53
Jim O'Shaughnessy: I totally agree.
1:18:55
Emad Mostaque: That's super important.
1:18:57
I think this is the other thing, what you saw is, so for example, someone took Stable Diffusion or image generating thing and combined it with the technological called dream booth that allows for pictures to be put into anything.
1:19:08
It started off requiring 48 gigabytes of VRAM.
1:19:11
So that's more than any consumer graphics card.
1:19:13
Within a week, it was down to 11 because the community just worked to it.
1:19:17
So the community also works on making things accessible when it's open source.
1:19:20
Whereas if you're a big company, you don't need to think about efficiency, right?
1:19:23
Have all the big chips and doodads that you need, whereas the community really cares about accessibility.
1:19:30
Jim O'Shaughnessy: Totally.
1:19:31
Emad Mostaque: Necessity is the mother invention, I suppose is the phrase.
1:19:35
Jim O'Shaughnessy: It certainly is.
1:19:37
One of the things I joked about when people were asking me about this obsession of mine.
1:19:42
I've been a journal keeper since I was 18.
1:19:44
And so I joke to somebody like, "Why are you interested in this?"
1:19:48
I said, "Well I'm going to take the 40," however many years.
1:19:52
Well gosh, we're getting even closer to 45 years, "Of these journals that I've written out.
1:19:58
I'm going to have them transferred over to machine readable text so the AI can explain myself to me."
1:20:04
Emad Mostaque: And it's just within reach, right?
1:20:09
Jim O'Shaughnessy: It really is. It really is.
1:20:12
And it's absolutely one of the most exciting opportunities that I've certainly seen in my lifetime. And I've seen a lot.
1:20:19
And I was around for the dot com craves and I was around for Web1, Web2, Web3, I think this is Web5.
1:20:29
And I think that as it diffuses through or diffuses throughout communities that are not strictly tech communities, that are broader communities, the results are going to be stunning and very hope inspiring.
1:20:42
Emad Mostaque: Well yeah, I think if- Jim O'Shaughnessy: Please.
1:20:45
Emad Mostaque: Yeah, I think Jack Dorsey's already taken Web5 actually, so we have to change it from that something else.
1:20:51
Jim O'Shaughnessy: Let's do Web6 then.
1:20:53
But it's like you know the old joke about the seven minute abs, and they didn't see six minute abs coming.
1:21:04
Well this has been amazing.
1:21:07
I cannot tell you how excited I am to be involved.
1:21:10
I cannot tell you how highly I think of you, and the team and the mission.
1:21:15
I think that, as people get to understand and know us better, we are going to see a huge number of people come to a Schelling point because it matters.
1:21:25
And because this innovation can go one of two ways, I think.
1:21:31
And the open way leads to an abundance mentality, leads to improving human outcomes, leads to improving children's rights. I love that notion.
1:21:43
And I think leads to us leveling up as a species.
1:21:47
So at the end of this podcast, I always ask my guests, and I've actually, last night I was thinking about what yours were going to be.
1:21:54
We are going to make you the Emperor of the World.
1:21:57
So you now have an additional title to your Emperor of GPUs.
1:22:03
You're Emperor of the World, but we are precluding you from killing anyone, from putting anyone in a reeducation camp, but we are giving you a magic microphone that you can speak two things into and incept the entire human population, completely nonviolently.
1:22:23
They're going to wake up the next morning and the two ideas that you're going to give me, they're going to think are their own idea and they're going to act on it.
1:22:32
What two would you whisper into that magic mic?
1:22:34
Emad Mostaque: Oh, that's a tough one.
1:22:35
I think the first one would probably be to whisper that they have agency.
1:22:41
People don't believe that they have agency, but the reality is that everyone has a potential to do just about anything if they put their mind to it and they believe that they can.
1:22:51
Doesn't mean that you can literally go from zero to a million, but you can definitely have control of your own thing because you are the main actor in your play.
1:22:58
And so I think that'd probably be one.
1:22:59
And I think the other thing is, like I said, you think about this a lot.
1:23:01
It is when you think about rights, again, frame it in the rights of the children, as it were.
1:23:08
When you're grappling with some of these bigger things.
1:23:09
I'd like humanity to think about that in general because forget about future generations.
1:23:14
They are the future generation here.
1:23:16
I don't think we spend enough time thinking about their rights and how to deal with them, because then things like climate change, food poverty, a lot of these other things really become a lot simpler, a lot more direct.
1:23:29
And I think we need to get on that same page, globally.
1:23:33
Jim O'Shaughnessy: I love both of them and I have been on the agency idea forever and so I love that one. I love them both.
1:23:43
But if people could just understand that they do have agency, wonderful things happen when people understand that they have agency and horrible things happen when they feel they don't.
1:23:53
Learned helplessness is a real problem.
1:23:55
And learned helplessness can only work if people believe that they have no agency. So I love both of those.
1:24:03
And the second one reminds me of, and I'm going to get it wrong because I'm doing it from memory.
1:24:08
Hopefully, when I have the AI on my phone, they'll do it for me.
1:24:13
But the African proverb that is, " We did not inherit this world from our ancestors for us.
1:24:20
We essentially are the custodians of this world for our children."
1:24:24
And I think that that reframing is a vastly better way to think about things as we plot our path forward.
1:24:31
Well, Emad I am incredibly excited about this opportunity. I am in awe of you.
1:24:38
I am delighted to be part of this grand, grand journey and thank you.
1:24:43
So I look forward to seeing you later this week.
1:24:47
And we're going out to San Francisco to have a bit of fun out there. And then Ad Astra.
1:24:53
Emad Mostaque: Ad Astra indeed.
1:24:55
All right, take care buddy. Thank you so much.