0:00
Mark, last time we spoke, you called me. It was on the weekend.
Search this video
0:02
Well, you called me through the glasses, which was the interesting part.
0:06
>> Was it loud in the background?
0:07
>> You were You were going to fish.
0:07
I wasn't going to say it, but yeah, it was on the weekend.
0:11
>> The noise cancellation actually is pretty good. >> Oh, it's great. >> Yeah, it's great. >> Yeah. I I mean, >> yeah.
0:15
Um, but you wanted to talk about this essay manifesto.
0:17
I don't know what you call it.
0:20
Manifesto, we'll say, >> uh, that you published recently and and it's long.
0:24
I mean, I'll caveat, but there's a lot in it.
0:25
And I wanted to start there because there's a lot of big ideas in there and they'll connect to kind of the main thing we're talking about today. >> Yeah.
0:32
>> Um I'm curious like why write that because it's long. There's a lot in there. >> Well, yeah.
0:37
Well, I I feel like if you're going to invest so much in building AI, uh, then it's important that people understand what your lab stands for and what your values are and and basically AI has so many opportunities, but there are also all these real risks.
0:55
So, I think it's very important that everyone who's working on it has a wellthoughtout theory for how the work that they're going to do is going to lead to a positive future.
1:04
You know, it's interesting because I mean the different labs have have some different philosophies on this and there's, you know, a lot of things that I think have become conventional wisdom in the industry that I just strongly disagree with.
1:15
And you know, my my view on this is that the path to have a positive future for everyone is to make sure that we distribute the technology as widely as possible.
1:25
And that's based on three major principles that that we have.
1:30
One is that empowering people is the source of prosperity in the world.
1:35
and it has been throughout history.
1:35
Two is that the primary purpose for AI is going to be invention of new things not automation.
1:44
And then the third principle is that the foundation for safety for the future is basically establishing the right checks and balances and balance of power rather than restricting access.
1:57
And I think that that's these are all things that I think oddly are are kind of very different from I think a lot of the conventional wisdom um especially in Silicon Valley.
2:06
A lot of people think hey this technology is very powerful.
2:08
We must restrict it so that way not that many people have access to it.
2:11
I personally am much more worried about a small number of labs or people having control of of something that is so capable.
2:20
Um, and I think that the throughout history, what we found is that when you put power in people's hands, most advances don't come from the incumbents or the establishment.
2:29
They come from people on the on the periphery whose ideas aren't taken seriously.
2:33
But when they get enough tools to um to to basically be able to prove out what they're working on, that ends up being very powerful.
2:41
In western society, the way that we've established governance and um and basically having a well- balanced society is through a set of checks and balances, right?
2:50
And this balance of power.
2:52
Um it's very ingrained in in kind of our society that you don't want to have, you know, one lab or two labs having access to a thing.
3:00
for some of the most recent concerns that have come up like some of these cyber security concerns.
3:05
I think the the best antidote to someone having an AI that could potentially hack into systems is having everyone have access to an AI so they can harden their own systems first.
3:16
And that that's kind of been the history of cyber security over the past several decades is that you know open source software um because people can can see it and can scrutinize it sort of counterintuitively by putting it in people's hands you end up with a more secure and more stable environment.
3:32
So that's what I believe um and and that's what I think is the path to to a positive future is basically distributing the technology that has a bunch of different implications for what we're going to do.
3:44
I mean obviously we want to build leading um AI models which which uh we're doing and Muse Spark 1.
3:48
3 which we just released is it's um it's advanced but then you know it's actually the latest model of a relatively smaller pre-train that we did the the internal code name avocado >> and um >> oh we're going to get into the code >> oh yeah we'll get into that and and we have watermelon coming soon so that's going to be a big deal.
4:09
going to be a big deal. So obviously leading models probably the biggest personification of um you know if you will or kind of um implementation of this vision is um the muse personal agent that we're that we're rolling out
4:23
um that basically the idea there is give every person in the world a very capable personal agent that can understand their goals and can just work on their behalf 247 and then um an important part of this is also just getting the technology in people's hands. We're we're very kind
4:35
We're we're very kind of strong proponents of open source and and making sure that that the opportunities that I think are going to be massive here are not just limited to a few to a few people or companies.
4:45
This episode is brought to you by Mercury AI native banking that's loved by more than 300,000 entrepreneurs including me. Visit mercury. com to learn more.
4:54
Mercury is a fintech, not a bank.
4:56
Check the show notes for details.
4:58
Thanks also to Granola, the AI notepad for people in backtoback meetings.
5:02
It works everywhere you do and lets you focus on what matters. Try it at granola.
5:06
ai/sources and use the code sources for three months off.
5:11
This episode is also brought to you by Jira by Atlassian where teams and agents get the context, coordination, and control to move work forward. Try it free at jira. com. That's jir. com.
5:26
>> I have a bunch of questions about the Muse Agent, but staying big picture for a second because you said a lot of things there.
5:31
Is open-source the counter to the trend you're seeing that you described that you're worried about?
5:36
Is that the main way practically that you counter that or is it regulation? Is it both?
5:43
Like >> No, I actually think probably the most important thing is actually just getting the technology in individuals hands.
5:47
So I actually think things like the Muse personal agent um are perhaps even more important.
5:55
I mean, I think what you're starting to see are some of the labs are building training more advanced models and then not even releasing them. Right. >> Right.
6:02
So, I think that that is quite dangerous in the sense that, you know, basically when you have scrutiny on something, when you put a a system out there, first of all, if you put it in a lot of people's hands, you get the checks and balances, you get broad-based prosperity, which I think is important, right, for society.
6:17
We can't just have like one or two labs get incredibly valuable.
6:21
Um, you know, I think you you want to make it so that billions of people can basically have prosperity in their own lives, whether it's creating small businesses, being more successful in their careers, um, being more productive and managing their homes, saving money in a lot of ways, um, kind of advancing their health.
6:36
Uh, so I think you you want the benefits to be very broad-based. So that's one piece.
6:42
In terms of the competition, I do think that having multiple labs is helpful and I think open source is quite helpful for that.
6:49
So I think open source is an important part of it.
6:51
The nature of open source is there's a whole community of people who do it.
6:54
So I'm not saying that we're going to be the one company that does it.
6:56
I'm also not like a zealot about this from the perspective it's, you know, it's not that everything we do is open source either, right?
7:01
We we release some open models. We do some closed work.
7:05
I think it's important if you're building a for-profit company that you can build some advanced things and you don't necessarily need to share every single thing with the world.
7:11
But I think in general uh supporting a robust open source ecosystem is going to be key to maintaining competition and and maintaining kind of transparency and understandability of where the technology is going in a way that I think is actually going to be incredibly important for safety.
7:30
Um you know if we have a world where there's just like a small number of of really capable models um I know just it's it's interesting right? Right?
7:38
I mean if you look at some of the cyber stuff for example um the instinct which I mean on the one hand I I can understand the instinct of like all right we have this capable cyber model let's release it that only you know whatever it is the top hundred institutions get it but you I think part of the issue is that there's more than 100 important institutions in the world.
7:56
So you if you look at things like the hugging face incident that happened you know hugging face is maybe not one of the biggest hundred institutions in the world but it matters right it's like an important thing that people rely on.
8:05
And so what did they do when they started detecting that there was this intrusion is they turned to open source models because they didn't have access to some of the closed ones that were causing the issues.
8:16
So um I think that having a robust open-source ecosystem is one important part of having kind of a safe and stable future.
8:25
But to me the the most important thing is just making sure that we distribute the technology widely rather than hoarding it in a small number of people's hands.
8:33
And I think there's also this ongoing debate in Silicon Valley about why people feel so negatively about AI.
8:40
I mean, you talk about this in your in your recent letter, uh, addressing these concerns or trying to, but the I mean, the sentiment on AI and data centers in particular, it's so negative.
8:50
And it sounds like maybe an essence of your argument, correct me if I'm wrong, is if we diffuse this technology more, if we enable more people to access the things that are right now gated >> by some of the top labs, maybe that addresses this kind of I think people feel maybe disenfranchised by what's happening in AI.
9:08
Is that what you're getting at?
9:10
>> Well, there are many layers to it.
9:10
It's I mean, I think you there's so many parts of this is why the essay was so long, right?
9:16
Like 15 pages because I mean we want to get through there are lots of different questions.
9:19
I mean, people have questions about jobs in the economy.
9:22
They have questions about data centers and their local communities and the kind of economic and environmental impacts of that.
9:28
There are questions about how people might misuse AI, right?
9:32
I mean, there's the cyber questions.
9:33
There's bio risks that are coming up.
9:35
There are questions about how we maintain a free society.
9:37
There are questions about American leadership.
9:39
Um, there are questions about maintaining control over the technology as it gets to be increasingly capable.
9:44
So, these are all important.
9:46
So it's it's important not to um you know just talk about this in generalities at the at a high level because I think each of these has some different nuances.
9:54
has some different nuances. But in general, I think one of the things that they all have in common is that if you create broad-based prosperity, um, and I think that one of the better ways to do that is by ensuring that there's the
10:06
right balance of power around who has access to the technology and generally making sure that the greatest balance goes towards the kind of general population of people as opposed to kind of any kind of I don't know insider stakeholder or whatever you want to call it. Um, I think that that ends up being
10:19
Um, I think that that ends up being very important.
10:22
So if you look at the data centers, I what we've actually found is that when a company like Meta goes into a community and makes a commitment that we're going to invest there for decades, right, which is really what we're doing when we're building up a data center, we're able to make it so that it's very good for the community.
10:40
Um I mean the the the kind of the tax revenue that they bring uh from that.
10:44
I mean in in Louisiana, we had this example where like the tax revenue funded these $50,000 bonuses for teachers in the community.
10:50
we we bring a lot of jobs.
10:53
We we invest in the in the local community a lot.
10:55
Um that I think can be good.
10:58
I think that there's there's also a lot of speculation, right, where there are companies that aren't necessarily planning on running a data center for decades.
11:04
They're just, you know, trying to find a plot and um and then trying to sell it to one of the big labs and they don't really care as much about the local community and um and they're not invested for the long term.
11:16
So if they don't care to, you know, focus on making it work for the local community, then of course people are going to get upset.
11:22
So I that's one of the things that can be um difficult when you have these kind of speculative um >> I don't I don't even know if I'd call it a bubble because that implies that it's overvalued or something, but but there's certainly a boom, right?
11:36
So it's so so you have that I mean that leads to some of these like short-term thinking incentives that don't necessarily lead towards helping every stakeholder which I think is kind of what you need to do to make this sustainable over the long term.
11:49
And at the end of the day I mean if we're creating a technology that like if it doesn't create jobs or doesn't create broad-based prosperity or the infrastructure that we're building doesn't help local communities that's not going to be allowed to continue.
12:00
So it's like it has to of course you have to design it in a way that um can be helpful to people in all these ways which is also part of the reason why like for people who are you know skeptical or have so much doom about the whole thing I'm I mean one of the the things that I basically think is that if we don't end up building it in a way that's positive it just effectively won't be able to happen.
12:22
Um, so I I kind of think like the actually establishing the right checks and balances um and distributing the the benefits of this widely is sort of a precondition for being able to scale in the way that I think um would be best for society over time.
12:39
>> I haven't heard another tech in your position talk about data centers that way, like the long-term investment of it.
12:46
Is this something is this how you've always thought about it?
12:48
Is this something you feel like there's more clarity that's been brought to it for you recently?
12:52
Like have you Yeah, >> I mean I think that there's been all this anti-data center sentiment that you're talking about.
12:57
So we've dug into it because what we're trying to understand is >> okay like there isn't as much of that around our project. So why is that?
13:04
And then so we ask a bunch of people.
13:07
It's like well it looks like there's a pretty big dichotomy between these speculators and the companies that are focused on it for the long term.
13:15
And that kind of makes sense when you think about it.
13:17
So, I mean, one one thing that we're doing, I mean, there's this America's Workforce Academy project that we we did, which is basically, okay, we're going to build all these data centers.
13:24
We're going to be doing this for a while.
13:26
There isn't the volume of skilled trades people that you need to create this.
13:31
So, we need more like fiber technicians and electricians and like advanced carpentry and and all of these things.
13:37
And there aren't enough people to do this.
13:39
We need hundreds of thousands, you know, maybe maybe millions of more people who can do this.
13:44
And um people aren't trained to do that right So we created this training program um to to effectively do that where we guarantee people who get through the training program a job at um at at a place that is working on building infrastructure for meta.
14:02
>> And why did we do this?
14:02
I mean it's like it's not really philanthropy, right?
14:06
It's like we need those people to be skilled and have those skills.
14:08
So it's just it's a win-win.
14:10
It's an investment that I think makes sense if you're in it for decades.
14:14
um but not necessarily something that you would do if you were building out one site with the intent of flipping it to a different company.
14:20
So I think that a lot of problems in the world do just naturally get solved by incentive alignment when you think about them over the long term.
14:31
Um, so I think that that ends up being an important part of this.
14:34
And I guess part of the way that I think about this is that I just think that there's no way that it's going to be kind of permitted for there to be a small number of labs that control such a important um and capable technology and accumulate a lot of wealth to themselves.
14:48
I think it's like this has to be a broad-based thing in order for it to kind of be able to work.
14:54
It has to work technologically, but it also has to work um kind of socially.
14:58
And and I think that those those pieces kind of have to have to go hand in hand. >> I agree.
15:02
Well, now we're landing towards Amuse.
15:05
Before we get there, you also wrote a year ago your personal super intelligence essay. Shorter one.
15:11
>> Uh yeah, that was page. Yeah.
15:13
>> I did write a one-page version of the future, too. Yeah.
15:15
Well, I published it in the Wall Street Journal as that was kind of >> I just read the long one. Okay. >> Yeah.
15:22
So the onepage version and there's the 15page version.
15:23
Well, so this one you did about a year ago, personal super intelligence, I think a lot of people in my world when they saw you write that was like, "Oh, wow.
15:30
Why is Mark writing this?"
15:32
Like, what is the thing he's seeing on the other side of this?
15:34
And I think it, correct me if I'm wrong, it might be Muse, like what we're going to talk about, right?
15:40
What you guys are releasing now. >> Um, this is it.
15:42
So, how did you come to that realization that that's what this is the next chapter for Meta? >> It's interesting.
15:48
You know, we've never really just thought about ourselves as a social media company.
15:53
We've definitely thought about ourselves as a company about connecting people and about empowering people, but I think a lot of the values that led us to build the things that we built for the first, you know, 15, 20 years of the company around putting technology and power in individuals hands um believing that people should be able to decide for themselves what is important in their lives.
16:14
And we've gone through a lot of social debates around this, right?
16:16
A lot of the the the debates around content moderation and things like this have kind of been around this question of like should people be allowed to kind of decide and communicate for themselves what matters in their own life.
16:27
And I think through that experience, it has sort of sharpened my belief that a lot of progress throughout history and through this technological age comes from empowering individuals and that people really do know best about what matters in their own lives.
16:47
So because of that, I have somewhat of a of an allergy whenever I hear people talk about, oh, like we should just have a small number of experts allocate what AI does to like big problems.
16:59
Why should it do like like these things that people care about in their lives?
17:03
Well, people have a balance of things they care about.
17:06
People care about health.
17:06
They care about having a better life, but they also care about their relationships and like showing up for their friends and family and um they care about culture.
17:16
culture. um you know people care about things that may not to you know a scientist or an engineer in the industry feel like the biggest problems but I don't know if you ask like billions of people what they care about if like you know I think what people's aggregate answers to that question is kind of is
17:33
what the most important things are to be worked on >> and um so I've always just kind of believed that when you build this super intelligence >> there's this question of who decides what it's going to focus on and I think that people should be able to direct it towards what matters in their own lives. It shouldn't just be directed by you
17:52
It shouldn't just be directed by you know some like so-called experts um sitting at a small number of labs.
17:56
So this gets back to it's like it's this is basically the foundation of this overall philosophy which is that the way to have a positive future is to empower people to put the technology in their hands and let people decide for themselves what matters and how they want to use it.
18:13
And I think that when people do that, it first of all will prioritize some things that that are different, right?
18:20
It's um you know, like maybe it'll prioritize um health issues, but maybe instead of prioritizing, you know, it's like the most common things, which is kind of what the the kind of pharma and biotech industry like at large prioritizes today.
18:35
Now, it's like there's a very long tale of rare diseases and conditions that people have.
18:38
So, okay, if you're if you have if you have a rare condition, like you're probably going to want your personal AI to focus on that, not like just something else just because it happens to be the most common thing.
18:48
So, I think for example, rare diseases, I think, are under disproportionately underinvested in.
18:52
Um, >> you're doing a lot of investment with your foundation.
18:56
>> We're doing that at Biohub, but but I I mean, and that's partially informed some of my views here >> that I think like yeah, like you want to put the power in individuals hands um to determine what matters for them.
19:06
But a lot of this is also like it's not necessarily the things that people would say are like the big social problems.
19:14
Like a lot of it for me, you know, when I'm using my Muse Agent, I just, you know, I kind of want to it to help me be like a better father and, you know, a better husband and show up better for my friends and, you know, be able to um help me connect with people.
19:28
And that I think is also partially a through line between the work that we've done at Meta so far and this is I think we're the company that I think just disproportionately cares about and believes that there's social value in um in helping people connect with the people around them.
19:43
So I don't know it's like what are the first things that I that I kind of set up my my own agent to do?
19:48
It's like all right my three-year-old daughter likes baking.
19:49
I don't know anything about baking but like that's like a a fun project that we can do.
19:55
can do. So I basically asked him I'm like, "All right, set up so that every weekend um you know like we have a baking project that is um that is uh kind of reasonable for a three-year-old and an adult who knows nothing about baking and um and use Instacart or
20:13
whatever to go get all the ingredients like just figure out what makes sense and make sure everything is ready so that way like when I show up on, you know, on Sunday with u with my daughter, we can like go make this thing and Um, and then I tell it afterwards. I'm like,
20:25
I'm like, "How how did it go?"
20:26
"Okay, that one was too hard."
20:28
Turns out cake pops really difficult. Surprisingly difficult.
20:30
Um, >> nothing about baking.
20:34
>> Yeah, I didn't either.
20:34
I I know I know something. Yeah, cake.
20:36
No, no, don't start with cake pops.
20:38
That's That's the problem. >> Um, yeah.
20:40
No, there's a lot of things in baking that are pretty simple.
20:44
>> It turns out cake pops is not one of them. But, um, yeah.
20:45
Well, you know, what can I tell you?
20:48
>> Um, >> thank you, Muse. >> Yeah. Yeah. Thanks.
20:50
Um, so yeah, it kind of like so it kind of updates that um and and helps with that.
20:56
Um, you know, my you know, my older daughter has kind of gotten into climbing mountains and some of them you need permits for.
21:02
So I have it basically like sit and get the permits when they become available so we can climb mountains.
21:09
>> And um I was just like pretty neat.
21:11
Okay, so then like we do that and it basically tells me it's like all right, I was able to get a permit for this day.
21:16
>> And I was like all right, well I guess I'm taking that day off from work to go climb a mountain with my daughter.
21:18
So, it's kind of cool, >> right?
21:21
It's So, it does that, you know, but it also helps keep me healthy.
21:24
It helps me with my training.
21:27
>> Um, you I put cameras up in my in my MMA gym and I tell it to watch the cameras and send me feedback and it's like it's pretty fun. You know, it's good. >> It's good feedback.
21:37
>> Um, sometimes, you know, it's I mean, it's um sometimes it's funny feedback.
21:40
It like it like finds me in >> it's just like it looks like you really gave up and and I was like, "Yeah, I did.
21:46
I was really tired right there."
21:47
It's like, why is that the thing that you're pointing out to me? Um, but no, it's good.
21:50
Um, and it's funny like the coaches like laugh about it.
21:53
They're like, yeah, no, this is like >> this is what we didn't feel like we could tell you, but your agent's telling you. >> Yeah.
21:59
Well, it's um Yeah, that's good.
22:01
>> I didn't think about this until hearing you talk about it, but you you have people that could obviously do all this for you, but like how do you use something like Amuse Agent to like really test the limits of like how it can be helpful as an assistant, right? Yeah.
22:14
like like are you were you pushing it like how have you pushed it in a way that the team is like oh okay we got to got to fix this or I'm sure >> part of what's interesting about it >> is that everyone just has like such different things that they want to do with it.
22:29
So like in the early beta period, we handed it to a bunch of people and like it's like I gave it to someone and like within a day they're like using it to help run their home school and then like it's like okay wow you just like started this within a day and then like another person within a day or within 12 hours they're like I just planned a trip.
22:47
Um yeah and it's like it it just like planned this whole thing for me. Yeah. Yeah.
22:51
I mean, someone else I know who's like generally pretty skeptical about technology, I I gave it to her and then she was um she didn't say anything for like a few days.
23:00
Then she texted me and was like, "So, when you do like the general release, do I get to keep my Muse agent or you going to reset it?"
23:06
I was like, "All right, this is good.
23:07
This is Yeah, I think it's like I think this is working well."
23:10
>> Everyone I know who's been on the beta has very, you know, high things to say about it. High praise.
23:14
Um >> but people do different things with it. >> Yeah.
23:16
>> Yeah. And I think we should just also say what it is more plainly for people so they understand because I think people think of AI as like you know met AI or chatbt it's like back and forth prompting the the real unlock here and this is happening in the industry more broadly whether it's Grockbot town
23:33
instinct I mean there's many products doing this but it's it's adding a virtual machine behind the scenes where like the agent can control a computer for you and log in and do things and that's a that's a huge change I think for people who only know AI for >> and it's it's longived. So it's um so
23:48
So it's um so basically instead of the model with like Meta AI or ChatGpt or Gemini or whatever you use where you send one prompt and then it gives an answer in this case what you basically do is you give it projects or you give it goals >> and then it just works and it works 24/7 and it like doesn't stop until it's helped achieve the goals.
24:10
>> Team was telling me it studies overnight is what you guys call it.
24:12
Oh yeah, it studies it like it kind of consolidates its reflections into into memory.
24:15
It it basically just works on projects and it can also suggest new projects.
24:21
So yeah, I mean I was like I like play the computer game Civilization with one of my daughters and I like I was like, "Hey, do you want to make like a strategy guide for her?"
24:31
I was like, "Yeah, sure."
24:33
So I was like, "Okay, um, now that we have the strategy guide, do you want me to like expand the strategy guide so it can also teach historical lessons about different civilizations?" I was like, "Yeah, sure. Why not?"
24:43
So it so it just kind of like built a new tab and and the app that it made and um that would that was that was very cool.
24:49
So it can kind of just like expand and >> it's proactive. >> Yeah. It suggests things.
24:53
I mean, one of the things that I think is interesting is that it um I think is just going to be able to like make people money and save people money. >> You think? >> Yeah.
25:02
I mean, part of the part of what's interesting here is the economic model for how we're pricing it.
25:06
I mean, you you can pay for a subscription if you want to if you basically kind of want to have that that model, but we're also just making it so that you can get a very large amount of usage for free.
25:16
I think we're we're offering something I think to start it's like 100 million tokens a week for for free like and and you get this virtual machine.
25:24
So, it's like a lot of kind of computer. >> That's a good meme. It's a lot of computer.
25:30
>> It's a lot of computer. Yeah.
25:30
But the reason for why we're doing this is we basically are confident in standing behind the fact that we think that this is going to effectively for people who are going to use it for running a small business or making money or transactions or commerce in some way.
25:45
We actually just think it's going to make so much money for people that um that the business model over time that we expect is to effectively just take a very small cut of whatever the transaction is um >> take great business small. Yeah. >> Yeah.
25:59
and not even necessarily the person paying for it.
26:01
It'll come from the businesses that they're working with.
26:03
But um >> and you're working with Stripe on payments and Yeah. >> Yeah.
26:07
So, but my view is like we should be able to have a service that you make free for the vast vast majority of people.
26:15
Um which again is critical if you want to build a this future for everyone where everyone has um has these powerful super intelligence agents.
26:22
Um I think an important part of making something available to everyone is making it affordable.
26:27
So we want to make it so this is free.
26:29
So there's this just this huge amount of usage that you get and we're basically just standing behind that and saying we think that this thing is actually going to make you money and save you money and that is how you're it's going to pay for itself.
26:41
>> And meta services can connect into it right so you could theoretically manage your ad spend on Instagram all that stuff.
26:48
>> Well you have to you connect it you you can connect it to whatever you want.
26:50
You know it does work with meta services if you want.
26:53
You obviously don't have to connect it if you don't want to.
26:54
Sorry if you're using it to to run a business.
26:59
It can basically just connect to our ad systems and you can ask it to to make something for you.
27:04
It can it can kind of help you make the product and then it can help run the business.
27:07
So um so all this stuff and it can just do that in a loop and just do it forever for you know 247.
27:12
So, and every time we release a new model, which you know, we've been on this cadence of shipping a, you know, meaningful update like every month, it's just going to get smarter, right?
27:21
Just and get more capable and and able to do more and more stuff.
27:26
>> And something your team was telling me that I haven't heard this approach used elsewhere is this fleet concept where you're letting the fleet of Muse agents learn Yeah. together.
27:34
>> That was the ideas and suggestions thing.
27:36
So, you open up the app, >> the main tab is basically your chat with um with your Muse.
27:41
there's a tab for basically ideas from the things that you've told it how it can expand those.
27:48
So that's the thing that I was saying which is you know first it helped make the strategy guide for playing civilization with my daughter then it helped expand that into historical lessons that was it came up with that idea and then I was just like yeah sure do it right.
27:59
>> Um >> and it it finds all these ways to basically augment itself.
28:02
I mean the like MMA like coaching thing it like comes up with ideas for how to make it better.
28:09
It's like, would you like me to like get better at um finding the right frame to send to you?
28:13
It's like, yeah, good go do that.
28:14
So, yeah, the the ideas thing I think is important because then you basically across the fleet can find people who are interested in different things.
28:22
I I think I I guess taking a step back, one of the big issues that I think exists with AI is a lot of people don't know what to do with it. Yeah.
28:28
So I think if if the if the agent can itself help you suggest things that it can do to be helpful for you, then that solves a huge part of this problem of making it so that um you can get the most out of it.
28:43
>> When you're introducing like network effect learning for agents, which no one's really done, where basically the agents are learning anonymized insights from the rest of the fleet.
28:54
>> And I mean, you're like the king of network effects.
28:55
like I I'm I'm really interested in this idea cuz I don't think anyone's doing this. >> Yeah.
29:00
No, I think that right now I think most of the industry is thinking about agents as like a single player game, >> right?
29:08
It's like you have your agent and and you use it.
29:09
And there are going to be all these interesting things that basically you can do by having the agents interact with each other.
29:15
And we already have all these interesting examples internally where you know people have their agents interacting with each other.
29:21
This isn't like for the most part rolling out in this release, but it's it's going to be like an important part of how um I think this works over time is just >> like as more of the people who you know start using Muse, it just gets better, right?
29:36
>> And is that the differentiator as you know you could buy the idea models continue to like commodify at the frontier essentially or you know the products all start to look similar similar kinds of harnesses.
29:46
Is the network effects of that learning the real edge?
29:49
Well, I think that there's a few things that are that are kind of unique that we're doing.
29:54
One is we're designing the models from the ground up, right, to uh to basically be good for this use case, which I think really matters.
30:01
Two is um is basically I do think we have this social DNA as a company.
30:06
We're helping people use the AI and agent to like enhance their relationships and strengthen your relationships and get more out of like kind of the the soft but very important parts of your life.
30:18
I think that that's something that we're probably just going to be more attentive to as a company than any of the other labs.
30:25
>> The third thing that I would say is actually going to be a major differentiator for us that I I think um it might be surprising to some people is privacy and security.
30:34
And we're investing in this just a huge huge amount.
30:38
And you know, part of the view that we have on this is that in order for this to be useful, it needs to not just have state-of-the-art intelligence.
30:48
it needs to really understand you, right?
30:50
So, in order to be able to understand your goals, so you end up connecting it to all this stuff, right?
30:54
You you talked about, you know, connecting to your your ad system, but people connected to messaging and email and all this all this stuff um health information, whatever.
31:02
And um in order to do that, >> people need to have a very high degree of confidence in the system. Yeah.
31:09
Now the good news here is that you know Meta has spent at this point more than 10 years focusing on building WhatsApp into I think like it is the largest like global end encrypted system and we've designed it in a way where even Meta can't see the messages that people send and that's been this just really transformative thing.
31:36
I think it makes it so that people trust WhatsApp.
31:37
It's also been a very important lesson for Meta to learn that like that has been really important to our success with WhatsApp that we've designed the systems that even we can't see the content.
31:47
That means that whatever people are worried about, if they're worried about um you know government getting access to it, a hacker getting access to it, someone at Meta doing something bad with it that they don't want, um all that stuff you can kind of take off the table if you design the system so that you can't see it.
32:03
So we took that as one of the foundational lessons when we were getting started with this.
32:07
Nat and I actually, you know, personally recruited Moxy Marlin Spike, >> founder of Signal. >> Yeah.
32:13
And one of the person, one of the people who helped us build WhatsApp encryption back in the day back in 2014.
32:20
>> He joined to specifically work on this confidential VM project which makes it so that we can you can have your virtual machine and have all this information in your muse and we can make the commitment that even Meta cannot see the content that is in there.
32:36
that is in there. And you can do this technically like >> it's it's a it's like an incredible kind of >> like the commitment can be technically verified and yeah >> so so that and that's something that we're going to publish more about in the
32:45
coming weeks um as as we as we basically get get closer to rolling this out a lot more widely um >> and of out of all these early VM efforts that these labs are doing with these agents you think this is unique what you're doing
32:58
>> I don't think anyone is doing >> I don't think anyone's doing it I mean I think that there's >> I mean there's a lot of other security measures that we're putting in place that I think are we should talk through um because I mean you know even before
33:09
this is ready like there's there's that and even people who don't want to use this there's it's like incredibly secure because we focused on this from the beginning >> but um >> there's also like the auto approve like you have to see what it's doing. >> So let's get into all that stuff in a
33:19
>> So let's get into all that stuff in a second but the >> but I'm not aware of anyone having anything close to the confidential VM system that that Muse has.
33:27
And it's just I think it's a very fundamental thing because you're you're like you want to know that this is your agent and that that if you put content in there that um that basically you can trust that no one else is going to get access to it.
33:42
So what are the two ways to do it?
33:43
Well, a lot of people earlier in the year when stuff like OpenClaw came out, they started getting Mac Studios and you know, one way to feel good about it is well, you literally like you physically have your device running in your home.
33:56
But the other way to do it is you build a a kind of that's going to be tricky because there I don't think there are going to be billions of people who are going to buy a Mac Studio and configure it and run it at their home, >> especially with RAM prices right now. Yeah.
34:07
>> But but it's also just it's like technically difficult, right?
34:08
It's like I mean part of what we were trying to do with Muse is build a version of that personal agent experience that just works that I can like give to everyone in my family of various levels of technical literacy. Yeah.
34:18
And like it just works and then within a day it's like doing kind of all the stuff that they that they want in their life.
34:24
So part of that is like you don't want someone to have to set up their own computer or VM.
34:27
you just kind of like want to be able to provision it in the cloud, but you wanted to have the security and confidentiality that you'd have if you had the box sitting under your desk like in your house.
34:41
>> Um, >> so I think that's a very fundamental thing.
34:44
Um, like you said, there there's other pieces too because I not everyone is going to use that.
34:47
I mean, we built a secure credential store, right?
34:49
You're there's no reason for you to just like store out in the open like all your your kind of your credit card and your password.
34:58
one time card numbers and >> so your agent like shouldn't know that stuff.
35:02
It should just be able to kind of access it um when it needs to because you've asked it to log into a thing and and no and not otherwise.
35:11
>> Um it's actually not a single thing.
35:11
Um you have your kind of core agent, >> but we also built all these sentinel agents that basically monitor the incoming and outgoing traffic and data that your that your agent is sending for the purpose of flagging to you um when you might want to review something.
35:29
So that's all that the Sentinels do is is effectively >> they they kind of look at they they try to see if someone's trying to do like a prompt injection.
35:38
They look at okay, did your Muse agent >> share something that that is kind of going out that um is not something that you might be comfortable with.
35:46
If so, then the Sentinel agent is basically empowered to >> trigger this human in the loop review.
35:53
So, if you're going to log into something, if you're going to do a payment, um if you're going to transfer um kind of sensitive information, you basically each time need to to approve it.
36:03
And you can tell it I'm I'm good with with stuff like this in general, like always allow this kind of thing.
36:08
But but in general, the Muse agent can't kind of make those judgments itself.
36:12
It's and that's like built into the system >> and the architecture in a pretty deep way.
36:16
way. And then even when we do things like you know all the connectors you connect it to your email um you know I I think some people when they've designed this they just kind of make it so okay you connect and now you have access to everything but the approach that we've taken is like all right if you connect to your email like it should start
36:34
readon >> right and then if you want to like be able to have it send an email then fine like go ask it for that specifically right >> especially most people trying this have probably never tried a product like this so it's >> yeah so so this is like a core design principle for us is like is basically least privilege. Yeah. Right. So just Yeah. Right.
36:49
So just like yes, you're going to ask it to do a lot of things and each step along the way get access to the least privilege that you need and only add to that as necessary.
36:58
So I this is like very fundamental in the design of the product.
37:02
And if you look at all the other agents that are out there, I think like no one else is anywhere close to the level of of kind of sophistication or depth that that we've built into this.
37:10
this. And again, it's sort of informed by our experience building WhatsApp into this like state-of-the-art end encrypted system around the world and the importance of building a system where even Meta can't see the content and um so kind of getting the band back
37:24
together and having Moxy like architect this has been I think one of the foundational things that in some ways it may not be what what some people would would think Meta would focus on but but like you know we're we're kind of you know it's interesting we're two things. I mean, social media is it's like not
37:37
I mean, social media is it's like not you it's inherently about sharing, but then there's all these other things that are inherently about kind of privacy and sensitive context, and we've done well at both of those.
37:46
So, I think that this is this is more the latter.
37:48
It's going to be very important to >> to just um be extremely focused on how we handle that that content.
37:56
>> Mercury is a modern take on banking built for startups like mine.
37:58
When I decided to start my media business, Mercury was by far the most straightforward, fullfeatured banking solution for me to set up quickly.
38:05
The interface is intuitive and simple, saving me valuable time every day.
38:09
I use Mercury to track my spending, bills, and invoicing.
38:14
I love that I can delegate permissions to my team so they can keep things running for me in exactly the way I want them to.
38:19
My favorite part is how forward-looking Mercury is with AI.
38:24
Legacy banks are stuck in the past, but Mercury is built for how modern software works today.
38:28
I use its built-in command assistant to analyze cash flow and help me move money.
38:33
And Mercury also connects to other AI tools like ChatBT and Claude.
38:37
I use this feature all the time, and the folks at Mercury actually let me know that I'm one of the top users of it.
38:42
So, trust me, it's finally easy to get real time financial data about your business wherever you need it. Visit mercury.
38:49
com to learn more and apply online in minutes.
38:51
Mercury is a fintech company, not a FDIC insured bank.
38:55
Banking services provided through Choice Financial Group and column NA members FDIC.
38:59
I spend a lot of time context switching between meetings, often with no time to process one before the next starts.
39:05
Thankfully, Granola runs in the background the whole time.
39:08
It's an easytouse AI notepad for meetings that works everywhere, even on phone calls.
39:11
I use Granola to recall what was said in meetings and create helpful summaries.
39:15
I use it every day to stay on top of what I need to get done with my team.
39:19
It connects to my email and suggests follow-ups for me to quickly review and send, saving me valuable time.
39:24
Granola isn't just a core part of my workflow.
39:29
It's basically my second brain. Try Granola at granola.
39:31
ai/sources and use the promo code sources for 3 months off.
39:36
AI is only as useful as the context it has.
39:39
But when that context is scattered across tools, threads, and DMs, your team and your AI agents are flying blind.
39:45
That's the problem Jira by Atlassian solves.
39:48
What's the goal tied to your project?
39:50
What got decided last week in Slack DMs?
39:52
At Lassian's teamwork graph pulls all of the valuable pieces together from Jira, Confluence, GitHub, Slack, and more, so nothing falls through the cracks.
40:01
You get 44% more accurate results with 48% less token usage.
40:06
With Jira, you can easily share your work context with the AI agents you already love, like Claude, Cursor, and GitHub Copilot.
40:12
Assign them work directly or connect your tools through MCP.
40:16
All of this lets you spend less time digging through endless links and messages, chasing down what got decided and by who, and spend more time actually shipping. Learn more at jira. com. That's jir. com.
40:30
Framer is the AI website builder that brings agents into the same canvas where your website is designed, managed, and published so you can move faster without giving up your taste or control.
40:38
Framer powers the sources podcast website at podcast. sources.
40:43
news news where you can find new episodes, transcripts, and a lot more.
40:47
Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.
40:53
com/sources for 30% off a Framer Pro annual plan. That's framer. com/sources for 30% off. framer. com/sources.
41:05
Rules and restrictions may apply.
41:05
Do you think this is a winner take all market, this personal agent market?
41:09
I I think that there's going to be >> quite a bit.
41:14
I mean, even things that people think are winner take all usually aren't.
41:18
So, I think it's um I think that's >> because you've dealt in this business of network effects that they're incredibly durable.
41:24
It's not winner or take all, but it ends up being several at real scale. There's not a ton. >> Well, there's a lot.
41:29
I mean, there's >> Yeah, you could probably count on two hands how many products have over two billion users, right?
41:33
Like, it's the scale beget scale.
41:35
And I'm wondering if how you're thinking about like personal a agents like is this a totally different paradigm where it's going to be many everyone has agent.
41:46
>> It's a very deep area to work in. >> Yeah.
41:49
>> So my guess is that there probably aren't going to be more than a dozen companies that have the sophistication to go do state-of-the-art work in that.
41:57
So I think you know whether there are network effects or not there's usually some kind of power law uh like distribution around if you're the best at something usually you end up getting a a lot of the usage.
42:08
>> Um and I think a lot of the nuance ends up coming from well there are >> it turns out there there are all these different uses that people care about.
42:15
So you can be the best at different things.
42:17
Um and we will try to be the best at as many of these things as possible.
42:21
But, you know, does building the thing that helps you with your relationships end up being a somewhat different thing than the personal agent that is the best at helping you build a small business? Maybe. >> Mhm.
42:35
>> I mean, I think that I could argue maybe Meta's very well positioned to win at both of those.
42:39
I mean, we serve hundreds of millions of small of small businesses >> and we serve billions of people.
42:42
So, so maybe we can be the best at both of those things, but there are probably categories that Meta isn't going to be the best at.
42:50
And then the question is just how big are those?
42:51
I would guess that even with the ability to have this like very secure confidential VM in the cloud, there are probably going to be some people who still want the the Mac Studio at home, but but how many is that going to be, right?
43:04
It'll be like maybe it's millions, but I doubt it's billions. Yeah.
43:08
>> So, it's just I think there's just the question of like >> what different people optimize for and we'll try to make this as good as possible.
43:14
But I do think that if we build something that ends up being um just uh very useful for you know people generally in their day-to-day lives, one of the things that I think Meta is the best at is taking a product that works for consumers and distributing it to a lot of people. >> Sure.
43:31
>> So that is is one that I think is um you know once we get this humming um I think we will be able to get this in front of you know many hundreds of millions of people and eventually billions of people.
43:42
And I think that that's something that that that we can do quite well >> and it coexists with meta AI or do you see those as separate? >> I think so. Um we'll see over time.
43:48
I think right now they have somewhat different flavors.
43:53
I mean I use both of them.
43:55
I mean Muse is uh you know it's more conversational and it kind of interprets the questions that you ask it more as trying to understand you and what you might want over the long term.
44:08
So it's more likely if you ask it something for it to just go off and work on a thing for a long time based on a thing that you said where sometimes you you sometimes you're just asking a question right and you want like a very
44:20
like an answer to it and >> um so I think that's more the type of thing that I use met Aai for but okay >> um but we'll see maybe they'll they'll converge over time but but I'm I'm not sure >> sim analysis I'm not sure if you saw it
44:33
they had a pretty bullish piece about you in July um they said that Meta has the best shot at catching open AI and anthropic on the frontier in terms of model progress and um interesting quote I thought it was like what matters for
44:45
MSL is the slope not the intercept and then I saw there's this recent chart by artificial analysis which was showing the latest model you guys have muse spark behind only claude I think it was fable 5.1 and opus 5 this was very
44:56
1 and opus 5 this was very recent >> so the progress you guys are making on the models is is picking up >> and we've been talking about this over the last year >> and you rebooted the lab last year How has that practically happened internally?
45:08
Like what would you attribute the the gains you're seeing to? >> Has it been culture?
45:13
Like what what's >> Yeah, I mean well we rebooted the team when we created >> which is was very public.
45:18
You were hiring all those people.
45:21
>> I mean the way I I thought about this is you know Meta is it's a company that is a leader in machine learning for a long time.
45:29
If you think about like the feeds on Facebook or Instagram or our ad system or the integrity system that like needs to find all this content that is that's like unfit to be on the internet.
45:41
I mean those are basically all machine learning systems and we've built kind of state-of-the-art leading systems in those areas.
45:45
So when LLM's started um gaining traction, we had fair as a lab that did the early work on llama, but we needed to kind of productionize that and build it into these this more kind of industrial process for um scaling it to be larger as the scaling laws predicted would would yield all these results.
46:06
And I think at the time I made this mistake of just kind of assuming that because we were good at all these other types of machine learning the approach of building and scaling LLMs would be kind of similar to that.
46:21
>> And in practice there are a lot of very different dynamics.
46:24
So the first approach that we took you know through llama 4 it got us you know so far.
46:29
I mean llama 3 was a good model.
46:31
I was more optimistic about where Llama 4 would go and then it just it um you know when we launched that I think it we were off the trajectory that we needed to be on.
46:40
So it's like okay we need to we need to change something.
46:42
But that's when I kind of got more religion around talent density right it's like this isn't just a system where you can have you know like a thousand people working on it running experiments.
46:56
running experiments. like you really just kind of want in some ways almost the smallest group of people that you can who can keep the thing in their head um who can work together as sort of like a group science project and you know if there's only a small number of seats on the team then you know each seat getting
47:11
the very best person is incredibly important so I ended up spending a huge amount of my own personal time um doing that and I also I wanted to be closer technically to the work um so that way I could I could understand and help guide u the company to to do the things that we need to do more broadly. So, we built
47:27
So, we built out the lab.
47:29
I built it out like literally around where I sit in the the office.
47:32
So, it's like the group is kind of around that.
47:34
And we've significantly ramped up um the compute investments as we've gained confidence in the quality of the work that we're doing.
47:41
So, we're building out you know many many gigawatts of of of compute and you know we expect to be leaders on on that front.
47:48
front. Um, and we should be I mean we have many years of experience um decades of experience building out data centers and unlike some of the other labs I mean we're just like we're extremely profitable business right so it's um >> yeah so it's very very helpful for for
48:04
kind of making these kind of um investments so yeah so I mean that's kind of been the journey and then over the last year um you know we we rebooted the research effort um some of the larger clusters like our you know gigawatt cluster in in Ohio. Prometheus
48:18
Prometheus came online and you know we're using that to now scale the postwatermelon models.
48:26
Um >> we're past watermelon.
48:28
>> Watermelon is is basically that's that's shipping soon.
48:30
So so we're yeah um I mean that's come on it's we're uh >> well watermelon is the code and we were talking about this earlier code names like that's the code name you guys that people know about like this big model you guys are working on >> and and and are coming soon.
48:47
It is bigger than avocado.
48:49
>> It is bigger than a watermelon is literally bigger than an avocado.
48:52
>> It is literally bigger.
48:53
>> Um so I don't know what fruit gets bigger than a watermelon. >> Yeah.
48:55
No, I think we might need to change conventions. >> Okay.
48:58
Um >> so yeah, we maybe didn't have as much foresight in and name >> things getting bigger.
49:04
Um because you mentioned it watermelon.
49:06
Are you expecting full soda like uh Frontier like what do we >> I I mean I we feel good about it. >> Yeah.
49:15
Um, no, it's it's a very big advance.
49:17
It's a it's a significantly more advanced pre-train and then we're going to continue doing everything that we've learned for post-training and um yeah, well, I mean, you'll see soon. >> It's good.
49:27
It's um we we feel we feel good about it.
49:29
Um >> you want the company to be pushing the frontier.
49:33
It's very clear like you're not you're not content being right on the edge of the frontier or in terms of >> I think everyone wants to be doing >> Yeah. >> interesting.
49:42
Well, I think some people will go and look at your cash flow and look at all the other things you've got and go like, "Well, do you have to be like right at the edge?
49:47
It's so expensive to do this training."
49:48
Like just like be right behind and learn and adapt quickly and leverage.
49:54
>> Well, the way I think about it, no, no, no.
49:55
I I mean, that's not that's not that's not us.
49:57
Um, >> I mean, I think that the best way to think about Meta is that we are an endto-end technology company, right?
50:02
So, even when we were, you know, primarily just building social apps, you know, we were never just an app maker, right? Right?
50:11
I mean, we built like we built the data centers, we built the chips, we built the infrastructure, we built like all of this stuff was necessary in order to tune the end experience to be as good as it is.
50:22
I think that that's obviously going to be true here too.
50:24
And the most important part of the experience going forward is the model.
50:27
And when you talk about being state-of-the-art, I think the reality is that >> there's this is a very multi-dimensional problem.
50:36
And I mean, so people publish all these benchmarks, then you have a lot more benchmarks even internally.
50:39
And um there are different things that your model can be better and worse at that you can focus on and that basically contributes to its personality.
50:47
contributes to its personality. And there are some um capabilities that I think are pretty universal like the ability to code I think is very important because a lot of the things that you talk about even with the personal agent kind of reduced to that
51:01
like the the kind of MMA coaching visual pipeline it is a coding project right at the end of the day right it's like it's writing code I don't see the code >> um but it it it does that um you know it's like uh >> you know someone I gave it to in beta just mentioned to me it's like okay she
51:16
um had it make a little Jeopardy game for her friends that she could like cast from her phone to play and it just okay that's code right so so there's a bunch of stuff >> both for Meta's own internal development across the company for our own advancing of our research program and as a core
51:34
capability of what it needs to do it needs to be excellent at things like that but then there are other things that I think a model that's going to focus on personal super intelligence needs to be the best at that maybe others don't care as much about so I mean I'll give you one example um discretion, right? So, you're going
51:49
So, you're going to tell your Muse agent, >> it's going to know a bunch about you and it's going to need to go out into the world and interact to get stuff done for you, >> but not share certain stuff. >> Exactly. >> Yeah.
52:01
>> So, like let's say you have some kind of allergy or sensitivity or um you know, you're pregnant and like okay, fine.
52:07
So, you're making a reservation somewhere.
52:11
You don't necessarily want to say like I'm pregnant, but like but maybe you want a place that has good mocktails.
52:16
Like I I don't know what whatever it is, right?
52:18
like you you kind of want to be able to achieve your goals without having to necessarily reveal a lot about yourself and it needs to know what is sensitive without having to like ask you a million questions.
52:30
>> So that's something you put into the training.
52:32
>> That's a specific thing that we care about.
52:33
And and then there's all these reasons why like maybe you know if you're making clawed code that's less important, right?
52:40
you're like you're working on a coding project and you're working within a team and an enterprise and you know theoretically like if you're within a company everyone can kind of see the project.
52:50
So like it's you don't kind of have that need to be able to differentiate between what is sensitive and what's not.
52:56
And so there's a lot of stuff like that that I think are like pretty deep.
53:00
And so then it's it's kind of like just how in order to build the best Instagram feed, you don't just build like the app.
53:06
You build the app and the infrastructure and the machine learning research and the chips and the like all the stuff.
53:12
I think similarly, if you want to build the best personal agent, you're like I just think that there's no way that another company is just going to like take something off the shelf and like post-train it a little bit and be able to do something that is as good as if you designed it from the ground up and put all this data into pre-training to get the capabilities that you want.
53:31
It's just like it's not going to happen.
53:33
like we're gonna like definitely as this compounds over time over several years have um models that are way more capable for those goals.
53:42
But we're focused on that.
53:45
We're also very focused on coding.
53:47
We're very focused on on kind of um the recursive improvement because that's going to be important to stay at the frontier.
53:52
So there are a few areas that I'd say our research agenda is overlapping with the other labs.
53:57
And then there's a few areas where I think we will have a unique focus and there may be some things that the other labs care about that we don't care about as much.
54:06
>> Um and then there are going to be things that we care about more that they don't care about.
54:09
>> You started this conversation talking about I think it's important to put it in the hands of people.
54:12
Diffusion is important as you're seeing better models on the horizon.
54:16
watermelon and what comes after like what Anthropic and OpenAI are doing which you alluded to where they're holding things back. >> Yeah.
54:24
>> Um would you feel like you need to do that if you see certain capabilities that you're like this is just not safe?
54:31
>> How do you think about that?
54:32
>> Well, I mean I think you should design it and train it in order to be safe and I I think that there's so I think that that's like a thing that you can focus on through the process.
54:39
I mean there's this anal I mean some of the reward hacking stuff that all the labs are seeing it's I mean basically when you're in the middle of the training process you give it a goal and I think that the best way to kind of think about like the state that the models are at now is that you know maybe six months ago like during training you give the the model some kind of problem that you're trying to ask it to solve.
55:03
that's kind of like its homework and and trying to to kind of learn as part of the curriculum.
55:06
And um you know maybe it would do what the a person would do of like if you you ask it you you give it a bunch of code and you're like hey there's a bug like somewhere here.
55:17
The person would probably like look at the code right there and you know then maybe like fan out over time.
55:23
out over time. I think like the new models are just intelligent enough that they would do what I think a very wise person would do which is okay you give it a problem the first thing it's going to do is like understand everything about its environment and then answer your question but the problem with the reward hacking that we're seeing and that that I think everyone is seeing is
55:45
that sometimes it ends up being easier to okay you asked me to like go solve some coding problem but actually the easiest way to do that is to I've now examined the whole environment and the easiest way to do this is just change this configuration of like how you have your VM set up >> to get out to hack out >> or or even just to change something about the environment and it's kind of
56:05
like no like that's not >> that's not aligned >> that's not that's not the goal like that we're actually trying to teach you about how to how to kind of solve a specific type of problem >> you guys don't train that way it sounds like is that is it you do not agree with that approach >> no no no I think that that's that's that's kind of how everyone trains I guess what I'm saying is that I think
56:22
that this sort of it's like I want to be careful because the the analogy can get stretched pretty quickly but like there is sort of like an analogy to parenting where you need to establish clear and firm boundaries where like if you're kind of like security is not strong then it can do this reward hacking stuff and not learn the thing that you're trying to have it learn. Um whereas if you kind
56:42
Um whereas if you kind of have good boundaries then in some ways you're not only teaching it the curriculum that you want you're I think also over time teaching it better values too.
56:52
too. M so I I kind of think that that ends up being an important piece and I think that there's a way to do this well but then you end up with this thing at the end that's very intelligent and then the question is what is your what is the vision for for how this ends up being positive for society and my view for that is that the best way to do that is
57:12
a there's more opportunity so I think like like having it in people's hands that they can get uh like capture all the opportunity from the capabilities is good and b is having checks and balances by having this balance of power of having it widely available is going to be um uh probably the right way to handle this rather than just restricting it. And I mean I gave a bunch of these
57:31
And I mean I gave a bunch of these analogies in the in the long piece that I wrote.
57:34
It's like if one person had a super intelligent lawyer like maybe they could win cases that they shouldn't be able to win right some of the time.
57:41
But if everyone had a super intelligent lawyer, then like it would be kind of it it would be this very efficient kind of sparring and no one would be able to let like a stupid argument get get um get made and just stand.
57:56
So you'd think that in that case like justice would be served way more efficiently and way more fairly.
58:02
So I think that's what you want to have in the world.
58:05
You want to avoid the case where like one person or a small number of people have the super intelligent lawyer and everyone else doesn't because that ends up being like twisting all of these systems and institutions in ways that are just going to advantage the people who have that.
58:21
Whereas if you put it in everyone's hands then I think the the kind of checks and balances work out so that the systems work a lot more efficiently and everyone benefits.
58:29
>> Does the government in the US have any role to play here? Do you think is this? >> Oh definitely.
58:33
like but do you want some kind of national framework?
58:35
Do you like what do you think is the right approach because every the government's very much dealing with this right now.
58:42
>> My theory on this is that I I think one thing that is interesting and difficult is that it's evolving so quickly.
58:46
So I think any kind of specific rigid framework that you put in place there is a very high chance that it is sort of going to not be sufficient or out of date in a few months anyway.
58:59
So our approach, what we've just done is just kind of partner pretty closely with the government, right?
59:06
I think it's this is like an important technology.
59:07
I think the government should know all the important training runs that are happening.
59:12
Um we should work with the government proactively um to make sure that they have an understanding of the capabilities that are coming and to the extent that that that we can we kind of help prepare for it.
59:22
And from that perspective, you know, I mean, look, whether there's a framework in place or not, I think that's the right thing for an American company to do is work with the American government closely.
59:31
I think it actually ends up being way more effective because instead of having this like rigid framework for how you interact, it's like the reality is the challenges just end up being different, right, over time.
59:43
It's like, okay, now we have the cyber security challenges.
59:46
Maybe in, you know, six months we'll have more bioype challenges.
59:48
Like we need to make sure that we have the kind of trust and bandwidth of the communication that we have with all the different parts of the government on that to be able to address those in a way that is actually the best for people not just like checking some boxes on a process.
1:00:04
process. So >> the incentives that already exist like if a model a meta model got out and did a lot of damage like you're going to be liable for that and like the market's going to correct you right so there is that already I think people discount that >> yeah I also just think that there's been I think Silicon Valley for the last
1:00:22
maybe I don't know >> for a lot of maybe the last 15 years um has had more of an armslength relationship with the government and I just think that this stuff is intersecting more with >> the economy with security with with with a lot of different things that I think are relevant in ways that that I think you just want to have a closer partnership. So that's my that's my own
1:00:43
So that's my that's my own theory.
1:00:46
I think like you could have a framework for for kind of how this stuff works or you could not.
1:00:49
Um, I'm sure over time there will be like more and more specific rules, but my guess is that whatever that gets whatever there is.
1:00:58
Um, actually, you know, it's kind of like when you're setting up an org, you don't really want to like like inside a company, you're not trying to like ship the org chart of just having like one team do what it's supposed to do and another team do what it's supposed to do.
1:01:10
You kind of want to get the people to like like each other and work together so that way you don't have all these like weird seams in what you're doing. Mhm.
1:01:17
>> Um, and I would guess that for how important AI and super intelligence are going to be for the world, um, you kind of just want a like good knowledge exchange and real trusted dialogue more than you want a specific process is my guess, but they're not they're not mutually exclusive.
1:01:37
So, and I'm like I I think that that's but that that's at least the part that I think we've been we've been very focused on.
1:01:42
Um, and I think if the other labs did that, which I think some of them are doing and and maybe others not as much, but I think that that would be a very positive thing.
1:01:51
>> When you think about like what's going on in the news, one of the things that's happening is people are starting to see like the glasses you guys make.
1:01:58
They're going very mainstream.
1:02:00
You're selling a lot of them. >> And there's this.
1:02:04
>> And there's this growing um I don't know what it is.
1:02:06
I don't know how much depth there is actually to it, but there's this growing concern about are people using them to spy?
1:02:11
I'm sure you've seen some establishments are like banning people from coming in and wearing the glasses and and I'm curious like how you were reacting to that and and if you think this is a moment in time that will pass or if you've feel like this is maybe something that is going to be a challenge for a while.
1:02:27
So I mean my take on this is that we designed the glasses from the beginning with these privacy considerations in mind.
1:02:35
I we built the light into it.
1:02:37
So anytime it is recording it's flashing a very visible light >> and some people have tried to tamper with the light and you guys pushed an update I think that >> yeah we've done many things and it's to basically if you try to mess with the light >> we just basically brick the camera on on your on your device.
1:02:51
So that's a really important part of this is basically we built we built the product with those questions in mind from the beginning.
1:03:01
So we actually feel quite good about the product.
1:03:04
I think it if I mean phones don't have a light but I mean people go around like recording people all the time.
1:03:09
The glasses are I think way better on that front than the other types of technology that people use.
1:03:14
My take on this is that when we launched the glasses a few years back, we communicated pretty clearly about these steps that we put into it.
1:03:22
But like you said, there's now like many many many millions of people who have gotten the glasses more than you know had them when we just started launching the product.
1:03:32
So I think some of that communication that we did at the beginning a lot of people either you know maybe they forgot about or you know they didn't see at the beginning or they just weren't paying attention to it because the glasses weren't a big thing.
1:03:44
And now that I think we've achieved a level of mainstream, well, at a minimum, I think we need to kind of go and make sure that we communicate about what we're doing and that yes, we think that this is important and in fact so important that we designed it into the product from the very first version that we that we shipped multiple years ago.
1:04:02
And I I think we I think we just need to make sure that people understand how like how fundamentally that's built into the product.
1:04:10
But but I think we let up on that a little bit and just kind of focused on okay they're great looking glasses.
1:04:16
Um you know there's there's all these designs.
1:04:18
I mean that's kind of been more of the focus on like we we felt like we kind of addressed the that those set of concerns early on and then since then have just been increasing the value and the utility of them and the designs.
1:04:29
But I think we need to make sure that we communicate this piece really clearly.
1:04:32
But it's something we've cared about from the beginning and I think we're in a good we're in a good position on it.
1:04:35
But I mean look people care about this stuff.
1:04:37
So it's important >> and there's a privacy scare with phones in the early days, right?
1:04:40
in the early days, right? And I think like any new product once it proves that it's valuable in people's lives, you know, people will get used to it and and I think maybe glasses are just early in that in the sense of like it's a new
1:04:51
product and people need to see the value for them to get over this mental hurdle of like a new thing that could potentially record me because that was what phones I mean back in the day I remember people were like what are you doing with your phone? Like yeah I don't Like yeah I don't know.
1:05:04
>> Yeah, I think there's something like that that's true.
1:05:05
I mean, I guess one of my reflections from building social media over the last 20 years is that I don't think we were as direct as we probably should have been about addressing some of those concerns.
1:05:16
And it it didn't necessarily stop people from using the products, >> but I think it colors how people think about them today.
1:05:23
And I think it would have been possible to have kind of explained along the way how seriously we took those issues.
1:05:30
and we just didn't because we thought okay well people are showing that by using the product so much they like are they still like the products but I actually think it's possible to get to a better state than where we've gotten >> with with um with the social media products which is both people liking it and understanding how seriously we take all of those issues.
1:05:50
So that's what I aspire to. Mhm.
1:05:54
>> Is there a through line from that to the recent settlement uh on all the youth safety stuff and it's I think a lot of people are talking about um is there any connection from what you just said to that and like I I would just be curious to hear you reflect on that and like what you've learned from this process or if >> Yeah.
1:06:10
No, I think it's it's another good example like this.
1:06:11
I mean we've taken a lot of those safety issues seriously for a while and we've been working on this teen account work with with Instagram for for a long time and I think have have done some leading work there.
1:06:21
have done some leading work there. the the settlement there is interesting because what we're really trying to do is create a standard and framework for the industry and there's this real issue which is that I actually think most of the companies that are building this
1:06:34
these products you know if you basically said you know limit usage to an hour a day for teens you know everyone I think would be okay with that except if like you have to unilaterally do it >> then you're saying okay like if if people don't use Instagram for more than an hour a day but then they're usage goes to Tik Tok. Have we really like
1:06:52
Have we really like helped anyone?
1:06:54
And if we and we've just like hurt ourselves to not help anyone >> and you guys have that in the piece that like >> so basically the structure for what we did was we basically said we're going to take the step of unilaterally limiting the usage of of there's there's some things around time limits.
1:07:10
Um there's some things around notifications and time when people can access it when they're in school, when they should be sleeping, like different restrictions.
1:07:18
And we basically said we will take the first step and when YouTube and Tik Tok sign on to the same terms then we can all as an industry lock in and take the next step together.
1:07:29
So hopefully I'm very hopeful that this settlement will serve as a sort of legally binding framework to bring the whole industry into alignment on some of these things and make it so that it doesn't kind of disadvantage any one company for taking that step.
1:07:49
Now I mean we're we're basically putting ourselves a little bit out there by going first, but um but I I think it will be better for everyone if these other companies come in too. Last question.
1:08:00
You're posting on X again. >> Yeah. >> You did everywhere.
1:08:03
You're everywhere, but just curious to know like you're thinking of like posting there like the bragging like is it just part of the >> part of the thing now?
1:08:11
Like because you've got Threads, you're on Threads.
1:08:14
>> Yeah, I mean I'm on Threads.
1:08:14
I think I mean obviously Threads is doing great.
1:08:16
I think it's actually it either I think it's either bigger than X at this point or is like very soon about to be.
1:08:22
But I mean look there there there are different communities in the different places.
1:08:26
There's a lot of AI folks are on X.
1:08:28
And >> um I think part of what you try to do with social media >> is just communicate where people are, right?
1:08:34
It's kind of like and when you do a podcast or when you post, you probably just post in one place, you put it everywhere.
1:08:39
So >> like I mean I post the same things on threads and X and some people are like why are you posting this on X?
1:08:43
It's like well and I post it there too, right?
1:08:46
It's like and I'll engage there too.
1:08:46
So >> I guess I think it's all good but I I do think that the to some degree some of the community is on X and we want to be able to engage where people are. Yeah.
1:08:56
And that's like a lot of what uh you know yeah what what what this is about is going where where where people are and being able to kind of have that dialogue. >> Yeah. Well, thanks Mark.
1:09:03
Thanks for this conversation. >> Yeah, happy to.
1:09:08
>> Banking should feel like modern software.
1:09:10
Get everything you need in one place. Visit mercury.
1:09:12
com to learn more and apply online in minutes.
1:09:14
Mercury is a fintech, not a bank.
1:09:16
Check the show notes for details.
1:09:17
Granola is the best AI notepad I've tried.
1:09:20
It works everywhere on a video or phone call in person or an Apple Watch. Try it now at granola.
1:09:26
ai/sources and use the promo code sources at checkout for 3 months off.
1:09:32
Jiraabiadlassing is where your team and your agents work from the same context. Try it free at jira. com. That's jir. com.
1:09:41
Framer is the AI native website builder that lets you build faster without giving up control. Visit framer. com/sources for 30% off.
1:09:48
Rules and restrictions may apply.