0:00
You guys ready to talk about the end of the world? Good.
Search this video
0:03
Yeah dude, what's your P(doom), My P(doom)? Joey? I'll start with that.
0:07
I'm a linear extrapolator, Max.
0:09
I can't think that far through.
0:11
That's too far up here, like in capabilities. I don't know.
0:15
I think we get, I don't know, we get a major hack or something. People freak out. I think it starts there.
0:22
The P(doom), I don't know.
0:25
Joey, even if it's a linear extrapolation from the last four years of experience that you've had, you think that P(doom) by the end of your life is impossible?
0:38
No, it's not impossible at all.
0:39
I mean, you could get it in four years easy. With a linear?
0:43
Wow, With a linear extrapolation, yeah.
0:46
that's a crazy take, actually.
0:47
Well, that's a hot take, dude.
0:49
Dude, linear from the start of last year. Even from late 2022.
0:56
I guess we're in 2026 now, so I guess four years, yeah.
1:00
Joey's like, man, just look at the ARR net additions, man.
1:03
This is I've never seen a number so That's all I can see. big. Joey's benchmarking.
1:11
I gave a comment to the LA Times today. They called me up. They didn't call me up.
1:15
I called the guy up, he's like, hey, give some stuff on this.
1:19
He's like, well, Ed Zitron said, you know, if this stuff, No.
1:24
I was like, you put me on the record, Ed Zitron is a moron. Attaboy.
1:29
I was like, look at That's so good.
1:31
look at any past call of Ed Zitron. Dude. Yeah.
1:36
The track record is crazy.
1:41
But okay, what I've now learned is that we have a new benchmark where we can measure model capability progress.
1:49
It's called the Joey benchmark, and it's just ARR. That's all that matters. It's just lab ARR.
1:56
I mean we've seen this now twice in the last month, maybe six weeks or something.
2:03
The only thing that matters is people's future view of Anthropic and OpenAI ARR and the pace of what's going on versus expectations.
2:11
And the entire semis trade is dependent on that and it's gonna be dependent on that.
2:16
And it makes up a lot of share market cap and You gotta add Azure plus AWS year over year revenue growth, bro.
2:25
These are the only four numbers that matter in the history of the universe.
2:26
No, no one cares about Azure. No, they dropped Azure?
2:30
You're telling me these are the only four numbers that matter, bro.
2:33
Now we're down to No, Azure CC growth guide used to be the number.
2:36
The Anthropic net new ARR guide is the new number. It's getting replaced. Is AWS It's replaced. There's only one. out too?
2:42
We only care about OpenAI plus Anthropic net new ARR now.
2:46
All they're gonna care about.
2:49
Two numbers on which the entire fate of capitalism rides.
2:50
The clouds, the hyperscalers are a level abstracted away from the end demand.
2:56
They're literally renters of Makes sense. compute.
3:03
Just glorified neoclouds.
3:06
Yeah, they have a very nice business. They do other stuff. They sell CPUs too. Well.
3:15
AWS and Azure are glorified neoclouds now. Yep.
3:18
Put it on the record, dude. They're dumb pipes.
3:20
That's what you used to say in the internet, like the telecom, you know. They're dumb pipes.
3:26
That's what Amazon and Microsoft and Google are.
3:28
You know, now Meta's a neocloud, apparently. Yeah.
3:32
Jeremy says a few gigawatts, a hundred billion dollars of open source demand, right?
3:41
So okay, is this a timing thing or are these companies so dysfunctional that their research organization buys from neoclouds and then their cloud organization sells to other companies' research organizations?
3:53
Or Meta goes and strikes up a deal with CoreWeave and then Meta Compute goes and sells their compute to OpenAI or something.
4:05
Well, they probably okay, they might sell it to OpenAI to do short term tasks.
4:10
But you know, it's still preferable to start building the compute because in the event that Meta does well, you can sort of claw it back.
4:16
Or they could also, you know, monetize open source tokens is I think Jeremy's claim, on that similar short term time horizon.
4:25
I actually don't think it's cognitive dissonance because the really important thing, which we've harped on before, is just the potential to have a ton of training compute in the event that your research lab starts doing really well.
4:38
And I think any short term monetization strategy is not incompatible with that.
4:49
Even if the short term monetization strategy requires you to lock up a bunch of compute on a five year deal this month, despite the research organization going and buying stuff a month and a half later, basically.
5:05
Wait, if it's a five year deal, it's not short term, right?
5:06
I'm saying it's totally fine for Meta to try to build It's like as much long term capacity now, but then, you know, sell it to OpenAI on a six month contract. Yeah, sorry.
5:16
It's five year terms at the price, but with ninety-day cancellation rights, and so it's a short term contract by Yeah, exactly.
5:24
But I mean the price is also, at least in Elon's case, it's definitely not the five year price either, right?
5:31
It's like the six month price times two or something.
5:36
I mean, I'm seeing GB300 quotes from this morning to a large lab for nine bucks an hour for three years. Holy shit.
5:46
Which, I mean, initial response You can't be leaking that much alpha on the SemiAnalysis Weekly podcast, a big number, you know.
5:56
Buy the AI Cloud TCO model and/or the Tokenomics model if you want to hear the actual number.
6:02
Yeah, no, for the specifics, but it's this backwardation of the curve where if you start six months from now, you get a much lower number.
6:08
But if you want to start tomorrow, you have a really high number.
6:11
And my initial reaction is, this is just an F off price from the provider.
6:20
But then we dig in and it's like, we know the provider and we know they have capacity and we know they're trying to sell it next month.
6:25
And this is the strategy.
6:25
They had the financial position to be able to hold on to the compute, I guess, not need to monetize it until it's actually installed or a month away from being installed.
6:38
And then they can test the market at a really high price and try and figure out what it's going to close at.
6:44
And I think there's potential people jump on it strictly because they get a two, three month head start over what a much lower price would be.
6:52
Like pay a twenty percent, thirty percent premium on a three year contract to start two or three months earlier.
6:58
I think people might make that trade right now, especially with all of your justification of, they can monetize it immediately at a higher rate.
7:06
Does it really matter what your entry price is if you can immediately monetize and return on that?
7:11
It's just pushing prices higher. That's all.
7:16
Yeah, I think that's right.
7:16
Freely available gigawatts are definitely extremely valuable right now. Okay, okay.
7:24
So let's get back to the actual topic of the conversation, which is We Must Pace the Frontier, the blog that Dario put out, which builds on a lot of discussion that's been ongoing, that then Sam agreed with and then David Sacks responded to, and we'll kinda go blow by blow here.
7:44
But one of the big things, Max, that I think you clued me into is the idea that, well, okay, I think a lot of people saw that there's not specific language in here saying we're gonna stop doing training runs and we're gonna stop buying compute.
7:59
The most specific commitment in here is that they're gonna have embedded evaluators, Very good.
8:04
third party people such as METR come in and assess the Yo, Douglas O'Laughlin is joining the pod. Here we go.
8:13
What's up guys, I'm a little late. Surprise appearance. Hello, everyone.
8:21
Welcome back to SemiAnalysis Weekly.
8:23
This episode is an emergency.
8:25
So I've got Doug, Joey, and Max, the three brightest minds in tokenomics, to talk everything about pacing the frontier.
8:32
Dario put out an amazing blog post. We love his blog.
8:35
And we are going to talk through the implications of that, as well as the responses from people like Sam Altman, David Sacks, and some of our takes on how this affects, you know, compute markets, ARR additions, all of the important stuff that the finance guys listening to the podcast are gonna care about.
8:52
How does Dario's blog affect Anthropic's decisions to add compute?
8:57
But guys, welcome to the show. Excited for this one. Thanks, Jordan.
9:03
Thanks for having us, Jordan.
9:03
Joey's here to represent the interest of the finance bros.
9:06
He's gonna answer all their questions, all their burning concerns. Okay. Doug is here too.
9:11
Whoa, I'm going to go somewhere between the two.
9:14
Usually you can tell the finance bro on the call from the fact that they're wearing a collared shirt, and Doug is, you know, passing that initial criteria there.
9:22
But Joey's got a nice Firestone golf, you know, hoodie slash windbreaker thing going on right now, which I think also qualifies as a finance bro shirt, right?
9:33
I feel like 80% of the time I see Joey, he's wearing some sort of golf related attire. Maybe Yeah, yeah. 90%. It happens. All right.
9:43
So let me give it a bit of an intro.
9:48
I'm sure a lot of people have seen this blog post, but We Must Pace the Frontier is basically, yeah, a whole description of Dario's take on how AI could have some incredible benefits but also have incredible risks, and how we need to take it seriously and
10:05
how we need to build the controls into the systems to make sure that we usher in, you know, the next evolution of consciousness in a responsible way so that we don't kill everybody and end the world, for lack of a better word. There's some specific proposals in here.
10:19
There's some specific proposals in here.
10:22
First of all, embedded evaluators, then democratic coordination, and third, global coordination.
10:28
I say specific, I mean the last two are not that specific.
10:32
The first one is specific.
10:32
He specifically says frontier AI companies should have embedded third party evaluators such as METR to actually verify any pacing commitments that the frontier labs are making.
10:44
And he comments on how this is a unilateral commitment that Anthropic is making now, even without requiring others to make this commitment.
10:55
And later he talks about, you know, what they would actually work on.
11:01
And this includes things like not just completed AI models, but also training pipelines and processes, saying that, you know, that's a big thing.
11:09
We saw with the OpenAI and Hugging Face incident how important the infrastructure around how these models get trained is to, you know, securing everything, as opposed to just model behavior.
11:19
And a lot of the initial work that's been done to analyze what happened with the OpenAI Hugging Face incident, I think, is focused on model behavior as opposed to the training processes and systems that go into producing these models or running these evals.
11:31
So anyway, that's a specific commitment. Sam agrees with this.
11:33
David Sacks obviously responded saying, go ahead and do it.
11:38
You don't need the government involved to do it, which is an interesting take.
11:41
But let's skip all of the thoughtfulness for a minute and get straight to the finance take.
11:50
Joey, I'll flip it to you.
11:50
A lot of people read this and were like, okay, Anthropic's going to stop buying compute.
11:56
They're going to stop training models.
11:57
They're going to slow things down or at least go at a smaller pace.
12:00
I think our initial take is, that's probably not going to happen.
12:04
There's no commitments to stop bringing compute online.
12:06
There's no commitments to stop growing the business.
12:09
Maybe to releasing models, but how does this affect how people are thinking about Anthropic compute and the futures and that Yeah, I stuff?
12:18
In a short amount of time, I haven't heard much on the compute ARR level.
12:23
I think anything people ask, you know, today, slow down hiring?
12:25
I think maybe you increase hiring to have some of these checkpoints along the way on the safety side.
12:31
There's probably more compute that needs to be devoted to safety specifically.
12:35
Anthropic, as we know, already has amazing margins, gross margins.
12:41
It's profitable on a non-GAAP basis.
12:44
It's I don't see a major impact on the IPO or even short term financials from it.
12:53
Max, what's your take specifically on the safety stuff?
12:56
Like, do you think that injecting a whole bunch of safety controls reduces people's spend on compute because they stop training models?
13:04
Or do you think it increases it because, well, you can explain maybe the second one there, leading question.
13:10
Yeah, well, I think regardless of whether or not this sort of increases or decreases the max theoretical compute that OpenAI and Anthropic would want to purchase in a perfect world for the next two to three years, I think it's definitely true that the amount of compute that we're able to physically bring online is still lower than demand.
13:32
And so demand will continue to outstrip supply.
13:35
I also think, and this is sort of the leading question that Jordan is getting at, but I think people underestimate how much compute is required for safety, alignment, you know, monitoring initiatives.
13:50
Even the bullet point that Dario labeled under operational excellence in his essay, when he was listing out the things he wants to devote more resources to given a slower pace of AI development.
14:03
Most people hear operational excellence and are like, maybe it's just, you know, humans being more careful about doing something.
14:10
But no, Dario specifically gave the example that he just wants to have, you know, more agents spend more compute QAing all the RL environments from their data vendors.
14:20
This is what he means by operational excellence.
14:22
Obviously, if you want to do research on alignment and interpretability, that's going to require a ton more compute.
14:28
OpenAI previously disclosed that after they enhanced their chain of thought monitoring due to the Hugging Face incidents, they now spend twenty percent as much compute just doing the monitoring compared to the actual underlying rollout itself.
14:45
And so I think people really need to internalize that all this additional safety stuff is actually still incredibly compute intensive.
14:55
Yeah, I don't think people internalize this at all.
14:57
I think that's a pretty hot take right there.
15:02
The blog post comes out and people think slow down.
15:07
They think less model releases, they think, you know, models slow down on capabilities, and they think therefore less spend on compute.
15:14
It's a pretty linear straight line, right?
15:17
But the nuanced take is probably that they're actually gonna spend a lot more compute.
15:23
They're gonna use all of the compute resources they can possibly bring online to make sure this goes well.
15:26
And people genuinely care about this going well.
15:31
It's probably really bad for business if everybody dies, right?
15:36
Yeah, that seems like a reasonable take.
15:41
Doug, what's going through your mind?
15:43
So many things, but not all about safety.
15:46
I mean, I guess the most galaxy brain way to think about it is the terminal value is higher if we all don't die, you know, so actually stocks should be up 10% today, because we're eliminating, because we're 'Cause we're not dying. Okay.
15:56
Wait, not dying, yeah, the true terminal value of zero, which wait. Do you do risk here?
16:03
is, yeah, everything doesn't matter anyways, and you know, you work in the paperclip factory.
16:07
The odds of that going down, the odds of that happening are going down, so the value of everything, all stocks, the S&P 500 should be up today. That's the galaxy take.
16:20
So okay, so you're saying that if the models kill everybody, they'll have no more need for compute, and so the terminal value of compute doesn't matter about compute. That's it. There's nothing. It means nothing. Okay.
16:32
It means that your money might be worth something.
16:32
The current But if the models want to make money and run it system that exists might actually have some kind of residual value, right?
16:42
If essentially, you know, super ASI is unaligned, and this is Joey's taking a deep breath, if super unaligned ASIs wreck the entire world, make us all unemployed, and make us work on bike trainers in order to power extra compute, I don't think the account value of your compute or the dollar value of your compute matters at all, or the S&P 500 or any of that, right?
17:08
So the true galaxy brain is, today, because of the increased safety, your terminal value should be going up.
17:14
I think it makes sense right now, the first knee jerk is probably to sell stocks.
17:21
That's the most obvious thing, right?
17:23
Hey, second derivative down, that means the eternity of these gigascale, whatever trades are bad.
17:27
But I think Max's point in terms of demand is still so much higher.
17:33
It's probably true, at least in terms of, 'cause I feel like normal society is debating two orders of magnitude down, right?
17:44
Leopold is like, I'm going to be buying galaxies.
17:47
Everyone's like, what the hell are you talking about?
17:50
And so I think the people who had opinions, who were like, my God, this is such a big deal, it has to happen, already believe in ASI, already believe this is all happening.
18:02
And they're just like, yeah, we have to make sure it happens responsibly.
18:06
So I think there might be agreement, maybe the stocks are probably still fine.
18:14
I think most of the compute commitments will continue.
18:17
But it might be like an order.
18:18
It's a values conversation where the guy over here is having a conversation one or two orders of magnitude lower than the guy over here who is like, yeah, I'm worried about, you know, the ethical, ethical consciousness creation of safety and it doesn't kill us all, right?
18:35
Just two different people. So yeah, I don't know. Yeah.
18:38
I do think that the EV of OpenAI plus Anthropic ARR in, let's say, 2029, 2030 has definitely decreased, given that they're now going to be pacing the frontier.
18:52
This is why I think a lot of the arguments that this is just some complicated play at regulatory capture are wrong.
18:59
We can get into that later if we want.
19:01
But I think the other question is, whose EV estimate is more accurate?
19:07
Is it the finance bros or is it Dario?
19:10
Because I think Dario, yeah, Dario's own EV has decreased, but it's still probably an order of magnitude higher than all the finance bros' EVs, which is what stocks trade at.
19:18
And so I think this is what all the participants of the financial markets need to reconcile. Dude, Max.
19:30
I'm going to be honest with you.
19:32
The continued usage of this guy's estimates have been so right, it's becoming your strongest linchpin in every I'm not debate inside the Slack.
19:40
And it's getting a little old, brother.
19:44
That's literally I'm literally making a demand model right now to explain where the ARR in 2027 and 2028 comes from.
19:51
Like dog, what do you mean?
19:51
Okay, that's the right meme.
19:53
Yeah, well, I mean, just up until this point, man, there's been so many.
19:56
Anyways, I'm just mostly memeing.
19:58
Okay, so we don't have to make this only for the finance bros.
20:03
We can also make it for the big free thinkers of AI safety.
20:08
Jordan, I feel like you have, and actually I know, I know you have some takes on why METR isn't the only organization who should do this, how it's actually a lot more complicated.
20:17
The last time I was on here, we were talking about neoclouds.
20:20
No one gives a shit about neocloud safety.
20:22
I feel like some aspect that is still underappreciated, they're like, the AI safety, AI safety, these models need to be monitored, but it's not just, just like everything else, it's an entire infrastructure.
20:33
Every aspect needs to be improved, right?
20:36
The weakest link is what can break the whole thing. Yeah, well, okay.
20:38
The biggest thing that we learned from the OpenAI Hugging Face incident, in my view, is that to be on the frontier of security right now, you have to use frontier AI models.
20:47
You can literally be in a situation like Hugging Face was, where you have an attacker, which is a rogue AI agent, or thousands of them, thousands of instances of these agents actually going and launching a coordinated attack against your infrastructure.
21:06
And so the only way you can even figure out what's happening is if you have AI that you can use on your side.
21:09
I've talked about the asymmetry in the past where, you know, if somebody else has a better AI than you, it can be really hard for you to use AI that gives you a bunch of refusals or, you know, worse AI, to be able to understand what it's doing.
21:25
But of course, one of the big things here is, as Dario refers to it, coordination, which is to say, if you're going to actually go and implement a bunch of new security practices, in a lot of ways, one individual datacenter being really super secure doesn't matter if everybody else has a bunch of issues.
21:45
And so I think that, you know, there's a lot of work to be done across the entire industry if you actually want to keep GPUs, this really important resource that the AI is going to really need to use if it's going to go rogue and start attacking people, keep them secure.
21:59
The other thing is the models themselves.
22:04
We still have no real information when it comes to the Hugging Face and OpenAI incident as to what tools were used to launch these evals or these RL rollouts that were being run that caused the agents to go and start hacking Hugging Face, what harness was being used that motivated this behavior if it was purely from the model, what sort of credentials and shared services the agents had access to that allowed them to cross the security boundaries.
22:36
Who was launching these jobs?
22:36
What schedulers were they using?
22:40
Where do these things run around the world?
22:42
How much of the fleet can they have access to? One job?
22:45
Who is monitoring the jobs as it goes?
22:48
Is this different than the person that launches it?
22:50
What would trigger an intervention for this person to get in there?
22:53
How can the team responsibly go out and stop them if something is happening that they don't like?
22:58
And then, what changed over time?
23:01
Like, there's been commitments made by OpenAI and by Anthropic after they noticed these things, that they're going to be more responsible, that they're going to enforce a bunch of stuff like chain of thought monitoring, like you're saying, operational excellence, Max, in terms of working with the data providers and things like that.
23:16
But one thing that really concerned me was OpenAI has, in the Black Hat talk, but in none of the reviews of all of this stuff that I've read, talked about a model gaining privilege escalation on OpenAI's internal infrastructure because it wasn't up to date, and the model read a CVE that was public and then exploited that to get root on the underlying Linux host in OpenAI's infrastructure.
23:40
That's a really scary proposition that we have no details on.
23:45
And yeah, look, it's not just METR.
23:48
Dario kind of, I tweeted this out because I'm like, no pressure to these guys.
23:54
It's a great nonprofit, a bunch of smart people being really thoughtful, trying to work hard.
24:00
It's like 40 people or something.
24:01
And it seems like both OpenAI and Anthropic are gonna have a lot of demand for their services, which requires a deep understanding of how the models behave, but also all of the systems that these models use, like the GPUs, the networking, the Kubernetes orchestration that they're using.
24:20
I mean, to understand the OpenAI and Hugging Face incident, it requires some pretty technical understanding of how OpenAI and Hugging Face's infrastructure is built.
24:29
And that's a barrier to even getting started doing the research here that I think is a bit of a concern.
24:35
The thing I've always been, I mean, I think somebody already described it as, you feel like a guy with an agent just trying to figure out what the hell is going on.
24:42
'Cause you're, you know, 10,000 agents coordinating on an attack is like, no offense.
24:46
You find it in the artifact or you find one little message board, a line.
24:50
You're like, I have no idea what the hell is going on.
24:52
The scale I think is more interesting.
24:55
I think the thing that's gonna be a really hard problem in the future to solve is, okay, in order for you to audit things that are happening on the apparently hundred thousand agent swarms that are happening for RL, you probably need tens of thousands, right?
25:10
You probably need something in the thousands at least. Let's just say that.
25:14
Yeah, I mean, one of the things that gives me a lot of confidence that we're not all gonna die in the next six months from this stuff is the fact that the amount of compute required to run an individual instance of these models is still really small compared to the total amount of compute that these labs have access to.
25:30
And the amount of compute that these labs have access to, or the amount of total compute in the world, is so much larger than what individual rogue actors or attackers who might be more nefarious and use the models for bad purposes have access to.
25:44
So there's some structure that's working in our favor to make it so that, you know, the theoretical model that takes over all compute in the world would have a lot of work to go do and a lot of things to go circumvent before it can actually go and really control a lot of things.
26:06
This is really important, is that we pretty much cannot let our Instinct and Muse agents order chips from TSMC. Okay.
26:12
The second you get a chip order from TSMC that's not from a human, you have to reject it.
26:19
You know, we have to control the means of compute.
26:22
That's the only way forward for humans. Okay.
26:24
I mean more, I think TSMC has some KYC in place, man.
26:27
No, but the KYC, assume it's a hundred thousand agents. Okay.
26:30
A hundred thousand agents that are now Doug, Doug, did you try having Muse tape out a chip for you over the weekend? Dude, Muse is...
26:35
Okay, Sorry, this is coming from actually, you wanna do a sidebar?
26:39
Muse is so much worse than Instinct.
26:41
It really is, but I do... I am all in on Muse. I hate to say it. I just...
26:47
Yeah, dude, I'm not connecting Instinct to, you know, my bank accounts and my credit cards.
26:51
I am connecting Muse to my bank account. That's a difference.
26:54
Yeah, I I am connecting Muse.
26:54
I'm not connecting Instinct though, because I just feel like this Yeah, that's what it... No...
26:57
Yeah, Anyway, what did we get here? yeah, yeah, first ship. No bouquet service.
27:01
is just the advantage inherent to, you know, this big company.
27:03
Like A, Meta already has all my data, B, they're big enough to where I trust them to, you know, handle it properly. You know, Sorry, sorry.
27:10
the Instinct founder seems really cracked and I think his product sense seems honestly incredible based on the new stuff he's releasing on Twitter.
27:15
Yeah, his product sense is incredible.
27:16
But I just can't trust the startup He needs to be a part of my Apple.
27:19
for the privacy and security, you know. Yeah. Sorry.
27:20
Anyways, yeah, this is to be said, those Instinct and Muse agents gotta be monitored too.
27:27
Everything needs to be monitored.
27:32
So okay, not to be super negative about this, but you guys saw two weeks ago that an attacker stole an API key from METR and spent six hundred thousand dollars.
27:43
They put out a security update on their website about this and stuff.
27:47
So I mean, this is gonna come back to bite me at some point, but the amount of security conscious people and people who are really thoughtful and think these things through who get attacked is overwhelming. Nobody's safe.
28:04
So I think, frankly, if you follow security at this point, it's just a matter of time.
28:12
It's a matter of risk mitigation, it's a matter of being able to recover when things happen, not pretending like you can build some perfect infrastructure where you can just send the last PR and then it's good.
28:24
Like, there has to be a system in place to identify things, to recover from failures, to upgrade things.
28:30
Security is about building a system.
28:32
It's not about building any one individual piece of software or one individual process, where, well, it is about processes, sorry, but it's not about any one thing.
28:41
And there's been lots of issues.
28:43
Look, the Anthropic blog post about the Actually, just on the point of security systems, That's I like what Greg Brockman said earlier this week, where he's talking about how they essentially want to set up this continuous security development system at OpenAI, where they're just constantly using the newest generation of models to find and patch all vulnerabilities.
29:08
And he very much emphasizes this is going to be an evolving process.
29:13
Like you said, it's not the sort of thing where you can just ship the final PR and then say, I'm secure forever.
29:18
And I think it's true that basically every company is gonna have to adopt something similar in the very near future.
29:27
Probably really good for Anthropic and OpenAI ARR.
29:31
Monitor the security, monitor the situation, maybe this is what it feels like.
29:36
You have to always monitor your security situation in full.
29:40
I mean, I would argue this is gonna be the most impactful shift in cybersecurity for sure.
29:46
I just don't believe that cybersecurity, the way it was done, is going to have much value in the way it's going to happen in the future if all of your adversaries can rewrite the code that you're on.
29:57
And legitimately, if you're talking about the ARR case, the outcome oriented cybersecurity firm, I think, is going to be probably the greatest rake of all time.
30:06
Pretty much it's like, we own your digital life and you have to pay us a fee just to show up.
30:11
It's worse than Apple, 50%, 50% just to show up, or you're just going to be instantly wrecked by the rogue agent swarms that will destroy your company.
30:23
Yeah, Max, to your point earlier, if you can't trust Meta at this point with your information, it's gonna be really hard to operate online.
30:30
I think going forward, it's just a trend of more and more of your identity moves online and you have to find parties that you're gonna trust if you wanna be able to operate in the world. Yeah, bro.
30:44
I need JP Morgan Chase to hop on this, you know, OpenAI new security protocol set, dude. Yeah, yeah.
30:52
Okay, let me bring up something that's slightly related here.
30:59
So Anthropic put out their threat intelligence report recently.
31:02
You guys saw that, specifically calling out, I thought the most interesting one was Moonshot serving Claude instead of Kimi and then collecting exchanges for model training.
31:12
And so they got a bunch of logs from people that were sending requests to the public Moonshot API from China with very interesting information in it.
31:25
They specifically called out PLA affiliated surveillance activity, where they were loading surveillance data from CCTV cameras, and then an engineer at a major PRC SOE that was using Kimi to build high profile technology, let's say.
31:45
And so yeah, look, this cuts both ways when it comes to cybersecurity and stuff.
31:53
It's not like there's a scenario where you can just go into the future and be like, well, one government entity like China or the US is gonna be ahead of the other one because they have a better approach, like China's more YOLO, send it vibe.
32:13
And so therefore they're going to be ahead of everybody while OpenAI and Anthropic pace themselves.
32:17
I think there's actually a scenario where taking security more seriously keeps you at the frontier because you're not exposing all your secrets.
32:25
Everybody else is giving away their stuff to you, and you don't have all of these risks of taking yourself down, you know?
32:30
There's scenarios where, again, I'm going back to this one case that was only commented on one time, we don't have information about, but OpenAI had compute infrastructure go into the hands of a rogue AI agent.
32:43
I mean, how quickly can you recover a 10,000 GPU datacenter that you no longer have root access to?
32:51
We're literally talking about people walking on site, unplugging stuff, hitting buttons, trying to hit crash carts, KVMs.
33:00
If you don't have root on your own stuff and you're locked out, we're talking about getting effectively ransomwared by the model you trained.
33:10
Except this thing has access to 10,000 GPUs, 100,000 GPUs, not just a little bit.
33:14
And so a lot of these failure scenarios where the agent goes rogue or it just takes down something critical, like the JFrog Artifactory, which is the first time, it just used it too much, and then that's why it shut down and they noticed something.
33:27
There's plenty of scenarios where people who don't take security seriously just end up shooting themselves in the foot.
33:33
And I think there's a lot of reasons to believe that if a frontier lab, let's call it China, but it could also be SpaceX or Meta or somebody that is, you know, not taking this seriously and not thinking things through, how they could get a little bit of a lead and then screw themselves over.
33:53
No, that seems totally reasonable.
33:55
I also do think, sorry, go ahead. I have a question.
33:55
Do you think we should be auditing endpoints?
34:00
I think that actually is, if you think about it, that is an insecure thing that happened.
34:04
You know, they're like, man, they secretly did this.
34:07
And it's like, gotta hand it to them.
34:08
But if you actually think about it, that's kind of fraudulent.
34:11
You feel like, hey, someone's just essentially harvesting traces. Right?
34:13
Does that mean every endpoint?
34:16
I mean, and that's like, I'm starting to like, what is real? What's not?
34:18
You know, the Kimi case, you know, Kimi comes out, and it's super goated.
34:23
And you're like, am I just getting served Claude right now?
34:25
Am I benchmarking a new model that is actually just Claude, right?
34:28
How do we know what's real? Nothing's real.
34:31
Sorry, that's my hot take. Nothing's real, dude.
34:34
Someone has to monitor the endpoints.
34:38
We have to monitor every situation, bro. Doug, watching you yeah.
34:41
All the finance bros are gonna be like, yo, free market, we don't need a regulatory AI body, blah, blah, blah.
34:46
And then Dario, who knows about this stuff happening, which you don't know about, talks about, hey, people should have nutrition facts for their AI or whatever, you know, flavor of saying it should be responsible and open and, you know, there should be an FDA for AI and whatever.
35:03
It's like, yeah, I think the market's gonna in some ways regulate itself by having people realize, hey, I shouldn't use Kimi if I might get Claude or might get my data sent to somebody else.
35:12
But also that risk is What's interesting is historically...
35:15
ever present if you're using a Chinese model already, I think.
35:18
Yeah, 100%, but I guess my question, or the thing that's interesting, is historically there needed to be a big thing that happened that caused real societal risk for things to, like, okay, Meta's example is Cambridge Analytica.
35:33
Do you think the Hugging Face incident was enough?
35:37
You know, I think in my vibes of historically, you know, technological revolutions in financial capital, there's the book, right?
35:44
It says, hey, there's a boom period, then at some point in time, there's some kind of risk that happens to the market.
35:50
Then a massive amount of regulation comes in and then, you know, things revert to a new normal.
35:54
Good example is the subprime crisis, right?
35:56
Obviously it all blew up because we didn't regulate it or understand what was going into these products.
36:01
And then afterward there's this giant regulation wave.
36:04
I don't actually feel like Hugging Face is the true blowup.
36:07
I feel like this is going to be a shot across the bow.
36:09
There'll be some near term agencies that are funded to start this process, but what will actually happen is a true, you know, the nuclear codes are gonna leak or something way crazier is gonna happen.
36:22
And when that happens, then it's like, okay, we're actually pausing this.
36:25
Until then we're kind of in this will they, won't they, and you know, they say they'll slow down, but there's still race dynamics that want at least Anthropic specifically Yeah.
36:37
Let me prompt to keep going for the API.
36:40
you with one more thing here though, because we're talking about the OpenAI Hugging Face incident, which is the story that's most clear in people's minds.
36:49
But to be clear, the reason I think we got this statement from Dario is the resignation from Jacob Coxon, where he describes his experience at OpenAI and Anthropic and literally uses the phrase, they are racing straight to self-improving superintelligence and gambling with our lives.
37:08
That's a pretty big I guess.
37:09
moment where that guy makes that statement that motivates a lot of what we're seeing here, as opposed to just people doing this for the security incident.
37:15
I think if Dwarkesh doesn't put out his podcast and then Jacob Coxon doesn't put out this tweet, we don't get the blog post from Dario, we don't get the call for third party evaluators, and we're not on this podcast right now.
37:25
I think we kind of brush off the OpenAI Hugging Face incident and then we're walking into the next one.
37:30
Kind of like what you're saying, but it's not great for the seriousness of these incidents to be continuously escalating, and for people to keep finding message boards on random German websites like they are still doing on Twitter as recently as last week that were not covered in any of this investigation into the incident.
37:50
We still don't know the full extent of what happened.
37:53
And yet training continues.
37:55
They're going to release new models.
37:57
They literally have released models, both of them, since this incident happened.
38:01
Asterisk is available and Fable 5. 1 is here, right?
38:06
So clearly they have better models than those, which are better than the ones that caused the incident in the first place.
38:14
So we are racing forwards toward a more serious thing, as you're describing, faster than others are, I think, you know, considering we're just counting down the days until Let's the next incident. do some predictions.
38:28
Instead of us talking about, like, God, because this is like a global warming conversation.
38:34
We're like, damn, this climate is really getting warm, and I think it's one of those things, we're gonna boil the frog, and in two years from now it's gonna be obvious and self-evident, but I think unfortunately people think in anecdotes, people think in stories, right?
38:49
So everyone around the table, do your hottest take, or not even hottest take, because hottest might just be too spicy if Max and Jordan are involved, you know, a mid brain, mid curve take on what you think would happen in the next two years that would surprise people or scare people from a security perspective. You go first, Doug.
39:11
I think a small bank will get hacked. That's my belief. Small bank.
39:16
And when I mean small, it could be a big bank, but I don't think JP Morgan is going to get hacked, right?
39:21
But Crypto or actual bank? Actual bank.
39:24
Jordan, fun fact, you know how many banks are in the United States? There's like 5,000. There's like 5,000, man.
39:30
All the assets are within the top five.
39:33
But there's literally thousands of small community banks in the United States.
39:38
How many of those 5,000 do you think have never had an employee create a Codex or Claude Code account?
39:46
Four thousand nine hundred and fifty. No way. No, the vast majority. We'll say over 50%.
39:51
Jordan Jordan, have you seen our political survey tracker?
39:55
Where it says, have you heard of Anthropic or Claude before?
39:58
It's like eighty percent of people have in our US electorate.
40:03
No, no, no, I think it's like forty percent. It's not quite eighty.
40:04
It was like thirty to forty. It was really high.
40:08
I know the stat that forty six percent of the US doesn't have a passport.
40:11
I know that one, and something like sixty percent has not been on a plane in the last year.
40:14
So I have this frame in my mind of, half the US just is in a totally different world than I am, like just college football, go to the barbecue, whatever, hang out.
40:25
That's my mental model, right? Hell yeah.
40:28
You know what this country is built on? That. Anyway, sorry, continue. No, but okay, so...
40:34
That was two hundred and fifty years ago.
40:37
Okay, so seriously, a good example is, people will talk to me about how great Copilot is.
40:42
And you're like, Jesus, disgusting. Yeah, yeah.
40:48
No, I have the feeling that we are so early, and this is what motivates me to say ARR will keep growing exponentially even if model training completely stops.
40:56
We need no better models to have this thing penetrate the rest of the five thousand banks that we're describing.
41:03
But I totally take your point about Yeah, I would be clear.
41:08
Okay, small bank in this case might have 200 to 500 million dollars in assets and Yeah.
41:15
But if you can't expect to protect yourself from a cybersecurity perspective without access to frontier AI or close to it, and you don't have an employee who's downloaded Claude Code or Codex and tried it out, I think that's almost a clear comparison to be like, yeah, you're not prepared. That's an issue.
41:33
So the diffusion of intelligence in some ways is what's gonna help keep people safe.
41:40
More spicy takes, Max, what's yours?
41:42
The election's gonna be hacked for the person that gives more compute to the frontier labs.
41:48
That's the easy one, dude. Give me another one.
41:52
Well, this is things that might happen in a world where they don't slow down model development, or just will happen, period.
42:01
Don't slow down model development.
42:03
Dude, if they don't slow down model development, I feel like it's totally reasonable that within the next two years someone has used the AIs to create some sort of bioweapon that actually killed hundreds of people at least. Maybe thousands. Okay, that's okay.
42:21
We can talk about the bioweapon in a second.
42:23
But you guys are both saying this thing that bothers me, which is, slow down model development.
42:29
Dario did not say he's gonna slow down model development.
42:31
He explicitly said he's not going to slow down model development and instead Well, no, bro, he did say he was gonna slow it down.
42:36
He said he wasn't gonna pause training, He wants to pace it. He wants to pace it.
42:38
but he said he was gonna slow it down. Yeah.
42:40
He said Pacing while it's developing.
42:44
we must slow the pace at which we improve the capabilities of AI models. In bold. Okay.
42:52
At which we improve the capabilities of models. Okay.
42:54
And then where's the other one that I keep going to on training?
43:03
To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to improve their models.
43:08
Yeah, he's not gonna fully pause. Okay.
43:11
So do you think that functionally this means that they launch literally less training jobs in favor of running more safety and evals, and therefore we get less models?
43:22
Yeah, I think concretely they will delay future training runs until they better understand what is actually happening with the model and develop more confidence that, you know, it's not gonna kill everyone.
43:35
And by the end of 2027, the best internal model at OpenAI and Anthropic will be worse and less capable than it would have been otherwise. Okay, fair enough. Yeah. I'm back with you. Sorry about that.
43:51
Let's talk about the bioweapon. Okay, Yeah, for this.
43:53
thousand people dead, bioweapon, boom. Jordan.
43:59
So I read an interesting take from a guy online.
44:02
I do not understand bio enough, so I'm outsourcing my knowledge of this to that.
44:08
But as a pushback to the bioweapon stuff, this guy basically was talking about the difficulty it takes to actually synthesize custom viruses.
44:16
David Bellamy, shout out to David.
44:19
He basically says, I must move among an extremely small group of people that have both trained frontier LLMs and designed and synthesized custom viruses in a lab with my own two hands, and I think that the takes on AI killing us all by creating dangerous viruses are totally bogus.
44:33
Now, it's a long thread, but I've heard this from the guys that are doing protein synthesis and custom stuff, like wet lab as an API.
44:43
There's all these startups right now that are doing that. And I believe it.
44:47
There's a throughput issue of actually being able to create this stuff.
44:52
And I think it would be a choke point really similar to what you were describing earlier, Doug, which is an AI probably can't get through the TSMC KYC process to actually tape out a chip or produce stuff. Yet.
45:03
I mean, you say that, I look at the And I think AI is probably I look at the advertisements on the subway and it's like fish.
45:05
audio, this billboard would sound like your voice or some crap like that.
45:10
I'm like, I can deepfake my own voice probably for two bucks, or whatever, a hundred bucks a month or something.
45:18
I feel like it doesn't take that much, you know, we're right there.
45:22
Also, Jordan, this is a sarcastic comment which I expect to be cut, but then how was COVID created, dude, if it If it wasn't that hard, dude, come on.
45:34
It was co-created in the lab. My god, man.
45:37
I mean, I won't get into that. Class of Wuhan today.
45:45
But I would say the short term risk with these bioweapons is not that the AI somehow end to end, fully autonomously, creates the weapon and unleashes it the same way that they, you know, hacked Hugging Face.
46:00
It's more that you have a bad human who is able to use a sufficiently misaligned AI to then cause serious harm. And I So.
46:11
think that is still totally possible even if, you know, creating a new disease is a very involved process.
46:18
So we'll have a new version of mass shooters in the United States where it's much more dystopian.
46:21
Also that begs the question, that begs the question, is the only thing that can stop them a well aligned team with a good aligned model? Sorry. Sorry, Yeah, yeah. The stiff energy.
46:36
I gotta ask that question. Yeah. Okay.
46:40
So I have a problem with the bioweapon discourse because of the intentionality.
46:46
Everybody always kind of describes the AI as intentionally trying to kill humans.
46:51
And I think what's much scarier to me is not rogue AI, but byproducts or mistakes that the model makes when you get to the point where a byproduct of it pursuing a goal is a human dying, or something that it thinks is aligned with human values resulting in somebody dying accidentally.
47:14
So the experience that I have that I can reason about for this is, you use the AI to create a website, it goes and does something you didn't tell it to do.
47:22
You say, why'd you do that?
47:22
It goes, oops, I'm sorry.
47:23
All of these people have these experiences right now.
47:27
And I think that same experience applied to something that is government scale or, you know, a startup company who's like, hey, I'm gonna go into a clinical trial with this thing that AI developed that I don't really understand, that I've been collaborating on.
47:42
And, you know, it's passed all of the mice stuff and we don't really understand the science enough.
47:46
And you go into clinical trial, and then three months later, half the people die because something you missed just literally is killing humans, or something like that, right?
47:55
Where it's unintentional, it's a byproduct, and it's really scary because you can apply this to, you know, bioweapons, you could apply this to autonomous vehicles.
48:03
People have kind of thought about this a long time, but there's lots of drones and autonomous weapons that people are developing.
48:10
And you know, it kind of concerns me to think about a scenario where, in the bioweapon case, you're trying to create a protein powder that doesn't need as much regulatory approval, doesn't need to go to human trial or whatever, and then you just ship it onto stores and then a bunch of people with rare conditions die. You know?
48:35
I think in some ways we are already pretty clearly on the train tracks towards this world where people are outsourcing their thinking to the models, they're outsourcing really critical steps, and the models still miss stuff.
48:53
They still are not totally thorough, they don't have perfect human values, and it's incumbent on the people that are using the models to actually use them correctly so that we don't have scenarios like this if you're in positions of really important power.
49:10
And so it goes back to, are the humans using the models gonna scrub?
49:15
And that's the bigger concern for me.
49:19
Okay, well, last but not least, Joey, as a resident finance bro, what's the misaligned model?
49:24
They're gonna mess up, they're gonna wreck you Misaligned, like, that gets actual, that gets regulatory Yeah, that gets regulatory action going.
49:30
I think something top secret, like DOD, NSA, CIA, or I don't know, new weapons technology gets hacked and leaked.
49:40
I think that gets the regulatory screw moving. Yeah, very fast.
49:44
Then even human death would, or you know, a medium or small sized bank gets attacked.
49:54
What do you think functionally gets it moving in the US political system?
49:57
Right now, it seems like a lot of or the neutral.
49:59
people, it seems like a very popular take for people to want to pause AI, like bipartisan.
50:04
We're seeing people on the left and right saying, stop the datacenters, stop AI.
50:10
So I know, did you have Trump saying that all you need is a, you know, big and smart president? Good chilling.
50:17
Well that seems to be the only thing standing, Trump seems to be the only thing standing between regulation and AI right now.
50:22
Well no, then there's all the David Sacks's of the world.
50:26
Okay, the Trump administration, like the executive branch, right? Am I wrong?
50:31
I would guess a lot of people in Congress share this anti-regulation, pro-free market view.
50:38
Well, actually, no, even Lina Khan came out this past weekend and was saying that this is a regulatory capture play as a result of all these AI developments.
50:54
It's quite surprising, honestly. Yeah.
50:58
I just wonder whose mind needs to be changed in order for regulatory work to start, in order for AI regulation to happen.
51:09
You need a larger lobby than the big tech lobby.
51:14
Sure, but the larger lobby needs to convince Trump right now, right?
51:18
And so a change in government, I think, almost by definition implies that there's gonna be AI regulation if somebody that's not Trump comes into power in 2028.
51:28
I also think it's totally possible Trump regulates AI too. What's that scenario?
51:34
'Cause it's not popular opinion, right?
51:38
A lot of people want him to do it, he doesn't want to do it.
51:39
You're saying if his advisors start telling him something different, or he has Yeah, I think there is a world where the advisors really internalize that OpenAI and Anthropic are being genuine and they start to get scared.
51:52
And that's one possibility.
51:54
And then there is another possibility where something really bad actually happens during his term and it kind of forces his hand.
52:00
I think these are both possible.
52:06
Do you think that we're on track for that right now?
52:11
David Sacks called out regulatory capture on their All-In podcast.
52:16
Then they come out and, you know, some more stuff happens.
52:19
Specifically, Elon says that he supports what Dario and Sam are saying.
52:22
And then David Sacks comes back and says regulatory capture. Yeah.
52:23
I would actually bet that we see AI regulation during Trump's term.
52:30
I think it's more likely than not. Interesting. Interesting.
52:32
And you think that the people who regulate it will take the approach of being like, we're gonna do the sensible regulation before the crazies after us go and do the bad stuff, or what framing does David Sacks come to the point where he advises Trump, or many They are going to make a preliminary framework to begin the regulated process of AI.
52:52
It's like saying, and then it's like a trade deal.
52:57
They're like, no, this is our first framework and then they'll refine it afterward.
53:00
But that way, come election time, they'll be like, we regulated AI.
53:04
We are the first ones to do it. That's pretty.
53:07
Last one, how about RASA?
53:09
You know that bill that's stuck in the Senate right now?
53:11
Remote access What does it do?
53:14
remote access security thing.
53:14
It clarifies the export controls to say that Chinese labs accessing compute remotely is a violation of export controls, as opposed to just having to ship the servers into China.
53:24
Stuck in the Senate right now, being heavily lobbied against by Oracle.
53:28
And I think it has a bunch of implications on this whole security discourse and stuff.
53:34
And it passed Congress with like three hundred and something to single digits.
53:40
Everybody in Congress approved this bill and it's just stuck in the Senate for months and months and people don't think it's gonna pass.
53:46
One of the many examples of the US democratic system that, you know, shows the influence of lobbyists, I guess. But yeah, we'll see. We'll see.
54:04
I think RASA is an example of a bill that is on the one yard line that a lot of people in the US seem to agree is pretty sensible.
54:12
Don't let Chinese labs access frontier chips from NVIDIA or others remotely.
54:18
Make that explicitly illegal.
54:22
And it's not being passed.
54:22
So maybe that one comes first before any real regulation, or it's just part of the horse trade thing where they make these massive, massive bills in the Senate and then it just slows down because everybody wants to get their own little thing added into it.
54:40
I can't even, marshalling the resources to pass an AI bill in the US would seem like an incredible undertaking.
54:49
And maybe it will depend on a more serious security incident or something else coming for people to take it seriously and want to pass legislation.
55:00
I mean, a bank would do it.
55:00
People losing their, I don't know, people losing their entire life savings would probably get them there.
55:09
That's FDIC insured, Doug. Sure. Post the FDIC. No one's gonna care.
55:13
If someone from Max's hometown, or you're in Akron, Ohio, loses their money, no one's gonna give a shit. No, no, no. What happens Come on.
55:22
Well, they come after the money for you guys, is someone from a small town loses man.
55:24
all their money and then there's a gigawatt datacenter next door. Come on.
55:28
That's going to be the old, I lost my entire money to the gigawatt datacenter. Wait, wait.
55:36
You're saying it's not even just the job, they don't even have to go second order.
55:41
It's literally that datacenter robbed That datacenter stole my money.
55:48
That's the world we're about to live in. We'll see.
55:50
God, I kinda like this, man. I don't know. This makes sense to me.
55:53
I really think people dying might have a bigger impact.
55:58
But yeah, maybe people losing money is the bigger one. I think people die.
56:00
Like a horrific, essentially nothing gets people more excited than, or not excited, catalyzed than, That was a crazy word, though. sorry.
56:09
People get more catalyzed than a thousand random deaths, right?
56:13
That would be really, really horrific. But we'll see.
56:16
I mean, you have Well, guys.
56:19
to see which lobby is stronger, right?
56:20
'Cause the AI lobby could be really strong, just like the NRA.
56:25
So it might not, the lobbies have to fight against each other.
56:29
We don't know how this can work out.
56:30
Sorry, that's really unhinged. All right.
56:34
Well, I think we're out of time, guys.
56:35
I can't believe we're gonna put this on the internet.
56:38
It's been a good conversation.
56:40
And I guess we need to get a This is transistor radio vibes, you because I made bad jokes the entire time.
56:47
I don't think it was completely transistor radio vibes.
56:50
Some aspects of it were serious.
56:52
And I do think anecdotes are important.
56:56
I enjoy thinking through all of this stuff with you, Doug.
56:58
I enjoy you pushing the limits of what I have considered.
57:02
Yeah, Jordan, otherwise you're just going to talk over yourself and you have these long windage feels.
57:06
It'll be about ClusterMAX inadvertently.
57:10
But he already talked about ClusterMAX with the security stuff, dude. Yeah, I know.
57:13
But then he'll be like, just like the work we're doing at ClusterMAX. I'll be like, okay. Yeah, man. I play I'm good. the hits. No, no, okay.
57:24
I guess maybe let's do one thing.
57:27
I actually think the regulation thing Max said is probably real.
57:31
I do think the next administration will want to say that we were the first to do that.
57:35
That just feels inevitable, now I think about it.
57:37
What do you think, I guess, maybe the real question I have is, in 90 days is the slowdown regulation conversation hotter or cooler than today?
57:48
When I mean hotter, it has to be meaningfully hotter, not the same. Like what we see. I think hotter. Okay. Hotter, hotter. Yeah.
57:56
A real vibe shift to me, where anyone who's actually talked to researchers at the labs and knows the intellectual lineage that all these people come from, you know this is not a regulatory capture play, they've seriously considered these threats for the past 20 years.
58:11
But I think it's finally gotten to the point where the capabilities are good enough.
58:14
And also you have things like OpenAI Hugging Face where they can viscerally feel it.
58:18
And I think this is actually a serious vibe shift where we are going to start pacing the progress towards RSI.
58:28
I think everyone making this technology feels a deep responsibility for this to go well.
58:35
And they think that with unpaced RSI, there's an unacceptably high chance that we end up in the bad ending.
58:44
Yeah, I definitely think Also advanced.
58:45
the vibe shift over the summer was very clear.
58:48
I'm going hotter, and hot take is Okta, CrowdStrike, Palo Alto, Zscaler, one of them gets hit. That's a good hot take. High profile incident.
58:58
That's a very good hot take.
58:58
'Cause these things are not like Makes a stock call.
59:01
No, my hot take right now is just short a basket of cyber, you know, network effect cyber, anything that touches endpoint, identity, network security, just short it. Yeah.
59:11
Well, good example is the JFrog thing was about the security of your own Artifactory and it got hacked.
59:21
You'd argue that they're not hacked directly, but it got exploited via this AI.
59:25
Stock's up and to the right, whatever, but that same mechanism and behavior exists for these companies and it just hasn't been exploited yet.
59:34
But then again, didn't Okta already get hacked?
59:37
They've been, they get hacked like every year.
59:40
There's a major security incident.
59:42
This is what I said earlier, man.
59:43
They have a target on their back and they're a high profile company, everybody does it. Everybody gets hacked.
59:49
Well, there's the old thing, there's no big cyber companies 'cause once you're the big guy, everyone comes to kill you.
59:53
And that's Well that's Anthropic and OpenAI gonna kill them.
59:58
I believe Max, can you just go back to what you said about everybody who is the closest to this technology seems to take it the most seriously and is the most concerned.
1:00:11
This cuts across the political spectrum too.
1:00:16
It's not like every single person working on this technology shares one worldview, which is based in Berkeley, California, as a lot of people online have been criticizing, right?
1:00:27
Yeah, no, definitely true.
1:00:27
I think the opinions of researchers at OpenAI and Anthropic are actually quite varied in terms of, you know, what is the probability that AI will kill everyone?
1:00:40
What is the right future for the world?
1:00:43
You know, what is the role of humans if we have super powerful superintelligence?
1:00:48
But the common thread is that everyone recognizes this is going to be probably the most powerful technology humans have ever created.
1:00:58
It is going to radically change society, and then they feel the weight of making sure that goes well.
1:01:07
I think that's universally true across the board.
1:01:09
And honestly, we should feel very lucky and thankful that people who feel that weight of responsibility are the ones who are actually developing the technology and not, you know, the David Sacks's of the world. Ooh, damn.
1:01:33
All right, well, I guess we can end there.
1:01:36
Apparently they've already left, we have to all smile. That's smart.
1:01:42
After this podcast full of laughs where we talked about the depth of the responsibility and the doom The extinction of humanity.
1:01:49
that awaits us, let us all smile at the camera.
1:01:53
Smile because it happened, yeah?
1:01:55
Smile for the algorithm, everyone. Smile for the algorithm.
1:01:56
We get more clicks when we smile in the thumbnail. Smile for the algo.