0:01
for our first ever advertisement.
Search this video
0:03
We're gonna be doing a lot more of this on future SemiAnalysis Weekly episodes.
0:06
SemiAnalysis is a reader funded publication. Over to Dan.
0:09
What are we advertising this week? Yeah.
0:09
Well we noticed that ninety percent of our viewers aren't subscribed so if you could hit the like button and subscribe and be sure to hit the alerts. Seriously though.
0:21
the real ad, we're hiring for a new group which is called Compute Capital Markets.
0:25
And we're gonna look at the loan demand how the financing markets are evolving.
0:31
for financing GPUs and we're gonna look at markets which talks about the tools that lenders and everyone in the market need to evaluate how GPU finance is going how the GPU industry is going that's rental pricing, residual value all those things that come with it.
0:48
and so we're looking for credit analysts.
0:50
We're gonna hire four slots.
0:52
We're looking for Singapore and New York City.
0:55
we're looking for a junior credit analyst about you know perhaps one to three years of experience In the credit space, investment banking in Singapore or the US.
1:04
We're also looking for a credit specialist.
1:06
That could be someone who's been a little more experienced probably four or five years credit trading credit sales, or credit investing.
1:16
You know, IG high yield would be great.
1:19
and also a credit analyst, anyone who's been a desk analyst buy side, sell side.
1:23
you know we're looking for people who are credit-minded. I think that.
1:29
if you enjoy this episode that's gonna come and all these structures and you know all these Z-spreads and T-spreads I think you're gonna really like this job.
1:37
So you know, click the apply button right here. Hello, everyone.
1:48
Welcome back to SemiAnalysis Weekly.
1:50
In this episode, we got Dan.
1:50
we're gonna talk about everything to do with capital and markets how we are funding the build out right now.
2:00
Dan has been digging in to everything credit capital, markets, compute.
2:04
specifically we put out an article on September 11th called Nvidia's Backstop Universe: Headwind or Tailwind where we talked about the eleven trillion dollar AI build-out Nvidia's backstop economics, and the limits of Nvidia's balance sheet.
2:21
So, you know, with that as a backdrop we're gonna talk through the contents of that article.
2:25
But at the pace things are moving right now that's like already a month old.
2:29
And so yeah without further ado, Dan, why don't you lead us in talking a little bit about Nvidia's backstop universe how you came to set up this, you know article and why you dug into This expanded view of Nvidia's balance sheet. yeah.
2:47
So let me show you a bit about Nvidia's backstop universe.
2:52
And indeed, it was just a month ago but things are moving so fast.
2:56
At this point, funded announced is around about like a hundred billion each month.
3:06
And this is how we looked at the balance sheet.
3:09
And the whole premise is Nvidia's been giving a lot of backstops and the question is why? Right.
3:16
there hasn't been a lot of visibility on how much issuance is going to come or how much funding needs there is.
3:22
so we defined that for the first time.
3:24
and what we did is we looked at the funding needs and we talked about whether Nvidia's balance sheet would be strong enough to support backstopping a lot of these funding needs.
3:31
we'll talk about that much more later.
3:34
But I think it's worth going through quickly the constraints and what sort of sort of led us to this point.
3:40
so if I go back to if we think about the constraints right, Jordan?
3:46
And you know, you asked me earlier about the Trinity right? Do you remember that? Yeah. Yeah.
3:52
We used to talk a lot about the Trinity.
3:55
And the Trinity, the AI project Trinity is basically capital offtake, and datacenter.
4:03
And Jordan earlier he was kind of asking me like, well what about compute right?
4:07
Isn't that the fourth leg?
4:09
which is a good point because it used to be a pretty big short a pretty big pinch point, and it used to be kind of a worsening pinch point.
4:20
but what's happened since then right?
4:21
Memory was approximately one of the hugest pinch points right?
4:25
So let's talk about constraints.
4:27
And the constraints have been moving.
4:28
They've been moving from compute constraints to datacenter constraints and now Financing is a new constraint.
4:35
so why what you know why are those more acute right?
4:40
So what's happened with memory?
4:41
Memory's been a shortage.
4:43
you can see that AI's percent of DRAM wafers was going to be 70 percent.
4:48
and then you can see that on the logic side accelerator is an ever higher share.
4:54
But what happened recently is there's been de-speccing.
4:59
So let me show you what what happened.
5:01
So if you look at accelerator perspective this used to be the pattern.
5:06
If you look at GPU ASP, it had been going up because of memory.
5:10
And it was set to go up in Rubin Ultra but that pattern actually stopped in its tracks.
5:15
And what we got instead was de-speccing. Why?
5:18
for really two reasons right?
5:22
Number one, ROI, and I'll just kind of flick to a quick a quick Bridge chart, right?
5:30
And so with this one what we saw is the cost was increasing for the VR NVL72.
5:38
and what they did to offset that is decrease SOCAMM but it's not just about price it's about availability.
5:44
The issue here is that there just simply hasn't been enough memory to go around to actually ship racks.
5:50
so the de-spec was necessary to deconstrain that.
5:53
so that's the chip constraint.
5:58
Let's move on to the datacenter constraint.
5:59
That was one of the first data constraints the first constraints that that people worried about.
6:03
but we've found a lot of creative solutions around that.
6:08
We've mustered a lot of capital to build datacenters.
6:10
we've activated more powered sites we've refurbished old sites, we've done a lot of behind the meter at to the point where by twenty seven twenty eight, it's much more balanced.
6:20
and this is no longer an obstacle.
6:22
So what are the new constraints?
6:24
So what we talk a lot about now is funding.
6:27
And that's a constraint people have learned about in just the last few months.
6:32
I'll show you how we calculated the overall constraint. Let's take a look.
6:37
So this is the cumulative total capex.
6:40
It's eleven trillion of capex that needs to be spent by twenty twenty nine. Sure.
6:44
Yeah, Dan, explain this chart just because I think a lot of people are throwing around the term of one trillion of hyperscaler CapEx.
6:52
How do you get to eleven point three trillion on this chart?
6:56
Yeah, so eleven point three trillion is cumulative capex from twenty eight.
7:01
Sorry, from twenty four through twenty nine.
7:04
And so it's escalating each year.
7:06
The way we did this, because we track all the GPUs we track all shipments, we track total cost of ownership of the racks the bill of materials.
7:13
and that's not just sketched across GPUs also XPUs, it's TPUs, AWS Trainium.
7:22
and if you take the sum product of the shipments across the total cost which includes the server, includes networking includes CPUs, includes everything you actually get to that 11. 3 trillion.
7:33
And by the way, you also mentioned hyperscale capex.
7:36
So that's actually a subset of this.
7:38
If you look at the legend, this is AI IT capex and datacenter capex.
7:41
And a lot of that capex is not incurred by the hyperscalers themselves it's incurred by independent datacenter companies.
7:48
So that's part of the equation of why.
7:50
one trillion in a year of hyperscaler capex becomes eleven trillion because it's hyperscaler, neocloud and then it's IT and datacenter. So Yeah.
7:58
And this is not covering necessarily other things in the supply chain which we do cover, which Dylan has talked about publicly Exactly.
8:02
which is, you know, power plants industrial equipment, stuff in the fabs that is you know, a precursor to AI IT in this measurement here So this should sort of like solve the problem.
8:10
like you know, fab equipment or just new fab.
8:10
Yeah, or sometimes like design.
8:19
Like there's a lot of stuff going into the build out right now. Yeah.
8:23
you said that there's a constraint related to funding.
8:27
Talk me through how people are actually funding this build out and how you just how is that where's where is everybody finding the capital?
8:38
Yeah, so it's a great question.
8:40
and what we said is there's gonna be seven trillion of debt outstanding by twenty nine.
8:44
and so what does that work out to?
8:48
It works out to what we did here is I put the net additions the net new funding.
8:52
so it works out to one trillion this year.
8:56
it's a little flat because whenever we forecast stuff forward we tend not to add capacity and deliver until it's confirmed right?
9:05
But yeah, and then there'll still be there'll be some debt.
9:07
That rolls off because all amortizing.
9:09
But you know, I would expect that this estimate will probably creep up.
9:12
So the green, the blue triangles are our estimates.
9:14
And if you add this up, that could see your seven trillion.
9:18
but Jordan, like where are they getting the money from? Right?
9:21
To answer that question let's look at how much money is put to work in this space.
9:26
So in the investment grade market it's about three trillion of issuance. That's the red box here.
9:35
And investment grade is just anyone with a credit rating of Triple B minus or higher.
9:38
And this is typically gonna be like hyperscalers SpaceX is also investment grade.
9:46
Typically like great companies Nvidia's investment grade, and then high yield would be companies that are a little more risky.
9:52
you know CoreWeave is technically high yield and you know, various other companies have a lot of debt or are a little bit more of a nascent business model.
10:03
And that high yield issuance is denoted down here.
10:06
It's about four hundred billion then leverage loans, which are loans that are of a kind of riskier nature or often used for acquisition and private equity is also four hundred billion.
10:17
By the way, this is IG bonds.
10:19
This is high yield bonds, and then I mean loans are kind of separate. Is this load up? Yeah, I'm functioning.
10:23
Yeah, so everything's growing both high yield and investment grade is both growing but take me through again where is the where Yeah, you want to try it over to the I think it's possible this one.
10:34
where's some of the individual projects that are responsible for this much debt being Yeah. raised? Yeah, just June. Yeah.
10:43
Which probably yeah absolutely.
10:45
So I can give you a few examples.
10:48
so you know, this like just since June since June alone, right?
10:52
I mean so to put things in context right?
10:56
So we're saying one trillion this year and we think about probably five hundred, six hundred billion was probably maybe around five hundred billion was raised through June, so we're pretty much on track.
11:05
but just since June, since June one until today is October seventh. in Asia least.
11:11
there's been about four hundred billion of capital like loans bonds of AI, you know, either raised or announced.
11:23
So there's been about call it 260 billion worth of major deals that have been raised, another 150 billion that are reported but not yet documented.
11:31
You know some large examples Blue Sky funding, which is a twenty four billion deal.
11:38
which comes with a six billion delayed draw term loan as well.
11:41
this is part of the Broadcom Google Anthropic TPU deal. Yeah.
11:48
Sopapia is another it's a Meta BlackRock JV El Paso facility with residual value guarantee for Meta. It's 12.
11:59
5 billion Hut 8 raised 4. 25 billion of paper.
12:05
You know, CoreWeave, like all the neoclouds have been added.
12:08
So the ones you just mentioned just hyperscale.
12:11
So now we're on a neocloud. So CoreWeave raised a 4. 2 billion convert. They raised 2.
12:14
6 billion of direct delayed draw term loan in August. Nebius just raised a 3. 5 billion convert.
12:21
Xancor in Asia did a three billion term loan one of the first in Asia.
12:24
Anthropic did a 15 billion pre-IPO revolver.
12:29
I mean, I c I could go on and I haven't even gotten to the hyperscaler bonds, right?
12:36
yeah, does that help Jordan?
12:37
I should talk about the hyperscaler bonds because that's what kind of kicked off this whole firestorm.
12:40
But you know, I could Yeah, I mean go through I have pages and pages of this.
12:45
Yeah, well I mean that's a nice little teaser taste there.
12:49
Maybe we I'd love to talk about Yeah.
12:49
But yeah, I think some of the specifics, both in terms of the actual projects but also in terms of what they are backstopping or what they are raising debt for.
12:59
Which is to say this is Yeah.
12:59
not like strictly for GPUs.
12:59
I think a lot of people might get confused by this and think like, you know, this is strictly just people raising money to buy Nvidia GPUs but there's so much more that goes into actually building a datacenter that can accept the GPUs.
13:15
And that means that Nvidia as well as all the hyperscalers are motivated to backstop people across the whole industry.
13:25
Yeah, it's the datacenter too.
13:28
And even though when you know those who are used to looking at total cost of ownership, you know, let me show you one of the total cost of ownership charts here.
13:37
So this is the full stack and what you'll notice we always say that most of the total cost of ownership is the capital cost versus the datacenter cost.
13:47
So if you look here, the operating cost per hour is sixty six cents and the capital cost is two dollars out of out of 280, right?
13:55
So if I do 213 out of 280 that's around 76% opex.
14:00
So you might think the datacenter is not a large part, right?
14:04
capex is around for a GB300 is around 31 million per megawatt, 31, 33 million per megawatt.
14:13
And then for datacenter it's about maybe 10 million plus right?
14:17
so if it's you know if it's 30 milli if it's like 30 billion 10 billion, that sounds you know kind of mismatch, right?
14:30
and the thing is right, the kind of trick here is that the capital side is a shorter term project, right?
14:38
It's a five-year depreciation whereas your datacenter is a 15-year right?
14:42
So on a cost of ownership it's like 75/25 but on a capex basis, it is more like 60/40.
14:54
Right, so datacenters even more because we're building more up front and this datacenter is gonna last fifteen years it's gonna do three projects of the AI IT CapEx.
15:03
Does that help putting context?
15:05
Like how much is needed for datacenters? Yeah. 100%. Yeah.
15:09
I think a lot of people don't really understand the breakdown in terms of like the chips are expensive.
15:14
They make up three quarters of the capital cost of the project.
15:17
you know, if you're depending how you model them right?
15:22
Which I think maybe is the critical question that's outstanding.
15:27
namely the residual value of some of these GPUs that are coming off their five year life cycle that people are expecting warranties expired.
15:36
Do they keep running them for year six seven, eight?
15:37
And how does that how do you view the cash flows for GPUs after the residual value period or the modeled lifespan is expired in terms of how it's enabling them to fund future projects? Yeah.
15:56
So I think that's a question people ask a lot.
15:59
and I'll go to the you know we'll we've gone through this many times but you know, this the one way to think about it right, to directly answer your question many people that are doing these projects they actually underwrite it to make money on a five to six year period.
16:16
So they're not actually banking on the residual value at all.
16:19
so and this gets into the lay of the land now because most of the clusters are actually you know built on long term off takes because that's the only way you can get funding, right? Remember the Trinity.
16:35
You need off take capital and datacenter.
16:41
So if you don't have a five year off take you're not getting financing.
16:44
And that's one of the biggest constraints.
16:47
And so because of that constraint because you don't get financing without a long term contract everything's typically like pre Preordained.
16:54
So typically on a five-year deal you're going to get an IRR of about 20 to 30% at least.
17:01
It's been higher in the past.
17:01
And what that means is by the time your project is over in five years, you'll have made on your capital investment 25% yield per year without banking on residual value right?
17:17
So most things are underwritten actually, interestingly, without regards to what happens after. It's like a bonus. Yeah.
17:25
So can we back up a second and talk a little bit about the broader industry?
17:31
Like we just keep saying datacenter and Mm-hmm.
17:31
and we you know we kind of talk about GPUs but when it comes to actually analyzing what is being backstopped by Nvidia for example. Mm-hmm.
17:47
you have this awesome chart that I've got on screen now.
17:52
Nvidia's off balance sheet commitments where you summarize the $497 billion of off balance sheet backstop commitments that they've made at this point.
18:00
And you go through seven categories here.
18:07
AI cloud agreements, where they are you know, signing deals directly with neoclouds.
18:12
datacenter leases where they're signing deals with you know providers that are building datacenters, guarantees and PPA guarantees again with more on the energy side cloud services agreements, which are not the same as necessarily a neocloud agreement in this measurement basically an accounting difference there. Data Yeah. Yeah, I know.
18:36
center leases for self-use, which is different than the previous categories for pre-assigned.
18:42
Supply and capacity commitments which is primarily around component suppliers like memory.
18:48
And then finally this residual value guarantee where we don't assign any value to it, but you know it's a category of something that they're guaranteeing for people.
18:56
So just give me a lay of the land of all of this. Like it's such a mess.
19:02
There's so much going on. Yeah.
19:05
So yeah, I'm happy to do that.
19:09
we can go through each of them.
19:11
I've got some nice diagrams that can tell you about you know, why each of them and how we modeled it and you know what the point is.
19:17
you know, good thing this is gonna be on YouTube because you're gonna wanna slow this down to 0.
19:20
75x when I go through it for all you listeners.
19:25
But before I do that, let me talk about like the why right?
19:28
Like why all this backs up because the problem is a lot of people just kind of skip the why.
19:34
And they go to the what, right?
19:36
and then they look at the what and they're like okay, this is circular financing.
19:42
now let's talk about the why.
19:45
why are we gonna hit an obstacle? Why is it a constraint?
19:48
other than rates going up which is the market telling you that it's gonna be tough right?
19:54
And the numbers I just showed you.
19:56
but there's something more.
20:00
Like I said, Jordan, most of the offtakes are five year most of them are hyperscaler backstop.
20:05
And what that means is let's say you've got a a Microsoft Nscale OpenAI deal.
20:13
What's happening is Nscale is selling capacity to Microsoft which is then selling it to OpenAI.
20:21
So Microsoft is effectively backstopping that sale because they're guaranteeing that capacity will be used right?
20:29
And some hyperscalers also do that directly like meta is directly off taking from neoclouds for instance.
20:36
There are many deals like that.
20:36
Yeah, so Nscale needs a guarantee to go out and take out a construction loan to buy and GPUs Exactly.
20:44
and to plug them in and turn them on.
20:45
And the guarantee is Yeah.
20:45
If they not good if it comes from OpenAI because they don't have an IG credit Exactly. rating.
20:45
And so instead of relying on OpenAI to pay the bill they're gonna say if OpenAI doesn't pay the bill Microsoft's on the hook. So why does Exactly.
20:58
But Microsoft sign up for this, Dan?
21:04
Well, it's a good question.
21:07
I think there's internal and external use right?
21:10
I think they also take a cut of it.
21:13
it allows them to expand and access capital without directly using their balance sheet, because it'll show up and you know as an upcoming work right?
21:22
A lot of it shows up as services of like contract which is under a certain accounting rule.
21:28
I forgot which one is it, Jordan? ASC 606. 606, not 842, yeah.
21:34
Okay, 606 is the yeah I was kinda about to be ASC 842 if I didn't remember that correctly. But yeah.
21:40
There's yeah, there's a number of different designations by the accounting standards committee or commission or something.
21:47
Anyway, the it's very material to these deals because the definition of a managed service is different than the definition of an operating lease in that a managed service does not end up on your balance sheet, an operating lease does. Yep. Yep. It's not a debt.
22:02
And so the technical yeah, the technical treatment of who's responsible for what in the datacenter is different than the accounting treatment and you know who can log into what or enter what you know room in the site or touch what or you know this is less relevant than how the accountants want to track or keep track of where these assets sit and how they depreciate right? Yeah.
22:33
I think Jordan, it's kinda like if it talks like a duck quacks like a duck, it's a duck.
22:40
So and I don't think the rating agencies, like, no one's stupid right?
22:43
but it takes a little more work right?
22:45
Because the problem is they don't do it for you.
22:48
They don't say like, they add up all this like Nvidia discloses their obligations and I think in some of the filings they will disclose it but it's not in the balance sheet right?
22:57
You have to go looking for it. Right.
22:58
And that's part of what we're gonna do at Compute Capital Markets CCM, our new group.
23:04
who we're hiring for, by the way we'll have a PSA later.
23:05
so if you love all things credit and like and we'll show you some charts, like if you like complicated charts you're gonna love this job.
23:12
But anyway the Jordan's point, it's kind of like you know it's something that they that's gonna be hard for them to walk away from.
23:20
But my point is even that, even if it's not debt like that is not unlimited, right?
23:24
It's there's gonna be a point where rating agencies where investors say Hey, you know, this is just too much for you to backstop.
23:29
Like, you know, how are you gonna backstop a three trillion?
23:32
Like no one has a crystal ball right?
23:33
Yeah, you specifically use the term balance sheet as a service. So like Mm-hmm.
23:37
right now it seems that all of the hyperscalers when we talk about this one trillion of capex are effectively putting their balance sheet to work as a service that has limits right? Yeah. Well yeah yeah.
23:51
So if you look at you look at Amazon right, for instance.
23:55
so they did a twenty five billion deal across six tranches is back in July right?
24:02
And it priced, you know w kind of well wide of where their benchmarks were right?
24:07
And this was kind of like the canary in the coal mine.
24:10
I mean, people are definitely aware but you are seeing the hyperscaler spreads blow up by quite a bit and the CDS spreads blow up by quite a bit.
24:20
I'll actually check for you right now.
24:21
We can maybe we can edit this a little bit but let me 'cause I wanna say a number if that's okay. You guys don't mind. Give me a sec. Model directory.
24:35
And I can fire up Bloomberg as well. Oops. Take your time, dude. No rush.
24:42
We can just clip this section. Yeah. Okay. So Amazon spread.
24:50
So if I look at if I look at July I'm just looking at July, right?
24:55
And their ten year you know Z-spread. restart, dude.
24:59
I'm only seeing the financing market obstacles. I'm not sharing it. I'm not sharing it.
25:03
I saw your cursor moving like you were sharing something. I know. No.
25:03
It's because it's over it's overlapping the deck. Great.
25:08
So I'll move the deck to another thing. Sure sure. Yeah.
25:13
So if I look at Amazon right, for instance, remember when they issued that bond their ten-year Z-spread would have been like around a hundred basis points.
25:27
So roughly if you have treasuries it used to be like four and a half percent then you add a hundred basis points or one percent.
25:33
Then that means the yield on Amazon debt is about five point you know five point five or five point seven five as the case may be.
25:40
And for five years, that spread was roughly around 65 basis points.
25:46
and then you know what sort of happened right after is it is it is it blew out probably by like in the summer it blew out by about probably 10 20 basis points from that issue which is kind of a lot of a move for IG.
25:59
I mean, things are relatively under control for now.
26:03
But you know you've definitely seen like Oracle has moved up quite a bit since the summer.
26:08
their entire curve probably moved up like 50 basis point 30, 50 basis points.
26:14
so you know you are seeing the debt market take note.
26:20
and you know, rating agencies like will see through this stuff.
26:27
Yeah, walk me through this no oracle increase of a hundred and five basis points and then compare that to CoreWeave which is well above investment grade at almost four hundred basis points higher. Yeah.
26:42
So Oracle, their spread is gonna be around like looking currently, ten year I've got them down for about two hundred and fifty Z-spread versus an Amazon.
26:55
I've got them only at a spread of a hundred hundred and ten for about ten years.
27:02
but this chart is a little bit about like as you said the relative you know, like how do we compare credit risk?
27:09
So what What this is illustrating is a different type of structures.
27:15
So what I'm showing you here is let's look at the structure where CoreWeave is selling an off-take contract to Meta.
27:23
And the question is, how much am I gonna charge to lend to CoreWeave for that?
27:27
So someone who's lending money to CoreWeave will say actually, this is meta credit risk plus execution.
27:33
So the meta curve at the time we did this for five years around 97 basis points Z-spread.
27:42
So now the actual lending to CoreWeave for this project was around 200 right?
27:48
It was 225, SOFR plus 225 which you know, when you do a little bit of math comes in slightly under a Z-spread of 200.
27:53
and so the difference between that 97 basis points and the 200 is actually the execution risk.
28:01
Because even though Meta is there right?
28:05
Off-taking or Microsoft is there you still have to stand up the cluster right? Which is why, you know.
28:11
I we should actually rename this to the ClusterMAX risk right?
28:13
Because if they don't have a platinum right? You know?
28:15
In fact, like there's gonna come a day Jordan, when it's like this is your S&P rating and this is your cluster That's not every Everybody wants to use ClusterMAX for things that max rating, right? it's not, man.
28:18
Everybody wants to do make stock picks from ClusterMAX.
28:28
Now you wanna rate people's debt from ClusterMAX. Okay. It is one data point.
28:31
It is Don't tell Ethan, he's gonna shut us down. my god man. No.
28:38
We're not branding neocloud debt product pricing as clusterback stand. Sorry, audience.
28:49
Jordan does not give investment advice.
28:51
There's a okay look, there's a lot of things that go into building a good quality cluster.
28:55
I think execution risk Mm-hmm.
28:55
on standing up a given datacenter on a given timeline and then maintaining a contract such that there isn't a cancellation or termination of the contract is typically bare metal and it's typically a very different sort of product that is being sold to people than the managed clusters that we're rating.
29:14
But actually I'm thinking this through and I'm not completely convinced in what I'm saying which is that.
29:19
Look, a lot of the experience that applies for building a great bare metal service certainly applies to the quality of the cluster.
29:29
Like people who build reliable clusters are generally able to maintain and deliver on their contract.
29:34
So all right, fair enough.
29:34
We can use this as a rating of some something towards execution risk.
29:39
Well, you know, Jordan, you say that but I've been on a roadshow with lenders.
29:43
I did New York, I did Singapore and a lot of them have said, Yeah this has been incredibly helpful to their process.
29:49
So I think it's like, you know i mean, I think it's look you know if someone's made it onto ClusterMAX gold, or platinum, like they're unlikely less likely to mess up right?
29:59
Like on execution generally.
30:00
It's just a stamp of approval right?
30:01
Just because, you know you got good grades You know, you got good doesn't mean you're a genius but like, you know, probably is more likely than not right? You know? Yes.
30:09
I think it's a signal and a data point but managed clusters and large bare metal datacenters are Sure. very different products.
30:17
And I think just because somebody's in platinum does not necessarily mean that all their datacenters are gonna be on time and vice versa.
30:24
Just because somebody's down the list or maybe ranked as like underperforming or something like that does not mean that their datacenters will be late or that they will be low quality in the future. True. True.
30:33
So I think that there's if anything I would be Asking people to assess a given project or a given site.
30:44
I mean, we've seen people structure every single site within a neoclouds portfolio as an SPV.
30:48
And I think that's the right way to think Mm-hmm.
30:48
about it, where there is a different in some cases, very different level of execution risk as you're describing it for a site in one location on one continent that happens to be you know, owned and operated by a given neocloud even though they can go to a completely different continent for a different project.
31:09
the next week, the next month and raise debt in a totally different way have a totally different team working on it.
31:14
And, you know, therefore things are a little bit different.
31:17
Dan, let's move on to Well well Jordan, I that's an interesting point right?
31:22
Because you know, a lot of the a lot of the a lot of the clusters a lot of the neoclouds we talk to that are unavailable, they're not unavailable.
31:32
It's because they're busy doing bare metal right? You know?
31:34
Like it you know it's to your point, it's not their focus right?
31:37
And so like yet they've landed large deals.
31:39
So it's not like, you know, it's just one data point and some people just don't focus on it.
31:43
Like some guys who are doing great have landed lots of deals just I'm like, hey, sorry guys, you know we just don't have the product for you to test right now.
31:50
And the whole market has changed.
31:51
I mean, we're constantly going back to this but when ClusterMAX started, buyers had five different providers offering them two-week POCs on 256 GPUs so that they could make their decision in an informed manner and sign up.
32:08
Now you gotta pick a dance partner six months before the GPUs get delivered and you gotta make a prepayment to these people that's worth 30-50% of your.
32:18
contract value in some cases.
32:18
I mean You said this earlier on this episode.
32:26
Effectively the buyers are just paying for people's cost of capital directly.
32:31
And so you're taking money that you've raised which has a cost of capital, and then you're giving it to somebody else and you have to pay for their cost of capital.
32:39
I mean, it's you know, it's stacking on top which is driving a lot of people to try to do self-build work.
32:46
And I think is what it's almost like this oscillation. It's like you start out.
32:51
Early and you do rental because you don't want it on your balance sheet.
32:53
You want to move fast, you the biggest cluster possible.
32:55
And then you start to grow and you realize hey, this is a bad deal.
32:56
Like I'm kind of getting ripped off.
32:59
I should own this stuff myself.
33:00
And then you start doing these self-build sites.
33:03
And then you reach this limit of what your balance sheet can handle how much money you can raise.
33:06
And then you start going the frontier labs go, Hey guys I need help.
33:11
I need to basically mortgage my future right now.
33:13
And I need you to build some stuff and I need to guarantee you some returns on this work because like Hey, I don't have the balance sheet to do it and I need to put your balance sheet to risk.
33:23
So it's an interesting curve where the frontier labs the way they're doing deals right now and trying to get everything off their balance sheet, and even the hyperscalers are trying to get stuff off their balance sheet.
33:31
It kind of looks like an early stage startup right?
33:34
They want the most that they can get right now.
33:39
And I think you've touched on like the why right?
33:41
Which is where we kind of got here right?
33:43
Like why is there all this backstop?
33:45
And you described the problems right?
33:46
Like you can't get you gotta book six months in advance you can't get the compute you want you gotta put 50% down.
33:52
and that's you know kind of why they've done the backstop.
33:58
I mean, let me share I did up this slide that describes some of the why.
34:02
and it's what you said broaden compute availability is what you just talked about.
34:09
now supporting the financing market is another thing because we just talked about how unless you have that IG backstop you can't do anything.
34:16
So it's very hard for like a neocloud it there's no incentive as a business model to say like, hey, I'm gonna fill my book with half of like various inference service.
34:25
because your lenders will go like hey, you know, I'm not gonna it was not IG credit like I don't understand this business well enough right?
34:32
Which kind of you know goes back to this earlier.
34:37
slide that I had on the willing the obstacles right?
34:42
They're still on the learning curve.
34:43
They don't understand the business well enough to underwrite anything other than IG.
34:47
And then they don't have the tools right?
34:48
And this is like a lot of what Jordan and I are working on.
34:51
You know, Jordan on the tools to assess the operational quality and and myself on assessing like what's happening in the market right?
34:58
That's why we have the GPU index.
35:00
That's why we're trying to you know create tools that lenders need.
35:04
To monitor risk, and you mentioned residual value.
35:06
This is one of the huge asks we had in our roadshow because banks think in terms of residual value.
35:10
They're lending against assets.
35:12
They're used to lending against property cars, and they have to know residual value which is very hard with GPUs.
35:19
It's not like you straight line depreciate.
35:21
I think the residual value is what you can earn on tokens or what you can reasonably rent it as.
35:25
And it's so distorted right now because I think in a normal market where you have like this contention between supply and demand where people produce enough GPUs based on the demand for them and they can always go a little bit higher then you have real pricing and you have real like settlement.
35:41
but what's happening right now is nobody has an understanding of what you know, the true theoretical H100 depreciation curve looks like as GB300 comes online as Vera Rubin comes online next year. And it outperforms that.
35:54
And then you say How can I run an H100 profitably?
35:58
Well, we already have models with InferenceX that show that you really can't.
36:03
If a GB300 is outperforming your hoppers by like seven times why would anybody want to buy a marginal hopper over a marginal GB300? You'd never want it.
36:12
You'd want the latest and greatest.
36:15
But this isn't the question people are asking about Hopper.
36:18
Instead, the question is: I have no option to purchase a B300 GPU because it's all sold out or I have to wait six months.
36:26
I want to do research right now.
36:29
What option do I have to buy?
36:29
And the answer is renew H100 contract with four year old GPUs for double the price of what I signed it for previously. And this is on screen.
36:37
Dan's got our pricing chart for the one year term length H100 deals that we know about.
36:44
I mean, they're approaching the prices that they were signed in the first half of 2023 today, like three bucks an hour for H100 GPUs on a one year term.
36:53
And look, if you believe in AGI frankly, I talk to all of these startups a lot.
36:59
And I basically give them the coaching of if you believe in your startup being successful, you have to imagine that somebody's gonna do something open AI and Anthropic can't, and it's gonna be you.
37:07
And so you have to, you know flippantly, you have to like believe in AGI here right?
37:12
And if you don't You know Jordan, but like I had a question 'cause you said that no one doing inference would ever rent an H100.
37:20
So like why are people why is the price going up then?
37:25
Well, because I think there's so much demand for it that the price goes up strictly because of it's the only option for a lot of people to do.
37:32
Second of all, there are some researchers who prefer H100 because it's familiar, they are more productive by the metric of jobs per day rather than you know, tokens per second.
37:42
It's like I don't need to spend time debugging I don't need to port stuff from BF16 to FP8 or FP4.
37:50
I'm doing training and you know I need numerical stability.
37:52
I don't care about these low precision data types that are used for inference.
37:55
I'm doing some new model where again I care about numerical stability. I don't want this risk.
38:01
So there's reasons why people want to do research with hoppers H100 and H200.
38:05
But I think the biggest reason why the prices are increasing today is just because GB300 sold out so people go to the B200, B300 right?
38:15
Those are sold out, they go to H200.
38:15
Hmm, so it cascades down. Exactly.
38:19
And I mean, if your answer is wait six months and have nothing or get started with H100s today a startup that just raised money has a pretty easy decision.
38:30
They go for the H100s they can get access to right now.
38:34
Yeah, that makes so much sense.
38:35
And that's kind of like why when we do this index like we do it survey based 'cause like, you know, there's a story behind every data point right?
38:42
This is not just, you know going like, you know, scraping a bunch like, you know, there is a story and those matter.
38:49
And you know, what I did, Jordan I actually decomposed.
38:50
do you wanna go or I can talk about this? Yeah, sorry.
38:50
Well, just one last thing. Yeah.
38:50
Sure If you go back to the previous one like we would sure.
38:50
we would have a more academic theoretical way to forecast the depreciation in the H100 pricing curve.
39:03
If you look at this chart and sometime around September or October of last year it just kept going down below a dollar eighty below a dollar forty. And we Mm-hmm.
39:10
actually I remember some deals being signed at that time for like a dollar thirty for an H100.
39:17
People were offering stuff that low.
39:17
Yeah, that was the bottom.
39:17
And but those were not great providers though. For sellers.
39:26
well no, I'm the guys who are I was a buyer.
39:26
selling at one thirty were not great we're not like great providers right? that fair enough. Yeah, exactly. Yeah.
39:35
but okay, but the point is like I know some sellers at that time who were evaluating dollar thirty and going should I do a one year at a dollar thirty or should I do a two year at a dollar twenty five or something like that?
39:47
And man, those guys are kicking themselves for not doing a two year then because they just renewed at like literally some of them at 250?
39:55
Like way more than they were paying this time last year.
39:58
And so anyway, the point is like we would have a real understanding for H100 residual value if we had seen the pricing curve depreciate like some other asset, like a car or something right?
40:07
But four-year-old H100s going into next year are going to be selling for likely higher prices than they are today if the trend continues.
40:17
And I mean, I expect it to because I expect demand to continue to rise.
40:21
and let me make two more cases for this that are not Based on like an AGI future, you don't need to believe in that.
40:29
You need to believe in datacenter delays or cancellations due to political posturing and moratoriums and execution risk as you were describing, which we see a lot of.
40:38
And you also have to believe in more demand from consumers on you know, Anthropic and OpenAI's API endpoints you which I think you can believe in even if you don't believe the models are getting better.
40:53
You just need to look at things like Instagram growing or Muse from Meta and just imagine a world where more people use AI driven products even if the models stop getting better. Right.
41:04
So the combination of like datacenters might take longer to build pl politicians may get in the way and stop them.
41:11
That will constrain supply.
41:11
More demand for the existing models and the models get better are all a recipe for increased prices over time.
41:21
And that's generally the case that I've been making to a lot of people. Yeah.
41:25
And the thing is right, if you did the scientific version we started this in twenty four we actually have a forecast and that actually looked like you know downward sloping, it actually started detaching.
41:36
We were like within two percent like a you know a good year and a half, right?
41:38
Good eighteen months, but then in it started dislocating for September.
41:42
Because the way we calculated the theoretical depreciation curve is the relative power, the relative compute power of new GPUs.
41:52
So, for instance, a GB300 had seven times more inference speed and three times more flops, right?
41:58
So it would basically those new capacity would push down the price of your old capacity.
42:01
In theory that's what should happen.
42:02
But what we always said is you know we understand supply, we track price but it is hard to track in real-time demand.
42:12
So what we do is we actually flip it on its head and we say let's observe price and let it derive demand.
42:19
And so You know, in this slide, this is exactly what I did.
42:22
what I did is I said, you know if you have a balanced, if you have a balanced neutral supply demand then what you see is it depreciating in value or rental price going down in you know in a scientific way right?
42:37
In line with new capacity right?
42:39
But what happened is in October right, that dislocated and what that implies right?
42:44
Because if your demand is higher than supply then your price surprised the upside right?
42:48
And so what I did is I reversed it and I said Hey, what does this imply for demand supply balance based on price?
42:55
And what we saw is it blew out completely in late twenty five and it's never looked back.
42:58
Does this kind of like tie in with what you've seen, Jordan? Like I assume so.
43:05
Like maybe not the theoretical stuff like what you observe the market.
43:09
Well, a couple of things happened around that time last year.
43:11
I think basically models stopped getting bigger.
43:16
Like the it took a while for the Chinese models to catch up but up until around this time last year when you know, reasoning models were taking off people kind of discovered we can trade flops for parameter count and continue to improve the quality of the model with a lot of innovations around RL. In post-training.
43:39
And they've just continued to exploit that where we may have gotten bigger models.
43:45
I think we likely did get bigger models you know, with the Astra and the Fable generation or the Mythos generation but we didn't get significantly bigger models that outpaced the improvement in hardware that came when the GB300 NVL72 came online which is to say significantly more memory memory bandwidth, and FP4 performance were able to keep up with the increased size of the models.
44:06
And so what that meant is that these These companies could continue increasing price per token on their APIs without necessarily increasing their costs to serve the model significantly.
44:18
And they're, you know, that like the result is just they're way more profitable.
44:24
Like way more profitable.
44:26
And you don't even need to know anything about the Anthropic group AI models to make this case.
44:30
You just need to look at the Chinese models and you need to look at what the performance of A DeepSeek V3 was on like an H100 H200, or even a GB200 when you know those were being brought online last year.
44:43
And then you need to compare it to DeepSeek V4 or 4. 1 Pro or 4. 1 Flash or GLM 5.
44:49
3 or Kimi K3, where you just look at all the innovations on sparse attention.
44:54
You look at the parameter count growing a bit like around two times, or three times in some cases but you know, flops not.
45:03
is increasing significantly and performance increasing so much.
45:07
And so they can command a pricing premium over the previous tokens because the tokens are more valuable, but it's cheaper to serve.
45:14
You get more throughput per GPU with these.
45:16
And so people serving Chinese models are enjoying gross margins of 80% right now on a GB300 NVL72.
45:21
And so it's just like it's like look obvious that There's going to be more demand for this compute this asset.
45:34
Now, the question or what gets confusing to me here in some cases, and maybe you have an answer for this is how nobody forecast this.
45:49
Like we did our best, we understood right?
45:52
And I think even the frontier labs who really deeply understand how these models work.
45:59
Did not expect themselves to be this profitable and to be able to serve this many users and to grow this fast.
46:03
And so even they were caught by surprise and had to do crazy things and sign big contracts at very high premiums just to get access to GPUs tomorrow to serve their users and to go for the bigger research jobs.
46:15
I don't view this as like an AI fast takeoff but it seems to be like a revenue and profitability fast takeoff for a lot of these labs.
46:26
Yeah, and I think you know, we've kind of in our discussion we've really hit on what is the debate going forward.
46:32
And it's the willingness and the ability.
46:36
And I think the ability is the profitability.
46:39
like a lot of people who are new to this are n don't realize just how profitable it is.
46:44
and that's one of the things that will determine whether we overcome this obstacle this kind of you know, funding wall.
46:53
Well, I think it yeah, it's also confusing which layers of the cake are profitable which is to say Nvidia Right, yeah.
46:59
Well we can look at that yeah.
47:02
Nvidia clearly had durable profit margins for a long long time. But Mm-hmm.
47:06
your yeah, your layer cake chart is great to explain.
47:10
Like clearly there has been an increase in the profitability of the neoclouds because the amount Mm-hmm.
47:16
that they can rent the GPUs for to people has increased.
47:22
recently when the GPU has not there's nothing changed on the operational costs or anything, right?
47:25
The people are selling tokens with more efficient models or higher prices.
47:30
And so therefore their profitability is increasing.
47:33
And then my personal experience and I think a lot of other people's experiences the value or utility that you're able to derive from using these AI applications is also increasing over time as they get higher quality.
47:43
And so I don't look at this and say you know, we're taking from one to give to another.
47:50
It seems like every layer of the stack in some ways is increasing profitability except for like the server OEMs where it's still a pricing war and they can't nine Well, no percent margins.
48:01
there's still a there's a bit for everyone but you know well I The true commodity, the tin box and the people to plug it in.
48:09
I can get into that, but I wanna you know, just so everyone stays you know Yeah, sure, yeah. organized, right?
48:11
I think again like everything we've talked like Jordan and I have talked to so far you can fit into two buckets the willingness and the ability.
48:21
What we're just talking about now is the willingness.
48:23
You know, what is the return?
48:25
Will the labs remain high?
48:25
Like how high are the margins?
48:27
You know, can and users and earlier on we were talking about the ability right?
48:31
We talked about the balance sheet of service.
48:33
we talked about Nvidia backstop which is trying to address the ability you know, and who's gonna fund them.
48:39
We talked a lot about that, but let's you know, why don't we why don't we dip into the willingness right?
48:43
Because that's what we were just talking about.
48:45
and we've done analysis where you can actually earn up to 100 million per megawatt generating frontier tokens off of GB300.
48:53
It may be a little you know optimistic in terms of utilization but when this thing costs about forty million per megawatt to stand up it's like, you know, if you're fortunate to sell Fable you make this back in half a year.
49:04
so it's incredibly profitable.
49:06
and let's compare that against what neoclouds are typically renting for a GB300 at least a few months ago.
49:13
about 12 million per megawatt or 12 million per gigawatt was how much you can how much neoclouds rent a five year contract.
49:19
But if you're doing frontier inference like I said you can do a hundred million per megawatt.
49:24
And people were shocked right, when they saw SpaceX and Google renting at 48 million per megawatt.
49:31
But you know that makes sense right?
49:34
Like because they're buying here selling here.
49:36
So of course they would but people were used to seeing this. so the profit is there.
49:42
and I think Jordan you mentioned you know where does the profit stay in each layer.
49:46
So what we did here is we actually organized the entire Kind of business as an income statement.
49:55
So you can think about this.
49:57
You can imagine you're an Anthropic right?
49:59
So this is your revenue and we put it in a couple different ways.
50:02
So million token, about $1.
50:02
85 per million tokens which is a good blend of Sonnet Opus with cache hits.
50:07
This is actually our own blended spending.
50:13
Even if Opus is five dollar input the cache hits will take it down. So about $1. 85 per million tokens.
50:21
Which is equivalent if you're generating I think we put around I want to say nine thousand tokens per second.
50:27
I can't remember what we used.
50:28
but it's equivalent to thirty-eight dollars per hour right?
50:32
And then we talk we already talked about the cost per hour of two eighty and then a neocloud, so the cost to own the system is two hundred eighty but the cost to rent from a neocloud is around it's probably around four dollars north of four dollars by now, four fifty almost five dollars.
50:47
So If we just simplistic look at that what is a gross margin? It's 90% isn't it? Right?
50:53
Because you're selling at 38 your cost is 380, and this is your profit.
50:58
So it's incredibly profitable. Yeah. At least for Frontiers.
51:01
by the way this was really cool.
51:05
Everyone everyone should read this like drop everything and read this now because you know Max and team.
51:11
We recorded a pod earlier today. You don't know. Yeah.
51:13
This is my third pod of the day.
51:16
I've been cheating on you, Dan.
51:17
But yeah, Max and Andrew went through this on the previous episode in case people my god, you're a pod polygamist. come on, man. Okay.
51:28
Well, anyway, Max and Andrew went through this on the previous episode.
51:32
Maybe for people who missed it do this one again, which is like for the all-in consumption plans on some of these on some of these yeah these businesses you can get a lot of value.
51:47
But it kind of depend if they're gonna be profitable at this layer or not kind of depends on how much people actually use the plans that they let people have access to.
51:57
Yeah, we peg ninety percent API margin and then I think in the model we use like fifty, sixty.
52:01
but the question I have is how much of these people are really using ten percent?
52:08
So I'm kinda you then there's a whole cottage industry in China right?
52:12
That like takes apples and makes applesauce right?
52:14
They take these twenty dollar plans and then just sell them as API right? On the great market.
52:18
And then you know, me, I'm on a twenty dollar plan.
52:22
Like I actually plan my sessions around like you know, utilizing I have loops so that, you know I use it even when I'm not there.
52:28
so I'm definitely on this bucket you know, but I don't know where everyone else is. Yeah.
52:35
I mean, Max made the case earlier that I think there's a lot of people who maybe get these through their business or you know, like the Mm-hmm.
52:43
company they work for and they're just like not really worried about the value trade exactly.
52:47
And you just kind of blend it over a bunch of users and it works out.
52:50
And the loud people are the ones pushing for more limits or the ones you're losing money on.
52:57
But well, you keep around because you don't know how to separate the wheat from the chaff and only sell the all-in plans to the most profitable segment of users which is the ones that don't use it.
53:11
Like, you know, the whole thing about Planet Fitness is marketing it to people marketing gym memberships to people who come once in January for the new year and then never come back.
53:21
That a profitable gym is the one that's empty.
53:24
They don't want anybody to show up.
53:25
I mean, kind of cynical there but That's the incentive structure for these guys to do all-in plans.
53:33
Whereas the exact opposite is true for the per token pricing where they're just trying to get people to consume as much as possible. So Yeah.
53:40
And then the next question becomes like okay, it's so profitable now but like what people ask is that gonna be the case in the future right?
53:49
let me find our token pricing.
53:51
gosh, where'd I put the token pricing?
53:54
and people say like how fast is token price dropping?
53:54
Is that gonna be the case in the future? yeah. I mean not to cut Sorry? There.
54:02
you off, but let me yeah.
54:02
Okay yeah, do this one, yeah.
54:06
Yeah, because 'cause my question for you for you, for you, Jordan, is like people ask me like how fast is token price falling?
54:12
Like, what do you mean it's falling? It's going up, right?
54:14
And in fact even if it's going flat, that's gross margin going up because while token price been staying constant, they've been generating more tokens right?
54:24
And then they're like, Okay well, great.
54:25
So I see frontier models been holding up but like, how fast are open models falling?
54:30
I'm like, Well, they're going up right?
54:31
And the other thing is when you look at you know, when you look at token expenditure it's like a changing basket.
54:36
So these indexes that track token expenditure like they don't adjust for that pri in my opinion.
54:44
I don't know how they work, but they don't adjust for that.
54:46
And they don't adjust for increasing you know, with agentic, it's been all about effective price and cache hits but I don't know what do you think of where do you think where do you think token pricing is going in the future Jordan? Yeah.
54:54
I mean, I had a little tweet thread where I went out David Friedberg from the All-In podcast.
54:58
He didn't respond of course.
55:00
But a lot of people are saying that the there's like this overtaking from open models overtaking closed models.
55:09
And I think the dynamic is just like people are consuming more of everything and they're consuming it at higher prices.
55:14
So you've got the price of input on screen here which I think is a decent metric but you know, that little drop down in the middle you can switch You should do the effective right? Well I mean you could.
55:20
It's just a different approach.
55:23
Yeah, you can do it for cache read which I think is the bulk of our spend is actually on the cache read tier because you know, agentic coding is just so much cache hits.
55:32
And then output is like the most expensive tier even though, you know, you don't actually generate a bunch of output.
55:37
That is what takes up all the time on the GPU when you're memory bandwidth constrained.
55:41
Anyway, the point is like I totally agree that model pricing is going up over time on a blended dollar per million token basis.
55:49
I think that Token efficiency is a question mark because you have to trade off efficient ability to solve simple problems with people's more grandiose expectations as they try to solve more complicated problems that weren't possible before with unlimited tokens and now are possible with a lot of tokens.
56:12
I don't know how you measure that trade-off.
56:15
But look, none of these dynamics to me point to anybody deciding I've had enough of AI, I'm going to use less of it.
56:23
I've had enough of these models let's make them worse and less efficient.
56:26
I've had enough of these new GPUs let's make the next one worse and less performant.
56:30
It all seems to be going the opposite direction right?
56:33
They're gonna raise prices over time more demand for it, the GPUs are gonna be more efficient and the models are gonna get higher performance. So we'll see.
56:39
there's a lot of ways in which this can break if people go and decide to YOLO for the 20 trillion parameter model with the new NVL72 and really use it.
56:52
If they start introducing, but you know ultra fast mode and some of these low batch size optimizations and stuff that can jack prices up.
56:59
But I think all of that is just like artificial constraints on what to me feels like continued overwhelming demand for these products that drive the usage of the GPUs underlying it.
57:12
Yeah, and we talked a bit about the income statement the explicit one, but what about the income statement above? Right?
57:18
What about the users, right?
57:20
That's part of the equation too.
57:20
What value are they getting?
57:20
So I don't see why anyone would step back when these are the kind of efficiencies they're getting.
57:27
like we built an entire database of you know, thousands of debt facilities and all these relationships you know, pure public domain and you know it's been myself, Oliver, Terrence you know, have been doing this and you know we're all part of current groups because you know, we're hiring, like I mentioned PSA.
57:46
but, you know it would have taken me four months and a team of four to do like half as good of a job as we've done so far in building up this database and you know relationship table so far.
57:59
and so these are, you know, we've showed this slide a lot.
58:03
We had a pod on this, but you know again, I don't see why anyone who is paying six dollars to initiate a company would ever go back.
58:11
You know, would you ever go back No, I Jordan?
58:11
Like or would you just stop using fast mode or like what would you I don't use fast mode. but no Me neither.
58:19
you'd you'd have to pry this stuff out of my cold dead hands man.
58:25
Even in my personal life now Mm-hmm.
58:25
with l some of these like support agents and stuff. It's like pick a wine.
58:25
Pick a wine to bring to dinner with my friends.
58:34
It's like okay, mom's for three dollars.
58:34
That's not what I'm talking about.
58:34
I still some stuff still has a personal touch, Dan, okay?
58:43
Like I'm not writing birthday cards with this stuff. Speak for yourself. Yeah.
58:46
I'm not that far off the deep end, but yeah, like I just got this refund on a flight when I was delayed on for four hours a few you know, there's no way I would sit on the phone trying to get this refund ever again, right?
58:57
I'm like checking, you know.
59:00
times at the gym or I'm checking like booking a flight or looking for hotels or checking on the grocery list or just check my email man.
59:10
Like what came in last night?
59:12
I mean it's a little weird but I talked to some friends who are not enlightened to the possibilities that having a personal assistant can provide.
59:27
And It does feel like everybody can never go back once they get the experience of the amount of time that you save and the feeling of like how much wasted time. Yes.
59:43
It's and I think Muse is gonna be eye-opening.
59:47
It's like why did I ever sit there for two hours and browse all these like airline booking sites?
59:51
Why did I ever e you know like I think there's gonna be that moment Yeah, it right?
59:58
it gets a little egotistical when you talk about it this way I think, because you know, you kind of anthropomorphize the agent then you view people who have personal assistance today as kind of exploiting like this thing.
1:00:08
But anyway, the my personal view is kind of like it's It's shocking how much time we waste on this monotonous stuff.
1:00:22
And it's shocking how you know some people that I'm close with do not value their own time.
1:00:29
you know I like yeah, I'm not gonna go into all the details there but you can imagine a lot of ways.
1:00:36
Well I think we're sold on willingness right?
1:00:39
We're sold on the you know AI is like useful, right? Yeah? Right? Okay.
1:00:43
Well, I look I felt like we're I felt since working at SemiAnalysis I felt like we've been at the tip of the spear for adopting a lot of this stuff Mm-hmm.
1:00:48
and just like feeling the ability to use it which both makes our research better and informs the underlying research that motivates some of the data that we see.
1:00:59
And it's really nice to see other people that we work with in my personal life adopt these things and then start to like regain control over a lot of things in their life. It's really cool. Really cool to see.
1:01:14
And I d I yeah, I just I get really happy when I see people have these good experiences using it and start to get motivated to like build a more positive and you know, high quality future together.
1:01:29
I don't know how to wrap that up Mm-hmm. Well Yeah but yeah.
1:01:29
Talk to me about the willingness to finance.
1:01:29
Do you think there's a huge element yeah.
1:01:29
Well we just we just talked about that. I think we're sold.
1:01:37
Like what I gotta convince you on is you know, again it's willingness and ability right?
1:01:40
And I gotta sell you on ability right?
1:01:42
Like I might have scared you with Well yeah.
1:01:42
So t this, but talk yeah, talk to me about the human element here right?
1:01:49
Because I think it's one thing I've also realized working at SemiAnalysis is how much of the this stuff that seems so data driven all w comes down to some human being making a decision.
1:01:59
Like Satya said pause Totally.
1:01:59
therefore pause, or Jensen said build therefore build, or backstop or whatever right?
1:02:06
So where does this break down to Genuinely one of the leaders of these companies having this experience using a personal assistant or an AI or just like having that like personal moment where you're like, my God, I can see the math discovery before it comes out and gets published, or I can see this progress we're making on this cancer drug or whatever.
1:02:23
And like, I just really want to bet the company on this and I want to do this work because it means something to me and this is like my life's work, as opposed to rationalizing all of the financial details to make it work.
1:02:34
Like can you even contend with explaining that balance between those two things?
1:02:41
Yeah, but you know, like I that is one of the big questions because we covered this is still under willingness right?
1:02:47
Because we covered we're sold right?
1:02:50
And I think what you're saying is are other people sold too?
1:02:53
And that's kind of what this chart is a little bit about right?
1:02:56
And I think the question like I would make one that's slightly different because what I would do is I stratify into consumer like API tip of the spear, people like AI natives and then the rest of the enterprises, right?
1:03:06
So I think we're convinced on the spear tip of the spear.
1:03:10
I feel like Muse is gonna prove the case in consumer but I don't know an enterprise because I sit at these t you know I do like a lot of these events and I meet a lot of you know, folks.
1:03:18
you know, we like we do talks and dinners on AI and then you know, kind of Fortune 500 they're like what would I use it for?
1:03:26
Like just summarizing stuff, like no it's so much so you know, I feel like there's just a lack of discovery in Fortune five hundred and lack of imagination and a lot of people They actually kind of don't come out this with an open mind.
1:03:38
They out this like, great AI, like how do I fire people? Right?
1:03:41
You know, like that's all they think about right?
1:03:44
And it's like Yeah, sorry.
1:03:45
so I just think the enterprises it's of the three, it's the one I don't know the most.
1:03:52
I don't know it because it's so much to like does the CO hey, let's embrace it let's trust Amazon, let's go full blast or they go like, no AI in your work. Like, what do you think?
1:04:01
Have you talked to Fortune?
1:04:01
Like where do you think they are on this?
1:04:05
Well, I think the dynamic that imagine happens with the Fortune five hundred is that the Fortune five hundred just slowly gets replaced with companies that are all in on AI and technology. that's dire.
1:04:16
Well, look at the top ten companies. But possibly true.
1:04:20
Look at the top ten companies in the world by market cap right?
1:04:22
I mean, these are all like nine out of ten of them are AI or semiconductor related or datacenter related at this point True. right?
1:04:25
Like when we say hyperscaler all five of them are in there.
1:04:31
Then you've got Nvidia, Broadcom TSMC.
1:04:33
the memory guys are getting close to the top ten.
1:04:36
So Fortune five hundred, I think we use that term to describe if you if you actually mean the five hundred biggest businesses in the world measured by revenue, I think a lot of them are just going to Maybe I should say corporate America, you know?
1:04:51
Legacy businesses, corporate America Legacy business, yeah.
1:04:54
government, just like people that are slow to adopt AI. And fair enough.
1:04:57
I think there are lots of people who will be a lot slower to adopt AI than the biggest companies in the world which I would lump in all the hyperscalers into their soon to be the AI labs.
1:05:08
If Anthropic goes in IPOs at two trillion dollars they're gonna be, you know, a top twenty five company in the world in market cap or something like that. Right.
1:05:16
So I mean, that's the proof point right there.
1:05:20
But you have to make this case of like going out two three years.
1:05:23
Do you expect that this next crop of neo labs that are pursuing all of these new innovations just like completely all get swallowed up and acquired and none of them pan out?
1:05:36
I actually expect many of them to do quite well across a number of different industries, continue to grow.
1:05:41
And if they do get acquired to probably help those businesses grow and those you know, take over.
1:05:46
And so I you know I don't view like a future where Visa and Bank of America and AT&T are just like so critical to the success or failure of the AI native companies that are trying to grow.
1:06:03
Maybe what I'm most interested in is the unlock of capital that the companies that may have been funding those companies in the past may turn and shift over and say Hey, I work in infrastructure finance I work in credit.
1:06:18
And for the next round of this I'm not actually gonna go do another you know project for another bridge or for you know some debt for some credit card payments provider or something like that.
1:06:33
I'm actually gonna go into this AI thing because I think that's upstream of all of those other businesses I've done and it actually is infrastructure and I'm gonna get these returns and I'm gonna have these guarantees and things like that.
1:06:42
And that's where I Ask you because you know these guys better than me about the human element of like for people who really just don't understand this and don't have the experience of using AI, who represent so much you know, capital that's still sitting on the sidelines.
1:07:00
We joke about stuff like Nvidia's revenue at if it hits five hundred billion dollars, is gonna be like two point five percent of the M2 money supply in the US, right?
1:07:14
Just unbelievable numbers we're playing with trillions of dollars of capex we talked about earlier in this episode.
1:07:19
But there's still so much left over.
1:07:22
Like what's the other ninety seven point five percent of the M2 money supply?
1:07:25
So Well, like let me yeah, I can share some stuff because now we're on kind of the ability, right?
1:07:31
and again one of the obstacles I talked about earlier was you know getting people like I'll I guess I'll show you oops, it's up here. let me go.
1:07:45
one of the key things on the ability is like understanding and one of the key obstacles is lenders on the learning curve.
1:07:52
Like so ch you know the people making decisions and lending like right now they're doing it because like okay it's Nvidia credit risk it's Microsoft credit risk.
1:07:58
Like I don't have to know.
1:08:00
But that's gonna change and that's why all these backs are there so they have time to understand this industry.
1:08:05
But let me size the problem for you little bit.
1:08:08
you know let's talk about you know what where this is going right?
1:08:14
Like obviously it's Yeah, sure we don't.
1:08:14
gonna be the second biggest debt financing market right?
1:08:19
It's already surpassed auto loans it'll surpass everything this year.
1:08:24
but everyone's done it without having to have a residual value model for GPUs because it didn't backstop and it's only second to the US mortgage backed market.
1:08:30
now Dan, sorry, move on from that chart. yeah, sure.
1:08:30
We've got some audio only listeners, but I just want to describe it.
1:08:38
it's such a compelling chart.
1:08:41
Auto loans, student loans, credit cards non-agency, CMBS, HELOC.
1:08:43
These US asset backed credit markets are just like flat lines like slowly going up to the right as people, you know You can't e you can't even see them going up yeah. actually.
1:08:58
And it they're It's just five flat lines at the bottom of the chart.
1:09:02
And then it's just this massive exponential line up to the right.
1:09:07
More of a straight line, but yeah straight line up to the right called AI debt financing.
1:09:11
And just explain to me the implications of AI debt financing in twenty six crossing over auto loans and student loans on this chart.
1:09:20
the implication is, and we already see it.
1:09:22
We already see long end you we already see ten year rates going up we already see credit spreads going up we already see an industry and the reason the credit market it's gonna be hard because they don't have the tools to understand like because it's all coming from like a different front.
1:09:38
It's a war on so many fronts because you're getting hit by hyperscaler neocloud, datacenter, everyone at the same time and It's like a zombie apocalypse but you don't know how many waves of zombies are coming right?
1:09:49
You know the Terminator you know how many Terminators give me your money.
1:09:58
Anyway, they're money zombies coming right?
1:10:01
And you don't know whether you're on wave five or wave fifty right?
1:10:05
so you know and to put things in in context as well, I've got a slide here that shows you what this is w like I told you I showed you what it is in terms of like the corporate bond market right? And that's just up here.
1:10:22
but on the next slide, let's talk about overall US issuance.
1:10:26
so it's one trillion, and it's gonna cross one trillion.
1:10:31
So if you think about it, right the total US corpor the US issuance this is federal, which is in the blue.
1:10:37
you know, for those of you listening that's about five or six trillion a year.
1:10:40
You've got MBS, RMBS, just looking ballparking for that it's looks like three or four trillion a year of issuance maybe two to three.
1:10:48
Corporate bonds we already talked about that, about three to four trillion.
1:10:53
total twelve point seven trillion.
1:10:55
And twenty six is the last year that AI debt is gonna be less than ten percent of all issuance right? All issuance in the US.
1:11:03
so it's gonna take believers.
1:11:06
and how do we handicap that right?
1:11:09
you know well this is maybe What we can do is we can jump into like you know back to the question you asked earlier.
1:11:17
It's like why is Nvidia doing these backstops right?
1:11:20
and let's just look at well sorry I'm gonna go okay, here.
1:11:26
Yeah like why do they have all these different if different backstops?
1:11:30
And again, it's they see this gap coming and they're figuring out how to plug it.
1:11:33
let's talk a bit about their AICP right?
1:11:38
So by the way, you know to start off, we projected Nvidia's balance sheet and we were looking at their ability.
1:11:44
To actually continue doing these backstops and how much they can do.
1:11:46
but let's talk about like an example of one of the backstops and how it accomplishes those goals.
1:11:51
This is the AICP program.
1:11:51
you know this actually started in Asia.
1:11:55
The first one was Firmus, actually the first one was Sharon AI then Firmus followed with another one.
1:11:59
you know, there's been Aolani there's been many others, GMI did one.
1:12:03
And the whole idea here is the way this works is Nvidia goes to neocloud says Hey, I'll give you a guarantee.
1:12:11
Like if you can't rent your cluster.
1:12:13
I'll rent it from you and I'll give you that guarantee for six years on 325 on average.
1:12:18
It's the actual table is different.
1:12:23
Each year is a different minimum.
1:12:25
But if you can't find clients you can always go to Nvidia and rent to them at the at this price.
1:12:30
Now the intention is not for that to actually happen.
1:12:33
The idea is you use this as a guarantee to then get financing. Remember the Trinity? Right?
1:12:37
So now you have an offtake and now you can get financing.
1:12:40
And then with that you can get datacenter.
1:12:43
With this in mind, lenders you see this little thing they it's prices Nvidia risk.
1:12:46
So they say hey, you know, I'm really lending to Nvidia plus execution risk.
1:12:52
and then what that allows the neocloud to do is now I don't have to rent the five-year off takes to whoever.
1:13:00
I can do you know, tokens of service I can do all these things.
1:13:03
It really opens up the market. That's the whole idea.
1:13:05
and they don't need to be IG that's the important part.
1:13:08
but this is only like an intermediate.
1:13:13
intermediate stuff because what it does is lenders have to understand that they're they're also taking the platform risk whether neocloud can build a good book of clients and they'll get to see it as they lend to neocloud.
1:13:22
They say like hey, can they build an independent business outside the optic?
1:13:26
They get to understand the customer profile the churn rates, the cancellation rates.
1:13:31
And they get to, you know, kind of do like a free look or a dry run a practice run until the real thing.
1:13:37
And that's why this is kind of really important why they're doing it.
1:13:38
Does that make sense Jordan? Any questions? Makes perfect sense.
1:13:40
Yeah, I mean, balance sheet as a service.
1:13:44
Maybe the one thing to talk about is like they at some point introduced a take on the percentage of revenue that the neocloud was able to I forgot about that. Mm-hmm.
1:13:49
get above the minimum that they promised.
1:13:57
And then they took that away and you know okay No, it's still there, right? explain that, yeah. Yeah.
1:14:03
So it's also a revenue model as well.
1:14:08
and it's fair, you know we're not communists after all.
1:14:11
so in exchange for the backstop Nvidia gets a 50% revenue share above the backstop.
1:14:17
and so for example, let's say you're in at $6 or let's make it easy, $668.
1:14:20
Then you have the difference is $3 so you gotta pay $150 to Nvidia out of the six in exchange for that.
1:14:30
So it's actually a meaningful revenue share for Nvidia.
1:14:34
And if you look at the other structures they've done this is the Lambda structure, they're also taking a revenue share out of Lambda in exchange for that guarantee.
1:14:42
So, you know, and another third reason Jordan, by the way, one of these backstops do is not just facilitate these deals happen, allow, you know in this instance, they're allowing Lambda to get financing because Anthropic not yet IPO, is not IG, right?
1:14:58
But with Nvidia supporting the lease for the datacenter they can actually get this done.
1:15:04
But there's another reason, right?
1:15:05
Let's look at Ports Pike.
1:15:05
This is a little complicated.
1:15:08
But the bottom line here is Nvidia is giving $105 billion of residual value guarantees to the Ports Pike datacenter complex in order to support this.
1:15:18
But the catch is it's gonna so what they've done is they've guaranteed that 4.
1:15:21
25 gigawatts or up to nine almost nine.
1:15:27
Across two phases, will be running Nvidia GPUs. So it also locks in.
1:15:30
It's like a kind of a land grab is another reason for it.
1:15:34
and so we looked at it we looked at all these different models and I'll talk about one last one.
1:15:40
And this is the kind of the most interesting one is the 500 billion capital partnership.
1:15:45
because this gets into how can we actually you know, to your point, right?
1:15:49
You talked about lenders as one homogeneous but it's actually very different.
1:15:53
There's bank lenders, insurance lenders they don't want to take a lot of risks.
1:15:55
You've got Private equity, they want to take a lot of risk you got private credit, they're in the middle.
1:15:57
Yeah, they all they all syndicate into these things as well right?
1:16:01
Well, it's about creating different products for different different types of investors.
1:16:08
So the five hundred billion capital partnership this is what we think it could look like.
1:16:12
And what you do is you can have an AI lab they don't have to be IG, but what you do is you tranch risk.
1:16:17
So you have equity and then you've got a B piece which doesn't have any residual value guarantee and then you have an A one, A two you could put one C to the other and then Nvidia is giving a twenty five percent residual value guarantee in this five hundred billion partnership.
1:16:30
Maybe you attach this 25% to this tranche or this tranche.
1:16:33
But because let's say there are losses in this program you know, in this business, then the equity will take the loss first then the B piece, and then part of the A2.
1:16:42
And so it will have to go all the way down to this.
1:16:47
The loss has to be so severe to affect the senior tranche which is how you can create different risk profiles and open up the market.
1:16:53
but anyway, all this depends right, on Nvidia having the capacity to do this.
1:16:58
So we analyzed the balance sheet.
1:17:00
You know, and you know, if you kind of love talking about this you know, this is all we're gonna do.
1:17:07
but you know, here's the kind of conclusion of this Jordan, right?
1:17:10
they we think they have the capacity this is not our forecast, but we think they have the capacity to support up to 46 gigawatts of capacity.
1:17:18
We haven't done it for the hyperscalers yet but that's still not enough.
1:17:20
There's two hundred and forty gigawatts is gonna come in.
1:17:23
And so what I'm saying here is the lending market has to evolve.
1:17:30
to take risks outside of backstops.
1:17:32
And that's what, you know, we're all about understanding that question.
1:17:35
And it's really the biggest question of our time to be honest, which is why we're starting an entire group around this question. Makes sense, man. Yeah.
1:17:44
If there's two tangible things I'm taking away from this it's one more education to the lenders on what they're actually Mm-hmm. getting into here.
1:17:50
you know, what the real deal structures are like and who the end user customers are and why they're structuring things this way.
1:17:59
And then two, well I guess two is twofold.
1:18:03
it's always incredible to see the size of these numbers and how much has already been done.
1:18:13
But also how much bigger it needs to get from here for things really. Yep.
1:18:15
This is incremental capacity adds.
1:18:18
So incremental sixty some odd gigawatts of capacity adds in twenty nine. Yeah, let's see.
1:18:25
When we first started this journey, global datacenter capacity is like 40 gigawatts maybe.
1:18:30
And now we're adding like 40 a year. Can you imagine that? I can, yeah. We yeah. With your usage. Excited for it.
1:18:43
Yeah, I know, I know, I know.
1:18:43
I was looking at the numbers.
1:18:43
Can you tell let's wrap up one little funny story?
1:18:49
When I got started working at SemiAnalysis I did not understand like anything about financing and how this stuff works.
1:18:55
I was trying to figure out who a lot of these lenders were what private equity was.
1:18:58
I'd known these names things like that.
1:19:00
And I remember working on this one deal.
1:19:03
I won't say, you know, who was setting up the deal but it was like private equity.
1:19:06
Did you think EBITDA was like a musical at some point Jordan? I can do the basics. That's fine. Yeah yeah. yeah.
1:19:18
I still get confused when you talk about project versus workbook IRR. I don't know.
1:19:25
I just know that it's really important project versus equity IRR. but equity IRR. Yeah, okay.
1:19:30
I just know IRR is you spend enough time with Dan, you know IRR is important. And it doesn't get Yeah. easier to say I-R-R. That is IRR, yeah.
1:19:42
that's the that was the pirate right?
1:19:44
Dan, when he started it's like he needs an eye patch because he's talking about his I-R-R. Anyway.
1:19:51
okay, let me tell this quick story before we wrap.
1:19:53
So this private equity company was setting up this deal.
1:19:56
We had to go do this technical DD on it we had to present to some of the lenders and the lenders were like, you know from all sorts of different institutions.
1:20:05
And I remember at the beginning learning like, private equity doesn't have any money for themselves.
1:20:10
Like they have lots of assets under management but they set up the deals and then the lenders come in and they actually like fund the deal and private equity like makes carry off of the top of that right?
1:20:20
If anybody in PE guys are listening to this is like yeah, of course that's a way. I don't This is Ha. anyway.
1:20:25
and so I had this meeting with this pension fund to explain like the work that we did.
1:20:31
And I remember sitting on the call it was the Ontario Teachers' Pension Fund.
1:20:37
And I was like, it's my mom's money.
1:20:41
Like, this is what's you know, getting in here.
1:20:45
So anyway the kind of this whole like loop back to thinking okay, yeah, like Canadian pension Ontario teachers' pension, lots of stuff in Quebec. Like, I'm Canadian.
1:20:57
So anyway the it like I ended up feeling like kind of proud of the work that we did not like I was helping these people who were just like rent seekers trying to make a little bit of money off of this.
1:21:07
Like it actually is pretty hard to set this things these things up make people comfortable understanding what they're investing in explaining the risks, understanding the technology what's the return, blah blah.
1:21:15
And yeah, there's gonna be more to do in the future because lots of people just like that are probably getting in and evaluating their first AI cloud opportunity or neocloud deal.
1:21:27
And it's gonna be a big part of the future.
1:21:30
We're on a learning curve and that's you know I think the more we help people get up there you know, talk about everything I think the more it helps everyone make great decisions. Yeah. Cool, man. Okay.
1:21:42
So you know one last joke you know why it's IRR? 'Cause No, why? pirate equity. Pirate equity.
1:22:02
Hey everyone, if you're still listening after this hour and fifteen minute long conversation about credit and markets you should consider applying to work at SemiAnalysis on credit and markets.
1:22:10
Dan is a very fun guy to work with.
1:22:12
Dan, who are you hiring for?
1:22:15
Well we're hiring credit analysts in New York and Singapore.
1:22:19
So we're hiring junior credit analysts.
1:22:21
So anyone who's done one to three years of experience could be investment banking ECM, well, DCM, you know and anyone who's focused on that.
1:22:31
also we're looking for more senior credit analysts anyone who's been a desk analyst or investment analyst on the buy side and also a credit specialist.
1:22:42
Someone who's quite in touch with markets has a great Rolodex, worked in credit trading credit sales, you know, buy side or sell side.
1:22:51
So love to have you apply.
1:22:51
Just click here to find out more.
1:22:55
also need a video editor.
1:22:55
If anybody knows about a video editor, shoot me a text.
1:23:01
Okay, this is your third interruption.
1:23:03
We're experimenting with ads on the SemiAnalysis Weekly podcast.
1:23:06
We also need a video editor. Akash is hiring.
1:23:09
Yeah, so if you know how to edit videos please hit me up.
1:23:11
We're looking for people. Please hit me up.