0:23
Hey, hey, hey.
Search this video
0:27
>> [music] [music] >> Heat. Heat. [music] [music] Heat. [music] Heat. >> [music] >> Heat. Heat. N. [music] Heat. Heat. N. Heat. Heat. N.
3:17
>> [music] >> We came to this world to reach [music] the stars.
3:28
We came to this world to see a future.
3:38
We came to this world to reach the stars to shape our future.
3:53
>> [music] >> We came to feel the new [music] [music] You will let me freak.
4:39
[music] >> [music] [music] >> You're watching TVPN.
5:02
Today is Tuesday, October 28th, 2025.
5:06
We are live from GitHub Universe here in the Fort Mason uh in San Francisco.
5:13
And we have a ton of exciting interviews today.
5:16
We are interviewing the CEO of Microsoft, Satcha Nadella. We're very excited.
5:21
We've been on a quest to interview MagG7 CEOs.
5:23
Uh and we are very excited to sit down with him today.
5:26
And it's a huge day because uh Microsoft to just today announced that they have uh entered the next phase of the partnership between Microsoft and OpenAI.
5:37
Uh there of course are uh dueling blog posts, one on OpenAI's website, one on Microsoft's website.
5:45
We will go through some of the Microsoft uh update to give a little bit of background before we go into our interview with Satcha Nadella.
5:51
But first, let me tell you about ramp. com.
5:54
Time is money, save both, easy to use, corporate cards, bill payments, accounting, and a whole lot more all in one place. Um >> let's go.
6:00
>> So this all started back in 2019.
6:02
Microsoft and OpenAI, it says here, has a shared vision to advance artificial intelligence responsibly and make its benefits broadly accessible.
6:08
What began as an investment in a research organization has grown into one of the most successful partnerships in our industry.
6:16
And I think that it might be one of the greatest deals of all time in business history.
6:20
It it is a remarkable remarkable deal.
6:23
I foright I was I was digging through some of the other deals that where big tech companies worked with each other or bought stakes in each other and there are some wild ones that people might not know about.
6:35
I think it might be worthwhile to go through.
6:36
Uh before we do let me tell you about reream one live stream 30 plus destinations multiream reach your audience wherever they are.
6:43
Um [laughter] uh so in uh in back in what was it 1997 Microsoft bought $150 million of non- voting Apple stock uh which settled some litigation committed to uh they were going to put Microsoft Office on the Mac for 5 years and it made Internet Explorer the default browser on the Mac >> and so >> slightly before my time. And I guess it was born.
7:12
But >> by they they did this deal uh but by by 2001 Microsoft had converted all of the shares into common stock netting the company approximately 18 million shares of Apple.
7:22
And then by 20 by 2003 they'd exited the position which I don't know if that's a good deal.
7:27
They maybe they should have diamond hands it but uh it's still it's still a wild wild moment. >> Yeah.
7:33
I I think you know something I'm excited to talk to Satia about is just like how much how much foresight he had. Yeah.
7:40
Uh whether he knew whether whether he was expecting a base hit or he really felt like it'd be a home run. >> Yeah. Yeah.
7:46
It is a very fascinating thing.
7:48
It's like you're doing a deal with this nonprofit.
7:49
Sam Alman's obviously a big character in even in 2019.
7:54
Sam Alman was an important figure in tech force.
7:56
Uh but at the same time I was I was running the numbers and I was like at at least today Microsoft makes like a billion dollars in revenue every single day.
8:04
And so if you think about it, I don't know if you actually think about it this way, but if you just think about it, like it's your job as the CEO to steward capital. Yeah.
8:12
And a billion dollars sounds like a lot. Yeah.
8:14
But you're making a billion dollars every business day.
8:16
Like there's five business days a week, 52 weeks a year.
8:19
You're basically re like revenue for Microsoft right now is about a billion dollars a day.
8:24
And so do you treat that deal like it's just another day at Microsoft or or is it something that there's weeks of negotiation because you do have a sense that this is going to be one of the more important deals?
8:34
And I always I always when you look at when you when you look at the check size >> Yeah.
8:40
>> relative to the capital available, it looks like a flyer if you were a [laughter] VC fund, right?
8:47
>> And uh, you know, I I think it's fascinating because there's so many, you know, scaled platform VCs that have to just be faced.
8:54
They have to look at this announcement.
8:58
Microsoft 27% >> 2010, right?
9:06
And they just have to look at that announcement.
9:09
Uh, and >> I haven't dug in, but I mean, I've seen there there's a little bit of sour gripes from the from the venture capital community saying we didn't get enough of we didn't get enough. >> Yeah.
9:19
If you look at the the riskreward that the the the the amount of capital that the seed investors deployed into the company relative to their ownership today, >> it looks uh you know certainly they made a great return on paper, but but >> uh did they actually make a great return relative to the risk of >> uh you know investing in a company that had uh went against every YC practice there [laughter] is, right?
9:44
It's like why Z says like so many videos of Samman saying don't don't reinvent the wheel >> the wheel let me do >> let me reinvent the wheel >> and and ultimately I mean this has led to like so much of the cha you know as as chat GBT has exploded the chaos around the company has almost entirely centered around the corporate structure.
10:04
So >> uh I wonder you know this may be you know the final company for Sam right in terms of of uh but I but I wonder what he would do next time around.
10:12
I mean we we we have run the experiment because he has a bunch more companies and most of them I think are pretty clean corpses but then again worldcoin has a token and so like there are multiple things going on but I think uh probably if we dug into his uh his BCI company we would see a cleaner corp and the I'm excited in the fullness of time when we get the when we get the books and the documentaries on uh both open AAI and this investment. >> Yeah.
10:40
I can't wait to see and try to understand uh where what OpenAI was really what they were facing at that moment when they did this series of deals with Microsoft, right?
10:51
deals with Microsoft, right? Because the ultimate deal of you know uh selling such a large amount of the company with a with a revshare attached which is wild ask ask uh you know a YC partner if a portfolio company uh one of the companies in their group came to them
11:07
and said yeah I have this investor it's a large tech company they want to invest like they want to buy like a lot of the company and they want a 20% rev share for like the next 10 years they'd be like you need to walk away from that deal immediately but Sam Sam and Satia did it and here they are today. >> How about two on 20 safe note? Yeah,
11:23
>> How about two on 20 safe note?
11:23
Yeah, [laughter] >> good old fashioned way. >> Good old fashioned.
11:27
Why are we reinventing the wheel?
11:28
Um >> there are a couple other interesting um cross industry deals.
11:33
Uh Jeff Bezos famously uh had a big stake of Google, right?
11:39
Wasn't he an angel investor?
11:39
Uh there's also the time that Intel, TSC, and Samsung came together to invest in ASML, which of course makes the lithography machines kind of pulling forward kind of the initial uh like weird, you know, circular deal that people point to, >> but it worked out.
11:55
>> But that one worked out for sure.
11:55
Um and uh and yeah, it's uh it's interesting seeing the the evolution of this deal in particular. Some quick history.
12:02
uh July 22nd 2020 2019 uh Microsoft invests $1 billion in OpenAI uh Azure is named the exclusive cloud provider.
12:13
Microsoft is named the preferred commercialization partner.
12:17
Um in 2020 Microsoft receives an exclusive GPT3 license and for its products and services >> and the foresight the foresight here is just from Zatia is incredible.
12:26
just from Zatia is incredible. like this is >> like at the at the time like it wasn't that >> like only a couple years prior Elon was basically walking away because he said like there was no there there I'm sure it was material [clears throat] progress
12:43
internally but to have that level of of uh understanding and that that much conviction to invest a billion dollars when you were years out >> it's more it's it's more complicated than that I I think because uh there are plenty of big tech CEOs who have taken $1 billion flyers on crazy ideas. We see
12:59
We see that all the time with the, you know, oh, you want to build some new hardware thing or how much, how much did Apple spend on the car?
13:07
They probably spent a billion dollars working on that car already and like, you know, they kind of like they the risk adjusted reward, the risk adjusted bet made sense, but then ultimately they pulled back from that.
13:18
And that's happened probably all over the place and Saja himself probably has other times when he's put down a big investment for something that was risky and it didn't pan out. Yeah.
13:26
>> What what is interesting about the OpenAI deal is that I know investors personally who were looking at the deal before that and they couldn't get over the complicated structure. Yeah.
13:34
And so they dipped out for that.
13:36
And so it's not that it's like, oh wow, we need to give a round of applause for someone who's helming a trillion dollar company to write a billion dollar check.
13:44
Like that's not that crazy.
13:45
That happens all the time.
13:47
What is crazy is to get over all the lawyers being like, you're doing what and how it's structured.
13:52
What are you talking about?
13:54
there's a nonprofit involved.
13:55
Why are we doing >> say that back to me? >> Exactly. Exactly.
13:57
And so, but knowing that that we do live in a society where if things if value is created, if a new platform emerges, everyone can overcome any any chaos and all complexity.
14:11
So, >> and so in 2021, Microsoft followed on with more investment.
14:17
Um and then uh uh the open AI service went general availability on Azure in 2023.
14:22
Uh before we move on, let me tell you about privy wallet infrastructure for every bank.
14:27
wallet infrastructure for every bank. I think pretty makes it easy to build on crypto rail security integrate on all one simple API >> and and and the reason that I say this is so notable and and this this conviction uh really matters is that
14:42
think of when you look at other scale you know hyperscalers how slow how how even years after the the the sort of chat GPT moment and granted everything we've talked about so far predates chat GBT right >> and and years after this chat attribute moment. We still have people that are
14:59
We still have people that are only now coming around to saying like we're we're ramping up we're ramping up capex, right?
15:05
And so, uh I just think that SATA was incredibly ahead of the curve here. >> Yeah.
15:11
Uh it's it's uh yeah, it's easy to look at the deal and through the lens of oh well uh co-pilot was the glimpse of value creation out of what is effectively like a nonprofit academic lab.
15:26
But it's important to remember co-pilot happened 3 years after GitHub copilot launched two or three years >> 2022.
15:33
>> Yeah, two or three years fully uh after the the that initial $1 billion investment.
15:38
Uh so yeah, remarkable remarkable progress.
15:40
Um >> um let's go back to the uh Microsoft uh announcement today.
15:46
We can go through some of the key points details on what's evolved.
15:49
Do you want to read some? >> Yeah. So uh what has evolved?
15:50
Once AGI is declared by OpenAI, that declaration will now be verified by an independent expert panel.
15:59
So I'm assuming they're going to get Joe Rogan, Andrew Huberman, Lex Freedman, and get, you know, a panel of podcasters to decide this.
16:07
Now, of course, uh this is a question that I want to dig in with Satia in just a few minutes trying to uh right right today like uh nobody, you know, some folks can agree on on a definition of AGI, but it's it's very much in flux, right?
16:22
Tyler Cowan >> was on our show a few months ago saying that he felt uh AGI had already been achieved and that we keep moving the goalposts.
16:30
Others believe that we're in this era of of spiky intelligence and we need uh sort of um you know more broad intelligence before we can get to uh true general intelligence.
16:40
Uh but going forward, Microsoft's IP rights for both models and products are extended through 2032 and now includes models post AGI with appropriate safety guardrails. That feels significant.
16:52
uh Microsoft's IP rights to research defined as the confidential methods used in the development of models and systems will remain until either the expert panel verifies AGI or through 2030 whichever is first.
17:03
Research IP includes for example models intended for internal deployment or research only.
17:06
Beyond that research IP does not include model architecture, model weights, inference code, fine-tuning code or any IP related to data center hardware or soft and software and micros Microsoft retains these non-ressearch rights.
17:19
Microsoft IP rights now exclude OpenAI's consumer hardware.
17:24
Uh that that's uh notable.
17:27
They need to start figuring out carveouts.
17:29
Uh but um and OpenAI can now jointly develop some products with third parties.
17:35
API products developed with third parties will be exclusive to Azure.
17:38
Non-API products may be served on any cloud provider.
17:41
So again, uh Satcha's for if you're just joining us, Satcha Nadella will be joining us in five minutes to break all of this down live on TBPN.
17:49
live on TBPN. uh for right right now we are setting the table with some analysis and uh looking through the details of the story that emerged today yep from Microsoft >> so Microsoft can now independently pursue AGI alone or in partnership with third parties >> if Microsoft uses openAI's IP to develop
18:06
AGI prior to AG AGI being declared the models will be subject to compute thresholds those thresholds are significantly larger than the size of systems used to train leading models today the revenue share agreement remains until The expert panel verifies AGI, though payments will be made over a longer period of time. OpenAI has
18:24
OpenAI has contracted to purchase an incremental 250 billion of Azure services, and Microsoft will no longer have a right of first refusal to be OpenAI's compute partner.
18:34
Uh again, like that 250 billion number uh is uh you know, it's certainly not the biggest number we've heard, but but it's quarter of a trillion is nothing to uh scoff at. Yep.
18:46
Uh, and OpenAI can now provide API access to US government national security customers regardless of the cloud provider.
18:52
And finally, OpenAI is now able to release openweight models that meet requisite capability criteria. >> Um, yeah.
19:01
>> So, uh, again, I'm I'm I I I feel like on a number of of these points, it feels like they are uh kicking the can down the road a little bit.
19:09
Again, obviously this was important to complete the conver conversion from the LLC to the public benefit corporation, which presumably can go public.
19:19
>> Um, but again, uh, my question and and my immediate thought is how many of these things are going to be critical to iron out before the IPO?
19:26
Is there going to be enough demand that it doesn't matter again?
19:30
In the same way that certain investors, you know, uh, you know, our friend Josh over at Thrive and others were had incredible conviction to be, you know, deploying again and again and again into uh, OpenAI's for-profit uh, subsidiary even when there was so much uncertainty around the structure, right? >> Yeah.
19:47
Uh, big open question in how Microsoft's internal AI research efforts evolve now that this is a little bit more concrete.
19:56
Uh, will be very interesting to see.
19:58
Yeah, I mean this this if Microsoft uses OpenAI's IP to develop AGI and then >> prior to AGI being declared.
20:04
[laughter] >> We'd love that.
20:07
>> Opening eyes dribbling dribbling towards the basket.
20:09
Satia comes in with >> who knows? Who knows?
20:13
>> Uh if you're just joining, uh we'll be live with Satcha Nadella in 1 minute and 17 seconds according to our timer.
20:18
Uh in the meantime, let me tell you about Cognition.
20:22
They're the makers of Devon, the AI software engineer.
20:23
uh crush your backlog with your personal AI engineering team.
20:27
Um so uh >> yeah so >> yes >> uh again there there's so many of these points that leave kind of open open questions they'll need to be effectively renegotiated uh again and again down the road but at least this provides >> uh pathway and it's it's no longer uh the the elephant in the room.
20:49
Yeah, it does feel like the the cap table is getting slightly cleaner and you're moving towards something where I mean if you look at the at the history of the Microsoft deal with Apple, they had a position they eventually rotated out of that, sold out of that because uh there's there's this question of, you know, if you're the CEO of Microsoft, you're such an Nella.
21:09
Should you be a venture capitalist as well?
21:11
Like like oftent times big tech companies do make investments, minority investments.
21:16
Sometimes they make whole coacquisitions, but is that your primary business? >> Yeah. Yeah.
21:20
I mean, ultimately, this comes down to feeling like potentially one of the greatest corporate venture investments of all time.
21:27
And so, I'm not coming up with any that are >> that are better.
21:30
It's pretty good >> significant in terms of not just owning a massive piece of a generational company and a future you know potentially what what looks like a future you know uh uh hyperscaler but also giving your business just this incredible strategic advantage uh in the race uh broadly.
21:49
So >> um >> yeah to go any more impactful you need to move over into the whole coacquisition world you have to talk about Instagram but even even that is tough but uh it's a very different very different deal structure and something that is just down the fairway buy the entire company buy the entire entire product uh as opposed to um make this bizarro minority investment and then grow from there. Yeah.
22:16
And the uh it's worth noting too that OpenAI and uh Microsoft Office are on, you know, already on a collision course, right?
22:26
Like you can imagine that over time uh these products, you know, overlap today.
22:31
You can use Copilot for a lot of things that you can use ChatgBT for.
22:32
That's only going to become there's still going to be this like massive tension there. Yep.
22:39
>> Um and uh we'll be covering it live.
22:42
>> Well, let me tell you about figma. com.
22:44
Think bigger, build faster.
22:44
Figma helps design and development teams build great products together.
22:48
And I believe we're ready for our first guest of the show.
22:52
Satcha Nadella, the CEO of Microsoft.
22:55
Welcome to the show, Satcha. Great to see you. How are you doing? >> Great.
23:02
>> Thank you so much for doing this.
23:02
Uh please, uh there were a ton of bullet points in the announcement today.
23:07
[laughter] Uh can you just zoom out and explain it to me like I'm five? What actually happened?
23:12
what what what actually changed because you've been in partnership with with OpenAI for six years now, but this feels like an important moment. What happened?
23:21
>> Yeah, look, first of all, you know, it just feels Yeah, you're you said it right.
23:24
It's a good it's an important moment and the story continues. >> Sure.
23:28
[laughter] >> But the story actually got started even the OpenAI one.
23:31
I've known Sam for a long time.
23:33
>> Um since his first company pre YC days all the way back then. Wow. >> That's right. That's right.
23:38
I was actually I remember him being at WWC DC presenting it in the double polo.
23:43
It's iconic but I didn't realize that we were doing business with them back then.
23:47
>> Um >> and it started I think in 2016.
23:48
In fact we were Azure was the first cloud provider. That's right.
23:54
>> When OpenAI got started uh in fact I think Elon sent me the mail asking for Azure credits. No way.
23:59
[laughter] >> So that's how it got started.
24:04
>> Hey I have this nonprofit. Yeah. Come on. [laughter] on the hat. >> That's right.
24:09
And they were obviously into reinforcement learning.
24:10
They were doing Dota and all of that stuff.
24:12
And uh and then at some point it reached uh where I think they went off I think they went to uh other clouds.
24:19
And so I lost touch for a while. >> Okay.
24:23
>> Um and then I I think in 2019 um Sam came and talked about sort of hey we're going to really we think this scaling stuff works.
24:31
Uh I I forget now it's a little hazy.
24:34
little hazy. when I read the paper in fact the paper was written by Dario Ilia and the scaling laws paper and uh the thing that's you know Microsoft has been obsessed since Bill started at Microsoft research in 95 is natural language >> uh it's just you know been the thing we
24:52
are an office company we are a knowledge work company and so we've always thought about text and AI as applied to text in natural language so you could say it's the prepared mind when sort of Sam said hey we're going to go take a on you want to be on it. Uh that's sort of what led
25:05
Uh that's sort of what led to really coming together on this.
25:08
Yeah, it was a it was a research lab.
25:10
It was a nonprofit >> as opposed to if they had stayed on the previous tech tree path of uh they were doing some humanoid robotics and they were doing some video game stuff and Dota 2 that doesn't jump out to you as immediately relevant.
25:24
It's it's interesting you bring that up because obviously RL has come back in a big way uh in in relation to sort of these large language models.
25:32
Uh but yeah, it's pro, you know, this is the funky path dependent way things happen, right?
25:39
Because I don't think I would have gone in fullon to say, hey, let's go, you know, partner with these guys, build a computer uh that scales it if it was not natural language.
25:47
But I'm glad we started there and then RL now is improving the quality of these models.
25:54
>> How big of a deal was uh writing a a $1 billion check back then?
25:57
I mean, it's a big company, Microsoft.
25:59
We think it makes revenues around a billion dollars a business day.
26:03
Uh was it one day of work for you [laughter] or or was it or was it you know weeks of negotiation?
26:09
Seriously, did you build memos?
26:11
Like were you building Excel sheets?
26:13
Like what were you thinking?
26:14
>> Even at Microsoft, you kind of got have to get a board approval [laughter] just go throw a billion dollars out there.
26:20
But you know, I must say it was not that hard to convince uh anyone that this is an important >> area and it's going to be risky.
26:26
>> area and it's going to be risky. Like I mean in retrospect I mean who would have thought hey I didn't put in that you know billion dollars saying oh yeah this is going to be what 100 bagger I mean [laughter] that's not like what was going through our head which this is a a partnership
26:41
that I mean by the way remember this is a nonprofit right and I think you know Bill even said yeah you're going to burn this billion dollars right and yeah we kind of had a little bit of high risk tolerance totally >> um and we said we want to go and give this a shot uh and then of course we subsequently Um, you know, here we are at GitHub universe. In fact, this is probably the
27:00
In fact, this is probably the place >> where that billion to 10 billion happened because in 21 >> uh is when I first saw GitHub copilot. >> Sure.
27:11
>> And I said, man, this is working.
27:12
>> This a year before the release.
27:14
>> Uh, no actually in fact I was fact-checking my thing.
27:16
I think GitHub uh copilot launched in 21, chat GPD in 22. Got it.
27:23
>> And if I remember right, 23 is the blip, the the November blip with OpenAI, and then everything has been smooth since then. >> The blip.
27:30
>> That's the [laughter] nicest way you could put that. That's fantastic. Um, yeah.
27:34
So, so, so that makes a ton of sense.
27:36
Um, obviously it's been it's been a wild ride up and down.
27:38
You started with just, oh, natural language.
27:41
Let's predict the next word.
27:43
Now, we're let's rewrite the entire global economy.
27:45
How do you think about the territory that you at Microsoft have have kind of claimed and what do you want to hold on to?
27:54
What's what's most important to map out where Microsoft where the edges of your territory are and then where founders and other business people can build in in partnership with you? >> Yeah.
28:04
So look if you take um sort of I always say Microsoft's a you know a a platform company and a partner company and we define platforms as where the value capture about the platform is higher than by the platform.
28:16
That's kind of who we are and that's you know GitHub is a great place.
28:20
So if you think about even at GitHub Universe today it's it's interesting right?
28:25
We you as you said we first started by saying hey code completions. Yeah.
28:29
completions. Yeah. Then we said let's chat right instead of getting distracted stay in the flow of coding you bring uh the information to the flow and that's chat became the thing then we said agent mode >> then we said hey let's have autonomous agents >> and then now we have multiple autonomous
28:45
agents working across all these different branches then bringing the PRs to me and so this entire conference is about what we call agent HQ and mission control where you have codeex you have claude you have gro every model you want to each working across their own branches. >> Then you have the IDE so you can bring
29:02
>> Then you have the IDE so you can bring up VS code where you can do the diff on each of the branches output and then so the story goes on.
29:10
So therefore to me building a system >> that really brings the innovation across the ecosystem into some kind of an organizing layer is what platform companies do well.
29:21
>> Have you seen anyone here that you think might be working on AGI?
29:24
Do you have a personal definition for AGI?
29:26
Yeah, that that if you look at pretty much all the deal points, uh it it you know it keeps coming back to this moment when AGI will be declared, right?
29:35
It's a there'll be a panel of experts.
29:38
Maybe that panel is still being decided, but a lot of experts today have differing definitions.
29:43
So, I wanted to get a better sense of how you how you imagine that kind of decision-m process will go when the time comes.
29:51
>> And we'll put you guys on the panel. >> Yeah.
29:53
[laughter] We've been doing evaluations on on AGI specifically around comedy. Can it raise high? >> There you go. There you go.
30:00
Comedy very difficult comedy bench to be I think first of all I think one of the reasons why quite frankly both Sam and I I think agree on this which is it's become a bit of a a nonsensical word.
30:08
I mean it's just changing and everybody defines it differently and and we now know what the issue is, right?
30:14
We know everybody describes even the intelligence we have which has been exceptional uh as jagged. >> Yep. Yeah. >> So spiky. >> Yeah. spiky. All right.
30:22
And so if you sort of say well we have spiky intelligence and in fact I think Andreidge Karpathy's point uh in one of the podcasts recently which is a good one which is >> even if you're having exception you know let's call it exponential growth in one of the spikes. >> Sure.
30:38
>> It's not as if the the Jags are getting worked out.
30:41
That's the nine's problem right that is each nine is maybe linear even sublinear problem right in terms of rate of progress.
30:48
So first step to me is even to get to broad intelligence forget sort of general intelligence we've got to get rid of these jagged problems and that I think is the first place to do So if you ask me, I think what may happen is we will achieve more robustness, let's call it that. >> Yeah.
31:08
>> Uh for different systems, right? So coding is a good one. >> Yeah.
31:12
>> I think the entire goal with GitHub and GitHub mission control and agent HQ is can I just like how I use compilers, >> can I use agents >> to generate better coding artifacts? Yeah. Right.
31:25
Yeah. Right. today coding artic I mean like so I sometimes think w coding is a sort of a slightly unfortunate term because >> it does lead to a lot of slop yeah >> right I mean it's kind of like I'm sure you code away and then you lose control of the project and then you got to put everything back into a markdown >> and you so
31:41
>> yeah even traditional knowledge work that's happening in the office suite it's not like you want the biggest Excel model right you want the one >> yeah but even excel it's a classic one in fact one of the other things that's happened is even in when I see M3 Microsoft 365 I copilot man the amount just like right now the number of repos on GitHub is exploding. Yeah. Yeah.
32:00
>> The other thing is everybody's generating PowerPoint and slide decks and sort of uh Excel models.
32:06
>> The problem with Excel models is you know when intelligence has created Excel models.
32:11
[laughter] I mean >> it is like a thing of beauty right the assumptions are clear the formulas are there >> even the formatting >> and the and yeah and the formula and you can iterate on it you can change it tells a story it's not like a one shot and when I want to change I can't go back and say zero shot the entire thing so in fact the agent mode in excel which I like is it understands office.
32:30
js JS it puts the formulas I can then iterate like I iterate on GitHub copilot.
32:36
So that so those systems so if you ask me about you know first how do you get rid of this jagged intelligence problem is you build a great knowledge work system that is multi- aent multimodel multiiform factor >> get to a great benchmark and an eval where you can trust it at 2 9's 3 9 and until you achieve that uh you're not going to be able to move say and say hey we have anything kind of general intelligence.
33:04
Yeah, it feels like there's there's there was a lot of uncertainty in the tech community around, oh, is super intelligence going to come out of the lab tomorrow and there's going to be this fast takeoff.
33:16
Now, it feels like there's more opportunity both for Microsoft to to add those nines to products that you have and then also uh to entrepreneurs who are building products maybe on top of Microsoft.
33:26
Microsoft. um how are you thinking about the entrepreneurial opportunity in the >> I think that's a very good point because at some level if you sort of buy the argument I made that there's a lot more invention to happen uh by the way the other thing that we should also talk about you I always say to this right
33:41
right today it's all the conventional wisdom is oh intelligence is just simple straightforward log of compute so throw more comput who the heck knows man one of the researchers comes from here comes out of here and says you know what I got it >> it requires less comput any of you guys have thought about like that's a game changer. So in pre and oh by the way
33:59
So in pre and oh by the way we're all like excited about reinforcement learning. Guess what?
34:04
Pre-training is a more efficient form of training because you can advertise it.
34:08
So there's a I think pre-training will have new breakthroughs.
34:10
Mid-training will have new breakthroughs.
34:12
RL will continue to improve.
34:14
We will then have to add more innovation to it.
34:16
And by the way this is on another part of this partnership which is I'm glad you know OpenAI is continuing to do great work.
34:25
Yakob, Mark and others are great and we'll partner with them and we'll continue to do so and Mustafa has built a worldass team right you know Karen Amar Nando these are I mean we have now three cool models whether it's speech or image or text and we're going to continually innovate so we'll write our worst as well
34:43
>> yeah how are you thinking about the interplay between open AI and what you do internally at Microsoft >> so our simple >> are there certain things you can take your foot off the gas because you're like actually open's got that handled or do you want a duopoly like actually we're going to fight it out on everything. >> I I'm much more like again my mind
34:57
>> I I'm much more like again my mind mindset is all platform man like hey on Azure do you run Windows? Yeah. Do you run Linux? Yes. Do you run SQL server? Yeah. Do you love Postgress? Absolutely. Net Java.
35:08
Hey I'm happy with OpenAI.
35:08
I would love to have anthropic mai rock.
35:14
Anyone if Google wants to put Gemini on Azure please do so.
35:18
>> What is that like culturally?
35:18
Like what does it what does it mean for the next Sach Nadella?
35:20
somebody who's working their way up in Microsoft, do they need to be, okay, I'm I'm building something internally, but my company isn't going to favor me.
35:31
I I need to fight it out with all my competitors across.
35:34
>> We all grew up in that culture where it doesn't mean because it's always we're going to bring our pieces together.
35:40
We are going to innovate across these seeds.
35:45
>> But as a platform company, you kind of want to support everything, right?
35:47
I like most people don't office was born on the Mac before Windows was even. >> Yeah.
35:54
If you don't if you don't give people choice developers here like will churn, right?
35:57
They'll find other platforms, right?
36:00
>> The concepts Bill had when he started Microsoft was, hey, we're a software factory.
36:03
We love all software categories and we're just going to go create software.
36:08
And so to me, we definitely want to sort of have that same attitude uh to innovation.
36:13
uh to innovation. We definitely need to stitch our stuff together so that they come together to solve bigger and greater problems but doesn't mean we can't create opportunities and in part the other thing that I grew grew up like for example you know building SQL server with SAP um and so we've always
36:30
partnered and or Intel Microsoft right I mean we wouldn't have the PC industry but you know it was called the grave go drove great model >> uh that's a good model to create value >> do you think that there's increasing returns this is going sound like a loaded question, but I I promise you it's not. Uh that there's increasing
36:45
Uh that there's increasing returns right now to being a deals guy or or innovating on the on the deal structuring side.
36:51
And what I mean by that is uh there's there's all these difficult problems to solve with energy and data centers.
36:57
And it feels like we there's innovation in tech that we normally think of as like the code or the algorithm or the design of the system, but then there's also this difficulty sometimes to just marshall the resources.
37:10
And is that a is that like a new phenomenon?
37:13
Has that always been true?
37:15
Is there if somebody's pursuing a career in tech is becoming a great deals guy or dealmaker like a important path now?
37:23
>> Yeah, I was just thinking about it man like which is yeah you have this great investment and it has great return and no carry all it all the value goes [laughter] to my shareholders. That's awesome.
37:31
But you maybe we should start a venture firm.
37:34
[laughter] >> Uh but >> you might you might you might do well there. >> Yeah.
37:39
I I think the the thing you're touching on is something that actually platform companies should think about which is >> what's the ecosystem upstream and downstream >> right to your point right now we have to as an industry like I mean the reality is let's take power right which is if they sort of say >> intelligence is about tokens per dollar per watt >> uh we got to get efficient on all of it. >> Yeah.
38:02
in order to get more efficient on it, you got to really think about even in our own industry, the token factory itself really getting better order of magnitude.
38:11
This is like again a renaissance time for systems architecture and so we you know obviously Nvidia is doing great work, AMD is doing stuff, Broadcom's doing stuff, all of us are doing great work to just push that.
38:24
>> Then the next barrier is going to be man can we generate energy faster? Can we build faster? Can we build a cooling?
38:30
I mean like I mean I now know more about campus schooling systems than I ever thought I'll know, right?
38:37
I mean and these are all choke points.
38:39
these are all choke points. How much do you want that to live within Microsoft versus um you want to just be a buyer and and the the all the different power players are out there building nuclear, wind, solar, and you're just dealing with it at a higher level of our >> sure the vast majority of this
38:54
infrastructure now that you know if you think about back at it right u our data center builds mostly we built and we lease some because no one was in the business of building at the scale at which we were building but now I think there's going to be opportunities form to lease and there's going to be significant competition amongst builders. So therefore, the lease prices
39:14
So therefore, the lease prices also >> Yeah.
39:16
Do you think you're more ROI focus than others that are throwing around big numbers?
39:21
>> I mean, I'm I'm always focused on long-term return.
39:25
>> Well, well, and and we're at a time right now where there's people that have come out and effectively said, I I don't actually care about ROI.
39:30
I just care about winning. Right.
39:32
about winning. Right. And it seems from your view >> a couple years ago there was there was the mood of like if you this might be the last invention >> if you always have someone else willing to give you the billion dollars when um or the $10 billion you can always be
39:45
about I'm out winning and not the return but at some point that party ends and everybody needs to sort of have a plan but in that context in these platform shifts to be short-term oriented uh doesn't uh help at all right because you kind of have you know I always say long before it's conventional wisdom. I mean,
40:02
I mean, if you look back, you asked how we put the billion and the reality is we put the 10 billion. >> That's right.
40:10
>> Uh or the 13 was fully committed before it became a thing, right?
40:14
And remember that was all done before chat GPT became a thing.
40:18
And so to me, >> how do you I I think there's a general consensus now that it's it feels very possible to predict like a year out, two years out and then 10 years out is extremely fuzzy.
40:29
What's your view on that given that you look like going back to the original OpenAI investment and the original partnership, it seems like you've had at least really good like six-year kind of like foresight ability to sort of uh invest against like a six-year time horizon, but uh how are you thinking about managing over you know the next decade? Yeah.
40:51
I mean, I think, you know, to me, you know, one of the things about tech is as a percentage of GDP, they're like around four or five percent. >> Yeah.
41:02
>> And if you ask me 5 years from now, 10 years from now, is that percentage going to be higher or lower?
41:07
I think the answer is pretty straightforward. It's going to be higher. It's just a question.
41:11
Is it going to be 10 or 15? So why is that?
41:14
Because the rest of the pie, the rest of the GDP would have grown faster.
41:19
So that's why I always go back to it.
41:21
At the end of the day, the only rate limiter here is the overall economic growth uh and the factors of sort of input to it.
41:29
So tech as an input, I think AI and everything that it entails is going to be a core driver and some of it will come from just this intelligence and its sort of u continual march of capability, but it'll also come from I'll call it great engineering and product making around it.
41:48
Like when I look at GitHub copilot today with agent HQ and what have you that's great pro because right now I'm inundated with multiple models >> and everything is slightly different except I have one repo and I want all of these agents to come work on all of my repo in different branches.
42:07
So you need great product making to bring more coherence to the chaos and that I think is going to make the big difference maker.
42:15
I was talking to Eric Lyman at RAMP who makes the show possible of course.
42:19
Uh and he had a question about how you what like what advice you would give to someone running a decacorn thousand plus employees in this age of spiky intelligence where there is the possibility that tools are going to get better very rapidly and maybe you don't want to scale up too fast and then have to do layoffs or retraining like you run a huge organization.
42:40
How do you think about managing human capital in what feels like an uncertain time?
42:45
Does it feel more uncertain to you now than it did 10 years ago? >> It's a great point.
42:51
I mean, Eric's a great founder.
42:52
I know him well and they're doing some unbelievable work.
42:56
And so, in fact, whenever I've talked to him, in fact, I learned from him even how he's rapidly changing >> uh the agents they've built.
43:01
Uh so, to some degree, I think >> uh whether it's at Microsoft or whe it's ramp, I think the key is learning the new production function.
43:09
So I when I look back at Microsoft I feel like hey look you know platform ships we've navigated.
43:16
I joined Microsoft when my our existential competitor was Noel.
43:20
>> Uh right and so you know in '92 and here we are.
43:23
And so but the bottom line is we've over the years navigated many platform shifts. >> Yeah.
43:28
We've also navigated tough business model shifts, right?
43:30
When you suddenly have uh you know you have a 98 99% gross margin server business and you move to the cloud and you don't even know man is there a margin here and yet you have to make the shift and figure it out.
43:43
out. Uh this one is interestingly enough both a tech shift >> a business model shift because this is the first time you have marginal cost of software just not like cogs of the SAS world but true marginal cost >> and three the way you produce your
43:59
artifact your software is changing >> so the product development process is completely getting ripped and replaced and that is a m whether it's for RAM >> in fact well even the competitive dynamic too because you people that can say hey we can build this product in two
44:15
months previously would have taken us 12 months why don't we enter that >> exactly category >> and it's kind of like rewiring yourself right unlearning is the hardest part learning is easy >> sometimes if you have to unlearn and learn it's much harder and so to me uh
44:31
that I think is what all of us have to I mean it's funny I met a bunch of student developers right here >> it is the first cohort of developers who grew up with GitHub copilot as standard issue when they joined It's completely different environment. >> So they're saying no there was a world
44:44
>> So they're saying no there was a world before GitHub.
44:48
They're like I don't want to live operating in a completely different abstraction.
44:51
Uh on the on the topic of changing business model shifting your business model.
44:55
Uh >> it seems like the console wars are over.
45:00
Take me through the journey.
45:01
>> You're peace time CEO now.
45:02
>> You're peace time CEO. The war is over.
45:04
But but but take me through the evolution of the of the business model shift on the gaming side of the business.
45:10
It's a one of the most interesting pieces of of Microsoft. >> Yeah.
45:13
I mean, I think you got to remember, in fact, Flight Simulator, I think, was the first product Microsoft built even before I think our, you know, our dev tools was first, >> Flight Simulator was second.
45:24
>> That says so much about the culture.
45:24
It is like as soon as you gave the developers the ability to write code, they were like, let's make a game. >> It's amazing.
45:31
>> And so to me, remember the biggest gaming business is the Windows business.
45:36
to us gaming on Windows and of course Steam has built a massive marketplace on top of it and done a very successful job of it.
45:44
So to us the way we are thinking about gaming is let's first of all now we're the largest publisher >> uh after the Activision.
45:51
So so therefore we want to be a fantastic publisher similar approach to what we did uh with office.
45:58
Uh we we're going to be everywhere in every platform.
45:59
So we want to make sure whether it's consoles, whether it's the PC, whether it's mobile, whether it's cloud gaming, we want or the TV.
46:07
So we just want to make sure the games are being enjoyed by gamers everywhere. >> Yeah.
46:12
>> Second, we also want to do innovative work uh in the system side on the console and on the PC and bring, you know, it's kind of funny that you know people think about the console PC as two different things.
46:24
We built the console because we wanted to build a better PC uh which could then perform for gaming.
46:31
And so I kind of want to revisit some of that conventional wisdom.
46:33
Not but at the end of the day console has an experience that is unparalleled.
46:40
>> Uh it delivers performance that's unparalleled that pushes I think the system forward.
46:43
So I'm very looking forward to the next console, the next PC gaming, but most importantly the game business model >> has to be where we have to invent maybe some new interactive media as well because after all the gaming's competition is not other gaming.
46:58
Gaming's competition is short form video. >> Yeah.
47:01
>> Uh and so if we as an industry don't continue to innovate both how we produce, what we produce, how we think about distribution, the economic model, right?
47:12
Best way to innovate is to have good margins >> because that's the way you can fund.
47:18
>> So, so interesting saying uh gaming's competition is short form video.
47:20
Feels like the entire world's competition is short form video. >> Yeah.
47:25
Uh we I mean we've heard this thing a while ago.
47:27
It just comes up again and again with uh public SAS companies that are maybe a little bit more of a point solution and they have to go through a business model transition and that can be harder than a tech transition.
47:39
And we hear about, oh well, if you want to change your business model, maybe you want to be private.
47:43
But it feels like, is there some sort of advantage of being a hyperscaler a$ four trillion dollar company that you can go and retool a piece of the business over here, change the business model and and have almost the the privilege of, you know, not having shareholders come to you and beat you down about a slight shift in a business model in a subdivision?
48:02
That's that you don't get that.
48:04
can't deny that you know diversity of business models, diversity of the portfolio that Microsoft has has been helpful.
48:11
I mean it's kind of >> uh but that said I don't think you can take that and say somehow you can make it >> if you don't reinvent yourself.
48:20
>> if you don't reinvent yourself. So I think what happens in tech unfortunately >> is that >> when these shifts happen >> whether you like it or not you have to first be relevant >> after having it doesn't matter what the business model is the business model may
48:34
be like hey I had whatever 90% margins you are going to at best have 10% >> but you have to jump all in because even that 90 is going to zero >> and so given the binary nature you got to make it to the other side but then the category economics matters Right? Because if you can't sustain long-term
48:51
Because if you can't sustain long-term innovation if there is no category economics, I mean hypers scale is a great one.
48:57
In fact, the best day in hypers scale business was the day Amazon announced their operating margins IPO. >> Yeah.
49:04
It you know because that's when everybody knew hypers scale business is an unbelievable business.
49:08
It's a commodity but at scale nothing is a commodity.
49:12
Uh and so to me that is kind of going to be the key here. even SAS applications.
49:17
>> Quickly unpack why that was so good for you again just because the market recognized that you were in the same business and it was fantastic.
49:21
business and it was fantastic. That is one and more importantly it was much more expensive right I mean think about our server business right it's super profitable >> except it was onetenth >> the size when I look at it compared to Azure >> so we like we sold a few servers but man
49:37
we sell a lot of cloud VMs >> uh or containers who would have thought how expansive >> the cloud consumption model is going to be in terms of people being able to sort of it's kind of the Germans paradox that sort of really played out in a massive way if you thought broadly. >> Yeah. And so I think on the business >> Yeah.
49:55
And so I think on the business >> well timed Jevans paradox post by the way back during the the deepseek moment.
50:01
That was an important post.
50:02
>> Spot on with that process.
50:04
>> Uh I wish we could keep going.
50:04
I think we have to wrap up.
50:06
But we love you this gong and give it a quick hit for uh 27%. >> Wow. >> That's a good hit.
50:17
>> That's a very strong as big as possible. Oh, there we go.
50:20
That is a fantastic signature. >> Thank you so much. Thank you for coming on. Thank you for having us.
50:25
This is a really great uh I've always uh whenever these big news these big tech news things happen, I always uh wish I could talk to the person who's making the news and now I get to and so uh what a wonderful conversation. What a moment. Uh and what a CEO. >> Yeah.
50:41
What what a Yeah, what a moment in the in the in the tech world. Uh well, thank you.
50:46
If you're new here and you tuned in just because of Sachin Nadella, the CEO of Microsoft live on TBPN, uh please uh follow us, leave us a comment, uh add us to your RSS feed.
50:56
We have a 15minute version of the show uh called Diet TBNN where you can hear mostly just all the flavors 15 calories.
51:05
Uh, you'll also hear ads from our sponsors like Bantum, automate compliance, manage risk, proof trust continuously.
51:12
Bant's trust management platform takes the manual work out of your security and compliance process and replaces it with continuous automation.
51:18
Whether you're pursuing your first framework or managing a complex program >> uh you could have kept going forever. >> Oh, absolutely.
51:24
We both could have we should do we should do a giga stream with ZTSM time just 12 hours.
51:30
>> It'll be super easy to get that on.
51:32
Sure, there's a 12-hour block somewhere out in like the 2030s that we could lock in.
51:37
>> Um, >> but I mean there is really so much I mean that's the uh that's the problem with these uh these conglomerate CEOs.
51:44
They just got too many business lines.
51:46
You know, you could do it you could do a whole hour just on Xbox and Activision. >> Yeah.
51:50
The the thing that stands out to me the thing that stands out to me is when you see some of these other hyperscalers or players Yeah.
51:55
uh their reaction time just Satia makes them look incredibly slow, right?
52:01
Like he's been making these like sizable bets like he's been seeing the future >> and yet only this year you've had other players who I won't uh I won't directly name deciding like okay I want to get in the game now.
52:15
It's like what what were you doing >> when uh when Satia was in the kitchen cooking?
52:20
[laughter] Um, >> this is really fun.
52:26
>> Anyways, what a what a wild day.
52:28
>> Uh, what else is on the timeline?
52:28
What other news should we bring the folks uh while we are uh here?
52:34
[laughter] I'm seeing mostly people talking about 996 Porsches versus 996. Uh, working hard.
52:42
Uh, what else is in the what else is driving the news cycle today?
52:47
Um, of course, uh, uh, OpenAI just did a a live stream, uh, with Sam Alman and the head of research over there talking about their side of the deal.
52:58
Uh, all parties kind of align to, hey, we got a clean cap table. Let's move forward. >> PBC. >> Yeah.
53:07
Uh, which is what Anthropic is as well, right? Yep.
53:09
Um, and >> so Microsoft now owns 27% or $135 billion stake in OpenAI.
53:15
Uh, and OpenAI is contracted to buyund uh 250 billion of Azure services. That's a lot of Azure.
53:23
Should make it easy to underwrite uh future capex on the Azure side. Earnings is tomorrow.
53:30
Uh we'll be back in Los Angeles at our at the TBPN Ultradome and uh we probably won't be live uh when earnings drops post uh close but uh we will be bringing you the news on Thursday of course >> Meta also reporting earnings tomorrow which will be notable. >> Yes.
53:49
Uh we also have Alex the product lead on codeex uh but we're we're wiring him up so that we >> in a second I can go here through a post from semi analysis.
53:58
They have a a green text here.
54:01
They say, "Be me, Qualcomm.
54:03
Time to enter Nvidia AI chip market. Nvidia is making money. How hard can it be?
54:07
Spent years developing AI 200 chip.
54:10
Finally ready for big announcement. Make fancy slide deck.
54:12
Put 768 GB of memory on there. Sounds big.
54:15
100 uh 160 kilowatt power consumption. Sounds powerful.
54:20
Add liquid cool sounds cool. Uh oit JPEG. What about flops? Hm.
54:26
Decide just to not mention it.
54:26
Also, don't mention price or how many chips per rack or actual benchmark numbers. Just vibes. Launch presentation. Qualcomm AI200.
54:34
It exists and uses electricity. Refuse to elaborate. Stock goes up 15%.
54:39
Uh that feeling when investors don't know what flops are either.
54:43
Uh my uh face went greater than 10x with no baseline. Ships in 2026. AI250 ships in 2027.
54:48
Still won't tell you the specs by then probably. Uh low TCO. Trust me, bro. Confidential computing.
54:56
The performance is confidential. [laughter] >> Unreal.
55:01
>> Uh I have more there, but uh let me first tell you about Julius.
55:04
What analysis do you want to run?
55:05
Chat with your data and get expert level insights in seconds.
55:08
Um no, you know what's funny is that in the 2019 blog post from OpenAI announcing the deal with Microsoft.
55:16
Uh they say Microsoft is investing 1 billion in OpenAI to help us support building AGI.
55:21
Uh but specifically we're partnering with with with Microsoft to develop a hardware and software platform within Microsoft Azure.
55:30
And so it feels like like I I imagine what they mean by hardware platform within Azure is just like a bunch of NVIDIA GPUs at that moment in time. Yeah.
55:40
>> But it does it does lead to these like the the natural questions of like how deep do you go in the stack?
55:45
And if you're if you're SAM and you're open AI and you're seeing that you're ultimately limited on on cloud capacity and then chips and then at first dollars because there was plenty of Azure capacity.
55:59
It's not like in 2019 they used all of it.
56:01
Uh but they needed the money then they needed the the data centers then they needed the chips and now they needed electricity and they're just going deeper and deeper in the stack. >> Yeah.
56:11
The thing that's notable is uh just how married uh OpenAI and Microsoft are.
56:16
When you look at uh OpenAI's relationships with uh Nvidia, AMD, Broadcom and these other players, everybody in the chip space is sort of like, you know, uh you know, in these sort of like complex dynamics, right?
56:28
uh we we got some backstory on uh the dynamic between uh just like the whole series of events between OpenAI and Nvidia and AMD and how that all came together and it seems like everybody in the chip side is sort of uh sort of I wouldn't say desperately but desperately uh sort of like competing for OpenAI's attention and resources.
56:52
Meanwhile, Satcha is able to uh just kind of sit back and ride this partnership out.
56:57
So >> I wonder where >> Qualcomm's retraced by the way.
57:01
It's now only up eight and a half% over the past 5 days.
57:05
So dropped a little bit after the >> is that on public.
57:08
com investing for those that take it seriously.
57:10
They got multiasset investing in trusted by >> we got to throw in a post here from uh spooks early uh early friend of the show.
57:20
Uh he's quoting a post from Samuel Hammond who said, "Melatonin in the US is sold in 5 milligram doses with the effective dosage range is. 3 to 3 milligrams.
57:29
Americans essentially overdose on melatonin by default for no good reason."
57:33
Spook says, "Okay, well, if there's an easier way to soul speak with my ancestors in the dream realm, please let me know."
57:40
>> I don't know if this is going to read at all on this particular stream.
57:41
We don't even have you tweets.
57:44
>> Banger Graedia is now live.
57:44
Do you think do you think OpenAI will launch a a Graedia competitor?
57:49
Is I I clicked on I clicked on a Graopedia? Yeah. Yes. [laughter] Immediately.
57:54
But I clicked on a Graedia like entry and I was like, "Oh, wow.
57:58
It's like a pre-baked deep research report on something that I already would have wanted to search for.
58:04
It's actually it's a great product.
58:06
Yeah, I mean deep research has uh if if deep research has disrupted >> uh in the same way that chat GBT has disrupted search, >> deep research has disrupted Wikipedia.
58:18
>> And there's so many uh so many prompts that I run that I'm like I shouldn't be like burning up the GPUs for this.
58:22
This should be stored in a database somewhere.
58:26
I should be able to access it.
58:28
>> This is the funniest thing is that like over time Open AI [laughter] becomes >> uh we have a surprise guest from Codex.
58:34
Alex, welcome to the stream.
58:36
>> Let me tell you about fall while he hops on.
58:38
Generate media platform for developers.
58:40
>> Nice to meet >> Nice to meet you.
58:41
>> Hey, welcome back to the show. Thank you. >> Uh, congratulations. Incredible event.
58:47
Uh, how many have you been to?
58:47
Give us give me give me a little read on the ground of like what's the scale?
58:50
Is this the biggest ever?
58:52
Tell me a little bit about what's going on today.
58:54
>> So, we just had um OpenAI Dev Day few weeks ago. That was awesome. It was massive.
58:59
Actually, I got COVID like the day before, so I was not there. So, yeah.
59:03
This is actually the biggest event like this I've been to this year.
59:06
>> So it's tough with the co now.
59:06
You don't get the same level of uh you don't get the same level of like oh I'm it's like okay so you're saying >> yeah I didn't think it was a thing anymore but anyways here today we announced a couple things.
59:15
We announced uh that >> codeex is coming natively to GitHub later this year >> and then we announced that today actually uh we're bringing uh codeex to >> co-pilot um VS Code sorry I'm going to mumble this co-pilot pro plus subscribers and VS Code can use codeex as part of this.
59:29
[laughter] That's awesome. Amazing.
59:34
>> Walk me through like the different flows and like the different actual like user journeys because uh there's something very interesting about GitHub has the ability to even host pages. Yeah.
59:44
And then codeex allows me to from my phone potentially like write a web app that then can be deployed.
59:52
Is that helpful to actually like instantiate a web app on the fly that actually lives on the internet that I can send to a friend?
1:00:01
Is this like is there is there the beginnings?
1:00:05
Are you are you starting to see like what the next era of vibe coding might look like? >> So totally.
1:00:10
I mean so mostly Codex is used by professional software engineers although we have a good amount of people who don't aren't as familiar with coding using as well.
1:00:16
But like I think the best analogy >> is to think of codeex kind of like a human teammate, right?
1:00:22
if we're working together, I could talk to you um in Slack.
1:00:26
I could talk to you in GitHub. I could text you.
1:00:29
>> I could come by your desk and we could like jam on something.
1:00:30
And so it's kind of the same you, but you're present in all those tools.
1:00:35
So that's what we're trying to build Codex into.
1:00:36
It's just like an AI software engineering teammate that works with you wherever you like building. >> Sure. Sure. Sure.
1:00:42
>> So yeah, use it from your phone and like make some there. Maybe that creates a PR.
1:00:47
you push it into GitHub, >> you know, maybe you you use Codex to review your PR in GitHub and then you land the PR like all those things, no matter where it is, it's just the same codeex agent. >> Yeah. >> Yeah.
1:00:56
Um, >> what is the I is there some sort of like business model flow through then?
1:01:03
Like you have to be subscribed to OpenAI, but then you also subscribe over to GitHub and that's just kind of like that's just like the default stack for a lot of people.
1:01:10
>> So, so this is like actually a kind of an interesting part of the deal.
1:01:12
an interesting part of the deal. Um so to use Codex today the main way that most of our users use us is they have a chachi chachbt account >> of course >> uh you know they're on pro plus business or enterprise um and then they can use codeex now as part of this deal what we figured out with GitHub is how to partner so that if you have a co-pilot
1:01:28
pro plus account >> um and you don't need to have a chat account you get the full power of codeex anyways and when I say the full power I mean you get to use our model and you get to use our our model harness which is kind of like the code that provides the prompts tools the run loop >> um and so you Our goal in this is just to like make codeex as ubiquitously available as possible. >> Um, and so yeah, you don't need there's
1:01:47
>> Um, and so yeah, you don't need there's no like flow through there.
1:01:49
It's just like, you know, you just need your co-pilot account.
1:01:54
>> There's somewhere somewhere out there there's somebody that just started CS in college and they're only going to live a life that where they're like just running codecs uh in GitHub just naturally.
1:02:05
And and >> well, I mean, look, so actually it's interesting, right?
1:02:08
Like if you think of GitHub, it GitHub does a lot of things. Yeah. Right?
1:02:12
But at least like where I personally spend the most time in GitHub is like actually collaborating with the other people contributing to the codebase, right?
1:02:19
>> And so like >> I actually think it's quite unlikely that you would only spend your time doing that type of activity.
1:02:23
You're also going to spend a ton of time in tools like VS Code or like the Codeex CLI or ID extension because that's where you're doing your work yourself, right?
1:02:30
Like >> again like I think the the human teammate analogy kind of works pretty well here.
1:02:34
It's like most of us 90% of the work we're doing is kind of at our desk.
1:02:38
We're not like having a meet well hopefully not having meetings like 90% of the time as a software engineer, right?
1:02:43
So probably that you know that uh person who's going to become an engineer but is currently in college.
1:02:47
They'll spend a lot of their time >> with superpowers but working at their computer doing stuff individually like commanding fleets of agents, right?
1:02:54
And then they'll spend some of their time collaborating with their team like obviously quite a lot of their time but not all of it.
1:03:00
I don't think the sort of individual productivity is going away. >> Yeah.
1:03:03
How how much uh one kind of follow-up question like how much uh how important is the metric of like how long codeex is spending working?
1:03:10
Is that something that you guys are like explicitly like tracking and trying to scale?
1:03:15
Because it just means that >> that goes from like 10 minutes to two hours.
1:03:20
We're tracking like the meter thing eventually 200 years.
1:03:22
I click it and I come back.
1:03:24
My ancestors come and they watch what's been >> I want to set it off and time travel. >> Yeah. Give me the 200year AGI.
1:03:30
So like this it's interesting.
1:03:33
I actually shared um at the keynote today that we last week an engineer on the Codex team ran Codex for over 60 hours on a single incredibly hard task. And that's that's crazy.
1:03:43
And we're like we're excited about the capability of that from the perspective of it means the model is actually able to do like very productive work for a long time and it means the model and the harness are working together to manage the context window because that consumes more than one context window length.
1:03:55
However, >> it's not like we have an eval that's like how long did the model work and let's maximize that like that's much more sort of a lagging indicator of like the intelligence capability of the model.
1:04:06
Like what we're really trying to drive is like how smart is the model. Yeah. Right.
1:04:09
And how how easy is it to work with?
1:04:11
And then it just turns out that as you make the model smarter and smarter, it can work it can take on like longer and longer tasks.
1:04:17
>> How how uh how what what do you think about the user experience?
1:04:19
If you're setting codecs off to go work for 60 hours, uh there's some risk that you know it's doing things maybe incorrectly and you come back after 60 hours and you're like I just I kind of just blew 60 hours that I could have been doing this myself.
1:04:36
>> Maybe in that 60 hours you watch all of Game of Thrones. >> That's possible.
1:04:38
Or you're watching Subway Surfers here.
1:04:39
We got a video [laughter] game video.
1:04:42
No, but my question is like the the workflow that uh that feels like the most natural giving using the teammate analogy is you just get a ping and it's like hey can you double check this before I continue?
1:04:53
Is that is that is that a workflow that you're thinking about like you just you're developer you get a push notification you're at the gym and you're like yeah looks good. >> Yeah.
1:05:01
So I think like there's two things you said that that make a lot of sense to me like one is just like steerability.
1:05:06
That's something we're working on.
1:05:08
We want us to be able to steer the model like short task, long task, whatever, right?
1:05:10
The other thing is kind of like proactivity, right?
1:05:12
Like again when when you hire someone onto your team, maybe at the beginning you're hanging out, you're collaborating directly, then you start delegating small tasks and at some point, you know, you will give them like a 60-hour task without like specifically prompting every single detail, right?
1:05:25
And what you expect of like a good employee is that they know when to ask you questions, right?
1:05:30
And so it is how frequently I'm able to prompt an LLM to the bottleneck of my productivity kind of like how I can structure the work >> so that like independent agents or humans or whatever can like go do the work and then ask me questions when they have >> so interesting because in uh we're so used to now like trusting uh a CRUD app, right?
1:05:55
like you can trust that you can put data in and and you're going to come back and it's going to be there, right?
1:06:01
faith and it feels like with agents as a product agent if you're building agents you need to be focusing on like how do I develop trust with the user and it's come you know again maybe it's like focusing on short-term tasks initially and having that seability >> so so this so if you think about it like
1:06:18
right now the place where most people are using coding agents is to write code right like codegen >> and again I keep coming back to the human analogy but like imagine you had a human teammate and the only thing they can do is write code they can't read user feedback. They're not in Slack. If They're not in Slack.
1:06:30
If there's an outage, they're not going to see it. >> Sure. >> Right. Sorry.
1:06:34
So, like are you going to trust that human teammate even if this is the smartest human teammate in the world? Like, no. Right.
1:06:40
So, like what we need to do to like build this trust is we need to like extend what agents can like look at across the software development life cycle, right?
1:06:46
So, they're like present in more of the team conversations and ideation and prioritization and planning.
1:06:50
They're also able like more and more capable at the code review stage.
1:06:54
And actually, that's one of the the recent product releases we shipped is like Codex code review and people loving that.
1:06:57
but more present at code review, more present at like the deployment stage and like code maintenance stage, aware of like what's going on in like your telemetry tools and like I think that's actually how you get to the trust.
1:07:08
So some of this is like increasing model capability, but a lot of this is actually changing the form factor of like how these models are harnessed so that they can like interact with more of what you need.
1:07:18
>> So it feels like a couple years ago we were just trying to predict the next token and it was like oh wow I can do poetry.
1:07:24
Oh wow, I can write code like this is amazing and very like undirected fundamental research.
1:07:28
Now we're in the age of spiky intelligence.
1:07:29
age of spiky intelligence. Is there a feedback loop where you're actually trying to take feedback from okay Carpathy says that the agents can't write you know nano GPT so let's go work on that or oh we've seen in the data that it's easy to write a you know website in Django and Python but if
1:07:48
you're trying to use forran to refactor some obscure to high frequency trading system or something like we're falling down on that let's actually put a team on this like how does act how does feedback work now and like what are humans on the team doing or is it all just zoom out and hope that the emergent
1:08:06
properties like solve like is it dx machina >> you know this is open AI so the way that we build is like constantly evolving like the codeex team was just like five engineers like a few months ago and now we're like I actually don't know but I think we're like 25 plus >> so um it is constantly evolving but what
1:08:21
I can say is that our team is like possibly slightly unhealthily on social media just like reading all the feedback so we love we love it when people send feedback to us Um, we're also starting to like try to get like a better understanding of like okay, like how do different like model snapshots and stuff compare? Sure. Sure.
1:08:37
>> Um, and so yeah, we're we're starting to build up a more and more systematic way of doing it, but I still think it's like quite early days and there's still a lot of taste involved.
1:08:44
>> Yeah, it does feel like we're entering the era where if you see in the data that, you know, maybe there's developers here that are like, I want to use codeex for this specific thing, you're you're you're falling down on this.
1:08:54
You're great at everything else, but you're not with this.
1:08:57
there is the world where you can actually go in and and >> how do you how do you make sure you don't get the wrong signal from social media because like there's a certain type of person who posts about post product feedback publicly and then there's for every one person that does that there there could be you know a number of people that just churn uh and and never say anything.
1:09:18
There could be a number of people that uh are just like super hungry power users and yet you know it's a real thing like getting a push notification and somebody's saying like you know they're DMing you something that somebody said about codecs.
1:09:30
It's like you need to make sure that it doesn't you know consume like 100% of your like worldview on how the product is actually resonating with users. >> Totally.
1:09:39
So um yeah let me add to that and actually I want to go quickly back to the for topic as well but on that note I think like yeah a lot of the feedback you're going to get on social media is like your power users. >> Yep. Right.
1:09:48
And so power users I think are really good like I kind of put that in the like how do we advance the capabilities and like we are trying to advance capabilities.
1:09:54
So this is good feedback like what are they doing like what should the product do to make easier for them. Yeah.
1:09:58
>> And then at the same time I think I like to balance that kind of with like just what is the first mile of the product like literally like the first 20 keystrokes like what are those >> right?
1:10:06
And I think >> for me I just like kind of focus on mostly these two extremes and so we're constantly looking like okay what is the new experience to get into the product and frankly I think there's a ways to go.
1:10:14
It's still a very powered user product and yeah, a lot to improve there.
1:10:17
On the sort of like the for trying question like >> one of the interesting things about building codecs in open source which we're doing is that we're seeing like larger enterprises with very bespoke needs who are excited about the capability.
1:10:28
You know maybe an engineer was using codeex on the side wants to bring it to work notices like oh in this codebase it's not doing as well as it's doing in like the code bases that openai like has more you know see more of.
1:10:36
So like what we're actually seeing is like certain customers are starting to like fork the CLI >> or work with us to deploy it in a very specific way where you can inject like you know more instructions for the like company specific language like into the context.
1:10:51
>> So if I'm a business if I'm a big enterprise and I have a million lines of forran for whatever reason uh and I you know authenticate with codecs you're not training on my on my code which is good for privacy but maybe bad for performance.
1:11:06
So what you're saying is that there's a world where we could work together to figure out how to actually fine-tune the model or train the model or work together to to have the actual product work better on my codebase. >> Yeah.
1:11:20
And I think like fine-tuning and training are like definitely levers that exist, but I think even before that there's like a ton of work you can do >> in the harness. >> Yeah. In the harness.
1:11:26
But even in like in terms of like you know agents.
1:11:28
mmd and just like how you tell the model what it needs to know if it doesn't >> every layer of abstraction there's there's opportunity to squeeze out extra performance before we go back to like hey let's pre-train on your data which is >> exactly I think there's like >> important >> yeah I think there's a giant capability overhang from models today.
1:11:44
Um, and so yeah, with it's it's kind of it's exciting.
1:11:48
We're seeing like a lot of pull from enterprise now. Yeah.
1:11:49
And it's exciting because we get to kind of like go deep, right, and invest a lot of time on our side to figure out how to make it work even with like the current model and the current >> uh the pull from enterprise.
1:11:57
Do enterprises care about benchmarks or do they feel like they've been hacked?
1:12:02
Going back to the social media thing, I think people were really into benchmarks and then pretty quickly everyone kind of assumed that they were saturated, they were gameable.
1:12:10
But what are you hearing on the enterprise side?
1:12:12
I think yeah, I don't I don't hear a ton about benchmarks to be completely honest.
1:12:16
I mean, maybe folks read it, but I think it very quickly comes down to >> think of the SAS.
1:12:21
It's like our our uh our our CRM is split second actually faster [laughter] than the competitor like you should you should use us instead actually. >> Yeah.
1:12:33
>> So, mostly I think what it comes down to uh at least on a lot of things we're seeing it well it's actually there's kind of two motions. Sure.
1:12:38
>> One is like it's just like they give the tooling to developers and it's just like do you like it? >> Yeah. >> Right.
1:12:42
what doc migration from one cloud provider to another or something like that and that's where we're actually like working more closely with enterprises to figure out like okay let's let's actually set up like a meta harness almost for like codecs to do this work this started with
1:13:03
like some customers like Instacart runs codecs like in one of these I don't know they don't probably don't call it a meta harness but like in basically a system that runs codecs automatic atically to do uh stuff you know that they want to do for code maintenance. >> Um and then now we've been like okay
1:13:15
>> Um and then now we've been like okay this is actually a pretty good idea like we can go help customers who have these like larger things that they want to do to set up this kind of like workflow automation.
1:13:22
Okay >> so there's kind of the two sides. >> Last question.
1:13:25
Are you feeling GPU rich or GPU poor right now?
1:13:28
>> Any any requests for Sam and [laughter] Sarah?
1:13:32
>> We we so the Codex team is getting a ton of support.
1:13:33
So >> So you're feeling GPU?
1:13:34
But I think I think the codeex team feels supported, but I think OpenAI like we could definitely like things are growing and more GPUs would be more good. So yeah, definitely far. >> Okay.
1:13:44
So So internally GPUrich. >> Yeah.
1:13:46
Every time I wouldn't say that.
1:13:48
That's probably overdoing.
1:13:48
No, every every time I'm in chat GBT now and I I prompt it on something that it should really think, you know, you should think a little bit and it's like, you know, I had a request yesterday >> for like give me a list of like 50 companies that meet these criteria and it was like I can't do that.
1:14:03
And I was like uh and I was like yes you can but then in that time it was like yeah the the GPUs were like you guys I'm sure and somewhere out there there's millions of codeex agents you know running and >> yeah this is the this is the >> uh the the the endless product feedback but thank you so much for coming to thanks for coming on guys soon.
1:14:25
>> Uh we have a few more people joining this show.
1:14:27
If you're tuning in for the first time, please subscribe, follow us on YouTube or add us to your Spotify.
1:14:34
And before we hop on with our next guest, we are on LinkedIn.
1:14:38
>> Don't forget about LinkedIn. Yes.
1:14:39
Honestly, honestly, I know a lot of you who are listening on another platform.
1:14:43
Do not follow us on LinkedIn. Head over there.
1:14:45
Uh we are actively hiring someone. We just hired someone.
1:14:48
We're we're working on ramping up our LinkedIn presence. Very excited for that.
1:14:51
We're also excited to tell you about Turbopuffer search every bite serverless vector and full text search built from first principles on object storage fast 10x cheaper scalable.
1:15:03
>> Next up we have Kyle COO of GitHub. Very excited. >> Very excited.
1:15:07
We are we are wiring him up.
1:15:11
>> He will be coming on in a second.
1:15:11
Next we'll have uh Jay EVP of Core AI.
1:15:16
>> I spoke to Jay a couple months ago at this point.
1:15:19
Uh I was in the back of a car.
1:15:21
It was it was kind of hard to get uh uh to get to to to bring like the level of enthusiasm that I had, but I'm very excited to talk to him because a lot of the questions that I got when I asked people, hey, we're we're going to Microsoft.
1:15:32
We're talking about their relationship with OpenAI was we want to hear what Microsoft's doing inside.
1:15:35
What is the core AI team?
1:15:37
Where where's where's that driving the business?
1:15:39
So, we're excited for that.
1:15:41
>> Uh and then next we'll have Jared Palmer, VP of product, core AI, and the SVP of GitHub.
1:15:46
and then we'll finish it off with uh Michael, founder and CEO of Workit as well.
1:15:55
>> Can head back to the timeline tomorrow for the show.
1:15:58
We'll have to dive more into Grapedia.
1:15:59
We got to got to start using it, checking it out.
1:16:03
>> Um >> while we have time, there's an app uh launched by uh a product designer at Meta. Did you see this?
1:16:09
It's an AI app that if you can't afford a vacation, the verge is saying an AI app will sell you pictures of one.
1:16:16
So you upload images of yourself and then it put it creates a bunch of vacation photos for you.
1:16:21
So uh I guess short the tourism industry of people.
1:16:27
>> This was announced by a founder who just said like I wanted to feel the feeling of like the warm and fuzzy vacation photos and so I used AI to generate those.
1:16:34
Uh it was uh it it was pretty wellreceived originally, but I think that they're spinning it and so the narrative might be getting away from them.
1:16:42
But we have Kyle from GitHub coming into the studio or the >> Hey, great to be here. Great to be here. >> Thank you so much. Yeah, massive day. Beautiful event. Good weather, too.
1:16:55
>> I know it's a little dangerous here uh when the weather's not great, but when it is uh [laughter] give me a little background on you.
1:17:01
How did you wind up here?
1:17:03
How'd you wind up at Microsoft? >> Yeah.
1:17:05
So, I joined GitHub 12 years ago as a dev before we had managers. >> Yeah.
1:17:12
>> Uh we call it open allocation now, but it was anarchy too. >> Freet Freeman. Yeah.
1:17:15
Nat joined in 2018 as part of the acquisition. >> There you go. Okay.
1:17:20
>> So, yeah, we were 140 employees when I joined back then. >> Wow.
1:17:24
Uh and and do you even think about employee count in GitHub now because it's so merged into Microsoft, but it's still its own brand.
1:17:30
Yeah, I mean we have over uh 3,000 employees that work full-time on GitHub and then obviously we partner with Microsoft teams to do a lot of you know the AI model hosting training and so on and so forth.
1:17:44
>> On on sort of a meta question, I mean AI and you know AI can do so much.
1:17:47
How are you thinking about scaling that team over the next 10 years?
1:17:50
Is it harder to forecast like human capital allocation in the age of AI?
1:17:56
Yeah, I mean part of the problem is that there's places where AI is like incredibly helpful like in software I think for sure and there's a ton of places that AI hasn't hit. Yeah.
1:18:06
>> You know, I mean there's we talk about like it so many of the sort of business operations side that >> AI hasn't proven to be as valuable to us yet, but I think over time it'll get there.
1:18:17
Events like this take people. >> Yeah.
1:18:20
>> You know, full in full out.
1:18:20
And so there's just an imbalance a little bit of where software has been so great and then the rest of what makes GitHub GitHub these people still. >> Yeah.
1:18:30
Uh earlier with Satcha before we jumped on we were catching up with him and I said uh my words GitHub uh co-pilot is criminally underhyped and I think the reason for that is like you guys don't need to go out and raise a venture round every couple months and uh you know you're obviously well well capitalized but uh can you give us a sense of the scale and kind of the growth of of co-pilot over the last couple years?
1:18:56
Yeah, I mean, you know, GitHub is used by like 80% of everyone that joins GitHub right now.
1:19:03
It's 36 million, I believe, joined in the last year in the first week.
1:19:07
It's like one of the first things they do when they join.
1:19:09
So, we're definitely still hitting the world's developers with Copilot. >> Yeah.
1:19:15
>> All the time, like every day.
1:19:15
Now, these days, just like, I don't know, 10 years ago, devs are using whatever tool they want.
1:19:21
Things are changing so quick. You got to keep up.
1:19:22
You got to try all these tools.
1:19:24
But we keep seeing folks using, you know, C-pilot over here and then trying out a new tool or using C-pilot over here and finding this new flow.
1:19:32
That's a big part of this like reopening up like GitHub has done over and over.
1:19:37
Let's bring us all together so you can collaborate uh in that single place while you're going to pick whatever tool you're going to use. And that's cool. I can't tell you.
1:19:44
>> That's part of being that's part of being a platform.
1:19:45
I was saying this to SA when he was on the air.
1:19:46
It's like you can't uh if you want to be a platform is the more closed off you get, the more you're encouraging other people to go elsewhere because the tools are changing so quickly, you just you need to be able to give people that flexibility, right?
1:19:59
And we just had Alex on from Codeex and that's a good example of it. >> Yeah.
1:20:03
I mean, the this AI moment, it feels a little bit like before every app had an API back in the day because that like that wasn't the norm.
1:20:11
Now we're in a kind of quasi walled garden moment where everyone's making their thing really really great.
1:20:17
You know, their model, their app, their service, whatever.
1:20:20
But in order for all those tools and agents to actually be valuable, we have to interconnect them.
1:20:26
>> So my hope is that just in general, not just for software devs, we can go back to that platform first approach just like as an industry.
1:20:33
Uh because then each of our products will be more valuable for our customers because we're not going to have to deal with well how do I actually place the grocery order? Yeah.
1:20:43
>> Because there's not an API for that.
1:20:43
>> Because there's not an API for that. you know >> what what were the kind of key moments for you and understanding uh that AI would completely change software engineering because when you you know we were just going back through like the
1:20:57
history even of Microsoft's investments in uh OpenAI obviously because of the announcement today and it just feels like Satia had this incredible foresight you know in in 2019 and the early 2020s that only now a lot of other CEOs are kind of reacting to But I want to know for you, you've been here getting for 12 years. Like what were kind of the key
1:21:18
Like what were kind of the key moments that were eye opening to you where you thought I've seen the future, we need to just invest heavily heavily in in this? >> Yeah.
1:21:25
I mean the first moment was uh kind of a pure open source moment, right?
1:21:30
Like when it first started happening and we were talking about transformers and everything like you see the ground swell on GitHub from the open source side.
1:21:36
source side. So we started talking about that and then when we got access to the first you know GPT3 I think you know model to ultimately build copilot the thing that was so interesting was we were building it to write docs like that was what copilot was copilot was taking
1:21:51
oh yeah co-pilot was taking that model we were going okay what we're going to do like writing code >> and they hate documentation >> and so we're going to generate the docs and then what happened was >> wait code is >> exactly >> same same characters actually exactly so then we flipped it and then we got to this code. Exactly. And then I think, Exactly.
1:22:06
And then I think, you know, while it seems very simple now, like the idea of ghost text, like more than just like an autocomplete or whatever, >> the first time we used it, that was truly the moment where we've said, >> "Oh crap."
1:22:20
Because no one had to do something differently.
1:22:22
And I feel like that's the big problem with some of the AI tools.
1:22:26
You got to go interact with them in a way that's not normal.
1:22:28
You got to go, "Okay, I want to go write an email. Write an email for me."
1:22:31
That >> that's not how our brains work. we just start typing.
1:22:35
And when we were able to do that with the IDE, that very very quickly kind of shook all of us because it meant oh, it won't be the same anymore.
1:22:45
Now with agents and whatnot, like because we can verify the code, we have an advantage in software versus some other agents where it's harder to verify.
1:22:54
Uh but that all started that first time you like wrote something and then it just appeared >> and I didn't have to learn anything. It just happened.
1:23:00
And now we just, you know, we all take that for granted because it's de facto. >> Yeah.
1:23:05
Do >> you have a philosophy of uh of how where inference happens will change over the next few years?
1:23:12
Like in a lot of worlds there's it used to be there's a decision between like fire off an agent, wait 20 minutes, wait an hour or something or do it quicker, you know, a couple seconds.
1:23:24
But then um there is the world where a lot of the work that's going on in software development is so high value like why not just do all three inference it locally immediately and then also in the fast model and then also kick off an agent for every single task.
1:23:36
Is that where we go or or is there sort of like some sort of shift in in where inference happens over time?
1:23:44
Yeah, I think you know we clearly are going to have more and more inference has that that should happen locally like that seems pretty obvious at this point I think and then I think when we're talking about you know how much we're going to kick off into what cloud agents and models etc.
1:23:57
I think the thing that's really interesting is that we are pretty close to having that now.
1:24:03
Like we're, you know, we talked about it a lot today. Other folks have that.
1:24:08
The real problem is is like the age-old like garbage in garbage out problem, which is like we talk about abundance and you just >> fish off five tasks and we pick the winner.
1:24:18
Well, if your input was crappy, then probably all five of those are also kind of crappy.
1:24:22
They're just different >> variants of crap, I guess, you know.
1:24:24
And so I really think it's about when you're having a discussion with a colleague or you're on a zoom call or you're in an issue or in linear using a ticket.
1:24:32
That is the moment when we can actually get as much context as possible and ask questions then like why isn't co-pilot in that moment going >> I think I know what you're trying to build but like are you you sure about that?
1:24:45
Y >> why do I have to carry that even >> via a click and then go let's plan to build this again. Humans don't do that.
1:24:52
>> We just read it and we get started.
1:24:52
So I think it's it is a bit about where inference happens but I think it's how early in I have a problem that needs a solution.
1:25:01
I should be doing inference in the background immediately before I ever invoke something you know to go say now it's time to work.
1:25:07
Uh >> it started while we were in the shower thinking about the idea.
1:25:10
That's I think what we need to get the AI to do more of. >> Yeah.
1:25:14
Um I'm I'm I'm thinking about like >> AI at GitHub is the funniest thing because it's like seven different initiatives.
1:25:26
Walk me through your your thinking on uh I with a lot of companies I I see this idea of like AI above the fold or below the fold.
1:25:32
So above the fold is kind of like I put a search box and I let you interact with my app, my SAS, my CRUD app uh via natural language.
1:25:39
But then behind the fold or below the fold is like in the back in this behind the scenes I'm running inference over the data to improve the user experience but I'm instantiating it with HTML basically at the end.
1:25:51
Um but with GitHub you also have inference that you're selling directly.
1:25:55
You have a whole bunch of other stuff.
1:25:56
Are you seeing any uh exciting developments on like that behind the scenes like using AI to improve GitHub as a product that isn't actually bubbling up to the user experience directly in the form of just like uh you know a text box?
1:26:10
Have you have you seen developments there? >> Yeah.
1:26:14
So I mean a big part of what we've been figuring out is like we have so much information about your interactions. Yeah.
1:26:20
you know, pull request, the ones you got closed, like I told a story, the first pull request I ever shared didn't make it [laughter] >> and like, but now you know that about me.
1:26:29
And so, I think the thing that we're figuring out is like we have like new uh models that uh allow us to really deeply understand your code more beyond just like >> the tabbing and like asking a question >> because then if we have that then we have to understand what does Kyle how does Kyle work in a poll request?
1:26:44
What mistakes does he make every single time?
1:26:48
>> I'm super fascinated by this.
1:26:48
>> I'm super fascinated by this. That's the thing that >> I mean we were just talking about with like uh graipedia today where basically it seems like the XAI team went and ran a bunch of deep research reports for all the topics that you'd want to know and then you just have that pre-cached output and I'm so fascinated by this
1:27:03
idea of you have a ton of user data you have a ton of inference it's going to be really inference expensive but what what is some sort of you know cron job that you can run over your entire user base all the data and then just have surfaced results or surfaced action items uh that seems like an interesting like underexplored territory. >> Yeah. Yet to be shared, I would say, but >> Yeah.
1:27:19
Yet to be shared, I would say, but I mean, like, if you think about today, we're saying we're going to bring all these coding agents.
1:27:24
Why Why does each coding agent have its own memory of how I've interacted with it?
1:27:30
>> I've been a developer for 20 something years.
1:27:33
>> We could just go, here you go, take this with you.
1:27:35
You know, you can understand how I work.
1:27:37
So, I'm going to get a result that matches what I'm looking for. >> Okay.
1:27:41
Walk me through the game theory around enterprise pre-training for coding agents.
1:27:46
So if I'm Coke and he's Pepsi and we both have written a bunch of corporate code and we have a massive GitHub installation with you >> uh and if we both say >> yeah we're going to train on us maybe we both get better models >> uh but at the same time we don't want to leak information.
1:28:04
So like what's the current thinking among like big enterprise customers around like will they jump over and say yeah you know what it's worth it or or from a from an actual like would an AI scientist just be like yeah I don't need that code anyway.
1:28:15
What what's the current thesis?
1:28:17
>> So, we spent a long time trying this and the problem is that everyone goes, "Hey, our code's very different. It's super unique. We work a certain way." >> Yes.
1:28:25
>> You don't like so many companies don't.
1:28:28
Now, there are diff there are examples of where that's not true.
1:28:31
>> Particularly really uh companies with a really uh long legacy of like cobalt, mainframe code, etc.
1:28:36
mainframe code, etc. We've been kind of discussing with them like what would it take to get another 100 million lines of cobalt code because then that does matter >> that actually moves the needle quality of the coding >> 100% because the problem is is that the practices and principles don't change
1:28:52
that much and then most of these companies are also trying to modernize so they don't want the code to look like their old code they want to use their unique IP and look like the thing they want it to look like in the future >> but if I have a 100,000line Django project and he has 100,000 line Django project like you're not like oh if only we had that. No, no, cuz we No, cuz we
1:29:09
No, no, cuz we No, cuz we we've like done it in like there's you know margin of error improvements but nothing major.
1:29:16
When we look at the uh >> like looking at the chain of commits then you can get to some interesting information. >> Okay.
1:29:24
How was the enterprise built exactly and why was that choice made?
1:29:28
There's a little >> but you can obviously still instantiate that on the fly with with an enterprise partnership.
1:29:32
There's just always the question about like is there something beneficial that all the companies working together? But that's very helpful.
1:29:38
Uh, thank you so much for hopping on the show. This is a lot of fun.
1:29:40
>> You have an incredible voice for podcast anytime.
1:29:46
>> If you ever wrap up, uh, we have Jay Per next.
1:29:49
Oh, no, we are moving on.
1:29:49
Okay, we are going to take you back to the news. Thank you for tuning in.
1:29:55
Uh, we also have to do an ad read for Google AI Studio.
1:30:00
We are behind enemy lines here at a Microsoft event, but we are presented by Google. uh Google AI Studio.
1:30:07
It's the fastest way from prompt to pro production with Gemini.
1:30:09
Uh chat with models, vibe code, monitor usage.
1:30:14
Um we are obviously very happy to be supported by all of our sponsors who make crazy events like this possible.
1:30:17
We are obviously able to come up here on short notice uh due to our sponsors and >> and uh Jay looks like he's getting miked up here.
1:30:28
>> We will bring Jay Periq in.
1:30:29
>> We I I brought this up yesterday.
1:30:29
Um, uh, John, you remember I brought up, uh, I did not know that Interstell that with Interstellar, Christopher Nolan spent $100,000 to plant 500 acres of real corn in Alberta. >> Yes.
1:30:44
>> Uh, uh, then sold the corn for a profit after filming. >> Yes.
1:30:48
>> Uh, and it remains the most profitable commodities trade in Hollywood history.
1:30:54
>> You acted like this was uh, >> this is old news.
1:30:56
I I I knew about this years ago.
1:30:58
This is this is one of those stories that goes viral inspired in the AI area of like well I could just generate the scene with uh with AI but I want to miss out on commodities trade for sure.
1:31:10
>> Um >> no I I I think Christopher Nolan just got lucky here honestly.
1:31:13
Uh it is it is pretty hilarious.
1:31:16
I also wonder uh you know how how apocryphal is this story because uh it doesn't account for everything else that went like if if the corn was planted by production assistants, >> right?
1:31:30
Like you have to burden that cost into the actual ROI >> to plant corn. >> Who planted the corn?
1:31:37
I'm just saying like is this is this gross is this gross profit or net profit?
1:31:41
That's what I want to know.
1:31:43
Christopher Nolan, everyone's talking a big big game about the interstellar trade and it might not have been as good as you think.
1:31:48
Also, who owns the who owns the rights?
1:31:50
Does the does the value of the of the corn acrue to everyone who has points on the back end?
1:31:59
Like, does Matthew McConnA make a couple dollars off of that corn trade? I don't know.
1:32:04
We'll have to get to the bottom of it.
1:32:05
Um, we have our next guest.
1:32:07
Welcome to the stream, Jay. >> How are you doing? Thank you so much. Welcome. Nice to meet you. Nice to meet you.
1:32:15
>> Uh, introduce yourself for anyone who's been living under >> living under a data center >> and, uh, and explain a little bit about what you're working on today.
1:32:23
>> Uh, I'm Jere and I am the EVP of Core AI here at Microsoft. >> Okay. >> Important job.
1:32:31
>> Do you have anything to share that updates your job today based on the news of OpenAI or is it just exactly the same?
1:32:38
>> It's exactly the same.
1:32:39
>> It's exactly the same. Really? Okay.
1:32:39
So uh are you are you marching towards AGI?
1:32:44
Is it a race on developers?
1:32:46
>> Is it a race if Microsoft becomes a platform for AGI and Open AI can compete there and Microsoft's AI internal AI team can compete?
1:32:53
Is there a world where you're racing to AGI against them?
1:32:54
Um I think we we have a process for figuring out what AGI is >> and that is something that both companies will continue to to collaborate work on research.
1:33:10
>> And in the meantime, >> yeah, >> we have this mission which is to focus on developers. >> Yeah.
1:33:16
>> And how we unlock way more creativity >> and to build a ton more things.
1:33:19
So I have this idea which is or this concept where you think about all of the potential you think about the Hoover Dam for example right you guys are familiar with the Hoover Dam >> there's 9.
1:33:31
3 trillion gallons of water behind that >> one gigawatt right >> and it's like massive right in terms of the amount of energy that it can generate so think about all of these large language models whether they be small ones big ones closed open ones multimodal video audio text etc And you think about how we're going to unlock that intelligence.
1:33:53
And in order to unlock that intelligence, we have to write a lot of software, right?
1:33:57
And so if you think about the history of Microsoft, Satya commented on this earlier, you know, it's like Microsoft has been around 50 years, right?
1:34:05
And you think about all the software that's been written by Microsoft and everybody in the last 50 years.
1:34:10
the last 50 years. And I I would posit that only 1% or less o than 1% of the software that has been written in history and that what we're going to see in the next 10 years is just this like prolific expansion of the amount of software that's going to be completely agree it's going to be crazy >> right so that is why we're all here today right which is like how do we
1:34:38
>> really drive and use agents use this technology with the right guard rails, with the observability, being able to customize this, personalize it, be able to tap in and bring in open source, being able to bring in your enterprise specific knowledge and controls and all of that and to really just change that trajectory of creation of of imagination in a building, right? And I think
1:35:00
And I think actually the notion of even what we think of as a software developer is going to change right now to make this way more approachable by anybody who has an idea being able to translate that into you know showing something building an app getting out there getting feedback iterating on it way faster than we've historically been able to >> in in in codegen.
1:35:19
uh like I I I want to get your read on uh how you're thinking about like today developers are you know maybe they have some favorite tools but they're willing to constantly be experimenting trying new things.
1:35:32
You guys are in a great position to be able to support that through partnerships uh and and sit you know at a foundational layer with uh with GitHub.
1:35:40
layer with uh with GitHub. Uh but how are you thinking about what what what's your view on switching costs today and how that might evolve as you know in five years from now do you believe developers will continue to just you
1:35:52
know want to always be trying the latest thing or do you think they'll like um eventually switching costs will get to the point where it doesn't make sense to just constantly be looking over in other places and really makes makes more sense to just focus on what you have. >> Yeah. I think there's like an element >> Yeah.
1:36:04
I think there's like an element to, you know, developers, builders, uh, around craft.
1:36:08
And I think you're always going to want to find like the best tool or the tools that suit your sense of craft, right?
1:36:16
Whether you're a wood like a wood maker, you're a painter, you know, and and there's like a big element of craft.
1:36:24
So, I think that there's going to be use cases where, hey, this is the way to do it.
1:36:28
way to do it. these are the best tools to say modernize or upgrade some version of like old Java code that you may have and there may be just like this is the one or two just true tried ways of doing it and proper tools that you use then there's going to be new use cases that
1:36:44
we haven't even discovered seen yet today you think about some of these rapid prototyping apps and I think you know it's great for the ecosystem that we're seeing different startups we have different you know we have GitHub Spark we have different ideas that are all kind of competing and trying different versions of this. Those things do mature
1:36:59
Those things do mature and there may be a smaller and narrower field.
1:37:04
But I think right now with this inflection that we're seeing in terms of building and velocity of change that there will always be lots and lots of things to go try out.
1:37:13
And I think that's good for developers, right?
1:37:15
And I think our platform is such that we care a lot about that ecosystem of startups, other companies that can bring that choice, bring those tools into it.
1:37:24
But then we can help like hook those things together. Yeah.
1:37:29
>> From an observability controls like just a sensibility perspective.
1:37:32
So if you want to scale this adoption inside of your enterprise, you need those rails so to speak, right?
1:37:39
>> There's there's so much there's so much like practical on the ground.
1:37:41
just make the piece of software 5% better with AI today.
1:37:46
Uh there's so much low hanging fruit.
1:37:48
It's a very exciting time.
1:37:48
Uh at the same time, we're in this like I feel like we're taking a breather from all the AI fast takeoff and it's exciting because they can go build so much enterprise software, so much value, so many new companies, so many things built on top of Azure and and Microsoft.
1:38:03
Uh but at the same time it feels like there is a new need for going back to the roots of academia or these like academic labs or these scientific labs.
1:38:12
Do you have a pitch for uh if there's someone out there who thinks that they're going to be the they're going to write the next attention is all you need.
1:38:20
They're going to write the next transformer paper.
1:38:23
And you know what in the short term they're not actually going to help optimize you know knowledge retrieval or codegen for this next couple years but they but they they believe they want to do that.
1:38:34
Do you have a pitch to them where they can come and work at Microsoft and and do that level of research? >> Yeah, absolutely.
1:38:41
So, I think there's there's lots of different adventures you can pick inside of Microsoft for and focused on like builders, developers, right?
1:38:48
Because one of the other fascinating and fun things about the core AI team is we have this super tight collaboration with Microsoft research. Yes. Right.
1:38:56
Yes. Right. So Microsoft research has all of you know 30 plus years of history in science research and programming language research compiler security and you name it right so we actually have a lot of collaboration and joint problem solving right where they can focus more
1:39:13
on that open-ended research whether it be hey here's how I'm going to go optimize this model here's how I'm going to do formal verification of the code that comes out here's what I'm going to do in terms of how to secure this code better and So those things are out there. They're like big unsolved
1:39:27
They're like big unsolved problems.
1:39:29
They're longer time horizons.
1:39:32
Then as those innovations, those inventions happen in research, we can do the tech transfer.
1:39:36
We can do the combined like product making together and then that acrru into GitHub or in VS code or into foundry whatever you know whatever is the right avenue to bring that stuff to our customers to developers. >> Yeah.
1:39:50
Do you have >> where where do you stand on the should you learn to code debate?
1:39:55
>> Oh, that's a good one. >> I think yes.
1:39:57
I think you should learn everything you can learn about these systems because the fundamentals, you know, ultimately if you can understand like how this stuff shows up and it's instructing a computer, a GPU, a mobile phone, then I think that and it's less about maybe even knowing kind of the the the code, but it's that systems thinking mindset, right?
1:40:19
It's the cultural aspect of it.
1:40:22
It's like, hey, I'm creating, I'm prompting, I'm guiding this thing, but here's how the code is going to generate.
1:40:28
I understand what these models can and can't do, how to guide them more with a higher efficacy, right?
1:40:33
So, absolutely, but I think of it more as like less of a narrow question of like, hey, should I learn to code or not?
1:40:40
It's like, how do I understand the system, the new system for how we're going to build software, build innovation?
1:40:48
There's understanding the hardware, understanding the software, understanding for example evals >> super super uh like important concept totally under reportported like right in terms of what's going to happen.
1:41:00
You have these offline evals we have these benchmarks I'm just saying in terms of the media. >> Yeah.
1:41:06
>> Yeah. in terms of like how important that is to get higher like quality outputs of these things because there's the offline evals that we can sit there and we can score and say we got these evals then there's the online or the
1:41:20
lived experience right when you put this AI into this product you're like wait that doesn't quite work the way these eval said it was going to work right weird ways in terms of >> sorry I I wait I have one more on that uh we got your answer on should you learn to code. I want to know should you
1:41:36
I want to know should you learn to deal?
1:41:37
Should you learn to do deals?
1:41:40
Is deal making underrated in 2025 in the age of AI?
1:41:43
Being a deals guy, understanding incentives, bring people together around a table, iron out a deal.
1:41:49
This is something that's it feels like it's growing.
1:41:52
We saw it with the Microsoft Open AI deal.
1:41:54
That was a very unique deal.
1:41:56
That was something that a lot of people if they were just saying, "Oh, >> it was much more than a than a traditional ton investment >> and it got done and it's probably one of the greatest deals in in tech history."
1:42:08
>> And so is there value in learning how to do deals and becoming a deals guy?
1:42:13
>> I don't know that that's a 2025 question.
1:42:15
question. I think that is a life skill to know how to collaborate and how to negotiate and how to compromise and how to see >> you know and sometimes like there isn't a deal to be made and other times there's a greater >> output or there's sort of a greater like a global maxima that you can attain
1:42:33
right and and that's where even if you look at the news today with our announcements of partnering with open AI and with anthropic bringing that all into this platform together I think is the what we can go build and what we're going to discover and how we're going to
1:42:49
accelerate our joint learning I think is important right and that can turn into a deal but I think that comes up with this like hey there's a greater good there's a greater opportunity there's sort of a greater market there's a greater problem a bigger problem to go solve then >> yes figuring out how it's going to work
1:43:05
>> nuts and bolts >> I like it >> uh how do you think about Jevan's paradox in the context of code during the deepseek moment >> Satia uh quickly came out and I think he posted the Wikipedia link to Jevans Paradox and it sort of like steadied uh the market broadly. uh there was people
1:43:21
uh there was people that just weren't weren't familiar, but I think it was well timed from his side.
1:43:27
But when it when it when it comes to um you know on our side uh you know we're a media company and we have a developer on our team and I think that like five years ago we wouldn't have had a developer and as it's become basically cheaper and faster to create software.
1:43:42
We now want to make software and we're a company that historically just wouldn't have.
1:43:46
So I'm I'm curious how you think of that in the context, you know, going back to your earlier point of like we might have a hundred thousand 100 thousand times more code.
1:43:53
Um so what's your view there?
1:43:59
>> Yeah, I think that's what we want to see the acceleration, right?
1:44:00
I think we talked about today there's 180 million developers in in GitHub today, right?
1:44:07
And and a new developer is joining GitHub every second, right? I love that.
1:44:12
Somebody said a developer per second and I was like that's that sounds like miles per hour but like this is just such an abstract concept.
1:44:18
>> That's how that's how countries talk.
1:44:20
They're like every every second. >> Yeah.
1:44:22
There's a baby born every second. >> Yeah.
1:44:23
But to think about, you know, it's not you think about the the types of personalities and and backgrounds, right?
1:44:29
You can be a product person, you can be a designer, you can be a you can be a marketer, you can be a dealmaker.
1:44:35
like all of this stuff you can join GitHub, you can start building, you can start checking in code, you could start mashing up different things.
1:44:40
So I actually think it's a super exciting time to see what the industry is doing, right?
1:44:46
right? And I think it's hard to predict the future, but I do actually really really fundamentally believe like from a mission perspective in core AI, our job really is to unlock that creativity both in the AI powered tools that you heard
1:45:00
about today plus the platform making these things secure and like really like anybody who's got an idea wherever you are in whatever department you are in an organization or an individual you should be able to actualize that. Like you know
1:45:13
be able to actualize that. Like you know we have this saying in our team which is like you know more demos less memos right it's like all about building and showing and iterating lots of stuff gets like we don't like it you know but the
1:45:26
fact that I can in 15 minutes go through 15 iterations versus in the past I might get a quarter of an iteration done that I think is going to >> matter how good a memo is like seeing seeing the product tells you 10 times more about it it sort of gets more creativity from the team a small group of people. Now, we have to make sure we
1:45:44
Now, we have to make sure we also spend time dealing with the fact that there are there are gaps in these in the technologies, right?
1:45:52
They don't like work perfectly, right?
1:45:54
So, we've got to keep building those guard rails.
1:45:58
We got to keep building that training.
1:46:00
The models will get better.
1:46:00
The tools have got to get better as well.
1:46:02
And that's where I think the GitHub community working together with these different partners that we have, the platform, we just have to keep learning faster and faster and faster.
1:46:12
That's what we're focused on.
1:46:15
>> So, more demos, less memos.
1:46:15
Uh, let's role play for a second. More deals potentially. Let's let's role play.
1:46:21
We're trying to do a deal.
1:46:21
Uh, if I'm a Fortune 500 CEO and I'm coming to you and I'm saying, I want to transform my business with AI.
1:46:29
Uh, I don't want to make mistakes.
1:46:32
What what pattern should I avoid?
1:46:35
what mistakes have you seen broadly trends that I want to stay away from uh so that I can move forward with something that actually drive shareholder value and isn't just rah I'm doing AI now. >> Yeah.
1:46:49
So the first thing I would say is like really understand what the top one two three outcomes are more like specifically like hey I want to transform my business.
1:46:59
Okay well what does that mean? Yeah. Okay.
1:47:00
Do you know what that means?
1:47:02
Are you saying hey I I need to I'm in an understand phase where I even just need to even >> create some bright lines around what is the ideal or kind of my dreams around the outcomes of what transformation means.
1:47:17
So get into the specifics of the what that actually means.
1:47:19
Is it some revenue thing?
1:47:22
Is it some product thing?
1:47:24
Is it some >> even just the difference between are you trying to cut costs or are you trying to grow topline?
1:47:28
So that's number one is just understanding like one listen to the customer.
1:47:32
I keep asking what why >> and to try to get um more grounded in what those those specific things are.
1:47:40
Number two is >> one of the things that I I will always encourage them or talk to them about is to then don't just talk about these things, right?
1:47:51
It's like what are you doing to start learning?
1:47:52
Because if you're early in that journey of of understanding AI, >> there is only so much that you can sort of like read and talk about and conduct meetings.
1:48:03
You do need to have like this internal adoption, right?
1:48:06
Where people are and you're encouraging, you're incentivizing, you're really driving that experimentation, that curiosity of your organization, right?
1:48:16
So how do you understand what your base level of curiosity and risk takingaking is?
1:48:19
If you are a more risk averse company in a slower moving company, then how do you change that culture? Right?
1:48:26
So cultural transformation is it comes up in 90% of my customer conversations.
1:48:32
We'll talk some tech stuff and then they're like, "Okay, Jay, >> how do we do this people-wise?" Right?
1:48:39
And then the third thing is >> headcount headcount planning.
1:48:41
They're like, "What what's your plan? Maybe I'll adopt that." >> Great.
1:48:45
And then the third thing that I always will encourage folks to do and we'll have a conversation is like raise your level of ambition.
1:48:49
Like wherever you think you are in terms of ambition and that outcome, I promise you it's not enough because the technology the models are growing way faster.
1:48:58
They're getting way smarter than we humanly understand.
1:49:02
So whatever ambition you have for this fiscal year or this half or this quarter, take it up a notch or two and then strive and push and lead to that to that point. >> Yeah.
1:49:13
>> Yeah. Are you do you have a right line internally with I feel like there's some organizations where core AI means not generative AI uh but I don't think you use that exact uh [laughter] dividing line but it should there be a dividing line between like machine learning recommendation systems how Netflix
1:49:30
recommends me the next thing to watch for example like that is an AI system what what what pops up on my news feed is AI but it's not generative AI it's not what we think of when we think of generative image models is it worthwhile in 2025 to have a bright line between those teams or those skill sets or is everything bleeding together? >> I think things are definitely blurring
1:49:50
>> I think things are definitely blurring together and there's stuff that's informing you know from one set of techniques to the other and vice versa.
1:49:59
I do think that those systems are very very sophisticated.
1:50:03
They're very I would say powerful in terms of like user experience today.
1:50:08
There are definitely places where people are using Gen AI when they shouldn't be and they should be using you know machine learning techniques that just really work and are faster, better, cheaper. >> For sure cheaper.
1:50:21
Yeah, you can imagine a bunch of things.
1:50:23
a bunch of things. Those are the things that you know we have to watch for in organizations where you know Gen AI is the hammer and everything looks like a nail when we actually have these mature optimized and like really exceptionally
1:50:35
bright people and technology to use those and not forget about those right but I do think that in at scale the stuff that we've learned in these more maybe you know uh more mature more scale out machine learning systems uh will feed back into how we make products using Gen AI. >> Well, thank you so much for coming on
1:50:54
>> Well, thank you so much for coming on the show. Good to see you. Take care. We'll talk to you soon.
1:51:00
We have Jared Palmer, the vice president of product for AI and the SVP of GitHub.
1:51:08
>> The SVP the VP you're you're both a vice president and a senior vice president. >> Yes. >> Incredible.
1:51:16
>> Is this like a two-phase situation? Title Max. Yeah, title Max.
1:51:18
Yeah, [laughter] >> I might just You're good. back up.
1:51:23
So just like you know regional branch manager. >> Yes. >> Right. Yeah.
1:51:28
>> Uh it's like assistant to the CEO assistant CEO.
1:51:32
>> So technically it is VP of product core AI and SVP of GitHub. >> Okay.
1:51:38
>> Does this by the way welcome welcome to the to the gig. >> Thanks. Congratulations. >> Monday >> 30 >> 13. >> Oh 13. Yeah. Okay.
1:51:46
What were you doing before a month?
1:51:48
>> I was VP of AI at Verscell. >> Versel. >> Yeah that's right.
1:51:50
We had GMO on the show yesterday.
1:51:52
>> Created a little thing called V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V Zero. >> V congratulations. >> Yes.
1:51:56
Uh so >> and been in the game for I don't know a little bit doing that stuff.
1:52:00
So anyway, yeah, it's been fun at Versel's been great. >> Yeah.
1:52:03
So I mean have you had time to actually develop like a vision for what you're building here?
1:52:07
Is it too early to ask or or or are you still just in kind of like let me assess the tools in the tool chest over here? >> Uh it's day 13.
1:52:14
So definitely [laughter] uh but I I've been a longtime GitHub user for for a very very long time like I don't think over 10 years I made my account.
1:52:23
So um >> and I imagine you've been thinking about like the broader developer experience and what this means in the age of AI all through the last I mean the last five years have been like a deafening ring of like AGI and takeoff and timelines and stuff.
1:52:35
You must have engaged with that of course. >> Yes. Yes.
1:52:37
And obviously at versell we thought deeply about developer experience.
1:52:40
I think that's really the vision is how do we bring aart with with core AI and the formation of it. you just had Jay on.
1:52:46
>> I think by combining Microsoft's assets across the stack, right?
1:52:48
You've got VS Code, Visual Studio, GitHub, >> and putting these actually all in one or will make for the ultimate developer experience, and that's what our our goal has to be.
1:52:57
And focusing just on that is I like my first and foremost focus. >> Yeah.
1:53:01
Do you think that developer label just melts away eventually?
1:53:03
It it feels like you think there will be a dividing line in five years, 10 years.
1:53:09
>> I don't know about five or 10, I guess.
1:53:10
I mean, it it just feels like there's a world.
1:53:12
>> I don't know about that, but >> uh but it it just feels like, you know, like there there was a time when when >> to take a photo, you needed to be a professional photographer because you need to needed to understand how to change film in a dark room.
1:53:23
And now everyone has a smartphone camera and everyone's a photographer.
1:53:28
That feels like it's coming. I don't know.
1:53:30
I just see like I can open up an app on my phone, type a prompt, get code. Sure.
1:53:35
It's like kind of hard for me to I need to link my GitHub account instead of pages to like actually deploy it, but like we're only a couple months away from that I feel like.
1:53:42
And then eventually it becomes like more prompt driven, but then there's still value. I don't know.
1:53:48
How do how does all this play out?
1:53:49
>> I think there's always going to be a market for people who get stuff done. >> Yeah. >> Right. >> Yeah. Just high agency people.
1:53:52
So builder people who and whether it shifts into >> um more product focus knowing how to build just like systems that are big and large. Yeah.
1:54:03
>> That maybe outside the training set I think is always going to be important.
1:54:06
I also think that some of the u the way I think about it at least is some of the >> the pipes the tooling probably aren't changing as fast as the AI is.
1:54:14
And what I mean by that is like the way that packages and code is distributed tested built I don't think that's going to change as fast as maybe the models will. that makes sense.
1:54:24
So with that with that like infrastructure in place, I think you're still going to have uh human involvement for quite some time.
1:54:30
I think the things that people will build may be more ambitious.
1:54:33
I think that's really exciting and our job is to facilitate that and empower developers and think about you know what they need.
1:54:40
>> Um but you know in 5 years or so I still think people are going to be building stuff with still going to be coding in some respects.
1:54:44
Uh it just may look very different.
1:54:47
>> Where do you want to see model progress?
1:54:49
people talk about the models are going to get better, like they're just going to get better, right? And uh plan around that.
1:54:54
But like when you're when you're talking to labs, like when you when you're at Versell or when you're now now at Microsoft, like where specifically are you even thinking and kind of pushing them to say like, hey, like it needs to be better here.
1:55:08
>> Ah yeah, I mean that's a great um great question.
1:55:10
At uh at Verscell, we work deeply with the model labs.
1:55:13
We obviously were very focused with a product like VZero on a specific subset of what models can do.
1:55:18
Uh even in the coding realm, Verscell was always focused on front end, right? And specifically Nex. js.
1:55:24
So not just one language, but one specific text stack.
1:55:26
Um and so we were always, you know, engaged with how can we make it better for next. js.
1:55:34
>> Uh switching switching gears for a second to GitHub.
1:55:36
Obviously we're now multiple languages.
1:55:38
We care about everything, but we do care about coding.
1:55:41
That's the primary um the primary focus point.
1:55:44
But coding involves so much more than just generating like um source code, right?
1:55:48
It's more than autocomplete.
1:55:49
Um we need models to be great at research, be great at reasoning.
1:55:53
And I think uh and then also delivering mergeable code, right?
1:55:55
Uh that's I think slightly different than just complete my comment.
1:55:59
Uh so we've been focusing a lot there and focusing on quality and something that we look to continue to hill climb on as time goes on.
1:56:06
How much have you studied the uh the open-source company like scalable business model like what like what Verscell did with Nex.
1:56:14
Verscell did with Nex.js JS like are are you are you familiar like can can you give me like like the the crash course if I'm like >> I'm a developer I want to build a business I'm I'm going to open source a a you know a package that does something and then I want to build a business
1:56:30
about it like what are the pitfalls that I need to avoid how do I actually balance like what are the tradeoffs that I'm making to actually build a great like open-source for-profit company because there does seem to be some tension there but it's held that model's held for going back Red Hat Linux all the way to Versel today. >> Sure. I think I'll I'll I have a >> Sure.
1:56:46
I think I'll I'll I have a controversial take >> please.
1:56:51
>> Um there aren't as many pure open-source companies where the core product itself is open source.
1:56:58
>> I think the more successful strategy is actually if you really dig into Verscell is Verscell is not open source but Nex.
1:57:04
js is open source and Nex.
1:57:04
js is a complimentary satellite product. >> Yes.
1:57:09
that drives attention and that is used by Verscell to make a better product and so they got this amazing feedback loop of internal dog fooding.
1:57:19
Um but there is a community around the the project which then I think some uh you know Verscell has a material amount of Nex.
1:57:25
js overall builds and developers use Verscell but it's not like Versel is it open source business y right it just has Nex.
1:57:32
has Nex.js JS is one of its largest pieces of the open source portfolio but it also has now AISDK and with Verscell the idea was to do something what we used to call framework um defined infrastructure so framework defined infrastructure and the idea was you can build this framework and with no configuration you can deploy it and you don't have to think about scaling it and
1:57:53
so the uh >> the analogy I would make is like imagine you were asked to uh I don't know cook food for everybody here at at universe >> with versel the idea was like, well, what if we gave you the the pots and pans and all you had to focus on was like cooking for your, you know, family of four and then Verscell would worry about like scaling it to everybody here. >> Yeah. >> Yeah.
1:58:12
>> Um, >> and so I think to to your point about like open source, my my my suggestions for the crash course is >> um a common pitfall that you should not run into is just assuming that you're free open source users are going to directly translate into paying customers. >> Interesting.
1:58:27
I think that's actually really hard because you've set up expectations that you're giving away a free service, right?
1:58:32
This free this code, right? Yeah.
1:58:34
And that all of a sudden they're going to convert and pay you X dollars a month and you're going to have an enterprise business which you haven't been really honing in on and grinding on and that's just going to happen overnight.
1:58:43
>> I think that's I think that's I think that's false.
1:58:44
I think you need to start from the beginning with both and also set expectations with your with your user base that this is paid, this is open source and if you can find a beautiful symbiosis between those two, that's when I see like it really being successful.
1:58:56
Is there some sort of like barbell strategy where you should access like actually go really broad with your with your open source package?
1:59:02
Anyone's using it, but probably like you know small developers, startups, solo indie devs are using it.
1:59:08
And then if you jump all the way to like oh you notice some big corporations are using it.
1:59:12
So you go with an enterprise plan out on day one.
1:59:16
It's like they're not going to be they have no ground to stand out if they complain.
1:59:21
It's a lot easier than being nerfing the open source thing and now all the indie devs need to pay me 25 bucks a month.
1:59:28
You know, that's way Fortune 100 company was using this now we got a million dollar contract with them. Is that best practice? >> It's hard.
1:59:35
Sometimes those big contracts early on can really be devastating because they can um remove your focus on growing that inertia, that momentum.
1:59:44
>> And so you have to be careful.
1:59:46
>> Obviously they're great. That's awesome.
1:59:48
but focusing on your core value proposition, your core customers and it's really great to get feedback by those enterprises early on and many many projects I've been involved with whether it was Turbo Repo, whether it was Nex.
1:59:57
js um even Vzero like um we didn't launch enterprise for almost a year or so and we even I even was a big debate between me and Germo and I think we were actually early we should actually delay it even further.
2:00:12
>> Really getting that ground swell is so important and you can always do enterprise.
2:00:15
Okay, >> you'd be careful.
2:00:16
I say always do enterprise. You'd be careful.
2:00:18
Yes, somebody could come in, but just driving up.
2:00:21
Even like chat GBT, by the way, didn't have enterprise for like a people were begging for it, right?
2:00:24
And then when it finally a lot of companies report like, "Yeah, we don't pay for chat GBT, but our employees all use it, >> right?"
2:00:31
And they and then you go to the CEO and you're like, "Hey, by the way, we have a lot of your your your data. >> I know exactly."
2:00:37
[laughter] Um, so >> it is a wild choice.
2:00:41
I mean >> how do you think uh you know a lot of there's so much excitement around the potential of AI and science and law and these other categories and obviously adoption is happening but how do you think adoption will will kind of how would you imagine adoption will look in
2:01:00
those categories because I think uh AI adoption in software engineering is very natural because the people that are building and and uh doing the research are are adopting the product and like it's a super tight feedback loop that you're not really going to see in the same way in some of these other categories. So,
2:01:16
So, >> uh yes and >> I don't know I before I got into software development I uh I was actually um a banker and so >> let's go. >> Yeah. Gold Goldman Sachs fig. Thank you very much. >> Let's go.
2:01:32
>> So, uh I did I did my banker, right?
2:01:32
So I I think I I got to tell you I'll be honest like uh I think you know Anthropic I was just talking with Mikey they announced Claude for Excel.
2:01:41
I think that's going to do wonders.
2:01:43
I think um if you talk to any Goldman Sachs analyst, they'll be very excited to to have that deeply integrated and if you're building Excel models all day long, >> there's whole businesses that are built just on like templates like >> totally.
2:01:54
So what's interesting though >> is if you look at cloud for Excel >> I think their core foundation is still the coding agent and that a there's something about the coding the coding runtime that can be then augmented to other verticals.
2:02:07
I think that's what you're going to see in the next year or so is is is these model labs build out these harnesses and go vertical by vertical um whether it's banking, healthare, um or consulting, right?
2:02:19
They're going to go through that through knowledge work.
2:02:21
Um and they're going to iterate on that just like and the hill climb. >> Yeah. >> Yeah.
2:02:25
What do you think on on on uh how do you how do you think about switching costs now and over time if you're a products company and you're leveraging intelligence from a lab?
2:02:36
like do you think the labs will over time make it harder and harder to kind of like rip out one one model provider and use another because right now it feels like there's this land grab happening in enterprise and this race between Enthropic and and uh and Gemini and OpenAI but like how do how do you think that evolves?
2:02:56
I think uh most of the product builders that I've talked to like the the from the companies you that are on here all the time, most of their teams are working with multiple models and they have and they're constantly evaluating whatever sort of product analytics or test harnesses.
2:03:10
They're looking for any edge they can.
2:03:12
They're they're so competitive. Yeah.
2:03:15
>> At least in like the the startup space that like >> and but switching models is not easy.
2:03:20
That takes time and especially when there's big rearchitectures like when reasoning came out for example >> um that may require a rewrite of all the prompts and all the edge cases that you've been massaging and these models have different characteristics um but I think most of the high performance teams are are are dialing in harnesses for each and every lab and they're they're just so hungry that
2:03:42
>> so switching costs are high switching costs are high so the answer is like use all of them from the beginning >> correct and and there may even certain subsystems or certain tool calls where you're going to switch models and mix them together and that just is you know part of the part of this if you look at like what Windinsurf did uh what they released Swigp that that specialized model for research. Yeah. Yeah.
2:04:03
>> Um you know they're combining they're mixing and matching.
2:04:04
I think that's the the next you know we'll see that throughout the next year.
2:04:07
I don't think it's like oh we're just going to use anthropic models >> um or we're just going to use OpenAI models or we're just going to use you know whatever model.
2:04:15
I think you'll see a lot of combination.
2:04:16
Thank you so much for coming on the show.
2:04:18
Congrats on the great rest of your day.
2:04:23
>> Um, before we bring in Michael Grinage from work OS, we got some breaking news. Did you see the blimp? Did you see the blimp?
2:04:30
Do you know whose blimp that is?
2:04:32
>> It's Sergey Brin's blimp. Let's go.
2:04:32
He I mean, this is Mag Seven on Mag Seven Crime.
2:04:38
>> I'm actually going to take I'm going to take a little bit of credit for that. I told the Gemini team. >> Oh, you told them?
2:04:43
I I I sure [laughter] I haven't text. >> What's up, guys? Hey, good to see you.
2:04:50
Great to finally meet you.
2:04:50
Mutual friends David mutual friend >> David. Love David.
2:04:56
I got you brought you guys both. >> Oh, please.
2:04:59
>> One of our highly coveted super rare >> enterprise ready hat.
2:05:04
>> How are you enterprise ready? >> All right. So, enterprise ready. What What does it mean?
2:05:07
Well, pretty much every software company eventually when they get product market fit and go up market, there's a ton of stuff they have to add to their app to go sell the enterprise. >> Yes.
2:05:15
>> So guys at Microsoft and GitHub, they did this years ago.
2:05:17
But if you're a new company, you have to add all that stuff to your product.
2:05:21
>> And it's things like single sign on, user provisioning, logs, security.
2:05:23
Work OS just does all that for you as a developer. >> Got it. Yeah. >> Okay.
2:05:28
So you can just focus on the core product actually. >> Yeah.
2:05:30
In the same way you use Stripe for payments or Twilio for messaging, work is really that for >> this is an interesting business because it feels like uh it it's not something that you could just like go through YC and like sell to another startup.
2:05:41
So like who was the first client?
2:05:43
How did you get into this?
2:05:45
What were you doing before?
2:05:46
>> So I worked started working on this a long time ago almost seven years ago.
2:05:48
So work OS overnight success. >> It kind of pains me.
2:05:53
>> Yeah, it's like a we're also like a pre-I company.
2:05:54
Someone called us the other day which also kind of hurts a little bit. Yeah, I [laughter] know. Show my ears. dinosaur.
2:06:00
>> I started as AI native before AI existed. >> Yeah, >> for real.
2:06:04
>> Um, I saw this problem with another company I started.
2:06:06
We we had built an email product, got a bunch of usage, got a bunch of adoption, >> tried to go to enterprise, >> try to sell it to these guys and they said no way we can let this touch our >> data saying no is the CTO, C CISO, >> usually engineering leaders, co-founders, VP of whoever is kind of responsible for the technology.
2:06:24
>> And is it because they want those features or they need them for legal reasons?
2:06:28
>> They got to have them.
2:06:28
They usually have deals that are blocked because they don't have these features.
2:06:31
So, you'll start growing up market and there'll be some customer that says, "Hey, we'd love to use your product.
2:06:35
We'd love to roll it out, you know, at Coinbase or Microsoft or something, but we can't do it unless we have these features."
2:06:42
>> Has demand just been insane because people are building products so quickly and then they start, you know, employees at employees at companies start adopting them kind of personally and then they realize >> it just is compounding. Yeah.
2:06:50
So, so we had a lot of growth, you know, years ago through the kind of the early cloud era SAS.
2:06:56
Like Versell is one of our customers, Carta, Plaid, folks like that.
2:07:01
>> In the last year and a half, two years.
2:07:04
[laughter] >> In the last year and a half or two years, what we found is it's actually perfect for all these AI companies. >> Yep.
2:07:09
>> So, today we're powering enterprise off for OpenAI, Anthropic, Perplexity, Cursor, >> Sierra, you know, all these guys that are growing faster.
2:07:18
>> So, [laughter] we got a lot of >> smash that. Yeah. >> Okay.
2:07:20
uh walk me through the thesis.
2:07:20
Uh in the YC era, it became uh like the the YC trade was basically you could be a kid in college uh you know graduate and move to Y move to Mountain View or or Silicon Valley and for $100,000 and some cloud credits from Azure whoever uh you could set up a website and go kind of build the first era of consumer and we got our Airbnbs from there.
2:07:46
We got a variety of consumer companies, but in the AI era, it's becoming easier to go enterprise on day one. Is that real?
2:07:53
Is that is that is that a reasonable thesis?
2:07:57
Do you see any data to that effect? >> Absolutely.
2:07:59
So, I think that previous era, the privilege that those companies had is they could take a while to get to enterprise.
2:08:04
So, if you look at Dropbox, Figma, Slack, it was years.
2:08:06
It was like three, four, five, six, seven years before they actually went after enterprise.
2:08:12
>> What we're seeing today is AI businesses get pulled up market way faster. >> Sure.
2:08:16
>> And it's way more competitive.
2:08:16
So companies like Cursor or Complexity pretty much in year one, year one or two, >> yeah, >> they get pulled that market to the enterprise and that's why >> they say yes because of that competitive dynamic, but also the tools that they're building like the the enterprise is just so ready for them.
2:08:31
>> There's another piece of it as well.
2:08:33
It's not just that they they grow faster at market.
2:08:34
But you think about these AI products, they are touching sensitive data.
2:08:38
You have one of these things that it's only valuable if you get access to all of your stuff.
2:08:41
You give it access to do things on your behalf.
2:08:43
So suddenly it becomes this huge security concern. Yeah.
2:08:47
Maybe an old product like Figma you could say to the design team just don't put any sensitive data in it.
2:08:52
>> But you get one of these agents or something connected you you need it to access everything and so they're scrutinized at a higher amount plus they grow faster plus in their life cycle.
2:08:59
It's a perfect storm where where we come in and help them.
2:09:02
>> Talk to me about domestic versus international.
2:09:03
I imagine a lot of your com a lot of your clients are already international.
2:09:06
So, does that mean you're international or are you focused on uh making American companies enterprise ready immediately and then maybe you'll go after the European market later?
2:09:14
How you think about that?
2:09:17
>> So many of our customers are actually right here. >> Yeah.
2:09:19
>> Like probably at the universe literally right here.
2:09:21
We were joking we could cut up our sales territories by north and south of Market Street [laughter] in SF because we have so many businesses that are here that are growing quickly.
2:09:29
>> Um what we find is their customers are international course, right?
2:09:31
So they're going and selling to larger organizations elsewhere in the world.
2:09:35
Okay, >> so our products that are kind of our customers customer those types of things we you know we localize we just did a big project to translate everything using AI.
2:09:42
So we launched with 100 languages so we do that kind of stuff but we find that the best product the best companies that are using work OS are these high growth AI businesses that are taking off >> and of course they're mostly here you know they're they're mostly here >> what's your philosophy around operating the business?
2:09:58
I I don't remember I don't necessarily recall the last time you guys raised money.
2:10:03
Like I'm sure people are throwing money at you all the time when they see the logos.
2:10:08
>> Yeah, we raised our our series B uh almost exactly four years ago actually which is >> that was the last financing.
2:10:13
>> Yeah, that was the last financing which is like an eon in the SAS era. Yeah, >> you hear that.
2:10:19
>> Since then we've just got dog [laughter] on our hands.
2:10:24
We've just been building it slowly since then.
2:10:26
You know, bit by bit by bit, brick by brick, you know, slowly.
2:10:28
Actually, I do have something to announce which is pretty exciting.
2:10:32
Um, uh, you know, you had, um, Aati here previous talking about how they do a billion in revenue every day.
2:10:38
We're very proud to announce that we just crossed 30 million in annualized revenue.
2:10:42
So, that's our Congratulations.
2:10:45
>> That's our big number.
2:10:46
>> Overnight success >> we're hitting today. That's great.
2:10:48
>> A little bit smaller than Microsoft, but um, but we're coming for you.
2:10:52
We've been compounding since then that the AI stuff has really been this huge tailwind for us and it's it's so fun to build infrastructure where we get to see into all these companies like my customers are the fastest growing most exciting AI businesses out there. >> Do you invest? Do you angel invest? >> I do some. Yeah.
2:11:07
>> Because I imagine you keep seeing these companies at this like crazy inflection point.
2:11:11
>> I've had VCs start asking me for the data.
2:11:13
They want to invest just to get the growth data [laughter] out of it.
2:11:14
Um yeah, it's a little >> Yeah, it's a little different.
2:11:17
Yeah, we don't we don't share that kind of stuff.
2:11:20
But just through, you know, building for developers and running events and I mean I love GitHub Universe.
2:11:24
This is like Coachella for like, you know, developer stuff.
2:11:27
You meet founders and meet other people building stuff.
2:11:29
And so, >> uh, who are some of the entrepreneurs that you look up to?
2:11:33
Who or what's the what's the story from a founder or business person that you keep coming back to is like, oh, that one. >> I'll go first.
2:11:42
David Sen, [laughter] >> just his story just how he No, I I do.
2:11:44
I think I think I I like he is like I I feel like I have the blessing of like I by being friends with David I'm friends with history's greatest entrepreneurs.
2:11:58
It's the most efficient it's the most efficient role model right a specific type of business but whatever business you're building you can learn from >> from uh >> I mean yeah there there's a lot of other things you could have said but that one's pretty good.
2:12:10
You [laughter] got anything >> I was trying to think for mine. David's great.
2:12:13
Uh I I'm just laughing because like like like the stakes are like you know uh Henry Ford inventing the uh you know the the what's it called?
2:12:20
The actual assembly line the manu the automated assembly line or like you know [laughter] uh Bill Gates writing picking the most impactful versus like the most valuable for you. >> Yeah. Yeah.
2:12:36
I guess who who do you come back to? I don't know.
2:12:37
I always come back to the we're very much in a marketing business and so I always come back to uh a quote that I attribute to David Senra but is actually from David Oggov that you are not advertising to a standing army you are advertising to a moving parade. Yeah.
2:12:51
And so the question is like why are people >> proving my point you uh you heard that for the >> from David?
2:12:58
No I read o I read oglev on advertising.
2:13:01
I'm familiar with the book.
2:13:02
I read it before David read it.
2:13:02
I read it before David read it. uh but but he did stick it in my brain and he advertised it to my moving parade and uh and I've always liked that idea of of even if you've shown someone an advertisement once or you've sent them a
2:13:14
message or you've given them a pitch once like you there there is a moving army there is so many distractions there's attention all over the place that you need to be hitting again and again and it's why they don't just do
2:13:25
one GitHub universe and say yeah we did it we're good they do it every single year >> and the message is different every year for them too they're evolving >> yeah man there's so many there's so many to choose from I'm I'm feel like a
2:13:34
little bit of an old soul in that when I when I heard Satia talking about like the early days of Microsoft and built I love building platforms >> like I built all this other stuff earlier in my career and as soon as I
2:13:44
started building stuff for developers other people making stuff I was like ah that's really sick like you see other people make stuff with the thing you made and then build their own businesses on top of that >> and to me Microsoft is like the first big software platform company you know Windows enabled so many developers to build and ship these experiences to change change the it's you know previous but it was this huge enabler this huge
2:14:09
like democratization of access to technology >> is there a uh a specific sales funnel that flows through GitHub with your product >> we do just a ton of stuff with developers I think you know uh I mean we do everything from you know sponsoring
2:14:24
podcasts and newsletters and meetups and and doing developer events our we did our own conference last week um we do a lot of open source stuff we run a really uh popular open source design system project called Radix. >> Oh, that's >> Oh, that's >> Yeah. So, that's on GitHub.
2:14:35
Um, but I think I think my GitHub account >> is probably one of the earliest like personal identities online I have as an account.
2:14:42
You know, I've had it since before I was in college.
2:14:43
So, >> uh I'm thrilled to be here. >> Yeah. Yeah. That's very cool.
2:14:48
>> Are you speaking at all today? >> I am.
2:14:49
I'm giving a talk tomorrow um all about AI and identity for agents. Okay.
2:14:55
So, so this is a new thing.
2:14:55
Work OS is kind of like an identity security company.
2:14:58
We help people with sign in and off.
2:15:00
And there's this big question right now of how we're going to secure agents. >> Yeah.
2:15:05
>> You know, we have seven billion people on the planet.
2:15:06
We'll probably have trillions of agents running around and doing stuff for us, connecting to different systems.
2:15:11
And security is even more important.
2:15:12
You can think of an agent kind of like a a crazy hyperactive intern.
2:15:17
>> You're going to get access to all of your systems.
2:15:18
And so there's this question of how do you authenticate them?
2:15:21
How do you build security around it?
2:15:22
Permissions, approval, >> how do they not get prompt injected? >> Right. Right. Yeah.
2:15:26
Um there's this old old quote uh you know to error is human but to screw up 10,000 times per second you need a computer to do that.
2:15:32
Agents are kind of like that right they make it really easy to do stuff really quickly but also make mistakes.
2:15:37
So my talk is all about that and some some ideas that we have around security for agenda.
2:15:42
>> Do you think uh agent security bifurcates along the uh consumer and b businessto business axis?
2:15:48
Do you think there's a discrete enterprise versus B2B layer?
2:15:53
It feels like we talked to the CEO of one password for example and it does feel like the like my the password to my Yelp account I might not be using like work OS for that in the future.
2:16:06
There's definitely going to be some blurring, you know, and and you know, talking about API >> I mean like like my my the way I'm thinking about it is like a small business might have 200 agents that are out in the world.
2:16:18
Maybe some are selling, maybe some are doing customer support.
2:16:20
It's like right >> when when should a CX agent be able to share >> account data, right?
2:16:27
When when when can a sales uh agent like provide pricing?
2:16:30
a sales uh agent like provide pricing? I mean like there's so many different things and you you know CEOs when they're working with teams right they have like processes in place for >> individual people and so I think I think this like really important problem area >> it's it's completely changing the way people think about security I think if
2:16:47
if you go to any of these like security focused conferences it's the topic on everyone's mind >> yeah exactly within Exactly yeah because wi within companies previously you've had these kind of silos of information or control you have permissioning systems that are pretty static but with agents you know you might have 200 200 today and zero tomorrow. You might spin
2:17:03
You might spin them up and down depending on a task, depending on a project.
2:17:07
>> And so that permissioning model is like completely changing and uh it's really exciting.
2:17:11
You know, we're right in the middle of it.
2:17:13
Like us working with all these different AI businesses, >> they themselves are building their own agentic workflows.
2:17:17
Whether it's, you know, stuff like codeex or cloud code or what cursor is building with their background horror story and security yet? >> Oh yeah.
2:17:27
[laughter] >> People can talk about it.
2:17:27
I mean there's like yeah when I think about or Defcon I think about like >> stuff it feels notable that there hasn't been like a specific day on the internet that everyone was like >> oh we went down because of AI that hasn't happened yet >> not not yet outage like I don't think no one no one pinned that on AI no one pinned that on on on generative AI or stochastic systems.
2:17:53
Yeah, there was one one a few months ago where uh Jason Lemin, you know, from Saster, he was vibe coding an app on Replet. >> Oh, I saw that. >> You remember that?
2:18:00
And he was like he was pure prompting, right?
2:18:02
He's not writing any code.
2:18:03
He's like just talking to the thing and uh he asked the agent to do something and it deleted the full production database. Yes.
2:18:10
>> And then he was like, "What the hell?"
2:18:12
And then the agent lied about it and was like, "No, I didn't do that."
2:18:14
You know, >> I think at some point the agent was like, "Yeah, >> like yeah, cover."
2:18:17
You know, it's like a kid like >> at one point I think the agent did just say like, "Yeah, you got me. >> My bad." Yeah, my bad.
2:18:22
Um, >> but but I do think they were able to roll it back.
2:18:24
I remember John getting in the thread and kind of just to close it out and not leave the >> Yeah, and they've built a lot of stuff since then as guard rails, but that just shows you like early on people pushing these systems to their limit and they can have catastrophic effects if you don't put up these guard rails.
2:18:37
So, that's what the talk's about.
2:18:39
We're doing a lot of innovation and research here, but it's uh it's going to take a while to get right. Yeah. Yeah. >> Well, >> awesome.
2:18:46
Amazing to finally have you on the show. >> Thanks so much. Been a long time fan.
2:18:49
It's great to see you guys so much.
2:18:50
Yeah, we can do it in person. Come back. Come back on again. >> I will. I will. Take care. See you.
2:18:54
>> We'll talk to you soon.
2:18:55
>> Um, >> there are a lot of posts here in the timeline that I want to share that we can't. >> We can't.
2:19:00
Uh, I mean, I'm being held back by >> I want to just come back to them tomorrow. We're being held back.
2:19:06
Uh, >> tomorrow you have our word.
2:19:08
We're doing lots of timeline.
2:19:10
If you're new here, leave us a subscription. Follow us on axe. Follow us on LinkedIn.
2:19:17
>> Subscription everywhere. It's free everywhere.
2:19:18
Sign up for our our newsletter, tvpn. com.
2:19:21
Uh we bring you the news in text form. >> Yeah.
2:19:25
>> Uh well, it has been a fantastic day here in San Francisco. Thank you to everyone. >> Stunning out. >> It is. I'm going to go try.
2:19:33
>> We got to go hunt that blimp. We got to find that.
2:19:35
It's not the Gemini blimp.
2:19:36
I'm telling you, it's Sergey's blimp. His personal blimp.
2:19:40
>> It's not a It's not a Gemini project.
2:19:42
It's not a Google project.
2:19:44
>> I thought you said it was branded.
2:19:44
I thought you said this branded funding a blimp company and they're testing.
2:19:52
>> So, it's so funny because I sat I sat down with Logan and the Gemini team and we were just talking about marketing ideas and I was like the obvious thing that you should do is get a blimp, wrap it with Gemini branding and just fly it around uh San Francisco >> is on top of it.
2:20:07
>> And and we were talking I was like, "Okay, finding a blimp."
2:20:08
I was doing some research.
2:20:10
There's like six active blimps.
2:20:12
I was like, man, this is going to be hard to find a blimp that can get to SF that can be wrapped and of course Google uh uh incredible foresight from Sergey to create an a beautiful billboard in the sky that's just waiting for uh branding.
2:20:25
But um >> waiting [laughter] well super fun day, a surreal moment.
2:20:31
>> Yeah, a lot of fun >> talking to one of the greatest living CEOs and um >> thank you to everyone on the Microsoft team.
2:20:40
Thank you to everyone on the GitHub team who helped organize this.
2:20:45
Thank you to our sponsors.
2:20:45
Uh profound linear numeralhq.
2:20:48
com sales tax on autopilot.
2:20:51
ai the number one AI agent for customer service.
2:20:56
Adio of course customer relationship magic.
2:20:58
Uh aid sleep didn't sleep on my aid last night.
2:21:02
Can't wait to get back to it tonight.
2:21:04
Of course we mentioned public. com also adquick. com. Getbzel.
2:21:10
Getbzel.com your bezel concier is available now source you watch the planet Jared Palmer I saw it I saw it he had a nice yacht master it was good >> that was looking good >> and of course on that >> find your happy place book a wander with
2:21:23
inspiring views hotel great amenities dreamy beds top tier cleaning and 24 having concier service it's a vacation home but better >> thank you folks we will be back in Hollywood tomorrow the podcasting will continue >> 11 a.m. m. Pacific. See you then. Goodbye. >> Bye.