0:00
Often when I look at what people are building out there, it's just like, "You're not quite getting it."
Search this video
0:02
Like, you know, if you were just like pushing yourself and just really imagining like all of this being roughly 10 times better than it is today, you know, like in a year, but you would build in a different way.
0:13
>> What do you think people aren't pricing in in terms of where things are going?
0:16
>> The majority of actions on the internet will be taken by agents.
0:18
If you want your product to be successful for agents, you have to build, you know, for a certain level of scale.
0:23
>> How many agents do you find yourself running in parallel just uh in the in your kind of your day-to-day work?
0:28
>> As I'm pushing the frontier, I find myself like building larger and larger teams of agents.
0:31
And then when we have the next breakthrough with models, like suddenly it's just like I'm like, "Oh, well, you know, a bigger agent can just do all of it."
0:38
And so I kind of shrink the team again.
0:40
And then it just kind of goes through this expansion and shrinking expansion.
0:43
>> It kind of makes me think about there's this whole loops thing and then this whole graphs thing.
0:46
>> Having to set up and fiddle with your loops is something that maybe people got excited about, but I don't think this is the way that it's going to work.
0:52
Over time, you just want to system that learns.
0:55
You don't want to necessarily think about how I'm just like, "I'm going to loop it exactly this way in order to get good results."
1:00
>> there's still more radical change happening to how we work?
1:04
>> to continue to change quite radically.
1:06
Even today, it still feels like a bit clunky, and I think it will feel clunky until it isn't.
1:16
>> I mean, Evo, thank you for doing this. >> Yeah. Big fan of you. >> Of course. >> Uh >> Nice room.
1:22
>> You launched uh a ton of stuff today. How how did your sleep? How are you sleeping? Are you doing all right? >> I'm doing great.
1:26
I'm fortunate to have a very good team. >> Mhm.
1:30
>> So, while I sleep, they were awake most of the night.
1:32
We have this amazing room uh at the office called the library, which we completely repurposed as like a mega war room.
1:38
And the vibes in there are just immaculate.
1:40
Uh people were there until like very, very late.
1:44
>> So, you've been a long-time engineer.
1:46
Are you are you coding at all anymore? Are you shipping PRs?
1:48
Are you mostly like in docs and in meetings now? >> What is coding? >> Exactly. Yeah. Are you shipping PRs? I don't know.
1:56
Is that >> Uh Yeah, yeah, yeah.
1:56
Merch Merging some code every once in a while.
1:58
Technically, like a lot of code is getting written for me to do all sorts of kinds of like analysis, like you know, understand like trends, understand a business, understand the next feature, you know, how well our previous launches are doing.
2:10
A lot of code is written by Codex for dot.
2:13
You know, I'm obviously not writing it by hand anymore.
2:16
Occasionally, at the in during the weekends, I'll have like a little therapeutic moment where I do a little lead code just by hand.
2:21
Um but that's the only actual hands-coding that I do.
2:26
>> How many agents do you find yourself running in parallel just uh in the in the kind of your day-to-day work?
2:31
>> I used to run a lot more in parallel and then we were, you know, fortunate enough to get um a breakthrough with ultrafast.
2:38
And so, now, you know, I feel like you know, I'm able to be in the flow again and so it's just having a faster agent helps me a lot.
2:44
I'm excited that we're getting, you know, 6-month-old ultrafast out there soon.
2:47
Um and then, yeah, I think it's it's it sort of varies.
2:50
Um as I'm kind of like pushing different here, I find myself like, you know, building larger and larger teams of agents.
2:56
And then, when we have the next breakthrough with models, like suddenly it's just like I am like, oh, well, you know, a bigger agent can just do all of it and I keep everything in memory and learn.
3:06
And so, I kind of shrink the team again.
3:07
And then, it just kind of goes through this like expansion and shrinking expansion. >> Hm, okay.
3:12
Can you say more about that?
3:13
It kind of makes me think about there's this whole loops thing and then this whole graphs thing.
3:16
Is like an evolution of that in some way or >> I think like having to set up and fiddle with your loops and, you know, figuring that out is something that, you know, maybe people got excited about, but I don't think this is the way that it's going to work.
3:28
I think the way that it's going to work is a what how we're positioning and and what we're shipping with dots, where you have an incredibly smart agent that works 24/7, understands your goals, understands your preferences, learns from feedback.
3:42
Um and, you know, it's like what we launched is like, no, perfect.
3:46
I will learn very much from, you know, making this available to our pro users.
3:49
But over time, just want a system that learns from, you know, what you want to achieve.
3:55
You don't want to necessarily think about hours like I'm going to loop it exactly this way in order to get results.
4:00
So, right now there's Codex, there's ChatGPT work, there's ChatGPT consumer, there's dots.
4:04
What I'm hearing is you think we're heading towards dots kind of being the primary way you talk to AI, and that kicks off all these other things.
4:13
>> Yeah, I think fundamentally if you take a step back and, you know, whether it's dots or not, um it's all about breaking free from the technology um and having this sort of like permanent active intelligence that knows, you know, everything it needs to do and is available through any client, any screen.
4:34
You can call it, you know, like maybe you walk into a meeting room, it shows up in the meeting, it takes notes, you know, you pick it back up on email, um and then, you know, you text it if you need it.
4:44
It's not that, you know, you need to be glued to a laptop or, you know, you're like always on your phone.
4:50
It's just like it's available when you need it and then it's also like gets out of the way when you don't need it.
4:53
And I can't wait, you know, for that to sort of like come to fruition because I think like just carrying your laptop as like this brick around everywhere is just like, you know, you're kind of like tied to the technology instead of the technology working for you.
5:06
>> This episode is brought to you by our season's presenting sponsor WorkOS.
5:07
What do OpenAI Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all have in common?
5:18
They are all powered by WorkOS.
5:18
If you're building a product for the enterprise, you felt the pain of integrating single sign-on, SCIM, RBA, audit logs, and other features required by large companies.
5:28
WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS.
5:35
Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS.
5:42
And that's because they are the best.
5:44
Whether you are a seed stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unblocking growth.
5:53
It's essentially Stripe for enterprise features. Visit workos.
5:57
com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions.
6:04
WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.
6:10
com to make your app enterprise ready today.
6:17
I was going to ask you about this.
6:17
It feels like so much has changed in how we work over the past like 2 years.
6:20
In 2 years, especially engineers, but a lot of roles have just the role your day-to-day life has changed significantly.
6:28
And I'm curious if you think we're kind of settling into what work will look like in the next, I don't know, 5 years?
6:34
Let's say 2 years, that's feels way too far to think about.
6:36
Or do you think there's still more radical change happening to how we work in >> No, I think it's going to continue to change quite radically.
6:43
Even today, you know, it does seem like the technology is finally coming together with voice, uh you know, multimodal inputs, outputs, but it still feels like a bit clunky and I think it will feel clunky until it isn't.
6:56
And then you're just like, "Wow, you know, it's like I can just talk to this thing the same way that we're talking right now.
7:02
It just remembers things perfectly.
7:04
If I want to, you know, ideate and brainstorm, I can just like, you know, squiggle a thing.
7:10
I have this collaborative surface.
7:12
You know, this is what we launched with ChatGPT spaces signal to various startup dot.
7:15
You know, you can imagine we have like a shared whiteboard, we can collaborate between humans, between agents.
7:20
And I think it's going to sort of like transcend conversations, transcend, you know, like a lot of the clients that you have today.
7:26
And like, none of this is just really working today, right?
7:28
So, I think that we're going to go through this big new transformation.
7:31
What I'm hearing here is dots is a big part of that future, this AI assistant that just kind of does everything for you and you don't have to think about Codex versus ChatGPT and >> Yes, and for Codex and ChatGPT, we're going to merge uh like the the we have we have chat, we have the work toggle.
7:45
Like you know, we heard the feedback loud and clear.
7:47
Like you know, people love the capability in work, but then also you know, you're like sometimes you want to use chat because it's a little bit faster.
7:52
And you know, more pleasant to use.
7:54
And so we're we're merging that.
7:55
We're reducing that complexity.
7:57
Eventually, we will ship all of the capabilities of Docs, you know, straight into ChatGPT as well and you know, lift the floor for like our 1.
8:03
2 billion users, which you know, at that time we'll see like how many we have.
8:06
Um and we want it to be really really seamless.
8:09
One thing that I'm very excited with with Docs is like they don't have a model picker.
8:14
There's just no configuration. You just talk to it.
8:15
Like the only thing that you have to configure is like which channels you want to talk to.
8:20
>> It feels very much like the Her kind of vision of the world. I don't know.
8:22
Do you think about that movie much?
8:24
Just this idea of this >> I watched it once. Yeah.
8:28
I think it's it's it's interesting, you know, to to to think about, you know, maybe more what what I think about more is like, you know, sci-fi from from, you know, 30 40 years ago and like how visionary it was.
8:40
>> Is there specific sci-fi you think about that's most influenced you?
8:43
>> Like you know, books like Neuromancer and then you know, the original like Star Trek.
8:48
Um you know, I think about that a lot.
8:49
And there's something about Star Trek which I think, you know, was like there was this technology that was like very very useful.
8:54
Obviously like, you know, you were able to just like talk to a computer and like it does things for you.
8:59
Like you can talk to your spaceship and it just moves, right?
9:01
It's like this is getting real, you know, like we're entering that era.
9:05
>> So we're recording this in front of a live audience.
9:07
There was this line out the door just people waiting to see you, to hear from you.
9:12
How do you just think about this, I don't know, role you're in, this responsibility you have to build what you're building and the impact it has on people's lives. I don't know.
9:19
Just what does that feel like?
9:20
How do you think about that?
9:22
>> I think it's very important to be part of a community and to build with people.
9:25
It was like, you know, we're we're discovering this technology and like what we can do together.
9:31
Um and so every time I you know, talk to people, I'm like I'm kind of like reminded of you're doing this thing which I hadn't even anticipated.
9:37
You know, it's impacting your life in a certain way and it's impacted like you know, this other person's life and then it's just like all together.
9:43
It's both very humbling, very inspiring, and I think very necessary in order to build something that is truly useful for humans.
9:50
I I don't know how we would do it without the community.
9:53
>> That's something I hear a lot with people building an AI is they don't know what it is until they've launched it and they see how people use it, see what emerges.
9:59
And it feels like kind of what you're saying is just build it with people versus like here's the vision, we've got it figured out.
10:05
>> Even the Twitter community has been very nice in general.
10:09
>> You're very good at Twitter.
10:11
Kind of antagonistic sometimes, but very good.
10:13
How do you How do you find time to even tweet?
10:14
I don't understand how someone in your role has time to sit on Twitter, tweet, reply.
10:19
>> I spend like maybe half an hour per day.
10:21
Yeah, just say rolling and then doing something when I feel inspired and I have something interesting to say.
10:24
Often it just comes like in the moment. Yeah.
10:28
>> Perfect Twitter skill.
10:29
>> And I have a We haven't released the dot the the the team dots yet, but you know today we're releasing the primary dot which you can configure, you can get used to like what a dot can do, connect it to your apps.
10:41
But we'll release the ability to create more than one.
10:42
And so I have one that's responsible for Twitter.
10:46
>> Okay, say more about that.
10:46
So right now everyone gets one dot. >> Yeah.
10:50
>> Right now they can't add more dots. >> That's right.
10:52
>> They will be able to have many dots.
10:54
Talk about kind of where that's going.
10:55
>> Yeah, so we wanted to start with one, the primary dot which is the one that you know, you'll probably also text that you know, learns your preferences most deeply as well and kind of see like you know, what people do with it, see how we
11:07
need to tune the system, you know, learn that working with the community and then very quickly soon you'll be able to add you know, a second one, a third one, a fourth one, as many as you need to create like your virtual team. And then
11:16
And then you can give them specific roles, right?
11:20
Is it It's not I don't find it absolutely necessary, but sometimes I have like a thing that is requires quite a bit of work, you know, such as like monitoring like Twitter for me.
11:27
And then you know, that's It's a thing is like like, this will be like so much work that is like the work of an entire dot.
11:34
Um and then you know, you can have a couple of those.
11:36
>> So if you think about all the launches that happened today, it feels like dots maybe like the sexiest.
11:41
I'm curious if there's something that you think is not getting the attention it probably deserves that will become a really, really important down the road that's kind of people are sleeping on that's like a vision.
11:50
>> Yeah, yeah, yeah, important.
11:51
>> the sleeper hit is ecosystem. So opening it all up.
11:54
Uh and the commitment, the deep commitment we have towards opening it all up.
11:58
So we have signing with ChatGPT 16 partners.
12:01
Very, very proud of that.
12:03
Like we started that like really it was sort of like in the moment last year.
12:07
Um I was talking to creators of Pi and open code and it was like, of course, you know, you should just be able to use the critics sign on and then you know, you use the usage and then you just we kind of like shook hands, you know, like virtually we're like, you know, it's just like use this this uh this off um and then you know, kind of trust you to not do anything shady.
12:25
You know, that grew obviously that became quite popular and then you know, now we made it, you know, an actual thing that was supported with many, many partners.
12:30
Um and I'm excited to grow that very quickly.
12:32
You know, the other side as well is like opening up all of the infrastructure and how we build for ChatGPT, you know, with plug-in extensions, also plug-in discovery, just allowing, you know, everyone to ship something through like maybe 1. 2 billion users, right?
12:45
You know, and benefiting from that distribution.
12:48
And we also have shared economics um that we didn't actually talk uh in the keynote, but you know, we will pay, you know, our plug-ins that are uh popular and you know, are seeing a lot of usage.
13:00
Uh they're going to get part of like the the revenue share as well.
13:03
Uh and so I think this commitment towards an open ecosystem is going to be very exciting. >> That's amazing.
13:07
Uh okay, so as an example, so say Notion, somebody's using the Notion plug-in or the Figma plug-in within uh ChatGPT work, Notion, Figma make money from people using it within the app.
13:19
>> Yes, so um we have, you know, all these subscribers that use ChatGPT, um they get like, you know, a certain amount of usage and then when have that, the when when the users use that usage with the plugin, or within the other product when they use sign-in with ChatGPT, um you know, we will have like some shared economics with them where they will get to remunerate.
13:38
>> One of the, I don't know, things I think that gets people excited about plugins and ecosystems is distribution platforms, ways to kind of get out there.
13:44
People discover my app, it blows up.
13:47
>> any advice for people that want their plugins to, you know, bubble up, to get discovered? >> Build a good plugin. >> Yeah.
13:53
>> The way that it works and we'll obviously tune uh the system, but we look at retention uh numbers, uh we look at um you know, how successful the plugin, the quality, and then you know, that's what we then start to recommend to users uh in conversations.
14:06
And so, your plugin can just like you know, get recommended to like you know, a significant slice of users.
14:10
Obviously, if your plugin is not quite good, they will stop being recommended. >> Right.
14:15
That's amazing that you're looking at retention of people using that plugin. That's right.
14:18
Um is it is it actually adding utility?
14:19
Is it, you know, enabling ChatGPT to do something else? >> Got it.
14:23
So, it's it's not so much an AEO kind of get the right words and get people writing about you.
14:27
It's people using it consistently, sticking with it. That's right. I think.
14:30
Okay, so you think that's kind of the sleeper hit that's going to become a big deal.
14:32
And I know this is kind of the second attempt.
14:35
You guys had an app marketplace that This one is the right one.
14:37
Okay, this is This one's going to work.
14:39
Maybe going back to dots cuz it feels like that's a big part of the vision in the future.
14:43
When did you guys start working on this?
14:44
Obviously, there's a open claw, a big open claw moment.
14:46
You guys Peter joined and there There's a foundation, uh and then Grok Bot, Muse, and Instinct, and all these things.
14:51
I guess how long have you guys been working on it?
14:54
Why did it take so long for you guys to get something out?
14:57
>> Do you remember a Codex Cloud?
15:00
>> Yeah, the first uh yeah.
15:01
>> That was like a year ago, right?
15:01
And uh if you look back at Codex Cloud and you look at the little animation that we used, uh I think that was the inspiration for Grok Bot. >> Um >> Yeah.
15:11
It's just like very, very similar.
15:13
>> Yeah, really similar, I would say.
15:14
>> But, you know, seriously, like on the research, long, persistent, long horizon task, this is something that we've worked on, you know, for more than 2 years.
15:21
Memory systems, that's like coherent, you know, like roughly the same amount of time.
15:25
We launched a lot of that straight into ChatGPT.
15:26
I think, you know, people just really enjoy that ChatGPT gets to know, like, you know, so much relevant information about you.
15:32
So many people I talked to have like an anecdote of like, "Hey, you know, I fell ill and, you know, ChatGPT remembered actually, like, you know, I had this barbecue and, you know, I had like lime and I was like making tequila and like, you know, it was just like probably like this burn that I have on my hand is actually like this lime, you know, it's like in some burn."
15:50
And he's just like, "How did it know that?"
15:51
And it's like, you know, it's just the memory works very, very well in ChatGPT.
15:53
And so, we've put a lot of research in that.
15:55
And, you know, all of this coming together on top of like the Quora harness and also the ability to just like run 24/7 with, you know, good productivity.
16:03
That's what we launched with dots.
16:05
And so, it's the work of like many, many months.
16:06
And then, one of the important things was also getting safety and security right.
16:11
You know, this is also why we launched with Astra, it's our safest, most aligned model.
16:14
And like a lot more effort went into like making dots, like, you know, just really safe and secure.
16:19
>> A question I like to ask people who come on the podcast is, where do you think human brains will continue to be valuable over time?
16:25
Where do you find yourself being necessary in the work that happens versus where AI is taking on more and more?
16:32
Where do you think human brains will continue to be useful, most useful over time?
16:37
>> Yes, I I think this is very much a function of how we build the technology in the way that we design everything at OpenAI is like to put humans at the center of it and build it as extensions of humans of of of your will and your taste.
16:49
And so, just really be like this super empowering thing.
16:53
Um, I think as long as we continue to do that and it kind of like enables you to just take whatever you wanted to do and, you know, your creativity and your taste and sort of like, you know, expand that, you know, in in a way that feels like awesome in the moment.
17:06
It's just like really this artistic tool, right?
17:07
And we may we may not have coders anymore, but we have more builders than ever.
17:11
And I think there's something that's going to remain, I I know, human about that.
17:14
Like humans want to learn and see, you know, what other humans are building and you know, I'm like much more interested I'm here I'm you know, I'm not talking to Dot you know, I'm talking to you, right?
17:24
I think that will remain true for like a very very long time.
17:26
>> One of the things that has emerged with engineers especially is in in how their lives have changed is there's a lot more context switching.
17:34
There's kind of this trend of loneliness that has emerged where they're talking to agents all day instead of other humans.
17:39
I'm curious how much you think about just like that part of the of the impact AI has on people's lives and just how you might Is there solutions to that?
17:47
Is there stuff you think about to make that less annoying? >> Yes, all the time.
17:50
Reducing, you know, configuration fatigue is, you know, one.
17:55
Um reducing the fact that talking to an agent is like sort of like a solo adventure and you just have this, you know, conversation just with with your one agent and then, you know, you have like many and then you're delegating so many things.
18:06
A lot of things are going to come together to make it a much more delightful and something that I spend a lot of time the team spend a lot of time thinking about.
18:12
So, I think, you know, >> For me, the ideal way to get things done is, you know, would be like to just have it in your physical space.
18:20
You know, we're just having conversation like we have now.
18:23
It's able to sort of like, you know, observe, you know, like hear the ideas that we have.
18:26
We can, you know, jot something down in a white paper on a on a paper, you know, I can maybe like, you know, take that and you know, start building in the background and then you're like, you know, actually have another idea and then you just like it starts building another thing.
18:36
You can just project it on a screen and then you just talk to it and it's just evolving in a conversation, you know, with other humans and you don't have this fatigue of like having to be like, you know, on this like screen and you're just like, you know, thinking about prompting.
18:46
It's just like becomes like super natural.
18:49
That's what we're trending towards, you know, I think we didn't fully get there today, but you know, we'll get there in the future.
18:55
>> One of the other kind of downsides that's emerged along those lines is just this kind of pressure to do more because we can do more.
19:00
Everyone's just like, come on, run 30 agents at once.
19:01
Why aren't you shipping more?
19:04
Everyone's shipping more.
19:06
Do you think that's solvable?
19:06
Do you think that's just kind of, I don't know, human nature?
19:08
We can do more, let's do more. Something there.
19:11
>> I think we we we heard Sam as well, you know, about the promise of of this as well, which is reduced the noise and you know, allow you to spend attention where you want to spend attention and you know, having like all these things
19:24
that you know, like maybe you know, maybe important and maybe not important but they're kind of vying for your attention and it's just sort of like tone that down a little and you know, just get you to focus on on on the things. I find it remarkable like every time I
19:36
I find it remarkable like every time I go on a holiday and you know, I I I I take that one week uh to just fully disconnect like you know, I start to thinking like different and more creative ways.
19:44
And I'm very eager to see like you know, can we bring that?
19:47
You know, can we just make this your day-to-day like you know, maybe you need fewer meetings.
19:51
You know, maybe you don't need you know, to do as much and you know, you would actually be more productive if you're like you know, better rested.
19:57
And so you know, we will have to you know, I think as an industry figure that out.
20:01
Um but I think that's the promise of AI and it's not you know, just one more prompt you know, per second.
20:09
>> There's a bot that I'm building right now that's kind of this energy audit bot that watches my calendar and asks me just like was this giving energy taking away energy?
20:15
Is it something someone else could have done?
20:17
I feel like you know, AI should be like hey Lenny, maybe maybe you can cut out these things so you could be a happier person.
20:22
Is there anything you've done really interestingly with AI recently that's just like a really cool use case of wow, that really was amazing.
20:27
It blew my mind or someone you've heard.
20:31
>> I can do was like something cool because the the live demo just failed, right?
20:35
You know, in front of everyone.
20:35
So there there was like okay, well, it's not so cool.
20:38
But actually uh my my my daughter was you know, it's like realized I was you know, I defected like you know, we have chat GPT production um went down and so it pinged me 5 minutes before the live demo.
20:51
Which was actually quite stressful because it was like hey, production is down.
20:55
And uh it's like oh well, you know, it's like do you want me to try and fix it?
20:58
And I'm like um I don't think you're there yet, you know, little dot but you know, thank you for trying and then you know, then you know, I got in touch with like the engineering teams and we started to look, you know, what was going on and you know, they did it's like fixed by now.
21:12
But, having this thing that just understands like okay, there's a pretty important thing happening. It's called DevDay.
21:18
There's like this production system. There's a live demo.
21:21
It's probably using this production system.
21:23
So, these two are things are connected and this is happening in 5 minutes.
21:26
So, I should ping him because like he probably wants to know about it.
21:28
I think that's quite remarkable. That is remarkable. Wow.
21:31
So, it just knew that this was coming and told you.
21:35
And I love that it wanted to fix it and you were like, not quite.
21:37
You're not quite there yet.
21:38
I don't think here's like, you know, please go impress me, but you know, that would have been a better story.
21:42
How do you feel about just giving access to all these things? I don't know.
21:44
Like did does it have access to the production code base and like, you know, It has access to some some of production systems with guardrails.
21:53
You know, so we we built We talked about specialist dots, but specialist dots are just like operated with additional guardrails, additional monitoring, and then, you know, on their own hardware.
22:04
So, we actually run some on Mac minis.
22:06
The fundamental difference in the way that we've built dots is like the harness does not run on the machine.
22:12
You know, and then this is like maybe people have not realized is like it can connect to as many devices as you want.
22:18
So, it has its own computer.
22:18
You can actually connect it to your own laptop as well.
22:21
Over time, you might connect it to like 10 different devices and it can control it all.
22:25
A little bit like an octopus.
22:27
So, you're basically talking your dot, it's living in some VM somewhere and it can Yeah.
22:29
Might be a VM, might not be a VM.
22:32
It's just like it's a thing, you know, and it can connect to many devices.
22:36
>> This season's supporting sponsor is DX.
22:39
Now that everybody at your company is finally using AI, the question that engineering leaders face is no longer are we using AI, but is it paying off?
22:49
With DX, you can measure and benchmark the impact of AI adoption across your full product life cycle.
22:53
DX shows you where AI is helping developers and where it's creating new bottlenecks.
22:58
You can also use it to evaluate AI vendors, monitor costs and licenses, and see exactly what's limiting your agents.
23:08
That's why hundreds of enterprises, including Snowflake, Sony, and BNY, use the DX platform to measure AI's impact on developer productivity. Visit getdx.
23:18
com/lenny to get a demo of DX. That's getdx. com/lenny.
23:26
So, they're really hiring a lot.
23:26
I know you're hiring a lot.
23:30
In your interviewing, what have you seen as skills that are kind of trending up in what you look for that is more and more important in people being successful now?
23:38
And what skills do you find are trending down that are just like, okay, we don't need that as much?
23:44
>> Yeah, I would say that skill that is trending down is typing fast.
23:45
No, that is not that useful anymore.
23:48
Skills that are trending up is, you know, just great taste, um thinking about the user, connecting to the audience that you're building for.
23:57
Um a lot of X founders are incredibly successful right now or like you know, there's like repeat founders.
24:03
We're also finding this like you know, when we're hiring at OpenAI, like you know, we have a lot of founders.
24:07
I think we have more than 120 XYC founders now currently at OpenAI.
24:12
So, it's like this mega startup.
24:14
And I think just this um this passion and this envy to build something that matters and then know what good looks like is more important than ever.
24:19
I've been arguing for a long time that PMs are kind of going to thrive in this time because that's basically the job of a PM.
24:26
Uh you know, they're like, where should we build? Help someone build it. And then is this right? Is this great? And iterate. So, uh I don't know. I agree with that. >> Yeah. >> All right.
24:35
>> Yeah, also roles are blurring.
24:35
So, you know, you might you might have been, you know, only into design or only an engineer and um you know, felt always a little bit uncomfortable, but you know, now it's just like it's your time to shine.
24:45
>> Do you miss the engineering part of the job?
24:48
I I was an engineer back in the day.
24:49
I was an engineer for 10 years. Little did you know.
24:50
And you know, there's this flow state.
24:52
There's this beauty of just building and just seeing it work and not work. Yeah.
24:55
Do you kind of missed out at all, or you're just like, "Okay, this is my old life."
25:00
>> I I I missed out for for a while, and then now with, you know, really fast speeds, I'm finding that back.
25:08
Uh and so, I'm like really excited to democratize that uh as much as much as possible.
25:11
It will take a while for it to be like, you know, absolutely ubiquitous, right?
25:14
And like, you know, available to like more than a billion users.
25:19
But now, you know, with with ultra-fast speeds, like we're able to achieve it for a 6-month sole is like, you know, like costs roughly the same as Astra.
25:26
And so, the progress there is like quite astonishing, you know, even even for us internally.
25:30
We're like, "Wow, you know, we can do these things."
25:31
Also because we are using Astra, and we're finding, you know, like we we can push it super far.
25:36
And when you have those speeds, and especially when you can able to like voice control it, you know, you kind of get back into this like really creative state of mind and endless flow.
25:43
It is different from the flow of the past the past, um but I don't miss it.
25:49
>> Yeah, it's like your last you launched videos, just people kind of standing around talking to their AI and just cooking or >> That's right.
25:55
>> sitting on a lounge chair. >> Yeah.
25:56
I asked people while I was kind of walking around the the event what to ask you, and one thing that came up a few times is advice for new grads, kind of people junior in their career.
26:04
I think a lot of people look up to you.
26:06
They want to be the next Thibault.
26:06
What advice do you have for folks kind of early in their career in terms of what to work on, what skills to build, what to do? Yeah.
26:13
I think the next Thibault is already at OpenAI.
26:14
Uh his name is Ahmed Ibrahim. I love working with him.
26:17
He was a new grad uh when he was hired at OpenAI.
26:22
He's now responsible for all of OpenAI compute fleet and applied uh which is a massive massive responsibility.
26:28
I've built a lot of the Codex harness.
26:30
And the thing that has made Ahmed um really stand out and successful is just his He's incredibly kind.
26:37
He's incredibly collaborative and always, you know, tries to solve the problem that is like really important, um but like, you know, a way that, you know, he's not putting himself first. And he is just a sponge.
26:49
He's like learning so fast and learning about everything, and you know using all of the latest technology to you know learn at you know speeds I haven't seen before.
26:57
Um and so he just in the span of like you know a couple of months and and and a year, you know, it's like his progress uh at the company was just absolutely fascinating to see.
27:06
Uh he's now someone I trust a ton uh in doing like you know the the the most gnarly launches at OpenAI.
27:13
All right, I got to get him on the podcast. >> You should. You should.
27:17
So kind of thinking about how things work at OpenAI, what would surprise people about what it's like to work inside OpenAI?
27:24
From the outside it feels like it's just like constant shipping code, things are kind of like, you know, chaotic but awesome. I don't know.
27:30
What would surprise people about what it's like inside at OpenAI?
27:34
>> I don't know if that's surprising but it is, you know, as I said, a lot of ex-founders um doing a lot of bottoms-up, you know, very exciting uh ideas.
27:42
You know, for example, decisions API, you know, came together very quickly.
27:46
We realized like we have a really good model with Luna.
27:48
Uh we can just do some, you know, constraint sampling and then, you know, have it, you know, shipped as like a different shape to the responses API and then um you know, when we do that uh it, you know, it's like it's faster.
27:59
It's quite delightful so for some cases and it's just if you were to like peel under the hood, you realize like, okay, there's a Slack channel that came together.
28:06
Um you know, initially it was like four people kind of hacking on it on a weekend and then it was made as accessible as like, you know, company food and then people got excited, started to build things and they're like, "Oh, this is, you know, we can support visual inputs.
28:18
Like this is, you know, even better than whatever is out there."
28:21
And then, you know, people get more and more excited.
28:22
People start pitching in.
28:24
And the way that is like, you know, feels completely unstructured and then um you know, take it all the way and like, you know, ship something.
28:30
And then uh we try to hold like a high bar for, you know, the quality and then you know, stop things, you know, before they go out or like, you know, maybe they need a little bit more baking.
28:38
There's many more things that, you know, we were maybe going to ship today at DevDay and then we're like, "Okay, like I was like I think this is a a already.
28:46
Let's kind of hold it back a little bit and you know, space it out.
28:48
So, we'll have more launches next week and and and and the week after.
28:51
But, it's just this incredible like bottoms-up energy and then, you know, people kind of assuming roles to kind of channel that in productive ways.
29:00
>> Something that's very clear and unique about OpenAI is it feels like you have a lot of autonomy.
29:05
You're just like pressing the reset button whenever you want.
29:06
You're just saying things on Twitter.
29:08
Uh it feels like a really unique culture where you have a lot of autonomy and I imagine there comes a lot of trust.
29:13
Maybe you can speak to just how that cuz I feel like that's an advantage at OpenAI.
29:16
It allows you to move faster, just that cultural philosophy.
29:21
>> You got a lot of autonomy and then, you know, you have to just own up to it.
29:23
You so, you know, we just kind of trust people to make great choices.
29:29
And then, you know, when things go wrong, uh you know, fix fix it quickly or, you know, learn from the mistake. So far, so good.
29:35
You know, it's it's just really worked as a very empowering um environment.
29:39
It is true that I can press the the reset button whenever you know, is necessary or whenever it feels right.
29:47
And, you know, that's like it that's a privilege.
29:49
Uh and also, it allows me to, you know, just really be close to the community in ways that I think, you know, would not be otherwise possible, right?
29:56
I don't have to run it, you know, through an echelon of approvals.
30:00
>> You said sometimes people screw up.
30:00
Is there something you screwed up in that in that I don't know, that the amount of autonomy and trust you have, something you messed up?
30:06
>> Yes, I I think at times um I could have, you know, made teams like, you know, more inspired to build like things that are less complex uh and just really continue to strive towards uh complexity.
30:19
I think we, you know, we early days uh in Codex, you know, we had, you know, a couple of like outages that I caused.
30:25
Very very early early is for me at OpenAI.
30:27
I took production down like day three.
30:28
You know, I just I was still employed after that.
30:31
Um and I learned my lesson.
30:34
>> One of the other questions people wanted me to ask you is how many resets should they expect in the next, I don't know, month or week?
30:40
>> Depends on times you break things.
30:42
>> Okay, so that's the philosophy.
30:42
Break something you reset.
30:43
That's kind of a >> Yeah, break something, celebrate things.
30:47
So maybe zoom in a little bit.
30:49
What do you think people aren't pricing in in terms of where things are going and what will change that they're not just like seeing as clearly as they should?
30:57
>> There are so many so many things that I feel are not yet priced in.
31:00
I think the majority of actions on the internet will be taken by agents.
31:02
Models are going to become more cheaper and faster at rates that are quite incredible.
31:09
We will finally be able to integrate all modalities together in a way that is very seamless.
31:16
I think, you know, those three I feel like you know, often when I look at what people are building out there it's just like not, you know, just like you're not quite getting it.
31:22
It's like you know, you're almost there but like you know, if you were just like pushing yourself and just really imagining like all of this being roughly 10 times better than it is today, you know, like in a year.
31:33
It's like, you know, you would build in a different way.
31:34
>> Do you see any kind of second order effects that come from that from this world where most actions are being Uh from agents.
31:39
Agents are probably most traffic's going to come from agents. I don't know. >> That's right.
31:43
>> You think about some second order effect that emerges out of that?
31:46
>> Yeah, there there there there are many.
31:48
Um you know, for for first of all, you know, if you want your product to be successful for agents, you know, it's like you have to build you know, for a certain level of scale.
31:56
Um we've worked very closely with Notion for example and like when they build their MCP, you know, suddenly it's like, oh well, you know, it's available to all these agents that you know, can actually do do the work and like use that MCP and so they saw like a ton of traffic kind of come in.
32:07
Um and that puts obviously a lot of strain on the system and you have to figure out the economics of that.
32:13
And so there's like this tension, you know, between like you know, if you're building products is like you know, do you build an interface or not?
32:18
And you know, you can kind of hold that back for a while but it is inevitable, right?
32:21
It's like you know, you're the majority of things are going to be used by agents.
32:25
Building towards that future I think is very important.
32:29
The other thing is, you know, on the flip side, you know, building like you know, absolutely delightful new experiences for humans that you know, really benefit from older modalities is something that I you know, I feel like is under invested in.
32:42
>> Is there something you've changed your mind about in the past uh let's say year?
32:45
Just something that you used to believe that you see differently?
32:49
>> I was expecting us to reach the level of capabilities the of the models that we have today, you know, say like Astra, you know, maybe in a year or two.
32:59
Um and so I had to sort of like you know, revise my my prior order.
33:03
Also, I did not expect to rely so much on voice.
33:09
Um and you know, I do so much through dictation or just calling my agent.
33:15
And I had not anticipated that, you know, how much how much better that feels.
33:19
So like I I had to change my mind there.
33:21
And then, you know, you talked about hiring and you know, one thing that I hadn't realized is like the incredible talent and energy and you know, the just you younger generations have um and how they would be the ones embracing like all this change first and like figure out how to harness, you know, all of it.
33:38
And so, you know, I had to change also my opinion on the hiring and the strategy there.
33:43
>> The just to double down on that, just this idea that new people new in the workforce actually have an advantage because they haven't worked in a certain way and they could just go all in on AI. >> Yeah. Yeah.
33:52
And I think I just also the ability to just like absorb and learn like super fast.
33:56
>> So you talked about models getting really smart fast.
34:00
Uh obviously there's a lot of discussion these days about what AI can do in the future, the dangers AI poses to humans, P doom, and all these things.
34:09
Uh you guys have slowed down some training.
34:11
Uh you guys everyone's pacing the frontier.
34:14
How do you think about this world of AI and the potential risk to the humans?
34:19
How do you think about that?
34:21
>> For me like pacing the frontier is just really investing ahead, right?
34:22
And also investing way more, you know, than we even think, you know, like we ought to.
34:29
Uh investing in the alignment and safety of the models, investing in uh security, investing in the guardrails.
34:34
We are spending more and more compute on sort of like secondary monitoring.
34:39
So, you have you know, you have all of the computer that's going to the primary system, you know, the agent that is like doing work.
34:45
And then you have all of the computer that's going into monitoring, you know, the primary agent to make sure that it's like not taking too high risk of actions or like not interrupting it if anything looks like you know, maybe it's getting prompt injected and just like you know, intervening with that.
34:58
And the majority of our investment on the API stack is like not actually going into the safety stack.
35:02
And so to me, you know, that is pacing.
35:04
It's like you know, you want to make sure that you know, you're extremely hardened. That's what we're doing.
35:09
Like we haven't yet released the next step up in capability beyond Astra.
35:12
Um we have released, you know, a level that is like similar, near Astra intelligence, but like you know, much more efficient.
35:18
And so we feel very comfortable about that.
35:19
But we're going to continue to like invest a ton there.
35:23
And I feel uh I feel very good um about our approach so far.
35:26
Um I think you know, it's like it was also in the news that you know, we had six one Astra and then we didn't release it.
35:32
Um and that is something I'm very proud of.
35:36
>> So, what I'm hearing is you're optimistic we will solve these problems of alignment and AI doing really bad things.
35:40
AI's been doing a lot of really really s- not good things lately with hugging face and all that stuff.
35:45
But what I'm hearing is uh optimistic we'll solve it. It'll be all right. >> Yes.
35:50
And it's also it's it's I feel, you know, obviously a deep responsibility, but also just, you know, if you think about the incentives for OpenAI, right?
36:00
you know, we build products for 1. 2 billion people. We can't screw that up. Right?
36:04
You know, we take it very very seriously.
36:06
Um what we put in the hands of so many people, you know, including people who are not very technical themselves, you know, you have to get it right.
36:12
And so, you know, we we don't gamble there. >> Final question.
36:17
Uh uh What's something that annoys you about the current state of the app that you're just like, "God damn it, we got to fix that."
36:22
Uh current state of the app.
36:22
Um I feel like the the the models are almost there, but not quite for the app to almost completely disappear.
36:30
And you know, I can't wait for you know, to just really get to this like super essence of simplicity.
36:36
I myself even get fatigued with the model picker and the reasoning efforts and you know, whether to use multi-agent or ultra or you know, what it even does and you kind of need a PhD in model pickers. Right?
36:49
Um and I just want to get rid of all of that um as quickly as we can. >> Amazing.
36:56
Ebo, thanks for doing this.
36:57
>> Yeah, thanks for having me. Yeah.
37:00
>> Thank you so much for listening.
37:00
If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
37:08
Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.
37:12
You can find all past episodes or learn more about the show at lennyspodcast. com.
37:19
See you in the next episode.