"The AI models that you're using today is the  worst AI model you will ever use for the rest   of your life, and when you actually get that  in your head, it's kind of wild. Everywhere   I've ever work..."

"Famously, you led this project  at Facebook called Libra. Libra is probably the biggest disappointment of my  career. It fundamentally disappoints me that this   doesn't exist in the world today becau..."

"Today my guest is Kevin Weil. Kevin  is chief product officer at OpenAI,   which is maybe the most important and most  impactful company in the world right now,   being at the forefront of AI and AGI ..."

"This is a very special episode and I am so excited  to bring it to you. If you enjoy this podcast,   don't forget to subscribe and follow it in your  favorite podcasting app or YouTube. If you become ..."

"When I was at Airbnb, one of the things that I  loved most was our experimentation platform where   I could set up experiments easily, troubleshoot  issues, and analyze performance all on my own.   Ep..."

"can fight rapidly shifting fraud and meet new  waves of regulation. Whether you're a startup   or an enterprise business, Persona has a plan for  you. Learn more at withpersona.com/lenny. Again,   tha..."

"thing that's happened in AI, which a high bar  since, I don't know, ChatGPT launched. Just like,   did you guys expect it to go this well?  What does it feel like internally? There have been a handful..."

"so you know each other and you're experts in  your product hopefully. And so there's some   sense in which if you're doing something  social and it's not taking off internally,   you might question wh..."

"and we're going to go through every single one.  So let's just get started. I'm just joking. Cool. I picked out the best and there's a  lot of stuff I'm really curious about. Well, it's 1 PM here. It ..."

"AI and the current thing that we use every day and  is just a part of our lives, that's an algorithm. It's so interesting because in the Bay Area you  see self-driving cars driving around and it's so ..."

"another 10 minutes, you're bored, you're doing  email on your phone, answering Slack messages,   and suddenly this miracle of human invention  is just an expected part of your life from   then on. And..."

"If I'm remembering the timeline right,  we communicated at Planet I was leaving   and I was planning to just go take some  time. I wasn't going to stop working,   but I was also happy to take the summ..."

"the storm. Again, you work at a lot of, let's  say traditional companies even though they're   not that traditional, Twitter and Instagram and  Facebook and Planet, and now you work at OpenAI.   I'm c..."

"There's something fundamentally interesting  about that makes life fun here. There's   also something we will maybe talk about evals  later, but it also really, in this world of...   Everything we're ..."

"your database works once, it works every  time. And that's not true in this world. Let's actually follow this thread on evals. I  definitely wanted to talk about this. We had this   legendary panel at..."

"Is it a simple way to think about  it, like unit tests for model? Yeah, unit tests, tests in  general for models. Totally. Great, great. Okay. And then why is this  so important for people that don't ..."

"ChatGPT just like chug for you for 25, 30 minutes.  It's not the immediate answers you're used to,   but it might go work for 25, 30 minutes and do  work that would've taken you a week. So as we   wer..."

"I'm just going to keep following the  thread that you're leading us down,   but I'm going to come back because I have  more questions around some of these things.   So you came to a space that I think..."

"could ever do that ourselves. We don't want  to. We if we did want to, and we're really   excited to power that for 3 million-plus  developers and way more in the future. Coming back to your earlier p..."

"but he also understands that with moving fast  comes, we didn't quite get this right or that we   launched this thing, it didn't work. We'll roll it  back. Look at our naming. Our naming is horrible. ..."

"So we have this philosophy, we call iterative  deployment, and the idea is we're all learning   about these models together. So there's a real  sense in which it's way better to ship something   even ..."

"And that's how you make sure that you're really  pushing the envelope and building new things. I had the founder of Bolt on the podcast,  StackBlitz is the company name, and he shared   this story tha..."

"why the industry moves so fast, but all respect to  the other big model providers. Models are getting   really good. We're going to move as fast as we  can and I think we've got some good stuff coming..."

"What would you say is the most counterintuitive  thing that you've learned after building AI   products or working at OpenAI, something  that's just like, "I did not expect that?" I don't know, maybe ..."

"but it was also the first time that a model needed  to sit and think. And that's a weird paradigm   for a consumer product. You don't normally  have something where you might need to hang   out for 25..."

"individual human and it works, which I don't know,  maybe I shouldn't have been surprised but I was. That is so interesting because when I see these  models operate, I never even thought about you   g..."

"This reminds me of something else you said  at the summit that has really stuck with me,   this idea that chat, people always make fun  of chat is not the future interface for how   we interact with A..."

"universal because it is the way we talk. I can  talk to you verbally like we're talking now.   We can see each other and interact. We can  talk on WhatsApp and be texting each other,   but all of thes..."

"I'm excited to chat with Christina  Gilbert, the founder of OneSchema,   one of our long-time podcast  sponsors. Hi, Christina. Yes. Thank you for having me on, Lenny. What is the latest with OneSchem..."

"a built-in validation layer that stops  any bad data from entering your system,   and OneSchema will notify your team  immediately of any data that looks incorrect. I know that importing incorrect dat..."

"How many product managers do  you have at Open AI? I don't   know if you share that number, but if you do. Not that many, actually. I don't know, 25. Maybe  it's a little more than that. My personal b..."

"they're just going to see a problem and go  do it. It's just a core part of how we work.   I think people that are happy with ambiguity,  because there is a massive amount of ambiguity   here, it is n..."

"report to you, your team doesn't report to you,  et cetera, but you also have the complexity of   a research function, which is even more sort of  self-directed and it's really important to build   a ..."

"because there's so much ambiguity, it's not  obvious what the answer is in a lot of cases,   and so having a PM that can come in and... And  by the way, this doesn't need to be a PM, I'm   perfectly h..."

"I'm not doing a good enough job of that. Just to give an example, why shouldn't   we be vibe coding demos right, left and  center? Instead of showing stuff in Figma,   we should be showing prototypes ..."

"And of course the model makes mistakes or it  does something that doesn't compile, but when   it doesn't compile, you paste the error in and you  say, go, go, go, go, go. And then you test it out   an..."

"And so I think you're going to want sort of  quasi researcher machine learning engineer   types as part of pretty much every team because  fine-tuning a model is just going to be part of   the core wo..."

"There'll be parts of our internal stack.  I'll give you an example. Customer support,   with 400 plus million weekly active users,  we get a lot of inbound tickets. I don't   know how many customer su..."

"We're using... At various places. Some of these  places, you want a little bit more reasoning,   is not super latency sensitive, so you want a  little more reasoning, and we'll use one of our   O seri..."

"in those situations and use that to at least  gain a different perspective on the problem. Wow, this is great. Because some of this really is talking  to a model. There's a lot of prior art   because ..."

"And do you find that AI... There's a  lot of talk about AI tutoring. Is that   something you guys are doing?  I know they're using Chat GPT,   I love all the photos you post where they're  playing wit..."

"... a great thing over the longer term. That  doesn't mean that there aren't temporary   dislocations or where there aren't individuals  that are impacted, and that matters too. So   it can't just be ..."

"To give people a sense of where things might be  going. That's a big question in a lot of people's   minds. So someone asked this question that I love,  which is, "AI is already changing, creative wor..."

"thoughts and put something into the model  and suddenly have output that I couldn't   have possibly done myself. That's pretty cool. Even you look at folks that are really talented.   I was talking to..."

"much more creative approach with an outcome that's  much better. And I did that assisted by AI."  My personal view on creativity in general is that  it's no one's going to... You don't type into   Sor..."

"upon the world someday. We kind of just talk  about the things we're working on and share   when we can and launch early and often, and then  iterate in public. I really like that philosophy. I love a..."

"models have come in just a couple of years. If  you went back to GPT-3, you'd be disgusted by   how bad it was, even though Lenny of two years ago  was mind-blown by how good these were. And for a   l..."

"the future is going to be very different than  today. The thing I try and remind myself is,   the AI models that you're using today is  the worst AI model you will ever use for   the rest of your life..."

"asked about this. So famously, you led this  project at Facebook called Libra, which is now   called Novi. A lot of people always wondered,  "What happened there? That was a really cool   idea." I kno..."

"launch a new blockchain. It was a basket of  currencies originally. It was integration   into WhatsApp and Messenger, and I think the  whole world kind of went like, "Oh my God,   that's a lot of chan..."

"I was looking at the history  of it, and apparently,   they sold the tech to some private  equity company for 200 million bucks. Yeah yeah and- They had to buy it back. There are a couple of current b..."

"better way of riding, and it still feels like  the future. So they've done an amazing job. That's awesome. By the way, I had the founder of  Windsurf on the podcast. It might come out before   this or..."

"not notice it over a week or even a month. And  a lot of people then kind of get dismayed and   stop. But actually, you keep doing it. The gains  keep compounding. And over the course of a year,   two..."

"I use that tip all the time actually. I  always... When I'm coming up with questions   for interviews and I use it occasionally to  come up with things I haven't thought of,   I actually type, "You're..."