All Segments
How OpenAI Builds for 800 Million Weekly Users: Model Specialization and Fine-Tuning
12 segments available
In this episode, a16z GP Martin Casado sits down with Sherwin Wu, Head of Engineering for the OpenAI Platform, to break down how OpenAI organizes its platform across models, pricing, and infrastructure, and how it is shifting from a single general-purpose model to a portfolio of specialized systems, custom fine-tuning options, and node-based agent workflows. They get into why developers tend to stick with a trusted model family, what builds that trust, and why the industry moved past the idea of one model that can do everything. Sherwin also explains the evolution from prompt engineering to context design and how companies use OpenAI’s fine-tuning and RFT APIs to shape model behavior with their own data. Highlights from the conversation include: • How OpenAI balances a horizontal API platform with vertical products like ChatGPT • The evolution from Codex to the Composer model • Why usage-based pricing works and where outcome-based pricing breaks • What the Harmonic Labs and Rockset acquisitions added to OpenAI’s agent work • Why the new agent builder is deterministic, node based, and not free roaming Timestamps: 00:00 Introduction 8:36 Horizontal vs vertical OpenAI 12:18 Why you can’t “disintermediate” the model 15:11 People build relationships with models 17:30 Not one AGI model, but many 20:10 Fine-tuning, RFT, and customer data choices 24:44 Prompt engineering isn’t the point anymore 28:06 What an “agent” really is 31:55 How OpenAI thinks about pricing 36:46 Why open-weights don’t kill the API 42:57 Different stacks for text, images, video 45:47 How the agent builder actually works Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: [https://x.com/a16z](https://x.com/a16z) Find a16z on LinkedIn: [https://www.linkedin.com/company/a16z](https://www.linkedin.com/company/a16z) Listen to the a16z Podcast on Spotify: [https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX](https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX) Listen to the a16z Podcast on Apple Podcasts: [https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711](https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711) Follow our host: [https://x.com/eriktorenberg](https://x.com/eriktorenberg) Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details, please see [a16z.com/disclosures](http://a16z.com/disclosures).
Segments Timeline
Introduction
"We want chat GBT as a first party app. First party app is a really great way to get 800 million wows or whatever now. >> Tenth of the globe, right? [laughter] >> Yeah. Yeah. 10% of the globe uses it >..."
Horizontal vs vertical OpenAI
">> Yeah. So, one place I wanted to start um is uh something that I find very unique about um OpenAI uh is it's both a pretty horizontal company like it's got an API like I would say we've got this mas..."
Why you can’t “disintermediate” the model
">> which is the the the problem historically with, you know, offering um a core services and APIs, you can get disintermediated, right? And so I can build on top of it, but then you know the user does..."
People build relationships with models
"is because of a relationship between the user and the model or do you think it's more of a technical thing which is like my eval work for like open AAI and it's you know and like the correctness maint..."
Not one AGI model, but many
"does that mean to a for AGI? [laughter] And the second is what does that mean for OpenAI? Like does that mean that like you end up with a model portfolio? Do you select a subset? Do you think this all..."
Fine-tuning, RFT, and customer data choices
"towards kind of more sophisticated use of things like you know like fine-tuning um which you know in a way you could read that as a bit of a capitulation that like you know there is product specific d..."
Prompt engineering isn’t the point anymore
">> Okay, you you said that views on prompt engineering have changed. >> Yeah, I wasn't actually I wasn't aware of that. All the other things I was aware of this one I wasn't. How >> I mean I think the..."
What an “agent” really is
"general take on agents is it's it's a it's an it's an AI that will take actions on your behalf that can work over long time horizons. And I think that's the that's the pretty general >> utilitarian de..."
How OpenAI thinks about pricing
"how h how h how have you evolved your thinking and how do you price these you know access to intelligence where you know you don't know how many people going to use it almost certainly usage based bil..."
Why open-weights don’t kill the API
"a while. >> So how do you think about open source? I mean, you know, I think you're the only big lab that's releasing open source. Is that >> No, Google has uh some of theirs. Yeah. Mostly smaller mod..."
Different stacks for text, images, video
"operationalize? >> Yeah, I think uh I think you're totally right. It's an anti pattern. It's pretty tough to pull off. Um >> uh I think honestly like props to Mark on our research team for like you kn..."
How the agent builder actually works
"like actually how you build agents and expose them has evolved too. So maybe you can talk a bit about that. >> Yeah. Yeah. I think um so at dev day this year when we launched our agent builder I got a..."