searchlore

Back to Resource

All Segments

Anthropic Co-founder: Building Claude Code, Lessons From GPT-3 & LLM System Design

Anthropic Co-founder: Building Claude Code, Lessons From GPT-3 & LLM System Design

17 segments available

Tom Brown co-founded Anthropic after helping build GPT-3 at OpenAI. A self-taught engineer, he went from getting a B-minus in linear algebra to becoming one of the key people behind AI's scaling breakthroughs. And his work is paying off. Today, Anthropic's Claude is the go-to choice for developers, and his team is overseeing what he calls "humanity's largest infrastructure buildout ever." On this episode of The Lightcone, he discusses his unconventional path from YC founder to AI researcher, the discovery of scaling laws that changed everything, and his advice for young engineers entering AI today. Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs Chapters: 0:00 - From Failure to Success 2:30 - Early Startup Days at Linked Language 4:12 - The Grouper Dating Experiment 6:10 - Making the Leap to OpenAI 8:42 - First Product Launch Challenges 10:12 - Self-Teaching AI Research 12:44 - Building GPT-3 Infrastructure 15:44 - The Anthropic Spinoff 18:23 - Early Days of Building Claude 20:21 - The ChatGPT Wake-Up Call 22:08 - Claude 3.5 Sonnet Breakthrough 24:13 - Why Benchmarks Don't Tell the Whole Story 26:20 - Claude Code's Secret Sauce 28:51 - Building for the AI Agent 31:11 - The Largest Infrastructure Buildout Ever 32:46 - Multi-Chip Strategy 34:38 - Advice for the Next Generation

Segments Timeline

1
0:00 - 2:32
2:32 duration509 words

From Failure to Success

"When we started out, we didn't seem like we were going to be successful at all. OpenAI had a billion dollars and like all of these all of this star power and we had seven co-founders in co like trying..."

2
2:32 - 4:12
1:39 duration358 words

Early Startup Days at Linked Language

"we ran the company for a bit. I ended up going back to to school afterwards and then when I left school I went to this company Mopub the mobile advertising thing right? Yeah. Yeah. I was like the firs..."

3
4:12 - 6:10
1:58 duration419 words

The Grouper Dating Experiment

"Yeah. So, so Grouper was a dating app only it was novel in that you had what three guys and three girls. Y this was before AI in a lot of ways. So there was like a set of a team of people who would ma..."

4
6:10 - 8:42
2:32 duration596 words

Making the Leap to OpenAI

"Yeah. So, I left Grouper 2014, June 2014, and I joined OpenAI I think a year later. I tried to like build up courage to make the switch to be a to try to learn AI research. At the time I was like, "Ok..."

5
8:42 - 10:12
1:29 duration349 words

First Product Launch Challenges

"and go like talk to someone new and they might just be like I don't want to talk to you. You seem weird. And so we solved that by just blind matching. Tinder came out while we were doing grouper and T..."

6
10:12 - 12:44
2:32 duration544 words

Self-Teaching AI Research

"got to. Yeah. Yeah. It turned out okay. First I did a contract actually with Twitch um and like earned like enough to have that six months of runway. So I did like three-month contract with Twitch and..."

7
12:44 - 15:44
3:00 duration615 words

Building GPT-3 Infrastructure

"Yeah. For GP3 for and how how what was that? Because you got from GPT2 was in TPUs, right? Yep. And the big breakthrough in GPT3 was like use more compute and using GPUs. Yep. So I ended up working at..."

8
15:44 - 18:23
2:39 duration560 words

The Anthropic Spinoff

"Can you uh tell us then how you ended up collecting the last infinity stone with? Yeah, with Enthropic because there's very few people in the world that have basically worked at OpenAI, deep mind and ..."

9
18:23 - 20:21
1:58 duration425 words

Early Days of Building Claude

"Hey, tell us about the early days of anthropic. So the the seven you broke off from open AI, you had a general idea of the sort of like long-term mission that you wanted to do to you know not destroy ..."

10
20:21 - 22:08
1:46 duration338 words

The ChatGPT Wake-Up Call

"This is summer 2022. Yep. And then chat GPD launched fall 2022 and then we relaunched our API after that and then claude AI after that also. I think it didn't seem like it was working basically until ..."

11
22:08 - 24:13
2:04 duration493 words

Claude 3.5 Sonnet Breakthrough

"and do you guys know like the day that you guys launched 3.5 sonnet did you know that you had something really special and this was going to be the turning point for the company or were you as surpris..."

12
24:13 - 26:20
2:07 duration493 words

Why Benchmarks Don't Tell the Whole Story

"somehow? I think that the benchmarks benchmarks are like easy to game where I think that all the other big labs I think have teams where they like their whole job with the team is to like make the ben..."

13
26:20 - 28:51
2:30 duration523 words

Claude Code's Secret Sauce

"Claude Code was um an internal tool also. So like try to help out our our engineers within Anthropic that uh yeah, Boris um had like hacked together. There's an internal anthropic engineer wanting to ..."

14
28:51 - 31:12
2:21 duration481 words

Building for the AI Agent

"I think I was kind of surprised that claude code like we we did build a thing that was like uh like the best in the market there too. It's not super clear to me what the big advantage was for us for C..."

15
31:12 - 32:46
1:33 duration303 words

The Largest Infrastructure Buildout Ever

"look at is just that humanity is on track for like the largest infrastructure buildout of all time. Now, this is going to be larger than the Apollo project, larger than the Manhattan project. It'll be..."

16
32:46 - 34:38
1:52 duration386 words

Multi-Chip Strategy

"I I definitely I feel like yes, yes, all all of those things. I wish I wish that nuclear was easier to build. Anthropic is the only major lab that uses not just one kind of GPU, but the GPUs from thre..."

17
34:38 - 35:48
1:09 duration221 words

Advice for the Next Generation

"Do you have advice for um kind of like a younger Tom version of yourself who now you've seen and went through this crazy journey? If someone was you back in the 20s living today and they wanted to rid..."