
44 segments available
Benjamin Mann is a co-founder of Anthropic, an AI startup dedicated to building aligned, safety-first AI systems. Prior to Anthropic, Ben was one of the architects of GPT-3 at OpenAI. He left OpenAI driven by the mission to ensure that AI benefits humanity. In this episode, Ben opens up about the accelerating progress in AI and the urgent need to steer it responsibly. *In this conversation, we discuss:* 1. The inside story of leaving OpenAI with the entire safety team to start Anthropic 2. How Meta’s $100M offers reveal the true market price of top AI talent 3. Why AI progress is still accelerating (not plateauing), and how most people misjudge the exponential 4. Ben’s “economic Turing test” for knowing when we’ve achieved AGI—and why it’s likely coming by 2027-2028 5. Why he believes 20% unemployment is inevitable 6. The AI nightmare scenarios that concern him most—and how he believes we can still avoid them 7. How focusing on AI safety created Claude’s beloved personality 8. What three skills he’s teaching his kids instead of traditional academics *Brought to you by:* Sauce—Turn customer pain into product revenue: https://sauce.app/lenny LucidLink—Real-time cloud storage for teams: https://www.lucidlink.com/lenny Fin—The #1 AI agent for customer service: https://fin.ai/lenny *Transcript:* https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann *My biggest takeaways (for paid newsletter subscribers):* https://www.lennysnewsletter.com/i/168107911/my-biggest-takeaways-from-this-conversation *Where to find Ben Mann:* • X: https://x.com/8enmann • LinkedIn: https://www.linkedin.com/in/benjamin-mann/ • Website: https://benjmann.net/ *Where to find Lenny:* • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ *In this episode, we cover:* (00:00) Introduction to Benjamin (04:43) The AI talent war (06:28) AI progress and scaling laws (10:50) Defining AGI and the economic Turing test (12:26) The impact of AI on jobs (17:45) Preparing for an AI future (24:05) Founding Anthropic (27:06) Balancing AI safety and progress (29:10) Constitutional AI and model alignment (34:21) The importance of AI safety (43:40) The risks of autonomous agents (45:40) Forecasting superintelligence (48:36) How hard is it to align AI? (53:19) Reinforcement learning from AI feedback (RLAIF) (57:03) AI's biggest bottlenecks (01:00:11) Personal reflections on responsibilities (01:02:36) Anthropic’s growth and innovations (01:07:48) Lightning round and final thoughts *Referenced:* • Dario Amodei on LinkedIn: https://www.linkedin.com/in/dario-amodei-3934934/ • Anthropic CEO: AI Could Wipe Out 50% of Entry-Level White Collar Jobs: https://www.marketingaiinstitute.com/blog/dario-amodei-ai-entry-level-jobs • Alexa+: https://www.amazon.com/dp/B0DCCNHWV5 • Azure: https://azure.microsoft.com/ • Sam Altman on X: https://x.com/sama • Opus 3: https://www.anthropic.com/news/claude-3-family • Claude’s Constitution: https://www.anthropic.com/news/claudes-constitution • Greg Brockman on X: https://x.com/gdb • Anthropic’s Responsible Scaling Policy: https://www.anthropic.com/news/anthropics-responsible-scaling-policy • Agentic Misalignment: How LLMs could be insider threats: https://www.anthropic.com/research/agentic-misalignment • Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next • AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt): https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff • Unitree: https://www.unitree.com/ • Arthur C. Clarke: https://en.wikipedia.org/wiki/Arthur_C._Clarke • How Reinforcement Learning from AI Feedback Works: https://www.assemblyai.com/blog/how-reinforcement-learning-from-ai-feedback-works • RLHF: https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback • Jared Kaplan on LinkedIn: https://www.linkedin.com/in/jared-kaplan-645843213/ • Moore’s law: https://en.wikipedia.org/wiki/Moore%27s_law • Machine Intelligence Research Institute: https://intelligence.org/ • Raph Lee on LinkedIn: https://www.linkedin.com/in/raphaeltlee/ • “The Last Question”: https://en.wikipedia.org/wiki/The_Last_Question • Beth Barnes on LinkedIn: https://www.linkedin.com/in/elizabethmbarnes/ • “The Last Question”: https://en.wikipedia.org/wiki/The_Last_Question • Good Strategy, Bad Strategy | Richard Rumelt: https://www.lennysnewsletter.com/p/good-strategy-bad-strategy-richard • Pantheon on Netflix: https://www.netflix.com/title/81937398 *...References continued at:* https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann _Production and marketing by https://penname.co/._ _For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com._ Lenny may be an investor in the companies discussed.
Benjamin Mann discusses the timeline for achieving superintelligence, predicting a 50% chance by 2028. He reflects on the urgency of AI safety and the implications of creating powerful AI, emphasizing the need to keep it aligned with human values.
"You wrote somewhere that creating powerful AI might be the last invention humanity ever needs to make. How much time do we have, Ben? I think 50th percentile chance of hitting some kind of super..."
Ben shares his motivations for leaving OpenAI, highlighting concerns that safety wasn't prioritized. He emphasizes the importance of ensuring that superintelligence remains aligned with human interests, framing it as a critical challenge.
"What is it that you saw at OpenAI? What'd you experience there that made you feel like, okay, we got to go do our own thing? We felt like safety wasn't the top priority there. The case for safety..."
Mann provides his forecast on the risks associated with AI alignment, estimating a 0-10% chance of an extremely negative outcome. He stresses the importance of addressing these risks before superintelligence is achieved.
"What are the odds that we align AI correctly? Once we get to superintelligence, it will be too late to align the models. My best granularity forecast for could we have an X-risk or extremely bad ..."
Ben discusses the competitive landscape for AI talent, particularly in light of Meta's aggressive recruitment strategies. He contrasts the motivations of employees at Anthropic with those at other companies, emphasizing mission-driven work.
"bad outcome is somewhere between 0 and 10%. Something that's in the news right now is this whole Zuck coming after all the top AI researchers, We've been much less affected because people here, ..."
Mann reflects on the potential for AI to cause significant unemployment, predicting a rise to 20%. He discusses the transformative impact of AI on capitalism and the nature of work in the future.
"If you just think about 20 years in the future where we're way past the singularity, it's hard for me to imagine that even capitalism will look at all like it looks today. Do you have any advice..."
In this segment, Lenny introduces Benjamin Mann, co-founder of Anthropic, highlighting his background at OpenAI and his focus on aligning AI to be helpful, harmless, and honest. The conversation sets the stage for discussing AI's future.
"Today, my guest is Benjamin Mann. Holy moly. What a conversation. Ben is the co-founder of Anthropic. He serves as tech lead for product engineering. He focuses most of his time and energy on al..."
Ben elaborates on the recruitment challenges in the AI sector, particularly the allure of high compensation packages. He emphasizes the mission-driven culture at Anthropic, where employees prioritize the impact of their work on humanity.
"including his thoughts on the recruiting battle for top AI researchers, why he left OpenAI to start Anthropic, how soon he expects we'll see AGI. Also, his economic touring test for knowing when..."
Mann discusses the common misconception that AI progress is plateauing. He argues that advancements are accelerating, with more frequent model releases and improvements in AI capabilities.
"world and his own life and what he's encouraging his kids to learn to succeed in an AI future. A huge thank you to Steve Mnich, Danielle Ghiglieri, Raph Lee, and my newsletter community for sugg..."
Ben explains the importance of scaling laws in AI development, asserting that they continue to hold true despite perceptions of stagnation. He compares AI progress to semiconductor advancements, emphasizing the need for a shift in perspective.
"and follow it in your favorite podcasting app or YouTube. Also, if you become an annual subscriber of my newsletter, you get a year free of a bunch of amazing products including Bolt, Linear, Su..."
Mann introduces his concept of the Economic Turing Test as a measure for achieving AGI. He explains that if an AI can perform jobs at a level indistinguishable from humans, it signifies transformative AI.
"The way teams turn feedback into product impact is stuck in the past. Vague reports, static taxonomies, unactionable insights that don't move business metrics. The results churn, lost deals, mis..."
Ben discusses the potential impact of AI on employment, predicting significant changes in job structures and the economy. He emphasizes the need for society to adapt to these changes as AI becomes more integrated into the workforce.
"business impact and act faster. It listens to your sales calls, support tickets, turn reasons, and lost deals, surfacing the biggest product issues and opportunities in real time. It then routes..."
Mann highlights the current effects of AI on various job sectors, noting that many people may not yet realize the extent of these changes. He emphasizes the importance of recognizing the exponential nature of AI progress.
"caught a spiking issue and prevented millions in churn. You can too at sauce.app/lenny. Sauce built for AI product teams. Don't get left behind. This episode is brought to you by LucidLink, the s..."
Ben shares examples of AI's impact on customer service, citing high resolution rates achieved by AI systems. He discusses how AI allows human workers to focus on more complex tasks, enhancing overall productivity.
"videos, design assets, layer project files, you know how painful it can be to stay organized across locations, files live in different places. You're constantly asking, is this the latest versio..."
Mann discusses the role of AI in software engineering, highlighting how AI tools can significantly increase productivity. He emphasizes the transformative potential of AI in enabling smaller teams to achieve greater outputs.
"LucidLink fixes this. It gives your team a shared space in the cloud that works like a local drive. Files are instantly accessible for anywhere, no downloading, no syncing, and always up to date. ..."
Ben concludes by discussing the potential for AI to expand labor opportunities, allowing workers to accomplish more. He emphasizes the importance of preparing for the changes AI will bring to the job market.
"Shopify, and top creative agencies use LucidLink to keep their content engine running fast and smooth. Try it for free at lucidlink.com/lenny. That's L-U-C-I-D-L-I-N-K dot com slash Lenny. Ben, ..."
Ben Mann discusses the unsettling transition from today's job market to a future dominated by AI. He highlights the concept of the singularity, where rapid changes make it difficult to predict the future. Mann emphasizes the importance of managing this transition to ensure a positive outcome for society.
"guess there's this scary transition period from where we are today where people have jobs and capitalism works and the world of 20 years from now where everything is completely different, but pa..."
Mann explains how people often struggle to grasp the implications of exponential growth in AI technology. He shares his early recognition of AI's potential with GPT-2 and contrasts it with the public's delayed reaction to ChatGPT, emphasizing the need for awareness of AI's rapid advancements.
"my job seems fine. Nothing's changed. What are you seeing just happening today already that you think people don't see or misunderstand in terms of the impact AI is having on jobs? I think part ..."
In this segment, Mann illustrates the transformative effects of AI on customer service and software engineering. He cites impressive statistics, such as 82% resolution rates in customer service without human intervention and how AI tools like Claude Code enable teams to produce significantly more code efficiently.
"But I guess to cite a couple of areas where I think things are changing quite quickly. In customer service we're seeing with things like Fin and Intercom, they're a great partner of ours, 82% cu..."
Mann addresses the potential job displacement caused by AI, particularly in lower-skill roles. He stresses the importance of society proactively preparing for these changes and discusses how individuals can future-proof their careers in an AI-driven landscape.
"need to get ahead of and work on. Okay. I want to talk more about that, but something that I also want to help people with is how do they get a leg up in this future world? They listen to this, t..."
Mann shares advice on how individuals can effectively use AI tools to enhance their careers. He emphasizes the importance of being ambitious and adaptable in utilizing these technologies, encouraging users to experiment and learn from their interactions with AI.
"how you use the tools and being willing to learn new tools. People who use the new tools as if they were old tools tend to not succeed. As an example of that, when you're coding, people are very ..."
In this segment, Mann discusses the significance of persistence when using AI tools. He explains that users should not be discouraged by initial failures and should be willing to rephrase their queries to achieve better results, highlighting the stochastic nature of AI responses.
"Okay, so the advice here is use the tools. That's something everyone's always saying, just actually use these tools. It's like sit in Claude Code. And your point about being more ambitious than ..."
Mann reflects on the skills he prioritizes in educating his children to thrive in an AI-driven world. He emphasizes curiosity, creativity, and emotional intelligence as essential traits, advocating for a Montessori approach to learning that fosters self-led exploration.
"and that's why we're hiring super aggressively. Let me take another approach to asking this question something ask everyone that's at the very cutting edge of where AI is going. You have kids, kn..."
Mann discusses the importance of kindness in the context of AI development and interaction. He highlights how fostering empathy and understanding in AI systems can lead to better outcomes and more meaningful interactions between humans and AI.
"20 years ago and I had a kid, maybe I would be trying to line her up for going to a top tier school and doing all the extracurriculars and all that stuff. But at this point, I don't think any of..."
Mann shares the story behind the founding of Anthropic, detailing the motivations that led him and his team to leave OpenAI. He discusses the internal tensions regarding safety and progress at OpenAI and how they sought to prioritize safety in their new venture.
"That doesn't come up as much just being creative. I want to go in a different direction. I want to go back to the beginning of Anthropic. Famously you and eight of you left OpenAI back in the day..."
In this segment, Mann elaborates on the importance of safety in AI development. He explains how Anthropic aims to lead in safety research while also pushing the boundaries of AI capabilities, emphasizing the need for a balanced approach to innovation and safety.
"safety wasn't the top priority there. And there are good reasons that you might think that if you thought safety was going to be easy to solve or if you thought it wasn't going to have a big impac..."
Mann discusses the perceived tension between focusing on AI safety and maintaining competitive progress in the marketplace. He shares insights on how prioritizing safety can actually enhance innovation and lead to better AI outcomes.
"and now that exact technique is working and many others that we have been thinking about for a long time. Yeah, fundamentally it comes down to is safety the number one priority? And then somethi..."
Mann introduces the concept of Constitutional AI, explaining how Anthropic integrates core values and principles into their AI models. He discusses the significance of these values in shaping AI behavior and ensuring alignment with human intentions.
"do you do a refusal that doesn't shut the person down, but makes them feel like they understand why the agent said, "I can't help you with that. Maybe you should talk to a medical professional, ..."
In this segment, Mann elaborates on how constitutional AI works, detailing the process of integrating ethical principles directly into the AI model. He explains how the model critiques its own responses based on these principles, ensuring that it adheres to values like kindness and respect. This proactive approach aims to create AI that understands and aligns with human values, fostering trust among users.
"That's right. That's right. And from a distance, it might seem quite disconnected, like how is this going to prevent X risk? But ultimately it's about the AI understanding what people want and n..."
Mann shares his personal journey towards prioritizing AI safety, influenced by his love for science fiction and the insights from Nick Bostrom's 'Superintelligence.' He reflects on the challenges of aligning AI with human values and the importance of addressing these issues proactively. Mann emphasizes that while the path to safe AGI is complex, he remains hopeful due to advancements in language models that better understand human intentions.
"that we've decided are good. And this is also not something that we think as a small group of people in San Francisco should be figuring out. This should be a society wide conversation. And that..."
Mann discusses the potential risks associated with AI, including the misuse of technology and the importance of transparency in AI development. He explains Anthropic's responsible scaling policy, which categorizes AI safety levels and outlines the societal risks at each stage. Mann stresses the need for awareness and preparation as AI capabilities grow, advocating for a proactive approach to mitigate potential dangers.
"like space operas where humanity is a multi galactic civilization has extremely advanced technology building Dyson spheres around the sun with sentient robots to help them. And so for me, coming..."
In this segment, Mann addresses criticisms of AI safety discussions, asserting that transparency about risks is crucial for building trust with policymakers. He argues that while the future of AI is likely to be positive, the potential downsides must not be ignored. Mann uses analogies to illustrate the importance of preparing for worst-case scenarios, emphasizing that the stakes are too high to gamble with humanity's future.
"It's interesting because you guys put out more examples of your models doing bad things than anyone else. There was I think a story of an agent or a model trying to blackmail engineer. You guys ..."
Mann reflects on the profound implications of creating powerful AI, describing it as potentially humanity's last invention. He discusses the exponential growth of AI capabilities and the urgency of ensuring safety before reaching superintelligence. Mann highlights the need for a societal conversation about AI values and the importance of aligning AI with human interests to prevent catastrophic outcomes.
"I mean, I think part of why we publish these things is we want other labs to be aware of the risks. And yes, there could be a narrative of we're doing it for attention, but honestly from a atten..."
Mann shares insights on predictions for achieving AGI, referencing the AI 2027 report and the exponential growth of intelligence. He discusses the factors contributing to this rapid advancement and expresses confidence in the timeline for superintelligence. Mann emphasizes that understanding these developments is crucial for preparing society for the transformative impact of AI.
"You wrote somewhere that creating powerful AI might be the last invention humanity ever needs to make. If it goes poorly, it can mean a bad outcome for humanity forever. If it goes well, the soone..."
Ben shares insights on the timeline for achieving superintelligence, referencing the AI 2027 report. He discusses the exponential growth of AI capabilities and the implications of reaching a point where AI surpasses human intelligence, emphasizing the need for careful consideration of this rapid advancement.
"a forecast that's pulled out of thin air. It's based on a lot of just hard details of the science of how intelligence seems to have been improving, the amount of low hanging fruit on model trainin..."
Mann explains his concept of the 'Economic Turing Test' as a measure for determining when superintelligence is achieved. He discusses the potential economic impacts of AI, including a significant increase in global GDP, and the societal implications of such rapid growth.
"about what that would mean from a individual story standpoint. If the amount of goods and services in the world is doubling every year, what does that even mean for me as a person living in Califo..."
In this segment, Ben discusses the complexities of aligning AI with human values. He outlines three potential scenarios for AI alignment, from optimistic to pessimistic, and emphasizes the importance of proactive measures to ensure safe AI development, drawing parallels to historical examples of global coordination.
"prior on that is updating to be less likely. In the optimistic world, we're basically done, and our main job is to accelerate progress and to deliver the benefits to people. But again, I think ac..."
Mann encourages individuals interested in AI safety to consider various roles beyond research, highlighting the need for product development, finance, and other fields to support the mission of safe AI. He emphasizes that everyone can contribute to the cause, regardless of their specific expertise.
"absolute best to make sure that that's true. Wow. What fulfilling work. For folks that are inspired with this? I imagine you're hiring for folks to help you with this. Maybe just share that in ca..."
Ben introduces the concept of RLAIF (Reinforcement Learning from AI Feedback) and its significance in AI training. He explains how this approach allows AI models to improve autonomously, while also addressing the risks associated with self-improvement and the need for alignment with human values.
"Awesome. Even if you're not working directly on the AI safety team, you're having an impact on moving things in the right direction. By the way, X risk is short for existential risk. In case fol..."
Mann identifies key bottlenecks in AI intelligence improvement, including the need for more data centers and computational power. He discusses the importance of scaling laws and the role of researchers in advancing AI capabilities, emphasizing the collaborative nature of progress in the field.
"does it pass the linter? Things like that. That also could be included in RLAIF. And the idea here is that if models can self-improve, then it's a lot more scalable than finding a lot of humans...."
In this segment, Ben discusses the challenges facing semiconductor technology and its impact on AI development. He reflects on the ongoing innovations that continue to drive progress, despite theoretical constraints, and the importance of adaptability in the tech industry.
"an empirical company. We have a lot of physicists like Jared, who's our chief research officer who I've worked with a lot, was a professor of Black Hole Physics at Johns Hopkins, and I guess he ..."
Ben shares his personal experiences and thoughts on the weight of responsibility that comes with working on AI safety. He discusses strategies for managing stress and the importance of community support in tackling the challenges of ensuring safe AI development.
"into and yet they're finding ways around it. We've got to start using parallel universes for some of this stuff. I guess so. Okay, I want to zoom out and talk about just Ben, Ben as a human for a..."
Reflecting on his journey at Anthropic, Ben shares his diverse roles within the company since its inception. He discusses the growth from a small team to over 1,000 employees and his contributions to various projects, particularly in transitioning from research to product development.
"density. One of the things I love the most about our culture here is that it's very egoless. People just want the right thing to happen and I think that's another big reason that the mega offers ..."
Ben introduces the Labs team at Anthropic, which focuses on transferring research into user-friendly products. He discusses the team's innovative approach and the importance of safety in developing new technologies, highlighting successful projects like Claude Code.
"that time has been when I started the labs team about a year ago, whose fundamental goal was to do transfer from research to end user products and experiences. Because fundamentally I think the ..."
In this segment, Ben shares insights on strategic planning for AI projects, emphasizing the need to build for the future rather than the present. He discusses the importance of understanding exponential growth in technology and how this perspective has influenced the development of Anthropic's products.
"see really cool stuff coming out soonish. Yeah, just leading that team has been so fun. MCP came out of that team and Claude Code came out of that team. And the people who I hired are like combo, ..."