Sander Schulhoff is the OG prompt engineer. He created the very first prompt engineering guide on the internet (two months before ChatGPT’s release) and recently wrote the most comprehensive study of prompt engineering ever conducted (co-authored with OpenAI, Microsoft, Google, Princeton, and Stanford), analyzing over 1,500 academic papers and covering more than 200 prompting techniques. He also partners with OpenAI to run what was the first and is the largest AI red teaming competition, HackAPrompt, which helps discover the most state-of-the-art prompt injection techniques (i.e. ways to get LLMS to do things it shouldn’t). Sander teaches AI red teaming on Maven, advises AI companies on security, and has educated millions of people on the most state-of-the-art prompt engineering techniques. *In this episode, you’ll learn:* 1. The 5 most effective prompt engineering techniques 2. Why “role prompting” and threatening the AI no longer works—and what to do instead 3. The two types of prompt engineering: conversational and product/system prompts 4. A primer on prompt injection and AI red teaming—including real jailbreak tactics that are still fooling top models 5. Why AI agents and robots will be the next major security threat 6. How to get started in AI red teaming and prompt engineering 7. Practical defense to put in place for your AI products *Transcript:* https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff *Brought to you by:* Eppo—Run reliable, impactful experiments: https://www.geteppo.com/ Stripe—Helping companies of all sizes grow revenue: https://stripe.com/ Vanta—Automate compliance. Simplify security: https://vanta.com/lenny *Where to find Sander Schulhoff:* • X: https://x.com/sanderschulhoff • LinkedIn: https://www.linkedin.com/in/sander-schulhoff/ • Website: https://sanderschulhoff.com/ • AI Red Teaming and AI Security Masterclass on Maven: https://bit.ly/44lLSbC • Free Lightning Lesson “How to Secure Your AI System” on 6/24: https://bit.ly/4ld9vZL *Where to find Lenny:* • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ *In this episode, we cover:* (00:00) Introduction to Sander Schulhoff (04:56) The importance of prompt engineering (09:01) Two modes for thinking about prompt engineering (12:02) Few-shot prompting (17:30) Prompting techniques to avoid (24:52) Decomposition (28:26) Self-criticism and context (40:29) Ensembling (45:59) Thought generation (48:23) Conversational vs. product-focused prompt engineering (51:56) Introduction to prompt injection and red teaming (53:37) AI red teaming competitions (55:23) The growing importance of AI security (01:03:39) Techniques to bypass AI safeguards (01:06:17) Challenges in AI security and future outlook (01:09:31) Common defenses to prompt injection that don't actually work (01:13:18) Defenses that do work (01:16:33) Misalignment and AI's potential risks (01:19:29) Are LLMs behaving maliciously? (01:26:05) Final thoughts and lightning round *Referenced:* • Reid Hoffman’s tweet about using AI agents: https://x.com/reidhoffman/status/1930416063616884822 • AI Engineer World’s Fair: https://www.ai.engineer/ • What Is Artificial Social Intelligence?: https://learnprompting.org/blog/asi • Devin: https://devin.ai/ • Cursor: https://www.cursor.com/ • The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell • Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika • Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder & CEO of StackBlitz): https://www.lennysnewsletter.com/p/inside-bolt-eric-simons • Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders | Guillermo Rauch (founder and CEO of Vercel, creators of v0 and Next.js): https://www.lennysnewsletter.com/p/everyones-an-engineer-now-guillermo-rauch • Technique #3: Examples in Prompts: From Zero-Shot to Few-Shot: https://learnprompting.org/docs/basics/few_shot?srsltid=AfmBOor2owyGXtzJZ8n0fJVCctM7UPZgZmH-mBuxRW4t9-kkaMd3LJVv • The Prompt Report: Insights from the Most Comprehensive Study of Prompting Ever Done: https://learnprompting.org/blog/the_prompt_report?srsltid=AfmBOoo7CRNNCtavzhyLbCMxc0LDmkSUakJ4P8XBaITbE6GXL1i2SvA0 ...References continued at: https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff _Production and marketing by https://penname.co/._ _For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com._ Lenny may be an investor in the companies discussed.
Sander Schulhoff is an AI researcher specializing in AI security, prompt injection, and red teaming. He wrote the first comprehensive guide on prompt engineering and ran the first-ever prompt injection competition, working with top AI labs and companies. His dataset is now used by Fortune 500 companies to benchmark their AI systems security, he’s spent more time than anyone alive studying how attackers break AI systems, and what he’s found isn’t reassuring: the guardrails companies are buying don’t actually work, and we’ve been lucky we haven’t seen more harm so far, only because AI agents aren’t capable enough yet to do real damage. *We discuss:* 1. The difference between jailbreaking and prompt injection attacks on AI systems 2. Why AI guardrails don’t work 3. Why we haven’t seen major AI security incidents yet (but soon will) 4. Why AI browser agents are vulnerable to hidden attacks embedded in webpages 5. The practical steps organizations should take instead of buying ineffective security tools 6. Why solving this requires merging classical cybersecurity expertise with AI knowledge *Brought to you by:* Datadog—Now home to Eppo, the leading experimentation and feature flagging platform: https://www.datadoghq.com/lenny Metronome—Monetization infrastructure for modern software companies: https://metronome.com/ GoFundMe Giving Funds—Make year-end giving easy: http://gofundme.com/lenny *Transcript:* https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis *My biggest takeaways (for paid newsletter subscribers):* https://www.lennysnewsletter.com/i/181089452/my-biggest-takeaways-from-this-conversation *Where to find Sander Schulhoff:* • X: https://x.com/sanderschulhoff • LinkedIn: https://www.linkedin.com/in/sander-schulhoff • Website: https://sanderschulhoff.com • AI Red Teaming and AI Security Masterclass on Maven: https://bit.ly/44lLSbC *Where to find Lenny:* • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ *In this episode, we cover:* (00:00) Introduction to Sander Schulhoff and AI security (05:14) Understanding AI vulnerabilities (11:42) Real-world examples of AI security breaches (17:55) The impact of intelligent agents (19:44) The rise of AI security solutions (21:09) Red teaming and guardrails (23:44) Adversarial robustness (27:52) Why guardrails fail (38:22) The lack of resources addressing this problem (44:44) Practical advice for addressing AI security (55:49) Why you shouldn’t spend your time on guardrails (59:06) Prompt injection and agentic systems (01:09:15) Education and awareness in AI security (01:11:47) Challenges and future directions in AI security (01:17:52) Companies that are doing this well (01:21:57) Final thoughts and recommendations *Referenced:* • AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt): https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff • The AI Security Industry is Bullshit: https://sanderschulhoff.substack.com/p/the-ai-security-industry-is-bullshit • The Prompt Report: Insights from the Most Comprehensive Study of Prompting Ever Done: https://learnprompting.org/blog/the_prompt_report?srsltid=AfmBOoo7CRNNCtavzhyLbCMxc0LDmkSUakJ4P8XBaITbE6GXL1i2SvA0 • OpenAI: https://openai.com • Scale: https://scale.com • Hugging Face: https://huggingface.co • Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition: https://www.semanticscholar.org/paper/Ignore-This-Title-and-HackAPrompt%3A-Exposing-of-LLMs-Schulhoff-Pinto/f3de6ea08e2464190673c0ec8f78e5ec1cd08642 • Simon Willison’s Weblog: https://simonwillison.net • ServiceNow: https://www.servicenow.com • ServiceNow AI Agents Can Be Tricked Into Acting Against Each Other via Second-Order Prompts: https://thehackernews.com/2025/11/servicenow-ai-agents-can-be-tricked.html • Alex Komoroske on X: https://x.com/komorama • Twitter pranksters derail GPT-3 bot with newly discovered “prompt injection” hack: https://arstechnica.com/information-technology/2022/09/twitter-pranksters-derail-gpt-3-bot-with-newly-discovered-prompt-injection-hack • MathGPT: https://math-gpt.org • 2025 Las Vegas Cybertruck explosion: https://en.wikipedia.org/wiki/2025_Las_Vegas_Cybertruck_explosion • Disrupting the first reported AI-orchestrated cyber espionage campaign: https://www.anthropic.com/news/disrupting-AI-espionage ...References continued at: https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis _Production and marketing by https://penname.co/._ _For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com._ Lenny may be an investor in the companies discussed.