All Segments
Chelsea Finn: Building Robots That Can Do Anything
15 segments available
Chelsea Finn on June 17th, 2025 at AI Startup School in San Francisco. From MIT through her PhD at Berkeley, where she pioneered meta‑learning methods, and Google Brain, Chelsea Finn has built her career around teaching machines how to learn. Now an Assistant Professor at Stanford and co‑founder of Physical Intelligence, she’s using that foundation to bring learning-driven robotics into messy, real-world environments rather than confined lab setups. In this talk, Chelsea traces the evolution of her team’s work—from early experiments on robotic grasping and vision to today’s ambitious efforts at folding laundry, tidying kitchens, and generalizing across tasks—all without hand-crafted code. Instead, they used scalable foundation models and massive datasets, teaching robots physical common sense as they learn by doing. She shares stories of the rocky setbacks, the surprises hidden in data, and the moment it all clicked: robots equipped with generalizable physical intelligence can indeed adapt and assist in the unpredictable world around us. Apply to Y Combinator: https://ycombinator.com/apply Work at a startup: https://workatastartup.com Chapters: 00:00 - General Purpose Robots 00:11 - Challenges in Robotics Applications 00:57 - Physical Intelligence: A New Approach 01:47 - Learning from Language Models 02:08 - Data Sources for Training Robots 03:32 - Training with Real-World Data 04:39 - Initial Successes and Challenges 09:10 - Breakthrough in Robot Training 11:03 - Improving Performance 15:43 - Expanding Capabilities 17:34 - Robots in Unseen Environments 25:54 - Handling Open-Ended Prompts 29:36 - Evaluating Robot Performance 30:03 - Future Directions and Challenges 31:27 - Audience Q&A
Segments Timeline
General Purpose Robots
"Hi everyone. Um, I'm really excited to talk about developing general purpose robots and how we might uh actually like truly develop and bring intelligence into the physical world. So, um, to"
Challenges in Robotics Applications
"start off, I'd like to talk about this problem, which is that if you want to truly solve a robotics application, you essentially need to build an entire company around that application. uh you need to..."
Physical Intelligence: A New Approach
"co-founded a company called physical intelligence, uh, that's trying to solve this problem. And in particular, we're trying to develop a general purpose model that can enable any robot to do any task ..."
Learning from Language Models
"how do we do this? Uh in this talk I'd like to talk about how we go about doing this. And if we were to take a lesson from language models we know that language models have taught us the importance of..."
Data Sources for Training Robots
"to say this conclusion is true, then you might look to certain data sources for largecale uh data. So for example, we might look at data from industrial automation and you get tons and tons of data of..."
Training with Real-World Data
"example of a data episode uh that we've collected. Uh this is uh in honor of our first anniversary, which was a few months ago. uh where we um here you can see a teley operator uh in person who's oper..."
Initial Successes and Challenges
"particular uh in this first part I'd like to talk about how we trained uh a pi zero foundation model to do this task which is to unload a dryer and fold laundry. Uh and to date I think this is the mos..."
Breakthrough in Robot Training
"point we actually had a bit of a breakthrough uh which was that um we found one thing that really seemed to make a difference in the robot's ability to do the task. And this was actually to take some ..."
Improving Performance
"were still training models largely um kind of we were pre-training uh and fine-tuning only on laundry data, and we weren't leveraging uh kind of pre-trained models in the community. And there were som..."
Expanding Capabilities
"um kind of clean up a table. And the robot's also able to successfully uh do this task uh despite the fact that we primarily were iterating a lot on laundry, but it's able to also apply this recipe to..."
Robots in Unseen Environments
"there's a number of limitations here and one limitation I'd like to point out is that these robots inevitably um in this case were trained in the environments that they were tested. Uh and so this mea..."
Handling Open-Ended Prompts
"And so just like we prompt language models, can we allow robots to respond to open-ended prompts and open-ended interjections? Uh so to do this and actually to do the past work, we're actually leverag..."
Evaluating Robot Performance
"so we also evaluated that um and we found that in blue the performance at following instructions and making progress on the task was substantially lower than the performance of our system which is sho..."
Future Directions and Challenges
"wrap up, um, and then we'll all have some time for questions. Uh, I talked a bit about how robots can do a variety of dextrous long horizon tasks with pre-training and post- training. How robots can s..."
Audience Q&A
"Happy to take some questions. Let's start on the left. Uh hi Chelsea. So, uh first I want to say thank you for all your work on robot learning. They're all really impressive. Yeah. And uh so mainly I ..."