0:00
One of the big surprises the last few years has been the coding agents across different domains.
Search this video
0:05
And now frontier researchers are showing that this includes controlling robots.
0:08
This led MIT Professor Philip Isola to suggest in a recent viral essay that we may be entering the era of robot use agents where general purpose models could make different robots more capable.
0:18
So today Francois and I invited the founders of Waddle Labs and RoboCurve, two groups of startups that are working at the very frontier of making robots more capable with LLMs.
0:35
Maybe you guys want to briefly introduce yourselves and just say a little bit about um what each of your companies focuses on. >> I'm Jaime.
0:40
I'm from Waddle Labs together with Vincent.
0:42
We work on uh building LLMs that control robots and we do this by doing two things: building a harness that allows the LLMs to do this very effectively, collecting data, and then using that data to train better LLMs.
0:54
>> Hi, I'm Jay, uh co-founder of RoboCurve.
0:57
We are an eval company for physical AI.
1:01
We measure everything, any robot, any model, uh including LLMs and also classical approaches such as uh VLAs, vision language action models, and world action models.
1:10
And we evaluate all kinds of environments like hands, um grippers, arms, um humanoids, quadrupeds, all kinds of stuff.
1:20
>> So in the last few weeks videos from both of you guys went quite viral on Twitter and I think inspire I think both of them are actually quoted in that Philip Isola essay talking about robot use agents.
1:29
Um the videos showed things like LLMs being able to unscrew caps, being able to communicate between multiple robots, being able to do tasks like uh uncapping a pen for example.
1:37
And so I thought maybe what would be fun is for Francois and I to dig in with you guys on some of the research that led to this moment to begin and then maybe we can also show some demonstrations of what this can actually do.
1:48
To start, why don't we talk about some of the early research around transfer between language models and robot control.
1:54
So I want to talk about um some of the early work on pre-training language models for decision-making and then also the RT2 paper, which established the VLA.
2:03
So, uh maybe Jay, do you want to tell us a little bit about some of this early work involving being able to use language models to do any kind of robot control?
2:11
>> One of the earliest successful approaches of using AI on robots is the RT2 paper, where they use a pre-trained language model on web uh text and images and use that to control uh robots.
2:25
And the interesting thing about that is that it's a fine-tuned version of uh language model and instead of outputting English, for example, they just output um what we call an end effector pose, which is the coordinates that you can then translate into joint commands that can control the robots.
2:46
Uh in some cases, it's um in some sense, it's very similar to what we see now with large language models, just that instead of fine-tuning, the models are good enough to just do it off the box.
2:58
>> This is actually quite similar to like the mapping that I have in my head is kind of like a the COT moment.
3:03
And so like when we did GSM8K and you said, "Susie had had $8. She spent five.
3:07
How much does she have now?"
3:09
Um it had to output like four hash marks and then the answer. And then EOS.
3:17
There was no ability for it to chain of thought.
3:19
And so then we allowed it to actually have like, "Okay, let me go 8 - 5 is 3."
3:24
Yeah, and so like hash hash hash hash 3.
3:26
And so you allowed it to do this kind of thinking before it actually gave an action or an answer.
3:31
Um and similarly now, like VLA is we're basically like an RT2 is basically like it has to output an action.
3:39
There's no like I can't allocate more compute for a more complex task.
3:42
And then now I have this code chain of thought thing that I can do.
3:46
And I can say, "Okay, even even if you were using the example where you're not actually outputting the code, you're just giving it to Astra and let it think think think think think and then output an action.
3:54
Similarly now like and then with code, it's a more Kolmogorov complexity or lower Kolmogorov complexity code code length to just use code and really compact code to just like here's the code. Just go.
4:07
>> I totally agree with you on that.
4:07
I feel like at the end of the day, it's a lot about the bit of less, right?
4:11
If you give the agent or if you give the AI model more autonomy and you if you unshackle it a bit more and give it more resources, it can actually do a lot of things that we fine-tuned it to do.
4:25
>> I'm just going to hop in here as well.
4:25
I think the the really interesting thing about the bit of lesson here is that um like VLAs by design architecturally they're built on top of language models as well, right?
4:35
They they can potentially reason, they can write code.
4:39
So perhaps the bit of lesson here isn't necessarily what architecture you build necessarily, but it's what data is most useful, right?
4:45
Like people have been sort of nagging at this VLA data bottleneck for years now and we're seeing very very slow progress.
4:53
And so the lesson here is maybe we we have a modality of data that we know works very well and this idea of transferring across different modalities, which Hamming and I are super excited about.
5:01
Like how do you take something that's traditionally out of distribution like robotics and make it something that's in distribution?
5:07
Like maybe the way of harnessing the bit of lesson is saying let's pick a data for which we know this modality is pretty blessed. There's a lot of data.
5:14
These language well these models understand it well and then use this as a way of uh unlocking a lot of other domains as well.
5:22
>> Yeah, and to add to that, the reason why RT2 is such a good model is because it is using a language model that taps into the modality of of all the data that a language model is trained on.
5:33
So all those um web images and web tags actually improves the VLA compared to just training a specific robotics foundation model without any pre-training.
5:45
>> So so you're benefiting in the case of RT2, these early VLA approaches, from the from pre-training.
5:49
Um I guess what's the limitation in the I guess action-taking fine-tuning approach that you can now get around if you can directly write code.
5:59
Like you're you're referring to like a difference in data there.
6:02
What exactly is that difference in data you're referring to?
6:05
>> There's a few things that models have got much better at since RT2.
6:06
Um like one, I mean they're much better tool use, for example.
6:12
So, you know, one one kind of data is now these models can uh well, I mean they they write code much better.
6:18
So, now they can uh write complex policies as code.
6:19
They're also much better using tools to explore um the kinds of environments they're in, like what arms they have access to.
6:24
And a lot of this comes from in-context learning.
6:28
>> Like the key difference between like these VLAs and or like GPT-6 or LLM isn't necessarily the architecture, but kind of the approach you take towards training.
6:36
We want to be bitter lesson pill, right?
6:38
Like we want to benefit from all kinds of data.
6:40
We want to pour in computer computer-used data into our robot models.
6:44
You want to pour in coding data into our robot models.
6:46
But when you do that, you just end up getting what's what we think of as like these general-purpose LLM agents.
6:51
And then so, to me it's like, okay, why not just build a really good foundational LLM and I use that to control robots, rather than training like a model that's specifically for robotics and that's more dependent on just robot data.
7:05
>> On that note, maybe one of the things that I think um if if we could wind the clock a couple years, may have been a good sign that we were trending in a good direction to this, is research on code as policies, right?
7:14
Do you guys want to talk a little bit about when the research community refers to code as policies, what exactly that means?
7:19
And I think it it's worth putting this in the context of when this paper came out, um which was coding agents just starting to work.
7:25
I think this was still in the era in which people were like putting comments and auto-filling, you know, Python code blocks, which you know, now we think of as agent history, but this was like two years ago.
7:34
Um so, yeah, how how does code as policies work and how did that inspire some of what you guys are now seeing as possible?
7:40
>> I'd probably rewind back to Voyager, where like Voyager was like the one of the first What is What is a coding agent mean?
7:46
Coding agent requires good tool use and on-the-fly tool creation. That's called code.
7:52
Okay, you have Python built-ins.
7:54
Let's just say I have only Python built-ins.
7:56
Those are tools and I have sort and I have like if and I have for and I have while and I have all these things and I need to like use those tools to assemble a new tool called a a new francois. py, right?
8:07
And like now that's a new tool and I get to use that.
8:10
That was Voyager and like yeah, I don't know I'm not going to say Voyager was the first one to do this.
8:14
Voyager was the most popular one to do this for Minecraft.
8:16
And then they created tools that they can invoke on the fly to help them play the game better and compressed thinking and experience into a new tool that they can later invoke.
8:27
And then I think because everyone in 20 24 had this insight was like okay, if we just pour if you know, Darian and Sam both all realized if I pour all of my heart and soul and resources and compute and intelligence into getting better at coding, we will have lift off.
8:43
Then I can automate the ML engineer, then I can have lift off and I'll have all the things.
8:49
I'll I'll solve all the things.
8:51
And so then that's what everyone did and I don't think that they had the insight I don't think they were so insightful to know that oh, if I did this, then we would have LLMs that would good enough to write policies for robotics and I can displace all of robotics with actual coding agents and I can displace artists with like, you know, a coding agents that are writing JavaScript to make beautiful like art.
9:14
Like I'm not sure that they had that insight to be honest.
9:18
>> Can we talk a little bit about what some of these early coders policy methods were even able to do?
9:21
So even you know, this is well ahead of us having coding agents that are widely available and people intuitively understanding how they work.
9:27
It's well ahead of people using RL methods to actually train those coding agents to be really good at tool calling.
9:31
What did some of those early ones maybe even before Voyager like coders policies um, is which I think the paper came out in 2022 at the end, right around when ChatGPT came out.
9:39
What was the capability of those compared to what you can see now?
9:43
>> Yeah, those, uh, Codas policy papers, especially ones from Google DeepMind team, were incredible because what they did is, uh, they created these kind of, um, functions like pick up an object, lift up, or like move to certain pose.
9:58
Uh, like they provided this list of functions to a coding agent, uh, in the form of like literally Python functions.
10:05
Uh, and then the coding agent would be able to write code, uh, involving these functions to then control the robot to do very complex tasks.
10:10
And what was very surprising, or what what this excelled was that coding agents can do this very one-shot.
10:16
They did not need additional robot data to in order to work with this code because they're already trained on so much coding data.
10:21
They already have a sense of like how to what what to do first, what to do second in order to like move a block into a bowl, for example.
10:28
And I think it's this one-shot ability, this in-context exploration ability, that really motivated a lot of later work to continue exploring, including us, to continue exploring how we can apply LLMs to robotics.
10:41
>> I know Francois, you have this, um, framework we've talked about of, you know, the various ways that learning can happen.
10:46
There's going to be learning that's embedded into the weights versus be in context learning and so on.
10:49
Do you want to quickly talk about that and then maybe we can think about where all the research of the last few years has fit into that framework, especially the direction it seems to be going.
10:57
>> I mean, I've been working on this experiment and actually I haven't really crystallized it into a paper yet, but basically it's like, um, what is the most efficient way from a, uh, intelligence per sample perspective to input a learning, a, let's say let's in this context a state action result or state action reward tuple back into the policy.
11:18
There's ICL, there is, >> in-context learning what we're describing.
11:23
>> learning where like I just like append it.
11:25
This is how most people are are are, uh, using LLMs.
11:28
They're they're like, "Oh, no, don't do it like this, do it like this." And I'm was "Okay."
11:31
And then it's it's still in the context and then I can remember and I can iterate.
11:34
And that really only works if you've trained the LLM or at least post-trained the LLM to be able to learn and improve.
11:40
And that self-refines a reflection like in literature from way back that like actually allowed multi-turn into the training set.
11:50
If you don't have that, it doesn't learn doesn't in context learn.
11:53
But even then there's a limit to how good ICL can go.
11:55
So, I've done this experiment where like you take an LLM that we've trained on and I've held out a task, let's just say GSMA K for for simplicity.
12:06
And and then I ICL it and I want to measure on the val set how much it improves on a per sample basis.
12:11
And it improves greatly very cheaply. It doesn't cost much.
12:15
I don't have to there's no SGD, so I don't have any flops.
12:17
And so, very quickly I can adapt and improve on my val set. Example example example.
12:23
One is non-monotonic improvement, which is wild.
12:26
So, it gets worse, it gets better, it gets worse, it gets better like pretty gra- aggressively.
12:31
Number two is that it caps out very quickly.
12:33
And so, after like 20, 30, maybe 40 examples, it is basically saturated and more examples back into the context does not improve.
12:41
>> So, you're like you're constrained by the model's ability to intelligently use all of its context.
12:45
>> Exactly, from the post-training.
12:45
How many multi-turns did it actually get to be able to improve and do self-reflection?
12:50
And then the most important one if for surely can't do is after it hits the context window context length of the model that it was trained on.
12:58
If it was trained on 100,000, really that means you have a context window of like 50,000.
13:01
Once you exceed 50,000, you don't improve anymore.
13:04
You actually just get worse cuz the model can't attend over everything.
13:06
Um there's a really good example you said which I didn't really think about is like, "Well, if I can rag or I can I can load in my act into active memory."
13:15
And this is very prime agent continual harness kind of like thinking where I can pull stuff in um on the fly example similar examples, then it's like me like, "Hmm, I I a I have an exam or I have an exam with a textbook.
13:29
I'm going to do better if I have the textbook to look up as reference.
13:30
And so that is um you know, definitely going to work and it's still and it also um scales I would imagine much better.
13:38
And then the the last two paradigms would be Laura with rank one or rank two or whatever.
13:42
Laura with rank 10 or rank 100 and then all the way to full SFT RL.
13:50
And if you're Tesla and you have all the data, infinite data and you're learning doing self-driving car by ICL, what are you doing? Like are you kidding?
14:00
It's like clearly we wouldn't do that, right?
14:01
Um and if you're figure or something like that.
14:04
But um it's amazing how good in the in low data regimes uh you can do with ICL and then even cooler we can compress the ICL into tool use into tools or like compress it into learnings.
14:17
And that's that's the hierarchy that we haven't really figured out that you guys are probably on the forefront of.
14:22
>> Yeah, do you guys want to elaborate on that a little bit cuz I imagine that's a central part of how you guys think about building models.
14:27
>> There's a few things that are quite interesting here I think to unpack.
14:29
Yeah, the the first thing is we've been sort of thinking about this idea of a hardness almost as a form of um like domain um specificity.
14:37
Like when when you for example deploy a robot in a new environment, maybe it's in a wet lab and it just needs to do a lot of um test tube picking up for example.
14:47
You can learn a lot of this in context, but then one way of consolidating this context for future agents is sort of packaging each of these skills that you've learned into specific um programs for example.
14:55
Like writing out these skills and writing out these memories is a form of consolidating.
15:01
It's it's like a form of distillation um from past experience for your future agents to use.
15:05
So this is quite interesting.
15:07
The other thing that's quite interesting is um it almost seems like there is some relationship to the broader meta-learning literature uh throughout in machine learning history.
15:18
Like you have this sort of bigger model that then programs a smaller model to do things.
15:21
And one of the very interesting things is that yes, these smaller models are going to learn in context.
15:26
They're sort of wrapped inside the harness and they're doing the task.
15:28
You can make them maybe smaller, you can make them run faster, as long as the bigger model can sort of do this domain specialization of your harness quite well.
15:36
So so this is something that we're we're pretty interested in right now.
15:40
And last point as well, I think the the the really interesting theoretical question is how far in-context learning can get you.
15:46
And I think it's really anyone's guess as to that.
15:47
Like there's papers that show that, you know, these models sort of can approximate gradient descent during in-context learning as well.
15:54
So it's this this line between weight space and symbolic space, I think it's is a little bit blurry.
16:00
>> YC's next batch is now taking applications. Got a startup in you? Apply at ycombinator. com/apply.
16:08
It's never too early, and filling out the app will level up your idea. Okay, back to the video.
16:14
>> So what exactly are we watching here?
16:16
>> Uh we're watching Ashra controlling the arms to pick a block off the table and put it into the bowl.
16:22
>> So here Ashra is using just these camera inputs, and presumably it's aware of the robot that it's controlling, and it's going to write code that controls the robot at its individual joints.
16:31
>> Yeah, yes, so it it does look at all the cameras.
16:33
It has all the camera feeds.
16:36
Um instead of code, it's more like a tool call.
16:38
So it sends commands to the robot to to control and put the block into the bowl.
16:46
>> So now in this situation we're using Ashra directly, but if we were using say Watched Harness, what's the kind of difference between directly using Ashra to control this versus doing this through Watched API?
16:56
>> Sometimes directly, like having Ashra directly command the pose the robot should go to, is not direct the optimal tool to use.
17:02
Um if it's a repetitive task, you don't want Ashra in the loop.
17:06
Maybe you want to write code that can run repeatedly very very fast.
17:07
Or if it's a task you've done a similar task before, you should be able to call a skill that has compiled and use that skill to do the task faster and handle edge cases better.
17:17
So, here we can see it seems to be approaching in on this guy. Oh, very dramatic.
17:22
>> Yeah, you can look uh you can notice that the latency is a bit slow because we are bottlenecked by the latency of Astra.
17:28
Uh but if you look at the trends, the latency of these models are improving very rapidly. >> Right.
17:35
>> So, one thing that we saw is that for stable class LLMs, their latency is improving by around 2x per month.
17:43
>> Which is very, very fast. >> Very fast. >> Yeah. >> Yeah.
17:45
If the trends continue, we could get real-time control by uh end of the year.
17:50
>> So, now that we saw this task go from end to end, why don't we walk through what that actually entailed?
17:53
So, there's a coding agent in the loop here running this. >> Yes.
17:57
>> Um what are the steps that the coding agent would have taken?
17:59
And let's kind of contrast the uh direct uh coding agent version versus the water harness in the looper. >> Yes.
18:06
So, what just happened was that um in a series of turns, Astra received the uh images from the cameras and it outputted the end effector pose for the robot to go to.
18:17
And it does it over multiple turns to complete the task.
18:23
This is less of code as policies but more of um tool calls.
18:27
>> The on-screen part is pretty repetitive.
18:29
So, you could actually automate a lot of that using code uh as as well as actually approaching the bottle, picking it up.
18:34
Like a lot of these things um are pretty uh deterministic once you've done the task kind of a few times.
18:40
Um the interesting part is where you're building the variation into the code's policy graph.
18:45
Um and so, we have a few points of variation.
18:47
This can come from, for example, when you detect an object.
18:50
Maybe you use a VLM as part of the tool call.
18:52
Uh or when something fails, for example, how do you check that it's failed?
18:56
How do you then um potentially do something else depending on the failure?
18:59
Like these kind of more uh flexible responses, we tend to put a VLM um inside the loop to ensure that, you know, while while while the coding graph is sort of deterministic, there's points of variation that allows to >> The biggest insight I think I've had a change in world view on how to perform machine learning and how to get the rest of the distance on AGI was like a lot of conversations I've had with Francois Chollet.
19:22
And in 2020, maybe even 2018 when he did on the measure of intelligence, he talked a lot about transduction versus is just wrong versus program induction.
19:32
And transduction just means I'm learning a function theta that maps from x's to y's. And why is that wrong?
19:39
It's just slow and it's information inefficient.
19:42
And so to go from x's to y's, I need a lot of pairs of x's and y's. >> Mhm.
19:47
>> And if I have a very small amount, then you need a lot of inductive bias and you what you need to act is a good generator to map from x's to y's.
19:53
So now my theta takes in three n n pairs of x's and y's and emits the function that maps from x to y. >> Yeah, yeah.
20:04
>> And that's what code is.
20:04
And so that's it like if you give me a coding interview with a whiteboard, you say, "Okay, here's a coding problem, here's some examples.
20:11
Okay, cool, write the function."
20:11
And then that we are emitting an f that will map from x to y.
20:17
>> And and this is quite interesting, right?
20:18
Because I think I mean this was a lot of the traditional program synthesis literature.
20:22
And I feel like a lot of reason why those methods didn't work as well was because that inductive bias as you said, right?
20:27
It's you're shifting the difficulty of the problem from finding that mapping to find the right inductive biases to map it to this like smaller code to then do the mapping.
20:36
But finding that set of inductive biases is super hard.
20:38
But maybe that's what Astra is buying us.
20:41
Like even in CogSci, there's a giant shift in the literature from, you know, specifically neurosymbolic methods to just using Astra to write code to map.
20:49
And so maybe that's a cool way to think about it. >> neurosymbolic. That is neurosymbolic.
20:51
We have neurons and then they're emitting symbols which are code.
20:56
>> I think this is actually a pretty good segue to the kind of final topic which is around inspired by this paper also from Filippo Ola that he titles the Platonic Representation Hypothesis, right?
21:05
Where Platonic because uh is a reference to Plato's Cave and you know, the idea that you know, there you're seeing shadows of uh various realities.
21:13
And I think here the um point he's making is that um there's extensive evidence to show that um language models and various representations of different data is actually learn these uh distance mappings between uh similar things under different training policies that are um overlapping.
21:33
And so, like the idea there would be that as you train these systems on increasingly large amounts of data, they converge to um a kind of consistent mapping of the world.
21:44
And maybe this implies that we would expect language models to get better and better at things over time.
21:50
That would make them useful for new tasks like robot control.
21:51
But maybe I'm I I imagine this representation hypothesis is actually pretty central to all of your guys's worldview or your view of your companies.
21:58
And so, maybe one of you guys tell me a little bit about how you think about this.
22:01
And also, maybe we can use that to make some guesses as to why we think models like Astra seem to be so much better at robot control than previous models.
22:09
And where, you know, make some predictions for where we might be going here. >> Yeah.
22:13
I think one way to make this concrete for the uh robotics models versus language models debate is that the very, very strong language models will have very similar representations of the world with very strong robotics models.
22:25
And if that's the case, then if you have a really strong uh language model, you also have a really strong robotics model. >> Right.
22:34
>> Uh using if we believe in the Platonic uh representation hypothesis.
22:37
And uh if that's the true, that then beta lesson is that is the best manifestation of the beta lesson, right?
22:45
Because you just need one really strong model regardless of architecture, and they would be outperforming any specific models uh that is slightly weaker.
22:56
>> The interesting thing that that's maybe not entirely intuitive and maybe I'd be curious to hear you guys all think about is why specifically it seems like these newest models, specifically Astra, seems to be so much better at spatial intelligence.
23:05
You know, we've all seen the demos online of it, you know, controlling Blender and making great 3D images.
23:09
Now, it seems like uh you know, U Show J in a benchmark that is like a meaningful step function improvement on um certain tasks.
23:16
Um where do you guys think the the pre-training and end post-training I guess of those models, what is likely changed about OpenAI's approach there that made them so much better compared to models even 6 months earlier?
23:28
Like, you know, that's still benefiting from the bitter lesson and we're still extremely good at code and we're still solving math problems, but haven't quite cracked the spatial intelligence needed here.
23:37
>> I think what Astra does incredibly well is it's like vision capabilities.
23:39
It was probably pre-trained on way more computer used data than ever before.
23:45
It's probably pre-trained on so much like CAD data.
23:46
And it's like all of these kinds of data probably teach the model like similar uh like understanding of like physical world as like a lot of robot data might.
23:58
>> So, the computer used data is an interesting point because it's not totally intuitive, I think, why computer used data is useful for understanding the physical world and that it you know, tends to be like a computer where you're clicking around.
24:06
Like, say more about why you think that is a big unlock because I totally buy they trained on way more computer used data than ever before. >> Right.
24:13
Um I think immediately if you were just training a model for computer use, that would not be able to apply to robotics.
24:17
But, if you feed a computer used data into a big model like Astra, um and by computer used data I mean like you drag a cursor around on a screen to orbit some CAD object in order to design in Blender, this tells you um like how to reason about spaces.
24:30
It at least tells you about like top down, left right.
24:34
All of these concepts that you need to control a robot, right?
24:35
Um and I think that's why this data helps so much for making these LLMs so much better at robot use.
24:43
I think like from the other direction, like a lot of people in the robotics community were trying more and more different kinds of data as well. Like, egocentric videos.
24:50
Kind of went from just teleoperation to to like, you know, a broader range of this data um because it's not just robot data that can teach a model how to use a robot.
24:58
Um so if we take this to the extreme, it's like why not feed every kind of data, uh coding, computer use, egocentric into the same model.
25:06
I think that's how we get to the most capable like robot use agent.
25:11
>> Yeah, it's so funny that like, you know, if you go to 1980s Xerox Park or whatever, like we literally made the graphical user interface to be more like the physical world so that we could interface with it.
25:20
And like what we ended up doing is building an environment that was actually helpful for robotics to learn how to use the physical world. Right?
25:29
We we have file systems, we have like files, we have folders, we have, you know, the GUIs for like SolidWorks and like Autodesk to like spin around stuff to make it simulate similar to the physical world.
25:39
And then like we couldn't get robots to work in the physical world, so we just trained it on that and then now it works in the physical >> Right, exactly.
25:46
There were incredible papers, uh I think from Princeton and also clip clap base robotics touches on this.
25:51
It's like we just if you design a right harness, where you make the tools that can face with the robot look like computer use tools, like you have a agent dragging cursor around to control where the robot goes, that improves how well the LLM uh is able to perform on these physical tasks.
26:05
>> Where do you guys see, you know, given like any reasonable guesses as to where the base models are going to continue improving and your guys' own investments in either evaluating these models or building harnesses around them?
26:18
What do you think is going to be capabilities that we maybe now see as challenging to do, but that are going to be increasingly possible or even trivial for a few months from now?
26:25
Like, you know, 6 months ago even the demonstrations that I've seen you guys post on your Twitter I think would have been kind of mind-blowing to imagine coming from an LLM.
26:32
>> There's some consensus within the frontier labs and also in the robotics foundation uh robotics companies that we will have general-purpose robots within the next 2 years or even earlier.
26:41
And this is something that society is probably unaware of or even unprepared for.
26:45
And when we say general-purpose robots, we mean something like if you give any natural language instruction, it can do what a competent teenager could do with their hands.
26:56
It's kind of like the chat GPT moment but for robotics in terms of capabilities where it can generalize to unseen tasks and unseen environments.
27:06
>> For you guys, for the Wild Labs guys, what does that mean for what you guys are building?
27:09
I think >> we want to take steps to get there in about 2 years time.
27:13
I think there are so many challenges that are very visible.
27:16
For example, what what people online talk about latency, this is a big problem.
27:19
I I think if you just have Astra being in the loop thinking at every step, this is really slow.
27:23
It's not going to be economically like useful.
27:26
So, how do you you know, consolidate kind of a first pass by Astra like this this this contextual learning into like a faster skill or policy that you can then run repeatedly at incredibly high throughput.
27:40
>> I think it's that the mappings to humans will get more and more like this where like the the the optimal thing to do may be to to put a new S AR state action reward back into context is very quick, but then there needs to be some like of go to sleep for a while like almost everything that is intelligent sleeps.
28:00
Like tell me an intelligent system that doesn't sleep, right? In some way.
28:02
And then during sleep compression happens.
28:05
The weird thing that happens from your hippocampus and it short wave ripples to both lobes and like there's weird you know, paths from memories that were compressed throughout the day to train the weights.
28:15
And similarly, maybe what the right thing to do is similar to dagger data segregation framework in our classic RL where you're you're going, you're collecting a bunch of data, and then you're somewhat reflecting on it, and then you're using it to update your weight file, and then you have distillation in from those experiences back into an updated weight file that maybe isn't Astra, but maybe it's your own models.
28:36
>> It sounds very Dream Coder >> Mhm. >> ask. >> Mhm.
28:38
>> But a lot of those >> specific tools I think that people used to build.
28:42
>> To take Dream Coder for example, right?
28:43
It had this library of skills, and then during the sleep phase, it basically refactored everything into more compact representation and so on.
28:48
I mean, we're seeing sort of similar things just happen not as rigidly as before in the space of programs, but now it's maybe like maybe you want to refactor traces.
28:59
Maybe you want to refactor skills.
29:01
A lot of these robotic code as policy papers, right?
29:02
They have this growing ladder of skills and it's anyone's guess really how you prune them, how you organize them and things like that.
29:08
So, yeah, I think as as world goes forward, like how you manage a growing contact of skills um of deployment data, all of that is going to be quite interesting.
29:19
>> I think with that, I think this was an excellent discussion.
29:21
Thank you so much Vincent, Hanmin, and Jay for being here.
29:24
I'm very excited for all the incredible robotics advancements I think we're going to see over the next few years and I think you're totally right Jay that I don't think broader society is totally aware of how much is coming and it's going to be a very incredible few years to come. So, thank you so much.