0:00
I told the team yesterday that the real cost of this 1.
I told the team yesterday that the real cost of this 1.
2 billion was not dilution.
Uh the real cost is is expectations.
Not only are from our investors because I think our investors they they understood and they agree on the long-term vision of the company, but the expectations you create in the outside world and if you raise 20 million and you do nothing for 2 years or or people don't see anything uh coming out of it for 2 years, then it's very very hard to survive.
>> So, we I actually met Alex.
Uh fun fact, we actually did the same YC batch.
Uh back then Alex was building Wit AI and we were building Algolia as I was in winter 14, quite a while ago.
Uh except Alex has done four companies, like one before, two after.
I've only done one, so like I cannot compare.
So, I'm super impressed by your like all your background and everything you've done.
What do you remember from Wit? Like wit.
ai, was that like the first AI company or something?
>> It was Yeah, it was the first . ai domain, actually.
Uh it was very hard back then to buy . ai.
It's it's a island, you know, off Australia.
>> Uh did you uh like did you know about it or did you found it by chance?
>> No, I found I found this and I thought, "Oh, it would be cool to have this domain."
Uh but nobody understood it was a domain back back in early 20 uh 13.
>> Yeah, that's how old uh we are.
>> [laughter] >> Um and then >> [clears throat] >> uh I was surprised because actually you sold Wit shortly after to Facebook, right? How did that go?
Like any uh any memory of that time?
>> Uh yeah, so um in 2015 I I received the actually an email from someone called Mark Zuckerberg.
Uh and I I after like half a second, I deleted the email without opening it.
it's it's a scam, you know, it's a spam.
And took about half an hour and then I thought, "Oh, but maybe it isn't, you know, I should check."
And I opened my spam folder and and it wasn't.
So, it started like that.
>> Uh and so, what happened after?
Like you met him, like it was real, you met him, and he made an offer? Like is that it?
>> So So, we actually followed the the YC playbook, you know, you're not supposed to >> If you don't want to sell, don't don't speak with the >> Yeah, you're not supposed to speak to anyone.
As soon as you start a startup, you receive lots of offer to talk.
But usually these are from the corporate development of large companies and they need to talk to everyone to understand the market and they need to know if you're desperate enough to sell for a low price.
And so, it's a very bad idea to waste your time and your attention.
So, YC tells you never talk to anyone unless you are selling your company.
>> Unless you want to sell.
But that was not the corporate I mean, or maybe the corporate arm of Facebook was so smart they got Mark to send the email.
>> Actually, the rule book is if someone really has a power to do something big, maybe and you you know, you can talk. So.
>> So, tell us what was the Within in a way, it feels like all your companies were very early in what they were doing.
Like Virtual was already conversational agents right before that name even existed.
>> So, this one was a really really too early.
>> But But Wit was also early.
>> Wit was not that early.
So, Virtual was my first company almost out of school was a chatbot company. I started in 2002. So, you are not born.
>> So, 20 20 20 years too early.
>> Yeah, 20 years too early.
But But still we made it, you know, we made something with it.
But But then people ask you So, I'm trying to sell chatbots for customer service in 2002.
And people ask me, "But what is a chatbot?"
And I say, "It's like live chat, but it's automatic."
And they say, "But what is live chat?
So it was really really uh too early uh and we we we spent 10 years with this company.
We survived uh long enough to and then there was actually a market uh in 2010 as a nascent market but it was definitely too early.
>> So you sold uh you sold Yext was you sold Wit then you spend a lot of time at Meta and then you created a third company uh Nabla and then at last uh Amilabs uh was at the target this year? >> Uh yes in January. >> January.
Uh it feels like every single of these companies was kind of early.
Like even even Amilabs right?
In a way you are going after a big new approach to AI that could feel very early today.
Is that something you like that's in you you always want to really uh >> I think it depends what is your strengths.
Um if your strengths as co-founders uh is go-to-market sales you know marketing then you shouldn't be too early.
You need a product to be ready you need the market to be ready to to receive your your go-to-market.
Um and so if your strength is to push a product to the market then being too early will kill you.
I think my strength and I think usually it's a cliche but in in France probably you know we are most famous for engineering and tech more than genius of marketing and sales.
Um so so in my case I think what I like to do is take very very early tech and and think about what product how how we can sell it through a product uh build this product understand the early market so I think my my skills are more on the early side of the go-to-market so that's why I prefer to take early ideas.
Uh but it really depends on on the what you want to to focus on. >> Okay.
Let's let's continue with Amilabs.
Uh it feels like it's the most ambitious project ever for you, right?
I mean, the first one were normal startups.
This one is not really normal.
Like, what led you to actually take that leap?
>> So so I mean it's I mean it's it's a it's a normal project.
It's just very very expensive because if you want to to build a foundational model, you need to buy thousands and thousands of GPUs.
And so you need billions of of euros.
So this is what is not normal in this project.
Um After So I've done several startups, always on the applied AI space.
So I'm using machine learning research or models that other other people have done.
Um and after all these year work having the opportunity to work upstream on the solution on the model that you know, if we are successful, millions of people will use is very exciting.
So moving upstream is what I think is the most exciting. >> Okay.
How many here know what word model is?
I like more than I would expect.
I had to research it myself just to make sure I understood what you are doing.
Because it's kind of like a very different take at the market than like LLMs are kind of like the one thing, right?
So in a way it's contrarian.
Doing contrarian for a while, why wouldn't LLM do everything?
Can you tell us a little more about what is the word model and what's your kind of like a like your bet on the market?
>> Yeah, so so first I'm happy to see that many people have heard about it. >> How many were honest? I don't know.
>> [laughter] >> So my my co-founder Yann LeCun speaks often.
So maybe >> You all heard from him.
>> So so basically as you know, large language model learns to manipulate language.
And so can read LLM will be trained by reading all the texts, all the books, all the internets in the world.
Uh and so LLM is learning what we humans have written about the world.
So, it's a kind of proxy.
The LLM doesn't work doesn't learn directly from the world, the real world.
We humans have wrote wrote books and articles about the world and then the LLM learns these texts.
An LLM is would like is like someone who has never go outside, never left the the room they were born in, and but read all the books every day for for centuries and centuries.
And so, this person would be seems like an expert in many things, but with no direct experience of the world, will have limited no common sense and limited ability actually to to find new solutions or to know what to do when facing a situation for the first time, lacking the direct world experience.
So, a world model is a model where we don't cheat.
We are not learning through text.
We are learning directly like humans, like animals, from the real world.
So, meaning video, audio, sensory data, uh playing with objects.
So, we are really starting from scratch like a baby, uh without any language.
>> So, what's the like the training data?
What kind of training data are you using?
>> Uh we are using a lot of video data because it's the easiest to to find, but um but it works with audio.
It would work with only like if you were born without sight or or hearing, you can still live.
Uh so, it's possible to to live through touch and it's the same in what we do.
>> Can Can you collect touch data? How do you do that?
>> Through through robotics. >> Okay. That's interesting.
So, once you get that model, what are the use cases?
Like what can it do that an LLM cannot do?
>> So, in the short term, there are lots of applications, especially in robotics.
Uh current robots are are very very narrow vertical robots.
They they do one thing in a in a you know, in one position.
Uh, and it takes a lot of time to train them to do only one thing always the same.
We are not ready at all to have these robots uh, be in an open environment whether it's in your house or in the street or, you know, and you see these videos of robots that start to dance and break everything around or I don't know if you've seen the one where a robot is doing karate demonstration and does like that and there is a child and the child is So so these robots are very very dumb.
The hardware is incredibly uh, evolved but but the brain is hasn't changed uh, and and we you cannot take these robots in an open environment.
It wouldn't be safe and it they wouldn't be useful.
So this is a a first um, application.
>> that differ from like VLAs for example?
>> The VLA is a very bad hack where you have a hammer, you have a LLM and you you try you see everything as a nail.
You are trying to use your your solution for for things that were not it was not designed for.
So a VLA uh, is is not accurate.
It's very slow but you need very fast, you know, very short latency to do these applications and it's very expensive.
You cannot pay uh, you know, $100 of compute for 24 hour of your robot home robot to work.
So there are many many problems with VLA.
>> And so like what would be a better solution?
Are they uh, easy like in terms of inference? Are they costly? Can they be real time?
>> So so they they are very costly to train like LLMs.
Hence, you know, the GPUs we have today.
>> More or more or the same? >> It's comparable. >> Okay.
>> [laughter] >> It's hard to compare because it's not the same structure or the same uh, but when they are trained um, they are supposed to be more lightweight for for several reasons but you have less parameters.
Uh, so inference should be uh, less expensive. >> Okay.
Uh, in the world where world model are like in production everywhere, uh, what is still good for LLMs? Like is that >> Yeah.
>> one replace the other or like do you keep >> No, no, I'm not saying LLMs are not good.
LLMs are will be are already very, very good as you we all know uh, for any um, everything that is language first, you know.
Uh, math everything that is a sequence of symbols like discrete and low dimension problems.
So, anything with language, mathematics, programming, these are all low dimensional sequence of symbols.
LLMs are designed for that.
They will it works we will never beat I think the LLM at that.
Uh, but then you have lots of high dimensional, very noisy, a very long horizon problems for which we think world models will be vastly superior.
>> So, we're speaking a lot about the future.
Um, like is there already world models in prod?
Like with some flavor of it at least?
>> There there are some ideas behind world models in in production.
For instance, in some autonomous driving companies are using some flavors >> Is is Waymo using a world model?
>> Some flavor of and then Wave also in the UK.
Some of the ideas we are we are using like a Jepa are have been around for at least 5 10 years.
You know what happened in 2018?
The theoretical idea behind LLMs like Transformers, the paper was there, but but Google did nothing with it.
Nobody did anything with it.
Nobody realized the the power. >> Is that the pattern?
>> It's it's and then OpenAI, you know, we we can criticize OpenAI for many reasons, but they had they took this and scaled it, re- solved some engineering, you know, scaling issues.
>> Is DeepMind is uh, Meta today like because it was developed there at first?
>> I I think all the large companies back then, you know, Meta Meta I was at Meta so I can I can I know exactly there are many reasons why in a large company you there are things you cannot do that as a startup you can do.
Uh you can take lots of risk.
If you don't take any risk, you're dead because the big companies are are are have more resources than you.
Uh they and they have a brand and they have many many things you don't have.
But the one thing you have that that they don't is the ability to to take crazy to make crazy things to take some risk.
And the reason when I was at Meta for instance, we we we worked on a on a chatbot which was a language model not large yet, you know, but the idea uh that we trained the >> Just out of wit. ai, yeah?
>> Yes, but trained statistically without rules but with trained from statistical data. We worked a lot.
We had a lot of fun and the first time we launched this chatbot, you know, in in in the lab at Meta uh we asked a question of how are you today and and the chatbot answered, "Oh, just chilling watching porn on my couch."
And and then I before that I wanted to release this chatbot.
>> And that was not a >> And then I thought, "Okay, I the legal team, you know, they will never ever uh want to have this discussion with me."
And so we knew we cannot release it. It's just an example.
Um and so OpenAI could do could take some risks with with GPT-1, 2, 3, you know, until ChatGPT.
That that no one else could could do.
So they have the ability to they have enough money and resources to scale this training.
>> [snorts] >> And but the ability to take risk at because yes, they they had this combination they they made it what became ChatGPT.
>> And so I just finish on that like what's the current bottleneck?
Is that training data then?
>> So you need three things.
You you need smart people, you know, talents to understand because not so many people, you know, understand that.
>> Most of them are working on LLMs.
>> And most of them are like LLM field like Yann says.
You need a lot of data and you need GPUs computes which is very even when you have money it's very very hard to secure GPUs.
>> [clears throat] >> you I mean you've already hired quite a lot of data talents and you brought a lot of money. So what's left data?
>> Data and compute even with money securing compute today is is difficult. >> Okay.
Um It's kind of like a contrarian bet.
Like what convinced you kind of to bet everything on it?
>> So I I don't think it's contrarian.
It was maybe contrarian a year ago but I think it's becoming less and less.
A year ago when I when we were saying the LLMs are are nice and powerful but they're not AGI.
They they don't lead to artificial or super intelligence.
Um most people disagreed now I think most people agree that LLMs don't lead maybe not most people but a good chunk of people agree that LLMs don't lead to a general intelligence.
Um so now you can disagree on what leads to >> Yeah I mean the definition is also fuzzy so >> The fact that the definition has changed so there's a very people who promised AGI changed the definition and stopped using the term.
I think it's a good sign that uh something you know is is is not working.
>> And you're doing that with Yann and you met Yann at Meta I guess.
>> Yeah yeah I mean so yes of course.
>> So you're working with him before so you kind of like knew him enough. >> Yeah.
>> How did that go like did he coach you to to for the job like like how did you end up working together?
>> Uh so so so Yann was an investor and a advisor in the in Nabla, my company prior to Ami Labs.
So, I saw him very often.
And one day they came to me and said, "Hey Alex, uh >> [snorts] >> I think world models are ready to to become very big and I think we could go faster if we do that outside of Meta as a autonomous, you know, startup.
Uh so, I want to do it and I need a CEO.
Uh >> [snorts] >> so, >> How you appointed?
>> [laughter] >> Yeah, I mean, you know, I spent as I my first company was doing chatbots as I said and after 10 years trying to teach a machine to understand human language and concepts for customer support, my conclusion was we cannot cheat.
We cannot a machine can speak about a cat, but if the machine has never seen a cat, you know, touched a cat, played with a cat, uh maybe be hurt with a, you know, by a cat, then it will never be very, very smart.
You need When we talk about a cat, it actually reactivates in our brain some past experience we had with cats.
This is my Um and even if you you read a book about some animals like that you've never like a dinosaur like you never seen, um the way you imagine a dinosaur is built on basic objects uh that you had experience with.
So, if you go, you know, so what we call grounding, um you need all your experience of things to be smart.
This is my It was my belief before 10 years ago.
>> Just need to see a juicy park and you're good. >> [laughter] >> Okay.
Uh And uh how do you like with uh with Yann, how do you split responsibilities then?
Like because he's still the inventor, like the researcher who kind of like came up with that uh J bar thing.
Uh are you more like on the execution side making sure you can uh build a product out of it?
How do you split the the roles?
>> Yeah, so so yeah, Yann Yann is the executive chairman of the company.
He has a scientific vision.
Uh and so he is leading the direction of the research.
And then my job is to make sure that there are people, we have electricity, GPUs, data, offices.
>> [laughter] >> So if it fails, it's because of you.
>> Everything is working, of course.
Uh and I love this And also, uh it's it's very hard to manage researchers.
You know, there is a joke that it's like herding uh cats.
Um if you have to leave You cannot tell researchers what to do. They hate that.
They will flee if you try.
Uh but on the other hand, if you leave researchers alone, they will just do very random things and run in all directions and you will never get anything of the of them.
And so, uh finding the very very tight balance between uh direction, enough direction with enough freedom, I think to in order to get a product one day, is what I like to do. >> Excellent.
Uh you also ended up raising the biggest ever seed round in Europe, right? Like 1. 2 billion.
Is that in dollars or in euros? Okay.
Well, close like 1 billion in euros, right? Uh walk us through that.
Like like Like first, maybe could you ever imagine raising that much money?
>> No, I I couldn't never imagine I think no no one could imagine but a few years ago that any startup would need so much seed money.
Uh the real crazy craziness is the cost of the of compute and because you you need a lot of compute, you need a lot of money and then all everything has shifted.
Uh so yeah, this is a new territory for for everyone.
What's difficult is when you start a company, I think having a culture of not spending too much money, be very careful, spend money as if as if it's yours, is very very important for the company.
Very hard to change later if you don't start like that, being very cautious with money.
Uh but but it's hard because if you spend 1 billion for GPUs, then everything else looks like very cheap.
So, it's it's very tempting to >> And how about finding the money?
Was that easier than expected or >> It was um not that hard actually. So, the >> There you go.
>> [laughter] >> You should have aimed for a billions, not millions.
>> the rule is not the amount It's not the the difficulty doesn't depend on the amount.
It depends on if it is your first time. >> Yes, of course.
I don't think anyone else could have done that.
>> First time you raise money for your first startup is hard.
And you you've been all, you know, through I remember, you know, my story your story.
You have to go through this uh unless, you know, you're very lucky or you're Um and then second time is is >> Just do IC that helps.
>> [laughter] >> Second time is easy and then after third time people just beg you to give you money to do anything you want.
So, but you have to go through this journey.
>> And because the first two times were successful, that helps too, I guess. >> Yeah. >> Yeah.
Uh I mean a seat with that size like kind of like Bobby said the expectations uh before you even start walking.
Uh I mean, does that money buy you buys you passions or does it put you on the clock?
>> So, I told the team yesterday that the real cost of this 1.
2 billion was not dilution.
Uh the real cost is is uh expectations.
Uh not only uh from our our investors because I think our investors they they understood and they agreed on the long-term vision of the company.
But the the expectations you create in the outside world.
And if you raised 1 billion and you do nothing for 2 years or or people don't see anything uh coming out of it for 2 years, then it's very very hard to survive.
So, the expectations is uh That's harder. Yeah.
>> And do you have a timeline?
Like like maybe you don't want to >> see expectations coming. [laughter] >> Yeah. Then it was course. You raised a lot.
Now you have to >> We we We have internally we have a timeline, uh but >> You're going to see anything this year?
>> We don't communicate about it.
>> [laughter] >> I I tried. I tried.
Um and you do that here in Paris.
And like where kind of like pretty much nearly everyone else is building like these labs in SF. Why Paris?
>> So we do this in Paris.
We also have a big team in New York, one in Montreal and one in Singapore.
Uh we we have these four cities because we had existing teams. >> I didn't hear SF. >> And no SF.
So it's a kind of >> That's a contrarian bet, too. >> A little bit, yeah.
I think it's uh when people join Amie, we hire a lot of people in SF in in the companies you can imagine, uh they have to move to New York.
It's a like a commitment and uh it's very important at this early stage to have people who join for with a real commitment and not just because uh they want to try.
So it's half >> [snorts] >> uh one day we'll have an office over there, but it's it's kind of on purpose.
>> And do you feel you you may miss anything by not having a foot there?
>> No, because we we we are over there very often, you know, we live over there.
We we are >> So you're you're still super connected to SF and so you kind of benefit from the ecosystem anyway. >> Exactly.
So I think it's uh it's fine. >> Okay.
Um Amie Labs is probably one of the most ambitious uh project or the most ambitious project you've ever done, but like uh sounds very ambitious.
Uh how could you What could you say to this room of talented kind of like builders uh who probably are picking less ambitious ideas than yours?
Uh how What could you tell them about how you learned about being ambitious?
And should they be that ambitious?
>> I think we are never enough uh especially in in France, in Europe, we are not ambitious enough.
You know, we don't need to say cliche, but it's true, we don't take risks.
Um I was telling you, you know, when I just 1 month after we were acquired by by Facebook, uh, I wanted to hire 100 people to do some kind of data annotation for machine learning.
>> You mean like a for with AI like >> For with AI, yeah. Yeah.
So, I wanted to hire 100 concierge uh, who work for real customers, you know, to book travels for instance, uh, and record everything they do in order to train my model.
So, it was in 20 15, 10 years ago.
Uh, and I prepared a memo to ask the permission to the cab board to hire 100 people.
They were not planning the budget of the company.
Back then, there were 6,000 people at Facebook.
Um, >> Yeah, so 100 was not nothing.
>> It's not nothing, you know, and 100 concierge in in Menlo Park is expensive.
Uh, so I sent when you meet the cab board, you have to send you have to send a memo 24 hour before at least or your meeting is canceled.
I arrived there and I was ready to argue to give every scientific argument why what I wanted to do was a great idea and we would and I I arrive at meet him and he tells me he had already the memo before and he tells me, "Alex, I love the idea. That's great.
I think we will kill Google because we will capture the intent and so with advertising and everything.
So, let's hire 10,000 concierge, not 100."
And then I I was >> [laughter] >> "No, no. Don't do that actually."
I I had to tell him, "No, no, no, no, Mark.
It's actually I'm not sure it will work. It's very dangerous.
There are lots of unknown in this project, so we should not um, to this day I don't know if he was really willing to hire 10,000 people or if he if he thought it was the best way to to get the real information about the what couldn't what could go wrong in the project, but I loved the the ambition and we hired 200.
>> Like 200 instead of 100, okay.
Like more than you expected.
Well, Mark has been well known for taking wild crazy bets on stuff.
Sometimes it works, sometimes it doesn't.
So, not that surprising he would go there, but yeah, kind of probably erased the bar from what you think is ambitious.
Um what should you what should the audience uh remember from that?
Like uh how high should they aim at their stage? >> What's that?
>> Uh how high should they like at at one point um people think too small when they're starting.
Uh they they try to find a safe idea. >> Yeah.
>> Or should they actually aim for something a lot more high potential, high risk?
>> No, I if I could go back you know, I think when you start uh you should look at a very narrow problem, but be very very ambitious in the in this narrow problem.
I think the the the including myself, the the one thing I did wrong is I wanted to solve to do a chatbot to do that does everything uh for everyone, so not narrow, very very hard to provide value as a very small team small small startup.
But if you choose a very narrow problem like doing one thing for one industry or for one set of customers, but in this narrow tunnel uh have a very big vision.
Don't be afraid to share your long-term vision.
Okay, I'm doing this today, but this is what we want to do later.
Um this is how I I I what I would do >> You have only one life. You're young.
You can take some >> you're very ambitious you are in a very wide field, then you're not credible. >> Makes sense. All right.
Uh just to conclude uh if Amie works, like like no, when Amie will have worked, uh what will the world look like?
Like 5, 10 years from now?
Can you uh can you give us a glimpse of the future you you envision?
>> Yeah, we we um first I think we'll have uh robots, you know, helpful robots, actually your helpful robots.
Uh I think most jobs will still be here, but not as uh dangerous or hard to do as they are today.
And yeah, I think machines with some sort of common sense will make a big difference. >> Awesome.
Thank you so much Alex for joining us today. >> Thanks again. >> [applause]