0:05
Welcome to Stanford, Luis.
Search this video
0:06
>> Speaker 2: Thank you for having me.
0:08
>> Speaker 1: We're so excited to have you here for the Last View from [LAUGH] the Top.
0:12
>> Speaker 2: Some of my executive team is just sitting there, okay.
0:14
>> [LAUGH] Well, we're excited to have them all here too.
0:18
We're so excited to have you here for the Last View from the Top of the year.
0:22
And it's a small world, because Duolingo's CFO, who is also here in the audience today, used to sit in my chair as a former View from the Top interviewer.
0:33
>> Speaker 2: He's right there, he's right there.
0:34
>> [APPLAUSE] >> Speaker 1: Luis, we have a lot that we want to get through, but before we go any further, do you mind providing a verification code for us?
0:45
>> [LAUGH] >> Speaker 2: Man.
0:46
>> [LAUGH] >> Can I tell you that it's really, really annoying when I fail them?
0:52
>> Speaker 1: [LAUGH] >> Okay, then we'll let you slip by with this one and we won't make you do it just in case.
0:59
Luis, it feels like from the start you were wired to be curious.
1:03
You taught yourself to code at the age of 8, and by 13, you knew you want to be a professor of computer science.
1:09
What drew you in so early and how did your environment shape that?
1:15
>> Speaker 2: Well, I learned how to code because of necessity.
1:18
I was 8 years old, and I wanted a Nintendo, and my mother bought me a computer instead, and so, I had to figure out how to use it.
1:28
At the time, it was a lot, this is the eighties, so, the computers were a lot harder to use, so I had to figure out how to use it.
1:35
And then, I at some point realized that you could actually get games for the computer, but those cost money, and my mom wouldn't get me games for it.
1:43
So then I had to figure out how to make my own very crappy games, that's why I learned how to code.
1:49
And then, that just made me kind of a computer nerd.
1:54
And then at some point, for some reason, I decided I wanted to become a professor.
1:59
And yeah, that's what I wanted to do throughout high school and when I went to college and then I became a professor, and it turned out I did not like being a professor, but that's what I wanted to be in my life.
2:11
>> Speaker 1: Well, I want to touch on that because you're right, you became a professor at Carnegie Mellon, but you tinkered a little bit.
2:17
>> [LAUGH] >> But you were tinkering on the side, captcha, recaptcha the ESP game.
2:25
So, tell us about that, when did you realize, you didn't just want to study those problems, you want to be out there solving them?
2:33
>> Speaker 2: I mean, when I was a grad student, I mean, usually you go to get a PhD.
2:39
About half the people who get a PhD in computer science become professors, so I wanted to become a professor.
2:43
When I was a grad student, I was building things, that's what I wanted to do.
2:51
I got a lot more reward internally from building things than from writing papers.
2:57
Over time, that's when I realized the job of a professor usually is to write papers, and I'm not getting as much reward from that, yeah.
3:06
>> Speaker 1: And part of those early papers that you were studying and your thesis was this whole idea of human computation.
3:11
And you had this interesting insight that gamification could be a tool for scaling.
3:17
What made you at that time believe that Play Not Pay, could be an unlock for solving some of these real problems?
3:23
>> Speaker 2: Yeah, that was very early, this is before the word crowdsourcing was invented.
3:30
I saw a talk by somebody, I don't remember who it was, that was talking about how computers were not as good as humans at solving crossword puzzles.
3:42
This is the year 2002 or something.
3:44
And it was an ongoing research problem to solve crossword puzzles with a computer.
3:49
That's funny, because now they're way better than we are.
3:53
But at the time, that was the problem that people were trying to solve, and I didn't know how to solve that problem.
3:58
I mean, that person was significantly smarter than I was.
4:00
But then I took a flight right after, and in my flight, I just turned around and everybody in the row was solving a crossword puzzle.
4:08
And I thought to myself, why do we need to get computers to do it when people are just doing it here for free?
4:14
>> Speaker 1: Probably different than what people are doing now on planes [LAUGH] >> Speaker 2: Yeah, so then that's when it occurred to me that maybe there's all kinds of things that computers could not solve, that we get people to solve by just turning them into a game.
4:23
And so, I started doing that, I started making games.
4:25
This was again, 2003 to 2004.
4:28
I started making games that could get people to solve problems that computers could not solve.
4:33
So, for example, again, another problem that back then was unsolvable by computers, giving it an image and telling it, asking the computer what's in the image, that was a very difficult problem for computers in 2004.
4:46
But what I did is I made a game that was played by actually millions of people where the game was just.
4:51
Here's how it worked, you went to this website and you got randomly paired with another person, just completely random.
4:59
And you knew that you were paired with another person, you knew nothing about them.
5:02
The only thing you knew is that you could both see the same image, and now you're told, type whatever the other person's typing.
5:07
And then when you both type the same word, you got points, that was the game.
5:12
It turned out this got played by millions of people, but what was amazing was that the words that people were typing to try to win the game were actually good descriptions of the image.
5:21
So we used it to collect a lot of descriptions for images that were then used to train computers to do it, that was the idea.
5:27
It was basically, tricking people into giving us data.
5:30
>> Speaker 1: [LAUGH] >> And what eventually happened with that game that you created?
5:34
>> Speaker 2: That game actually got sold to Google and it was then renamed.
5:38
I called it the ESP game because it was an extra sensory perception, I thought that was a good name.
5:43
Then Google bought it and they called it the amazing name of the Google Image labeler.
5:48
>> Speaker 2: [LAUGH] >> Much less fun, but yes, it actually helped with all kinds of things with image search.
5:55
>> Speaker 1: Well, Luis, this is just one of the various creations you had at that time.
5:59
And eventually, you also had tenure at Carnegie Mellon.
6:02
You were pretty set, a lot of us would say.
6:05
So why were you so hungry to build something new with Duolingo?
6:09
>> Speaker 2: A couple of reasons.
6:11
I really wanted to do something related to education, I've always wanted to do something related to education.
6:17
The other thing is I had a PhD student named Severin, who needed a PhD thesis.
6:22
And so, we started coming up with things that he was going to work on, and that we were going to work on together.
6:28
And then, we quickly realized a few things, we realized that we both like to educate, good.
6:34
Then I told them, if we're going to do something related to education, I want it to be accessible to everyone.
6:38
Because I grew up in Guatemala, and particularly in poor countries, you really see the difference between those who get an education and those who don't.
6:45
So I want it to be accessible to everyone.
6:47
Then we started thinking, well, what should we teach?
6:51
And we agreed on teaching languages in particular because of teaching English.
6:56
Actually, at first we wanted to teach math because we're both math nerds.
7:00
But then we realized that teaching languages was better, in particular because of teaching English.
7:04
So we agreed we're going to do a thing that teaches English for free over the Internet.
7:08
That's what we agreed on and then we thought, may as well teach other languages if we're going to teach English.
7:13
And so that's kind of where we went but really we were looking for a PhD thesis topic for him.
7:17
That's what we were doing.
7:19
>> Speaker 1: Well, I think we're glad that PhD thesis became a lot more.
7:22
>> Speaker 2: It did, he finished his PhD thesis after we started Duolingo.
7:27
He told himself that he was going to, this is much later, that he was going to finish it by writing every Sunday for a year. And that's what he did.
7:35
Every Sunday for a year he wrote his PhD thesis and then he finished it.
7:39
I did not think he was going to finish.
7:40
>> Speaker 1: [LAUGH] Well, what's so great about all of this is you had a partner in crime and a shared mission for this.
7:48
A lot of people here at the GSB at some point would love to find a co founder the way you did.
7:53
So what advice do you have for us for building that kind of partnership that lasts as long as yours has.
7:58
>> Speaker 2: Become a professor at Carnegie Mellon.
8:00
And find a Swiss student >> Speaker 1: [LAUGH].
8:04
>> Speaker 1: We're friends, that matters, but that's probably not what matters the most.
8:15
I think what matters the most is that you have compatible ways of working, and that you respect the other person and that you really deep down know that they're going to treat you fairly.
8:28
One of the ways in which I knew he was going to treat me fairly was very early on, this is before anything had started.
8:34
Now, bear in mind, I was a professor and he was a student, I was his supervisor, we were going to start a company together.
8:40
And he came in with a piece of paper that he wrote by hand, that said, it's called something like our agreement.
8:53
And it was not illegal or anything, he just wrote it by hand and it just had bullet points, like first bullet point.
8:57
It's, we are going to share all decisions on hiring.
9:02
We are going to share all decisions on firing.
9:04
He had just five bullet points, and then he said, sign and I signed it, and I thought that was actually quite good, he made it clear.
9:13
>> Speaker 2: Well, I think that kind of transparency and the trust that you guys were able to build so early on, was probably really critical at the start because Duolingo was trying to change the status quo a little bit.
9:23
At that time, there was already Rosetta Stone that cost hundreds of dollars, but you guys wanted to make language learning 100% free.
9:31
So, how did you convince those early investors you could be a billion dollar company?
9:34
>> Speaker 1: [LAUGH] >> Speaker 1: They were just investing because I had sold my previous company to Google.
10:04
So they were hoping that I would do something that would make money, but they later told me they did not think Duolingo was going to be a good business.
10:11
>> Speaker 2: [LAUGH] >> Speaker 2: Well, they were wrong.
10:15
>> Speaker 1: They were wrong and I'm happy about that.
10:17
We stuck with it, but really the main belief at the time was that education was not a good thing to invest on.
10:24
And I can understand why, there's a lot of reasons why it's not a good thing to invest on.
10:29
So that's why they didn't believe in us.
10:32
But then we launched and then it just started growing a lot, it started becoming an app with a lot of users.
10:40
And that's our second investor, by the time they invested, we already had, I don't remember, but probably about a million active users.
10:47
So, we already had a bunch of users, and so, we were more credible.
10:55
>> Speaker 2: Well, you've taken that million to now over 46 million daily active users. >> Daily, yeah.
11:01
>> Speaker 2: You guys are worth over $14 billion, and you have- >> Speaker 1: You got to check the stock.
11:05
>> Speaker 2: I do need to check the stock price, that's the latest numbers that I have, but I might be wrong, Luis >> Speaker 2: [LAUGH].
11:10
>> Speaker 2: And you guys have 10 million people, I believe, that come back to your app every single day for at least a year straight.
11:18
What have you learned about human motivation that sometimes traditional educators miss?
11:23
>> Speaker 1: People don't read, that's probably the biggest thing.
11:28
Really people don't read, you would think people want to read, they don't.
11:33
Short attention spans, so our lessons have gotten shorter and shorter over time, they're about two minutes long each.
11:39
People love filling progress bars.
11:41
If you see a progress bar that is 75% full, you will continue to fill it, whatever it is.
11:49
Competition and cooperation, we have that both in the app.
11:52
So we have competition, we have these leaderboards, I've learned about leaderboards, by the way.
11:57
Leaderboards were this really interesting thing, I've learned about leaderboards that leaderboards with your friends are really not very good.
12:05
We first thought that we should have a leaderboard with your friends, they're not good.
12:08
Maybe for other stuff, but for us, it's just rare that your friends are equally committed as you are.
12:13
It is much better to have a leaderboard with strangers that are equally committed as you are. >> Speaker 2: Sure.
12:18
>> Speaker 1: So we learned that.
12:19
Animation, just beautiful animation, the app really helps in mood metrics.
12:26
And then the other thing is our notifications are very quirky, we've spent a lot of effort on making them quirky, and they really do get people to come back.
12:38
I think people give us a little bit of license to be weirder with them because people think, this is actually for my own good, because Duolingo is actually teaching them something.
12:51
So I think if we were a company just trying to sell vaping or something, our notifications, we couldn't use the notification we use, but for education, we can.
13:00
>> Speaker 2: Yeah, I don't know if we want the owl vaping >> Speaker 2: [LAUGH].
13:03
>> Speaker 1: It'd be funny, but no, not going to do it. >> Speaker 1: [LAUGH].
13:07
>> Speaker 2: We'll definitely talk about your marketing because that is a whole part of Duolingo.
13:11
So, don't worry, Luis, we'll get there.
13:14
But one thing that's interesting is you guys started off, obviously, as a free app, and as time has gone on, you've added on more premium features.
13:21
So, how do you decide what stays free and what gets monetized?
13:28
>> Speaker 1: Over time, what goes into the decision has changed quite a bit.
13:35
At first, we were just free, there was no way to make money on Duolingo.
13:38
I mean, there were no ads and there was no way to pay us, we were just, we made the joke that we were not a non-profit, we just didn't have profit.
13:49
This is before we hired Matt, our CFO over here, he wouldn't have liked that.
13:54
So, at first, we didn't make any money, in 2016, one of our investors at that time, we got an investment and Duolingo was valued.
14:03
We had zero revenue, absolutely zero, and we got a valuation of something like half a billion dollars.
14:09
An investor based in San Francisco, flew to Pittsburgh, where we're based in Pittsburgh, Pennsylvania, just to tell me that we needed to make money.
14:21
This is after she closed the investment, she went there, she knew this was not going to be that easy.
14:25
So she took me to a bar, got me drunk, and then said, you have to make money.
14:31
So we started trying to figure out how to make money, and it was this amazing thing, I had never seen anything like this, nobody in the company wanted to make money.
14:39
We had hired a bunch of people who were like communists or something, nobody wanted to make money because everybody who came to work for Duolingo came for the mission, okay, it's about free education.
14:53
So when I stood in front of the whole company, which at the time is not that big, it was maybe 100 to 150 people.
14:58
And I said to everyone that we needed to make money, nobody wanted to do it.
15:04
And people started asking questions like, why?
15:09
And then my answer was, well, how do you think I'm going to pay our salary?
15:15
And then they rightly would say, I don't know, but you've been paying our salary, so.
15:20
So anyways, eventually it took six months for people to be okay with making money.
15:27
And what we came up with was this freemium model where the idea is that you can learn as much as you want for free, without ever having to pay us, but if you don't pay us, you have to see an ad at the end of a lesson.
15:39
And if you want to get rid of the ads, you can pay us.
15:42
So we came up with this model, one thing that I like about the model, by the way, is that it's a small form of wealth redistribution.
15:47
Because what ends up happening is that, remember we're communists, what ends up happening is that the people who pay are usually the wealthier people.
15:56
They usually are from wealthier countries, they have a stable job.
16:00
And the people who don't pay are usually from poor countries, so they may not have a stable job.
16:05
And so what I like is basically the rich people are paying for the education for everyone. So I like that.
16:10
>> [LAUGH] >> And then what goes into the features?
16:14
We just ab test a lot into what goes into the subscription and what doesn't.
16:21
My sense is that the users would like everything to be free, the investors would like everything to be paid, and our employees would like everything to be free, but also be millionaires.
16:32
>> [LAUGH] >> So you have to manage all of that.
16:36
Correct me if I'm wrong ma'am.
16:37
[LAUGH] >> Speaker 2: Well, you're not just stopping with the app in the language world.
16:43
You guys are going into other spaces.
16:46
You launched the Duolingo English Test to take on legacy giants like TOEFL, which actually many people in this room have taken.
16:53
>> Speaker 1: The TOEFL or the Duolingo English Test, Hopefully Duolingo English too.
16:56
>> [LAUGH] And with that, you're redefining language proficiency.
17:02
Can you tell us what that means and what it takes to basically build a new universal standard?
17:07
>> Speaker 1: Yeah, so we have this test called the Duolingo English Test.
17:09
The idea is to compete with the TOEFL and the IELTS and these Standard English Proficiency Test.
17:17
The reason we started it was actually because, two reasons, actually, but one of them was that our own users were sending us emails saying, hey, thank you for teaching me English, but now I need a certificate that says that I can speak English.
17:31
Because I need to apply for a job or whatever.
17:33
Can you give me a certificate?
17:34
And we looked into this and what we found was this crazy market where billions of dollars are spent every year by people who have to take a standardized test.
17:42
This was the year 2014, what was crazy at the time was that these tests had to be taken in a testing center.
17:49
And we thought that we could do better by having people take the tests online.
17:53
At the time, people thought we were crazy that, you could take a test online.
17:57
It turns out you can take tests online. It's not that hard.
18:01
But we made a test that was online and we started trying to compete with the TOEFL.
18:06
The people at the TOEFL laughed at us at first because they're like, what are you trying to do?
18:10
It turned out that it was actually quite hard to get universities to accept our results.
18:16
But we were lucky that we got the first university that accepted results was Yale Undergraduate Admissions.
18:22
So we were very lucky that one of the most prestigious universities in the world accepted that.
18:27
And from then on, it just kind of for the next several years spread by now, 98 of the top 100 universities, according to US news and World Report in the US accept the results.
18:39
So we're very happy with that.
18:41
And this year, we expect to surpass the total in volume, so we're very happy with that. It's also cheaper.
18:47
>> Speaker 2: Yeah, I'm sure a lot of people have that.
18:49
[LAUGH] >> Speaker 1: And you can take it from home and you don't have to travel.
18:51
But what I'm really excited about is that now we have the score, which is the score for the Duolingo English test, which is from 0 to 160.
18:58
I want this to become the proficiency standard for English.
19:01
We just put the score in the app.
19:03
So if you use the app from not for every language, for a lot of the languages, you get the score right next to it.
19:09
What we want, particularly for English, is that this is how people refer to how good their English is.
19:14
So rather than saying, I'm intermediate, we want people to say, I am Duolingo65.
19:21
And if we are able to get there.
19:22
We're not there yet, but if we're able to get there, I think that's a very defensible position.
19:26
>> Speaker 2: Yeah, that's incredible.
19:29
And you guys are expanding beyond languages, too.
19:31
You said that you guys wanted to start with math, but you ended up going with languages.
19:34
But now I can learn math.
19:36
>> Speaker 1: We've added math. >> And music. >> Speaker 1: Yes.
19:38
>> Speaker 2: And most recently, chess too there.
19:40
>> Speaker 1: It is, in fact, rolling out today.
19:41
So some fraction of you, if you have an iPhone, can now learn chess.
19:45
That fraction at the moment is very small.
19:47
Like 1% of you can learn chess, but over the next week or so will roll to 100%.
19:53
>> Speaker 2: But why those?
19:54
What makes a skill or subject the right fit for Duolingo's learning approach?
19:58
>> Speaker 1: Yeah, it's a really good question.
20:00
We debate a lot about what subjects to teach.
20:04
There's a few things that we need them to do.
20:06
First of all, we want very large demand, so, like, hundreds of millions of people need to be able to learn it.
20:11
There's a reason for that. We're an app.
20:16
We can't really charge you $20,000. We can't do that.
20:20
The most we can charge is, like, 20 bucks.
20:23
So in order for this to be a good business, for a new subject to be a good business, because we can only charge you 20 bucks.
20:28
And we're also freemium, so not everyone pays.
20:30
It has to really have this very large demand.
20:33
So most things don't have that large demand.
20:35
For example, even coding, there's only about 20 million people in the World that either want to learn coding or are learning coding.
20:42
That's just not a very large number.
20:43
If you actually start whittling it down 20 million people in the world, how many are we going to capture?
20:48
Let's say 10%, that's 2 million people.
20:50
How many of those are going to pay?
20:51
Well, actually 10% of those pay us.
20:52
So that's only 200,000 people that will pay us.
20:55
So it's not a very good business for us.
20:57
So this is why we're looking for things that have hundreds of millions of people or billions of people. So that's one thing.
21:01
Another one is we want it to be good for the world.
21:05
So turns out a lot of people want to learn like Pokemon cards.
21:08
>> [LAUGH] >> We're not going to do that.
21:11
So we want it to be good for the world.
21:12
We want it to be hundreds of millions of people and we want it to be the case that we can do a good job with a mobile app.
21:19
So you can't really teach everything with a mobile app that well.
21:23
So those are some of the things, the last thing, which is more about the team.
21:28
We need somebody or a small group of people inside the company to be excited about this, to actually go work for it, work on it.
21:35
So those are the things that end up happening.
21:38
>> Speaker 2: And I think I've heard you say that a big enabler of you expanded these other areas has been AI. >> Speaker 1: It has.
21:44
>> Speaker 2: And it's been so exciting watching the company lean into that new era with your product strategy.
21:49
We are going even further.
21:51
Just last week, Duolingo made big news when you release an email that you send to all of your employees [LAUGH] about AI's new role in the company and its day to day operations more specifically.
22:03
Including things like moving away from contractors, limiting new team headcount and favor of automation.
22:08
Can you tell us about that vision and what you think AI will enable Duolingo to do now that even you couldn't have imagined a few years ago?
22:15
>> Speaker 1: Okay, AI, completely transformative, at least for our business.
22:22
In particular for language learning.
22:24
I mean, it's AI, the big advance in AI over the last couple of years is large language models, the middle l is language.
22:32
We are a language learning company majorly so that has a big impact.
22:37
There were things that I did not think were going to be possible for at least another 10 years.
22:40
For example, we've always wanted to get people to practice conversation on Duolingo.
22:46
Before more than two years ago, you could not practice conversation on Duolingo and the only way in which we had to get you to practice conversation was to pair you with another human.
22:55
I never wanted to do that.
22:58
We tried it a few times and the reality is most people don't want to talk to another human in a language that they are not comfortable with.
23:05
They may tell you they want to.
23:06
In fact, everybody tells you they want to, but they are lying.
23:09
>> [LAUGH] >> They do not want to.
23:11
I mean, because we put it in the product, we would tell people here, you can use it. Very few people use it.
23:18
5% of extreme extroverts use it.
23:21
And then nobody else used it.
23:23
So we always knew that we wanted to have something that was not humans, but we couldn't do it with the technology was just not there to practice actual conversation.
23:33
But now with large language models, you can actually practice conversation.
23:36
We're very excited about that.
23:37
The other place we're very excited about is education needs content.
23:41
That's what people are going to learn.
23:42
It turns out we can generate content really fast, really high quality and iterate a lot on it with AI.
23:49
So all our new subjects, the content is generated now with computers. AI, sorry, chess.
23:56
The chess course which we are launching literally this week, we still Started working on it eight months ago.
24:03
There was zero work on it before eight months ago.
24:05
So we started working on it eight months ago.
24:07
It got started by two people, neither of whom knew how to code and neither of whom knew how to play chess.
24:14
>> [LAUGH] >> Speaker 1: I'm actually not kidding about this.
24:17
>> [LAUGH] >> Speaker 1: And they got a course done.
24:20
Now, of course, after they prototyped and everything, after a while, about four months ago, we started putting engineers in there so that they could productionize it.
24:29
But they really did spend about, call it four or five months, just the two of them, prototyping, until we had a course that looks just like the one now made by them.
24:39
And so that's the amazing thing about AI.
24:41
This was done by non-engineers who didn't know chess.
24:44
>> Speaker 2: I mean, it's amazing hearing how it's basically helping you guys push new frontiers. >> Yeah.
24:48
>> Speaker 2: But Luis, there are cultural implications of it.
24:51
Duolingo is known to be quirky, mission-driven, deeply human.
24:56
Do you see any risk of that changing as AI takes a bigger role in the company?
25:00
>> Speaker 1: I don't think so, I mean, for example, last week, when I sent my email to the whole company, there were negative reactions.
25:09
Not inside the company, but outside, there were negative reactions.
25:13
And of course, it got spun into Duolingo firing all its employees.
25:16
>> [LAUGH] >> And just hiring robots somehow.
25:19
That is not what we're doing, we're not laying anybody off.
25:22
That's not what we're doing at all.
25:25
But we are just going to be a lot more scalable.
25:30
And of course, we're going to have a whole human element, and what we're going to have our employees do is basically go up a level.
25:37
All this stuff that, for example, a good example is animators.
25:41
Animators spend most of the time just making sure that the movement is just precise.
25:48
That's going to be done all by computers.
25:49
And then they're going to be able to just come up with better things to animate.
25:54
And it's going to be the same for Illustrator, it's going to be the same for engineers, it's going to be the same for everything.
25:59
I think we're just going to bring everybody up a level so that they can spend their time on the more creative tasks rather than these repetitive things.
26:05
So I'm very excited about it.
26:07
I know we're not laying off anybody.
26:08
>> [LAUGH] >> Speaker 2: And a lot of maintaining that kind of culture integrity that you're talking about really starts at the top with you.
26:15
So I want to make sure we spend some time to hear about your leadership.
26:18
And here's some of the lessons you picked up along the way.
26:21
As you've evolved from inventor of a product to now CEO of a public company, what's changed in how you lead?
26:29
>> Speaker 1: For the first 50 employees of Duolingo, I was a micromanager, and I actually claim that was not that bad.
26:36
I think if you're starting a company- >> [LAUGH] >> You should be a micromanager up until about employee 30.
26:41
>> [LAUGH] >> I took it too far, I went to about 50.
26:45
I no longer micromanage, not because I don't want to, but because I can't.
26:50
>> [LAUGH] >> Speaker 1: It is just not possible to micromanage that many people.
26:53
So I've learned not to do that, I've learned to delegate.
26:58
And I think at this point I also have learned that most of my job is culture, carrier, mascot, and just making some of the kind of tough philosophical decisions.
27:12
The other thing that I've learned, this took me many years to learn.
27:15
The things that I'm good at or that give me positive energy, I should spend my time on that.
27:19
And the things that I'm not good at or don't give me positive energy, just don't do those things. >> Yeah.
27:23
>> Speaker 1: So, good example, two of my executive team are sitting here, Head of People and Head of Finance.
27:29
I am neither good at those things- >> [LAUGH] >> Nor do I get energy from them.
27:33
>> [LAUGH] >> Speaker 1: So they have all the freedom in the world, but our poor Head of Product- >> [LAUGH] >> Does not have a lot of freedom.
27:43
>> [LAUGH] >> Speaker 1: Matt laughs because it's true.
27:47
>> [LAUGH] >> And so what I'm hearing is it's really about the people that you're putting around you and you're building up.
27:53
So I want to ask about that.
27:55
How do you think about who to bring in or who not to as you scale Duolingo and help the company continue to grow?
28:04
>> Speaker 1: This is really one of the most important things you can do if you're building a company, making sure that you bring the right people.
28:10
Obviously, we try to really bring people who are very good at whatever their function is.
28:16
So if they're engineers, we try to look for really good engineers.
28:18
That's kind of table skates like, sure, you do that.
28:21
We also want to have a real good culture.
28:26
At the moment, we've changed that because we changed it, not the spirit of the rule, but we changed the actual rule.
28:32
For many years, we had this rule that was, if you're trying to fill a role in the organization, it's usually because you have a hole in the organization that you're trying to fill.
28:42
The rule that we had is it's better to have a hole than an A hole.
28:47
>> [LAUGH] >> That was an internal rule, we still have that.
28:52
But let me tell you, we actually really, really lived by it.
28:56
The best example that we had was when we were searching for a CFO. This is Matt's job.
29:02
>> [LAUGH] >> Speaker 1: This is before Matt showed up.
29:03
We were searching for a CFO, it took us a year and a half to find a CFO. It's a very long time.
29:09
After about a year of searching, now bear in mind, I had the board breathing down my neck, just saying you have to find a CFO, etcetera.
29:17
So about a year had passed, we were looking for one.
29:22
And we interviewed a person that everything looked good, passed the interview, the resume was excellent.
29:32
They came from a fancy company.
29:33
It was good, this is the person, I was like, we're going to hire this person.
29:38
During the hiring meeting, it came out that this person was not very nice to a couple of people.
29:43
One, his driver who picked him up at the airport, he was not nice to that person.
29:48
And then another person who was a very junior employee, and then we didn't hire him.
29:53
The boost, by the way, broke my heart because I was like, no.
29:56
>> [LAUGH] >> But we really live by this rule.
29:58
And I think he has created a really good culture that toxic people at Duolingo either are not there or don't last long or repress it.
30:11
>> [LAUGH] >> Speaker 2: So many good one liners from there.
30:18
>> [LAUGH] >> Speaker 1: Maybe they are toxic people.
30:21
>> [LAUGH] >> They are repressed.
30:22
>> [LAUGH] >> Speaker 2: And for you guys, culture, I mean, culture is important for every company, but for you guys, you live and breathe it. >> Speaker 1: We do.
30:29
>> Speaker 2: And we're lucky as the public because we get a front row seat to it with whatever it is your marketing team is doing.
30:35
And so, Luis, we want to take a quick minute just to talk about that.
30:39
For those of you who are not as familiar with Duolingo, if you're years at the app, you'll probably get incessant reminders and notifications.
30:47
>> Speaker 1: They're not that incessant.
30:48
>> Speaker 2: No, no, no, not at all.
30:49
>> [LAUGH] >> Speaker 2: And if you're not on time, they can get a little angry.
30:53
>> [LAUGH] >> They have an unhinged TikTok from giving ladies advice for the club.
30:59
>> [LAUGH] >> Keeping up with pop culture.
31:03
>> [LAUGH] >> And even making celebrity appearances at Coachella.
31:06
>> [LAUGH] >> Which is funny because I actually saw them when I was there a few weekends ago.
31:09
>> Speaker 1: You were at Coachella and you saw them?
31:10
>> And they were having a lot of fun.
31:11
>> Speaker 2: Yeah, they were, yeah.
31:12
>> They were having a great time.
31:12
>> Yeah, this is what we pay our marketing people to do to go to Coachella.
31:15
>> [LAUGH] >> Speaker 1: So if you need a job at marketing at Duolingo, let me tell you [LAUGH].
31:20
>> [LAUGH] >> Speaker 2: And probably the most viral is you guys even faked killed your L.
31:27
>> Speaker 2: We did not, look- >> [LAUGH] >> We can't control what that guy does.
31:34
>> [LAUGH] >> Speaker 1: He faked his death.
31:37
>> [LAUGH] >> Speaker 1: Can't control him.
31:39
>> [LAUGH] >> Speaker 2: Luis, this is not normal corporate behavior.
31:44
>> [LAUGH] >> Speaker 2: How did chaos become your brand?
31:47
And why does it just work so well?
31:50
>> Speaker 1: I wouldn't say our brand is chaos.
31:51
>> [LAUGH] >> Speaker 1: We call it wholesome, unhinged.
31:56
>> [LAUGH] >> Speaker 1: This. It took many years.
32:00
It's not like we knew from day one that this was going to be like that.
32:04
The first thing that happened was we had a mascot.
32:06
We decided to have a mascot.
32:08
We just thought that an education thing should have a mascot.
32:12
I don't know, we just thought that.
32:14
No great reason for that.
32:16
Then after we had a mascot, we made the mascot our app icon.
32:21
So if you've seen our app icon, it's a blown up face of our owl. So we did that.
32:26
What that does is it makes it so that all the notifications that you get on the phone look like they're coming from the masco, because every notification has the app icon in it.
32:35
So it looks like your notifications.
32:37
So that gave the mascot a voice.
32:38
But we didn't really understand that we were giving the mascot a voice.
32:41
We were just like, well, it's a notification, so we're going to write it down. So that gave it a voice.
32:44
Then over time, AB tested different notifications.
32:48
At some point, it occurred to me that here's what was happening.
32:52
We were sending notifications for you to come back.
32:55
If you ignore us and don't come back, we send you another notification the next day.
32:59
And if you ignore us and don't come back, we send you another notification the next day.
33:02
But after five days, we stop, because we're nice.
33:05
>> [LAUGH] >> But after five days, it occurred to me, I wasn't really trying to come up with anything to get people to come back.
33:13
It occurred to me that on the fifth day, we should tell people that we're stopping because we're nice.
33:19
So we changed the fifth day copy of the notification to say something like, these reminders don't seem to be working, we're going to stop sending them for now.
33:30
>> Speaker 2: I've gotten a few of those. >> Speaker 1: Yes.
33:32
>> Speaker 1: [LAUGH] >> Speaker 1: I did not realize how powerful that is to get people to come back, because people feel like we're giving up on them.
33:40
So it's like your parent being like, I'm giving up on you.
33:43
>> [LAUGH] >> Speaker 1: And it really gets a lot of people to come back.
33:46
So that was extremely passive aggressive.
33:49
That's not how we wrote it, but it came out that way.
33:53
>> Speaker 2: And I went back every time.
33:54
>> Speaker 1: Yes, it works.
33:56
And then what that did is the Internet started coming up with memes about how passive aggressive our owl was.
34:04
And then they started coming up with more memes about how the owl was willing to go through any psychological terror towards you to get you to come back.
34:13
And then also, the Internet came up with the fact that then the owl's willing to go to anything to get you to come back, including kidnapping your family.
34:20
And so all these memes that started coming up, but this was not us.
34:25
Eventually, what we decided to do was to lean into this.
34:30
So our April Fools campaign from six, seven years ago was we made a video.
34:40
You've seen all these crappy lawyer commercials on local TV, where they're basically telling you they're going to get you money for your accidents or whatever.
34:51
We decided to make a video of a crappy lawyer claiming to get you money if the owl had kidnapped your family.
34:58
>> [LAUGH] >> Speaker 1: And that was our April Fools.
35:00
By the way, one thing we learned about that April Fools campaign was we actually went and hired one of these companies that makes the crappy lawyer videos.
35:07
And we showed up there and we're like, we want you to make us a crappy lawyer video.
35:11
And they were like, what do you mean?
35:12
>> [LAUGH] >> Speaker 1: I am not kidding.
35:19
They were like, we make great videos. We're like, okay.
35:23
So in the end, we actually did not work with them.
35:26
We actually worked with an extremely fancy video production company who fully understood we wanted a crappy lawyer video.
35:31
>> Speaker 1: [LAUGH] >> Speaker 1: But anyways, that's kind of our first thing.
35:35
We leaned into that, and then we just started leaning in more and more.
35:39
And every time we leaned in more towards the memes, it worked.
35:45
So that was kind of one thing.
35:46
The other thing that happened was our head of marketing at the time told me, hey, I need you to let me hire a few people that are very young, straight out of college, because I just think they have ideas that we older people don't have.
36:01
And I'm like, sure, you do that.
36:03
And then she brought up the resumes of a couple of people that she wanted to hire.
36:08
And I looked at the resumes, and I was like, absolutely not.
36:12
These were not like the other resumes at Duolingo.
36:15
Most resumes at Duolingo look like all of you, super fancy school, perfect GPA, etc.
36:23
These resumes, not like that.
36:26
But anyways, I trusted her and I said, sure, I'll let you hire them.
36:30
And she did, she hired these two people, and one of them, at some point pitched to me.
36:38
I was in the pitch that they wanted to start a TikTok account and I'm like, why?
36:44
And then they showed me some of the videos that they wanted to do, and I'm like, this is not going to work.
36:49
So I did not think it was going to work, but they begged me to let them do it.
36:52
And I thought there was no downside to let them do it, because I'm like, look, nobody's going to watch this, so it doesn't matter.
36:58
You're going to put your funny content.
37:00
It's not even funny, just weird.
37:01
Your weird content out there, nobody's going to watch it.
37:04
So there downside, go ahead.
37:07
It turned out I was wrong.
37:09
The Tiktok is now, if it's not the, it's one of the most followed, top five most followed brands on TikTok in the world.
37:19
There are weeks where we have the single most viral video in all of TikTok and it now accounts for something, like 20% of our new users. I was wrong.
37:32
>> [LAUGH] >> Speaker 1: I was wrong, I was wrong.
37:39
But yeah, that's kind of what happened.
37:41
And over time, we really embraced it.
37:44
>> Speaker 2: And I think you leaning in, it just speaks to your leadership.
37:48
It's bold and you're willing to do things a little differently.
37:51
>> Speaker 1: That's because I thought there was no downside.
37:52
>> [LAUGH] >> Speaker 2: An interesting thing that I've heard from your team that you like to say is that from time to time, every leader has to be unreasonable to move things forward.
38:04
So when have you been unreasonable at Duolingo?
38:07
>> Speaker 1: Wow, they told you that?
38:08
>> [LAUGH] >> I don't know about unreasonable.
38:14
I think I definitely use this, whenever some change needs to happen, I definitely act more unreasonable than I actually am, and I know I'm doing that.
38:28
And actually, the people that work with me closely know that.
38:31
But it really kind of scares some people when I just give these kind of edicts that I just do that.
38:36
And sometimes it's a lot more unreasonable.
38:37
For example, I don't know, what's a good example?
38:42
What's a good example of some reason, unreasonable thing? I don't know.
38:46
I don't find them that unreasonable, but other people do.
38:49
But anyway, sometimes I feel like it's necessary.
38:54
>> Speaker 2: Well, unreasonable or not, we love your leadership and the company is just something that's so fun for us all to watch.
39:00
Luis, we have one last question for you before we open up to audience Q&A, and so we're asking all of our guests.
39:06
So our theme for this year is leaving your mark.
39:09
So, Luis, how do you want to be remembered?
39:13
>> Speaker 1: I really want to have a positive impact on education.
39:18
I think that one way of viewing Duolingo is, actually the most common thing that happens when you ask people why they're using Duolingo is they'll tell you something to the effect of, well, I used to X.
39:31
I used to play a lot of Candy Crush, I used to scroll a lot of Instagram, etc.
39:35
But now some of that time is spent using Duolingo, and at least I'm learning some Spanish.
39:42
So I really want it to be the case that we can show that screen time is actually useful for the world.
39:50
And I think we can make a big change.
39:51
And I think for the first time ever, with phones, we can reach billions of people and educate them.
39:59
And I think Think we can do that.
40:00
So that's what I would like.
40:02
>> Speaker 2: And I think we're excited to watch that happen.
40:05
We have a couple of mic runners coming around with a mic.
40:08
If you get it, please state your name, your year in the program and then your question.
40:13
>> Speaker 3: My name is Layla, I'm an MD, MBA, second year MBA, and I have a two-part question.
40:19
So, let's see, I'm on day 2602 on Duolingo. >> Speaker 1: Nice.
40:25
>> Speaker 3: So seven years. >> Speaker 1: Okay. >> Thank you.
40:28
>> [APPLAUSE] >> Speaker 3: So I get the dopamine hit and it works for me.
40:34
But the thing is, for me, past getting one extra day on my streak, that's where it ends.
40:39
The challenges don't do it for me.
40:40
I have too many lingots with nothing to do with them.
40:43
You stopped giving us Duolingo outfits, which we'll talk about at launch.
40:48
>> Speaker 3: [LAUGH] >> Speaker 3: [COUGH] So my first question is, are you planning new rewards or challenges that could push longtime users like me to do more just quick than the one lesson a day?
40:59
And then the other part is the streak really does work.
41:02
But the thing is in poorer countries, the connectivity thing, there's no Internet, there's no power, a lot of time that wastes a lot of precious learning time.
41:11
And then the ability to download lessons and do them offline, that's behind a paywall.
41:16
So is there any way around that in places that need it the most?
41:20
>> Speaker 1: Yeah, great question.
41:21
Okay, so rewards, we're working on that.
41:23
It is incredibly difficult to come up with good rewards, at least for our team.
41:28
There's going to be a huge revamp over the next few months that's going to allow us to give us some much more interesting rewards.
41:37
So hopefully it'll work for you, I don't know.
41:39
Not everything works for everybody, but hopefully this will work.
41:42
So we are working on rewards.
41:43
That is something that we actively work on and it is something that the product organization at Duolingo has been frustrated about for the last ten years.
41:51
We know our rewards are not good enough.
41:52
So yeah, we're frustrated about it, we're working on it.
41:55
In terms of connectivity and stuff like that.
41:58
By the way, it is not the case anymore that offline lessons are behind the paywall.
42:04
Everybody gets access to offline lessons now, so that has changed and we're working quite a bit.
42:09
For poor countries with connectivity, what matters the most is the Android app.
42:14
And so we have a whole team making sure that our Android app really uses as little bandwidth as possible.
42:22
It's as small as possible, actually, the size of the Android app is quite small.
42:27
So we're trying what we can.
42:28
We're still not there in terms of, I mean, there really are a lot of people who just have spotty Internet.
42:35
But my hope is that this will resolve itself without us having to do much and we'll just concentrate on the teaching component.
42:42
>> Speaker 4: One of the things that Duolingo managed to do really well in the EdTech space is solve the retention problem a little bit.
42:48
And I was just wondering how your time in CMU labs or your time in more technical, I guess, positions helped you to get gamification right.
43:04
>> Speaker 1: I don't know exactly how that helped, but you are right that a lot of people ask, what is it that we do well?
43:11
I think the main thing we do well is motivation.
43:15
It's just really hard to stay motivated to learn something by yourself.
43:18
So we spend a lot of effort on that.
43:20
I mean, the way we solved it, by the way, was just through trial and error.
43:23
I mean, some of us have by now some intuition about what is good for gamification, but the reality is that most of it is just trial and error.
43:32
We have run in the history of Duolingo 16,000 AB tests.
43:37
So each one of those is just trying to figure out, do we change this little thing, this button to be red or blue to try to get you to come back.
43:46
And so the combination of 16,000 AB tests has made it so that our retention is actually outstanding.
43:53
At this point our user engagement is starting to get comparable to social networks.
43:58
It's not there yet, but it's starting to, and so we're very proud of that.
44:03
>> Speaker 5: If you could go back in time to when you first started building out Duolingo, what would be the one thing that you would change about how you initially approached building out the company?
44:12
>> Speaker 1: Very easy, I know that because I've regretted it so much.
44:17
We spent the first, call it five or six years on Duolingo absolutely not monetizing.
44:24
We should have monetized after year three.
44:27
It was okay that we did not monetize for the first three years, that's fine, vut after that we're just wasting time.
44:33
And if we had monetized after year three rather than after year six, we would today be three years ahead.
44:39
Simply whatever is going to happen three years from now, we would have done it today.
44:42
So yes, that was a mistake of mine because I thought that making money was evil.
44:47
>> Speaker 6: Hi, I'm Philip, I'm a second year in the MBA program.
44:50
You started out your career back in the stone age of the Internet, getting humans to create data that was useful to train computers in ways they couldn't otherwise understand.
44:59
Today, as the computers and models have become much more powerful, you now have 46 million active users generating a ton of data.
45:06
And every overpriced consultant says data is the quote, unquote new oil.
45:09
Do you see a way to monetize or leverage that data to help the computers get better and kickstart AI progress?
45:16
>> Speaker 1: Yeah, I mean, that's something we already do, but we could probably do more of.
45:20
I mean, we have a ton of data of how people learn every day on Duolingo.
45:24
I don't know the latest number, but it's something like 1.
45:27
3 billion exercises are sold by our users every single day.
45:30
So we get to see what they get right, what they get wrong.
45:33
And we do use this data to build models, but we probably can do it more.
45:36
And yeah, I mean, there's definitely active work stream at Duolingo to use this data as much as we can, yeah.
45:44
>> I think we have time for one more question.
45:47
>> Speaker 7: Hi, Luis, my name is Naza, I'm a second year MBA student.
45:51
I'm really inspired how Duolingo is able to preserve some of the lesser known languages.
45:56
For example, there are 42 times more learners of Irish language than there are native Irish speakers.
46:03
So from that perspective, how do you see the role of Duolingo as a tool to preserve some of the languages that are close to extinction or just lesser known languages?
46:13
>> Speaker 1: Yeah, it's a really good question.
46:15
I would like to do more of that.
46:17
At the moment we have a few kind of minority languages.
46:22
We have Irish, we have Welsh, we have Esperanto.
46:25
Well, Esperanto's constructed language, but we have a bunch of these.
46:29
I would like to add more.
46:32
There's two limiting factors, basically return on investment.
46:37
Not that many people are learning them compared to other languages.
46:40
I mean, if you actually look at what languages people are learning on Duolingo, 50% of the people on Duolingo are learning English, about 20% are learning Spanish and about 10% are learning French.
46:51
So just those three languages account for somewhere like 75 to 80% of our users.
46:56
So we spend most of our efforts there.
47:01
Then the other problem with the kind of minority languages is that AI is just not as good at them as it is for Spanish or whatever.
47:09
But I think over time that's going to get better.
47:10
And once the cost of adding a new course becomes low enough, we'll probably really add as many languages as we can.
47:20
But that day is not today at the moment, because AI is not very good at pick whatever language you want that is not very popular.
47:31
AI is just not very good at it, so we're not doing it at the moment.
47:34
But I would say if you give it two, three years, we'll probably add more.
47:39
>> Speaker 2: Luis, before we let you go, we have one last tradition here of eat from the top, and that's our rapid fire. >> Speaker 2: Okay. >> You ready? >> Speaker 1: Ready.
47:47
>> Speaker 2: Luis, what is your highest streak on the app?
47:50
>> Speaker 1: I don't know the exact number, but it's 3,300 something, yeah.
47:55
>> Speaker 2: Companies love to say that they're Duolingo for blank.
47:58
What's the strangest one you've heard?
48:00
>> Speaker 1: Duolingo for anger.
48:03
I don't know if they're teaching you to be angry and why would you do that, but I get the ads on social media for Duolingo for anger. I'm confused about that.
48:14
>> Speaker 2: [LAUGH] You've opened up a restaurant, Duo's Taqueria. >> Speaker 1: Yes.
48:19
>> Speaker 2: What's your favorite thing to order? >> Sorry.
48:25
[LAUGH] >> We don't have that dish yet. So, I like everything.
48:31
We opened a restaurant, by the way, we're very proud of it.
48:35
Except the only person that doesn't, it's not the only person.
48:37
One person who doesn't like it is our CFO, because our restaurant is not an excellent business like Duolingo.
48:43
It loses money, but I'm going to say the tacos al pastor, they're very good.
48:49
>> Speaker 2: Amazing, and finally, Luis, be honest with us, how often do you get locked out of CAPTCHA?
48:58
>> Speaker 1: 15 to 20% of the time.
49:00
>> Speaker 1: [LAUGH] >> Speaker 1: And it's really annoying.
49:03
>> Speaker 2: [LAUGH] Amazing, well, thanks so much, Luis, this has been incredible, we've loved it. >> Thank you. >> [APPLAUSE] [MUSIC]