"- On one access, you have more hardware coming in. On the other hand, you have an explosion of innovation in AI. And so what happened with both TensorFlow and PyTorch is that the explosion of innovati..."

"It's been, I think two years since we last talked, and then in that time, you somehow went and co-created a new programming language called Mojo. So it's optimized for AI. It's a superset of Python. L..."

"and then having the world's power and the innovation of all the smart people in the world get unleashed on something can be quite different. And so really where Mojo came from was, starting from a pro..."

"- Well, and like you take a step back and look at what file extensions are, right, they're basically metadata, right? And so why are we spending all the screen space on them and all this stuff? Also, ..."

"For example, in Swift, you can do that for sure. So an example we gave at Apple was the dog cow. - Yeah. - So that's a classical Mac heritage thing. And so you use the dog and the cow emoji together, ..."

"- So optimize through simplification by having one thing. So you mentioned Modular. Mojo is the programming language. Modular is the whole software stack. - So just over a year ago, we started this co..."

"- So Mojo is a programming language that allows you to do the higher level programming, the low-level programming, like do all kinds of programming in that spectrum that gets you closer and closer to ..."

"And so I think that TensorFlow and PyTorch and these systems embracing Python really took and helped Python grow, but I think that the major thing underlying it is that Python's like the universal con..."

"And so if you look at something like PyTorch or TensorFlow or, I mean, even a simple use case, like you define a class that has the plus method, right, you can overload the dunder methods, like dunder..."

"- [Chris] Yep. - But it's really messy. - It's super messy. It was accidentally, I mean, different people have different interpretations. My interpretation is that it was made accidentally powerful. I..."

"And so, again, one of our goals with Mojo is to make things really easy to use, easy to learn, and so there's a natural stepping stone. And so as you do this, you say, okay, well, I have to do program..."

"Like, they grow macros or something, right? List right? - Yes. - I know your roots, right? (Lex chuckling) You know, and this is a powerful thing, right? And so, you know, if you go back to list, one ..."

"- Maybe can you describe auto-tuning and how do you pull off, I mean, I guess adaptive compilation is what we're talking about is metaprogramming. How do you pull off- - Yes. - auto-tuning? I mean, is..."

"A lot of the algorithms that you run are actually the same, but the parameters, these magic numbers you have to fill in end up being really fiddly numbers that an expert has to go figure out. And so w..."

"- So for auto-tuning, for like tile size, you mentioned 120f for TPU. You would specify like a bunch of options to try, just in the code- - Yeah. Yep. - Just simple statement, and then you could just-..."

"So first of all, when we say that, we're talking about what's called CPython, it's the default Python that everybody uses. When you type Python 3, that's like typically the one you use, right? CPython..."

"- Before I talk about some of the interesting stuff with parallelization and all that, let's first talk about, like, the basics. We talked the indentation, right? So this thing looks like Python. It's..."

"- So it's kind of optional, it's not strict typing, like, you don't have to specify type. - [Chris] Exactly. - Okay, so it's starting from the thing that Python's kinda reaching towards right now with..."

"why do you have to rewrite it? Well, you have to rewrite it to deploy. Well, why do you wanna deploy? Well, you care about performance, or you care about predictability, or you want, you know, a tiny ..."

"- And, you know, people can ignore Mojo. And if it's not their thing, that's cool. But if they wanna use Mojo, I don't want them to have to rewrite all their code. - Yeah, I mean, this, okay, the supe..."

"can't use 'em for performance. And so it's really- - I mean, that's a big one, right? So you can't utilize for the compilation, for the just-in-time compilation, okay. - Yep, yep, exactly. And this al..."

"And so that's really cool because, you know, to me as a idealizing compiler language type of person, what I wanna do is I wanna get magic out of the compiler and put in the libraries. Because if someb..."

"- So one of the exciting things we should mention here is that this is new and fresh. This cake is unbaked. It's almost baked. You can tell it's delicious, but it's not fully ready to be consumed. - Y..."

"if you work with the community, right? And so, yes, it can be frustrating, can be challenging for lots of people involved. And, you know, if you, I mean, you mentioned our Discord. We have over 10,000..."

"And so you pass that pointer to an array around to things. If you pass into a function, they may take that and scroll away in some other data structure. And so you get your array back and you go to us..."

"You have to be careful to implement it in an efficient way. - Yeah, is there a performance hit that's significant? - Generally not if you implement it the right way, but it requires a lot of very low-..."

"This is- - You just kinda have to know that. - You just have to kinda know that, right? And so you roll out version one of the database. You just kinda have to know that. Of course, Lex uses his own d..."

"so I've worked with it in Swift, is you say, okay, well, let's provide value semantics. And so we wanna provide the view that you get a logically independent copy, but we wanna do that lazily. And so ..."

"And so it's, again, it's not novel research at all, right? The magic is getting the design right so that you can do this in a reasonable way, right? And so there's a number of components that go into ..."

"so that you get the power of something like the Rust programming language, but you don't have to deal with it when you don't want to, which is a major thing in terms of teaching and learning and being..."

"Like, once you create, it has to be where it was, right? Now, if you look at many other examples, like a database handle, right, so, okay, well, what happens? How do you copy a database handle? Do you..."

"And so it's not that anybody's right or wrong, it's about how do we build one system that scales? - By the way, the idea of an atomic number has been something that always brought me deep happiness, b..."

"enabling unparalleled efficiency. By the way, the use of unparalleled in that sentence... Anyway. (Chris chuckling) Enabling unparalleled efficiency, scale, and the reliability for the largest workloa..."

"And so I think that we as an industry have been struggling. And if you look at what deploying a machine learning model today means is that you'll have researchers who are, I mean, wicked smart, of cou..."

"but you've got Python on one side and C++ on the other side. Python doesn't really deploy. I mean, can theoretically, technically in some cases, but often a lot of production teams will wanna get thin..."

"in terms of their capabilities and gigantic scale, many researchers using PyTorch, right? And so PyTorch doesn't have those same capabilities. And so what Modular can do is it can help with that. Now,..."

"And why is that? Well, it's because, again, these systems have been built up over many years. They haven't been rethought, there hasn't been a first principles approach to this. And so what Modular's ..."

"in terms of all this really cool technology across compilers and systems and runtimes and heterogeneous computing, like all this kinda stuff. And like, all this technology has been, you know, I wouldn..."

"Like, we've built auto-tuning in many systems. We've built programming languages, right? And so like, have implemented C++, have implemented Swift, have implemented many of these things. And so, you k..."

"but I have sympathy for the poor software people, right? I mean, again, I'm a generally software person too. I love hardware, but software people wanna build applications and products and solutions th..."

"of a general purpose programming language that can then be compiled across all these kinds of hardware. - Well, so it's not really about a programming language. I mean, the programming language is a c..."

"and much more general research hackability together, right? - And that's enabled by the ASICs. That's enabled by certain hardware. So it's like- - Well. - Where's the dance between, I mean, there's se..."

"And on the other hand, you have, I don't know how many pieces of hardware there are out there. It's a lot, (chuckles) okay? It's not even hundreds. It's probably thousands, okay? And across all of edg..."

"multi-layer abstraction of hierarchical parallelism, hierarchical, asynchronous communication and making that, again, my enemy, is complexity. By getting that away from being different specialized sys..."

"do matrix operations over there, right? But then you realize you get a little bit more complicated because you can do matrix multiplication on a GPU, you can do it on a neural net accelerator, you can..."

"Let's actually do a search of the space to decide, well, what is actually better, right? Well, then you get into this problem where you realize this is not a small space. This is a many-dimensional hy..."

"- You started and did a little bit of a whirlwind overview of how you get the 35,000x speed up or more over Python. Jeremy Howard did a really great presentation about sort of the basic, like, looking..."

"but you get bottleneck sending data back and forth to memory, whether it be local memory, or distant memory, or disk, or whatever it is. And that bottleneck, particularly as the training sizes get lar..."

"each version of the chip comes out with new features and they change things so that it takes more time or less to do different things. And you can't rewrite all the software whenever a new chip comes ..."

"It's built by different teams, it's built by different hardware vendors, it's built by different systems, it's built by different internet companies. They're trying to solve their problems, right? And..."

"So it's running its own bike codes, and that's how it provides full compatibility. And then it gives us CPython objects, and we use those objects as is. And so that way we're fully compatible with all..."

"that's really important to you, you can move it to Mojo. You get massive performance benefits on that and other advantages. You know, if you like static types, it's nice if they're enforced. Some peop..."

"And again, we're not working forwards from making Python a little bit better. We're working backwards from what is the limit of physics? - What's the process of importing Python code to Mojo? Is there..."

"Mojo just gives you superpowers, right? And so if you wanna stay with Python, that's cool, but the tooling should be actually very beautiful and simple because we're doing the hard work of defining a ..."

"And it also means we don't wanna repeat the Python 2 to Python 3 transition. Like we want people to be able to adopt this stuff quickly. And so by doing that work, we can help lift people. - Yeah, the..."

"Back in time, project I worked on is called Clang. Clang, what it is a C++ parser, right? And when I started working on Clang, it must have been like 2006 or something, less, or 2007 something, 2006 w..."

"and argue with them and try to get them to rewrite their code, right? - Yeah. - And- - [Lex] 'Cause that breaks a lot more things. - Yeah. The reality is like nobody actually even understands how the ..."

"- You mentioned Guido, the benevolent dictator for life of Python. What does he think about Mojo? Have you talked to him much about it? - I have talked with him about it. He found it very interesting...."

"- I would not say that. I think that would drive a lot of people really crazy. - Because of the PTSD of the 3.0, 2.0. - I'm willing to annoy people about Emacs versus Vim or- - Not that one. - [Chris]..."

"or a new thing in Swift, or you build a new thing in Mojo, then you want it to be end up being used on the other side, (chuckles) right? And so then you need to work on integration back the other way...."

"- So for a package, port it to Mojo, and then create a Python interface. - [Chris] Yep. - So when you're on these packages, NumPy PyTorch TensorFlow. - Yeah. - How do they play nicely together? So is ..."

"I love Python, I love TensorFlow, I love PyTorch, right? This is about making the world better because we need AI to go further. - But if I have a process that trains a model and I have a process that..."

"- You're saying this is an irrational set of life choices I make or what? (chuckles) (Lex laughing) - Did you go to the desert and did you meditate on it? Okay, all right. No, it was bold. It was bold..."

"of different projects, which I'll skip over, right? One of which was this Swift for TensorFlow project, right? And so that project was a research project. And so the idea of that is say, okay, well, l..."

"Second thing is that, you know, a lesson learned is that Swift is a very fast and efficient language, kind of like Mojo, but a different take on it still, really worked well with eager mode. And so ea..."

"I wish it did, of course, right, but, you know, it's a research project. And so you're there to learn from it. - Well, it's interesting to think about the evolution of programming as we come up with t..."

"Like, I'm not saying Python is bad. Like, I hope Python, CPython, like, all these implementations 'cause Python ecosystems, not just CPython, it's also a bunch of different implementations with differ..."

"for why you should try Mojo would be. So practically speaking- - [Chris] Yeah. - it seems like, so there's entertainment. There's the dopamine hit of saying, holy, this is 10 times faster. This little..."

"It's very hard to predict, of course, but, you know, if you look 10 years from now and Mojo's super successful. - [Chris] Yeah. - What do you think would be the thing where people like try and then us..."

"the people that don't actually care about the language, that just care about getting stuff done, those people do like learning new things, right? - [Lex] Yeah. - And so you talk about the dopamine rus..."

"in the deep end and rewrite everything and, like, whatever, that's cool, right? But I think the middle path is actually the more likely one where it's, you know, you come out with a a new idea and you..."

"Today, we have CPU, GPUs, graphics. And AI, because it's so important, because the compute is so demanding because of the smart cameras and the watches and all the different places that AI needs to wo..."

"a few major libraries that go Mojo first? - Well, so I mean, the Modular engines on Mojo. (chuckles) So again, come back to, like, we're not building Mojo because it's fun. We're building Mojo because..."

"to rethink these things. - So the two big gains are that there's a performance gain and then there's the portability to all kinds of different devices. - And there's safety, right? So you talk about r..."

"and iterating and evolving things. And so you look at this LLVM thing that I worked on 20 years ago, and you look at MLIR, right? And so a lot of the lessons learned in LLVM got fed into MLIR, and I t..."

"people wanna download it around locally. So we're working on that right now. And so- - So that's the goal, to be able to download locally to it. - Yeah, that's what everybody expects. And so we're wor..."

"I signed up to work for Apple because I love Objective-C. Why is there a new thing?," right? - Yeah. - And so now what that meant practically is that the push from launch to first of all the fall, but..."

"If you're interested in what we're doing, we'll do it in an open way and we can do it together, but don't use it in production yet. Like, we'll get there, but let's do it the right way. And I'm also s..."

"in a sense, right? - Yeah. - And so, I mean, if you go back in time when we started Modular, which is just not yet a year and a half ago, so it's still a pretty new company, new team, small but very g..."

"I think people are excited about ownership and taking a step beyond Rust, right? And there's people that are very excited about that and there's people that are excited about, you know, just like I ma..."

"It's all his fault. - Well, there's a kinda very refreshing, pragmatic view that he has about machine learning that I don't know if it, it's like this mix of a desire for efficiency, but ultimately gr..."

"And so Jeremy has been really pushing the limits. And one of the things I'll say about Jeremy, and there's many things I could say about Jeremy, 'cause I'm a fanboy of his, but it fits in his head, an..."

"And as you say, he then brings it and teaches people- - [Lex] Teaches it. - And his mission is to help lift, you know, his website says "making AI uncool again," like it's about, like, forget about th..."

"- So from the perspective of, if you want to create your own package, and then - Yes, yeah. - or you wanna build on top of a bunch of other people's packages and then they get updated and things like ..."

"- So almost a tangent to that question from the user perspective of packages. It was always surprising to me that it was not easier to sort of explore and find packages, you know, with, with PIP insta..."

"like search and discovery as YouTube calls it. - Well, I mean, it is kinda funny because this is one of the challenges of these like intentionally decentralized communities. And so- - Yeah. - I don't ..."

"They can go figure that out. - But the point is to have an interface that's usable, that's successful to people of all different skill levels and- - So, well, and again, like what are the benefit of s..."

"- So any kinda packages that are hybrid in nature would be a natural fit to move to Mojo, which- - Which is a lot of them, by the way. - Yeah. - So a lot of them, especially that are doing some intere..."

"Why does Python not have function overloading? - So I can speculate. So Python is a dynamic language. The way it works is that Python and Objective-C are actually very similar worlds if you ignore syn..."

"and a function for strings. Well, even if you could put it in that dictionary, you'd have to have the collar do the dispatch. And so every time you call the function, you'd have to say like, is it an ..."

"that program Python might be interested in is def versus fn. So it's two different ways to define a function. - Yep. - And fn is a stricter version of def. What's the coolness that comes from the stri..."

"but you don't have to use it. And this is a way that Mojo is both compatible 'cause defs work the same way that defs have already always worked, but it provides a new alternative that gives you more c..."

"And if you typo something, it doesn't matter. It just tells you it's fine, right? And so I think that the trades are very different if you're building a, you know, large scale production system versus..."

"- Yeah, yeah. So the way to think about it is, think about a function that doesn't return anything, just as a simple case, right? And so you have function one calls function two, calls function three,..."

"- Exactly. And so the competitor does all this for you. And I mean, one of the things, if you dig into how this stuff works in Python, it gets a little bit more complicated because you have finally bl..."

"actually really great and for certain things, right? And so these are also called closures. Closures are pretty cool and you can pass callbacks. There's a lot of good patterns. And so- - So speaking o..."

"and I'm not gonna go into ring theory or something, but the, you know, you can say it's arithmetic. Like if you can add, subtract, multiply, divide it for example, right? And so what you're saying is ..."

"And so we're making incredible progress. By the way, Mojo's only, like, seven months old. So that's another interesting thing. - Well, I mean part of the reason I wanted to mention some of these thing..."

"If you look at C++, the way this works is that if you define a variable or a set of variables within a function, they get destroyed in a last in, first out order. So it's like nesting, okay. This has ..."

"It's the same thing as the Swift team, right? And this is where one of the reasons we released Mojo much earlier is so we can get feedback and we've already like renamed a keyword data community feedb..."

"- I mean, it's funny you come back to the flame emoji file extension, right? You know, we have the option to use the flame emoji, which just even that concept, 'cause for example, the people at GitHub..."

"Swift has a lot of syntactic sugar, which may may be a good thing, maybe not, I don't know. But because it's such an easy and addictive thing to do, sugar, like make sure blood get crazy, right? Like,..."

"And syntactic sugar just makes all that more complicated. So. - And yeah, list comprehension. Is that yet to be implemented? Yeah. And my favorite d I mean, I dictionaries. - Yeah, there's some basic ..."

"And I've built a lot of teams, a lot of different places. If you zoom in from the big problem into recruiting, well, so here's our problem, okay. I'll be very straightforward about this. We started Mo..."

"And so what we've done is we've said, okay, well, let's build an amazing culture. Start with that. That's usually not something a company starts with. Usually you hire a bunch of people and then peopl..."

"And so we have a clear vision of what we're doing. And so I kind of grew up at Apple in my engineering life, right? And so a lot of the Apple DNA rubbed off on me. My co-founder Tim also is like a str..."

"Yeah. But famously, if you go back to the iPhone, for example, right? iPhone 1, every, I mean, many people laughed at it because it didn't have 3G, it didn't have copy and paste, right? And then a yea..."

"and we get the whole company together periodically, and then we get work groups together and we plan and execute together. - And there's like an intimacy to the in-person brainstorming. Yeah, I guess ..."

"that happens that's Zoom. - And you're bonding. Yeah. - You're bonding, you're bonding. But through that bonding, you get the excitement. There's certain ideas are like complete. And you'll see that i..."

"- What about people that are kinda specialized in very different aspects of the stack working together? What are some interesting challenges there? - Yeah, well, so I mean, I mean, there's lots of int..."

"And so I think that's really important. - That's team. But what about building a company as ambitious as Modular? So what are some interesting questions there? - Oh, I mean, so many. Like, so one of t..."

"And so that's also pretty exciting and quite nice, but one of the ways that this manifests is it makes it easier to make decisions. And so one of the challenges I've had in other worlds is it's like, ..."

"- This is awesome. So I have to ask you about one of the interesting developments with large language models is that they're able to generate code recently. Really? Well, yes. To a degree that maybe, ..."

"There's mistakes. They're interesting that you learn from, but I just, it would love to get your opinion first high level. Yeah. Of what you think about this impact of large language models when they ..."

"And just like, you know, I think we all try to scale through delegation and things like this, delegating rote things to an LLVM I think is an extremely valuable and approach that will help us all scal..."

"or the human and the compiler, roughly, right? And so I think the LMS will have no problem learning whatever keyword we pick. - Maybe the fire emoji is gonna, oh. - Maybe that's gonna break it. It doe..."

"I mean, I think that would be interesting. - Is that wise? - Well, well, I mean, it would be very expensive. So compilers run fast and they're very efficient and LMS are currently very expensive. Ther..."

"and that could be interesting. But if you've chased that rabbit hole down, the question then becomes, how do you express your intent to the machine? And so maybe you want LLLM to provide the spec, but..."

"So it's just, I have to ask you about the big philosophical question about human civilization. So folks like Eli Kowski are really concerned about the threat of AI. - Yeah. - Do you think about the go..."

"- Well, that's in the physical space. I do think in the digital spaces, the stuff that's built on top of LLMs that runs, you know, the millions of apps that could be built on top of them, and that cou..."

"to improve our lives, and I want that to go faster. - So what do you think the future of programming looks like in the next 10, 20, 30, 50 years? That alums, LLMs and with Mojo, with Modular, like you..."

"And so one of the things that I'm a little bit concerned about is right now the big companies are investing huge amounts of money and are driving the top line of AI capability forward really quickly. ..."

"- And now the herd is using Python. So if you wanna be a rebel, go check out Mojo and help Chris and the rest of the world fight the arch nemesis of complexity 'cause simple is beautiful. - There we g..."