"That's not even a question for me - whether  we're going to go take a swing at building   the next thing. I'm just incapable of not doing  that. There's a bunch of times when we wanted to   launch fea..."

"hands. It's a big step forward for Meta AI. But I think if you want to get under the hood   a bit, the Llama-3 stuff is obviously the most  technically interesting. We're training three   versions: an..."

"H100s? How did you know that you’d need the GPUs? I think it was because we were working on Reels.   We always want to have enough capacity to build  something that we can't quite see on the horizon  ..."

"analyses trying to connect the dots forward. You've had Facebook AI Research for a long   time. Now it's become seemingly central to  your company. At what point did making AGI,   or however you consi..."

"to make it better on all these things even if  people aren't asking primarily coding questions.  Reasoning is another example. Maybe you want  to chat with a creator or you're a business and   you're ..."

"If you're doing $10Bs worth of  inference or even eventually $100Bs,   if you're using intelligence in an industrial  scale what is the use case? Is it simulations?   Is it the AIs that will be in the..."

"You mentioned AI that can just go out and do  something for you that's multi-step. Is that   a bigger model? With Llama-4 for example, will  there still be a version that's 70B but you'll   just train..."

"it's just inherently brittle and non-general. 
 When you say “into the model itself,” you train it   on the thing that you want in the model itself?  What do you mean by “into the model itself”?  For ..."

"What is the community fine tune of Llama-3  that you're most excited for? Maybe not the   one that will be most useful to you, but the  one you'll just enjoy playing with the most.   They fine-tune it..."

"Although one of the interesting  things about it, even with the 70B,   is that we thought it would get more saturated. We  trained it on around 15 trillion tokens. I guess   our prediction going in wa..."

"maybe those bottlenecks get knocked over pretty  quickly. I think that’s an interesting question.
  What does the world look like where there aren't  these bottlenecks? Suppose progress just continues..."

"It's just like 10x bigger than your budget? I think energy is one piece. I think we   would probably build out bigger clusters than we  currently can if we could get the energy to do it.  That's funda..."

"Llama-3, and maybe Llama-4 onwards? As in, you  put this out and if somebody has a ton of compute,   then they can just keep making these things  arbitrarily smarter using the models that   you've put..."

"Let's zoom out a little bit from specific  models and even the multi-year lead times   you would need to get energy approvals and so  on. Big picture, what's happening with AI these   next couple of d..."

"the things that they want a lot more. So maybe not overnight, but is it your   view that on a cosmic scale we can think of  these milestones in this way? Humans evolved,   and then AI happened, and th..."

"Obviously it's very difficult to predict  what direction this stuff goes in over time,   which is why I don't think anyone should be  dogmatic about how they plan to develop it   or what they plan to ..."

"I think that there's so many ways in which  something can be good or bad that it's hard   to actually enumerate them all up front. Look at  what we've had to deal with in social media and   the differ..."

"It seems to me that it would be a good idea.  I would be disappointed in a future where AI   systems aren't broadly deployed and everybody  doesn't have access to them. At the same time,   I want to b..."

"time a year or two years, let's say you just have  one or two years more knowledge of the security   holes. You can pretty much hack into any system.  That’s not AI. So it's not that far-fetched to   ..."

"that I don't hear people talking about quite as  much. There's the risk of the AI system doing   something bad. But I stay up at night worrying  more about an untrustworthy actor having the super   st..."

"That seems plausible to me. If that works out,  that would be the future I prefer. I want to   understand mechanistically how the fact that  there are open source AI systems in the world   prevents so..."

"then that could be a risk. That's one of  the things that we need to watch out for.  Is there something you could see in the deployment  of these systems where you're training Llama-4 and   it lied to..."

"lot of what we have to spend our time on as well. I found the synthetic data thing really curious.   With current models it makes sense why there might  be an asymptote with just doing the synthetic d..."

"It has to be the past? Oh yeah, it has to be the past.
  I'm really interested in American history and  classical history. I'm really interested in the   history of science too. I actually think seein..."

"Now I think that there can be things that are  better about being physically together. These   things aren't binary. It's not going to be like  “okay, now you don't need to do that anymore.”   But ove..."

"of my life. Our family built this ranch in Kauai  and I worked on designing all these buildings. We   started raising cattle and I'm like “alright, I  want to make the best cattle in the world so how ..."

"I'm not sure but I'm actually curious  about something else. So a 19-year-old   Mark reads a bunch of antiquity and  classics in high school and college.   What important lesson did you learn from  it..."

"I don't want to strain the analogy too  much but I do think that a lot of the time,   there are models for building things that  people often can't even wrap their head   around. They can’t understand..."

"take a bunch of your money. But then there's the  qualitative version, which is actually what upsets   me more. There's a bunch of times when we've  launched or wanted to launch features and Apple's  ..."

"will be better because it's open source. There is one world where maybe   that’s not the case. Maybe the model ends up  being more of the product itself. I think it's   a trickier economic calculation..."

"should share the upside of that somehow. Regarding other open source dangers,   I think you have genuine legitimate points about  the balance of power stuff and potentially the   harms you can get rid..."

"I actually think the real harms that need more  energy in being mitigated are things where someone   takes a model and does something to hurt a  person. In practice for the current models,   and I wou..."

"By when will the Llama models be  trained on your own custom silicon? 
  Soon, not Llama-4. The approach that we took is  we first built custom silicon that could handle   inference for our ranking an..."

"Final question. This is totally out of  left field. If you were made CEO of Google+   could you have made it work? Google+? Oof. I don't know.   That's a very difficult counterfactual. 
 Okay, then th..."