All Segments
Francois Chollet — Why the biggest AI models can't solve simple puzzles
53 segments available
Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today. I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through. It was really fun discussing/debating the cruxes. Enjoy! Check out ARC-AGI Prize here: https://arcprize.org 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒 * Transcript: https://www.dwarkeshpatel.com/p/francois-chollet * Apple Podcasts: https://podcasts.apple.com/us/podcast/francois-chollet-mike-knoop-llms-wont-lead-to-agi-%241/id1516093381?i=1000658672649 * Spotify: https://open.spotify.com/episode/7bmeJQOvXGy4LYl6YoiYYP?si=obUSUEwjSA6tkB8EBcb18w * Follow me on Twitter: https://x.com/dwarkesh_sp 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 – The ARC benchmark 00:11:53 – Why LLMs struggle with ARC 00:19:43 – Skill vs intelligence 00:28:38 – Do we need “AGI” to automate most jobs? 00:49:11 – Future of AI progress: deep learning + program synthesis 01:01:23 – How Mike Knoop got nerd-sniped by ARC 01:09:20 – Million $ ARC Prize 01:11:16 – Resisting benchmark saturation 01:18:51 – ARC scores on frontier vs open source models 01:27:02 – Possible solutions to ARC Prize
Segments Timeline
Introducing the ARC Benchmark
Francois Chollet introduces the ARC benchmark, a unique IQ test for machine intelligence designed to resist memorization. He explains how LLMs, which excel at memorizing static programs, struggle with ARC puzzles that require core knowledge and novel problem-solving skills. This segment sets the stage for understanding the limitations of current AI models in achieving AGI.
"LLms are very good at memorizing static programs If you scale up the size of your database, you are not increasing the intelligence of the system one bit. I feel like you're using words like memo..."
The Challenge of Novelty in AI
Chollet discusses the critical difference between memorization and true intelligence in AI. He emphasizes the need for models to adapt to novel tasks they haven't encountered before, which is a hallmark of human intelligence. This segment highlights the challenges LLMs face in synthesizing new solutions on the fly, a key requirement for achieving AGI.
"have not been doing very well on it. In fact, the approaches that are working well are more towards discrete program search, program synthesis. First of all, I’ll make a comment that I’m glad tha..."
Why Intelligence Can't Be Static
In this segment, Chollet explains why intelligence must be dynamic and adaptable rather than static. He contrasts human intelligence, which evolves to meet new challenges, with the limitations of AI models that rely on pre-trained data. This discussion underscores the fundamental differences between human cognition and current AI capabilities.
"then I would be extremely interested. I would think this is on the path to AGI. The advantage they have is that they do get to see everything. Maybe I'll take issue with how much they are relying..."
Understanding ARC Challenges
Chollet describes what an ARC challenge looks like, emphasizing the need for core knowledge and reasoning skills. He explains that each puzzle is novel and requires a fresh approach, which LLMs struggle to provide. This segment illustrates the unique nature of ARC puzzles and why they are a significant test for AI models.
"Instead, we have general intelligence. We are born with extremely little knowledge about the world. We are born with the ability to learn very efficiently and to adapt in the face of things that..."
The Role of Multimodal Models
Chollet discusses the potential of multimodal models in improving spatial reasoning and understanding abstract concepts. He raises questions about their ability to tackle ARC-type challenges and the empirical nature of these claims. This segment explores the future of AI and the importance of developing models that can handle diverse types of reasoning.
"One view is that in the next year, as we gain models that are natively multimodal capability rather than as an add-on, they will understand these kinds of patterns because that's something we 'd..."
Performance Spectrum in Humans vs. AI
In this segment, Chollet compares human performance on ARC challenges to that of AI models, highlighting the spectrum of intelligence in humans. He discusses how even average humans can outperform LLMs on these tasks, emphasizing the need for AI to bridge the gap in understanding and reasoning capabilities.
"really what LLMs are struggling with. Before I play more devil's advocate, I just want to step back and explain why I'm especially interested in having this conversation. Obviously there’s the mi..."
Unlocking AI Performance with Fine-Tuning
Chollet explains the significance of test-time fine-tuning in improving LLM performance on ARC tasks. He contrasts this approach with static inference methods, highlighting how active inference can enhance AI's adaptability and problem-solving abilities. This segment delves into the technical aspects of AI training and performance measurement.
"There's a bunch of interesting points here. There is indeed a branch of LLM approaches spearheaded by Jack Cole that are doing quite well. They are state-of-the art in fact.. But you have to look ..."
The Confusion Between Skill and Intelligence
Chollet addresses the common misconception that scaling up AI models equates to increasing their intelligence. He clarifies the distinction between skill and intelligence, arguing that current benchmarks often measure memorization rather than true reasoning capabilities. This segment challenges the audience to rethink how we evaluate AI performance.
"inference to LLMs. That's working extremely well, actually. So that's fascinating to me. There are so many interesting rabbit holes there. A lot of the scale maximalists share your broader perspe..."
Redefining Reasoning in AI
In this concluding segment, Chollet redefines reasoning in the context of AI, contrasting program fetching with on-the-fly program synthesis. He emphasizes the complexity of true reasoning and the challenges AI faces in replicating this human-like ability. This discussion encapsulates the overarching theme of the conversation regarding the limitations of current AI models.
"They're basically these big interpolative databases, interpolative memories. Of course, if you scale up the size of your database and cram more knowledge and patterns into it, you are going to b..."
Generalization vs. Skill in AI
Chollet discusses the misconception that scaling up AI models leads to general intelligence. He clarifies that while larger models can handle more complex tasks, this does not equate to true general intelligence, which involves the ability to adapt and learn efficiently across a wide range of problems.
"Isn't that like the same kind of thing? You can't just see one example and now you have the program. You actually have to drill it. These models also had to drill it with a bunch of pre-training d..."
Learning New Languages with AI
Chollet references Google's Gemini 1.5 model as an example of how AI can learn new languages efficiently by leveraging pre-training data. He contrasts this with human language acquisition, highlighting the differences in adaptability and the limitations of LLMs in synthesizing new knowledge.
"more general ways of reasoning? Absolutely. But then isn't that intelligence? No, it's not. If you scale up your database and keep adding more knowledge and program templates to it, then sure it b..."
The Limitations of LLMs on ARC Puzzles
Chollet argues that LLMs struggle with ARC puzzles due to their design, which resists memorization. He explains that despite their vast knowledge, LLMs fail to solve these simple tasks, underscoring the limitations of their reasoning capabilities and the reliance on memorization.
"They would give the model, in context, the grammar book and the dictionary of a language that has fewer than 200 living speakers. It's not in the pre-training data. You just give it the dictionary..."
Memorization Patterns in AI and Humans
In this segment, Chollet compares the memorization patterns of LLMs to human cognitive processes. He discusses how both rely on memorization but emphasizes that humans possess a unique ability to adapt and generalize in novel situations, a capability that LLMs currently lack.
"to resist memorization. This is the only thing. This is the huge blocker for LLM performance. If you look at LLMs closely, it's pretty obvious that they're not really synthesizing new programs on..."
The Role of Novelty in Programming
Chollet highlights the importance of novelty in programming tasks, arguing that LLMs cannot effectively generalize to new problems. He discusses the limitations of LLMs in adapting to unfamiliar situations, contrasting this with human programmers who navigate novel challenges daily.
"I think this is true of humans as well. Humans use memorization pattern matching all the time, of course, but humans are not limited to memorization pattern matching. They have this very unique a..."
The Complexity of Human Problem-Solving
Chollet asserts that human problem-solving involves navigating a wide range of novel situations, which LLMs are not equipped to handle. He emphasizes that true intelligence requires more than just memorization and pattern matching, highlighting the adaptability of human cognition.
"their program? At what point do they have to use this so-called extreme generalization capability? Forget about Google software developers. For every human, every day of their lives is full of no..."
The Future of Software Engineering with AI
Chollet discusses the potential future of software engineering in a world where LLMs are prevalent. He questions whether LLMs can replace human programmers, emphasizing the need for creativity and problem-solving skills that go beyond mere memorization.
"Take self-driving cars, for instance. You take a self-driving car operating in the Bay Area. Do you think you could just drop it in New York City or drop it in London, where people drive on the ..."
The Essence of Programming
In this segment, Chollet reflects on the essence of programming as problem-solving rather than just syntax. He argues that while LLMs can assist with coding tasks, they lack the deeper understanding and mental models that human programmers possess.
"I’m denying the premise that that's the only thing these models are necessarily doing. Suppose you just subbed out a remote work with an LLM and they're a programmer. What is the first point at w..."
The Role of Memory in AI Development
Chollet discusses the role of memory in AI development, arguing that while LLMs can fetch templates from their training data, they lack the ability to create novel solutions. He emphasizes that true intelligence involves more than just recalling information.
"I agree we're not at a full AGI yet. These models have fewer than a trillion parameters. A human brain has somewhere on the order of 10-30 trillion synapses. If you were just doing some naive math..."
Creativity and Interpolation in AI
Chollet explores the relationship between creativity and interpolation in AI. He suggests that while larger models may exhibit higher levels of generalization, true creativity involves a blend of memorization and reasoning that current models struggle to achieve.
"If I look at how people are using LLMs in their software engineering job today, they're using it as a Stack Overflow replacement. They're using it as a way to copy-paste code snippets to perform..."
The Limits of AI Generalization
Chollet concludes by discussing the limits of AI generalization, particularly in relation to novel tasks like ARC puzzles. He argues that while larger models can learn from their training data, they still face significant challenges when confronted with unfamiliar problems.
"levels of generalization. GPT-2 couldn't do grade school-level math problems that required more generalization than it had the capability to do. GPT-3 and GPT-4 can. Not quite. GPT-4 has a higher..."
The Challenge of Novelty in AI
In this segment, Chollet explores the implications of a multimodal model potentially solving ARC tasks at a high level. He questions whether achieving this would indicate progress towards AGI, emphasizing the importance of training models on core knowledge without anticipating specific test sets, which is the essence of the ARC challenge.
"If you present something that’s actually novel that is not in a steady distribution, like an ARC puzzle for instance, it will fail. Let me make this claim. The program synthesis is a very useful ..."
The Nature of Intelligence and Automation
Chollet discusses the distinction between automation and intelligence, asserting that while LLMs can automate many tasks, they do not equate to true intelligence. He emphasizes that intelligence is necessary for dealing with change and uncertainty, and that relying solely on memorization is insufficient for complex problem-solving.
"but after having only been trained on core knowledge-related stuff. But human kids, we’re necessarily just trained on what we have in our genes… Let me rephrase that. I want it to be only trained ..."
Pathfinding as a Metaphor for Intelligence
Chollet uses the metaphor of pathfinding in a game to illustrate how intelligence operates in uncertain environments. He explains that intelligence involves navigating future situations with incomplete information, contrasting this with the limitations of pure memorization, which cannot adapt to new challenges.
"you can just brute force intelligence. If the world, if your life, were a static distribution then sure, you could just brute force the space of possible behaviors. There are several metaphors fo..."
The Spectrum of Human Learning
In this segment, Chollet argues that human learning is a blend of memorization and reasoning, challenging the notion that LLMs simply memorize. He discusses how humans learn skills through practice and application, suggesting that the processes of learning and reasoning are more complex than mere memorization.
"I feel like you're using words like “memorization,” which we would never use for human children. If your kid learns to do algebra and then learns to do calculus, you wouldn't say they've memoriz..."
The Future of AI and Job Automation
Chollet contemplates a future where AI can automate many jobs through learned skills, questioning whether this automation represents true intelligence. He asserts that while economic value can be generated through automation, it does not equate to the intelligence required to navigate dynamic environments.
"Tell me if this is wrong. Is it compatible in your world if all the remote workers are gone but they're doing skills which we can potentially make synthetic data out of? We record every single r..."
Generalization vs. Memorization in AI
Chollet discusses the limitations of LLMs in terms of generalization, arguing that while they can perform tasks through memorization, they lack the ability to handle novelty effectively. He emphasizes that true intelligence requires more than just the ability to interpolate from known data.
"the moment you have to deal with change, novelty, and uncertainty. As long as you're in a space that can be exactly described in advance, you can just rely on pure memorization. In fact, you can..."
Grokking and Generalization in Deep Learning
Chollet introduces the concept of grokking, explaining how deep learning models transition from memorization to generalization. He highlights the importance of the minimum description length principle in understanding how models can learn to generalize effectively from limited data.
"learning can learn, there is a regime where it generalizes if you have an overparameterized model. We don't have that in comparison to all the tasks we want these models to do right now. Grokking..."
The Role of Fluid Intelligence in AI
In this segment, Chollet discusses the concept of fluid intelligence and how larger models can achieve higher levels of generalization. He explains that as models grow, they can learn to generalize across multiple skills, but they remain limited by their architecture.
"That's correct. LLMs are not infinitely large. They have only a fixed number of parameters. They have to compress their knowledge as much as possible. In practice, LLMs are mostly storing reusab..."
Deep Learning vs. Program Synthesis
Chollet contrasts deep learning with discrete program synthesis, explaining how each approach has its strengths and weaknesses. He argues that while deep learning is efficient for certain tasks, program synthesis offers a more data-efficient way to learn generalizable solutions.
"It's intrinsically limited because the substrate of your model is a big parametric curve. All you can do with this is local generalization. If you want to go beyond this towards broader or even ..."
Creating a Hybrid AI System
Chollet envisions a future hybrid AI system that combines deep learning with discrete program search. He explains how this system could leverage the strengths of both approaches to create a more efficient and adaptable intelligence capable of solving complex problems.
"quickly, but it is very data inefficient. In order to make it work, you need a dense sampling of the operating space. You need a dense sampling of the data distribution. Then you're limited to o..."
The Role of Intuition in AI Reasoning
Chollet discusses how deep learning models can provide intuition to guide program search in AI. He emphasizes the importance of combining intuition with reasoning to create a more effective AI system that can adapt to new situations and solve problems efficiently.
"The outer structure is going to be a discrete program search system. You're going to fix the fundamental limitation of discrete program search, which is combinatorial explosion, with deep learni..."
The Challenge of System 2 Architecture
Chollet reflects on the challenges of developing a System 2 architecture for AI, arguing that while scaling up LLMs is relatively straightforward, creating a robust reasoning system is much more complex. He emphasizes the need for deliberate engagement with learned material to achieve true intelligence.
"that was synthesized using very, very little data. Something like this would solve ARC. That's actually a really interesting prompt. There’s an interesting crux here. I talk to my friends who are..."
Intelligence and Memory: A Complex Relationship
In this segment, Chollet explores the intricate relationship between intelligence and memory, referencing Jean Piaget's definition of intelligence. He discusses how human intelligence often relies on prior experiences and memory, raising questions about the adaptability of AI systems that lack this depth of memory.
"that. I'm curious why. My intuition is that this whole System 2 architecture is the hard part. It’s the very hard and unobvious part. Scaling up the interpolative memory is the easy part. It's l..."
Genetic Factors in Human Intelligence
Chollet addresses the genetic basis of intelligence differences among humans, suggesting that innate factors play a crucial role in cognitive capabilities. He emphasizes that simply providing more training data cannot transform someone into a genius, highlighting the need for better algorithms and architectures in AI.
"Here's one way to ask the question. In the brain, supposedly we do program synthesis, but it is just synapses connected to each other. Physically, it's got to be that you just query the right ci..."
The Importance of Architectural Improvements
Chollet discusses the significance of architectural advancements in AI, noting that models like Gemini 1.5 Flash demonstrate improved efficiency compared to previous iterations. He reflects on the current state of AI research and the potential for breakthroughs through innovative architectures.
"That's a great question. There is extensive evidence that differences in intelligence are mostly genetic in nature. That means that if you take someone who is not very intelligent, there is no a..."
The Birth of the ARC Prize
Mike Knoop shares his journey into AI and the inspiration behind launching the ARC Prize. He recounts his experiences during the pandemic that reignited his interest in AGI and the ARC puzzles, leading to the establishment of a competition aimed at advancing AI research.
"you're running this prize with François. Tell me about how this came together. What prompted you guys to launch this prize? I've been AI curious for 13 years. I co-founded Zapier and I’ve been r..."
Why ARC is Unique and Challenging
Knoop explains the unique challenges posed by the ARC benchmark, emphasizing its resistance to memorization and the need for innovative approaches to solve it. He highlights the importance of making ARC more widely known in the AI community to inspire new research directions.
"saturation effect that MMLU and GMS8K have. When I looked at the scores and the progress over the last four years, I was really shocked to see that we'd made very little objective progress toward..."
The Need for New Ideas in AI
Chollet and Knoop discuss the stagnation in AI research due to the closing of frontier research and the dominance of LLMs. They argue that the current focus on LLMs may hinder the exploration of new ideas necessary for advancing towards AGI.
"that and start being able to tackle problems that you haven't memorized or seen before, you need to try new ideas. ARC is not just meant to be this sort of measure of how close we are to AGI. It..."
The Impact of Open Research on Innovation
Chollet reflects on the historical importance of open research in driving AI innovation, lamenting the recent trend towards closed research practices. He argues that this shift has stifled creativity and slowed progress in the field.
"that are actually working against that happening. We’re actually less likely to generate new ideas right now. One of the trends is the closing up of frontier research, right? The GPT-4 paper fro..."
The Future of AI Research and the ARC Prize
Knoop outlines the structure of the ARC Prize, including the prize pool and the goals for participants. He emphasizes the importance of public sharing of solutions to foster collaboration and accelerate progress in AI research.
"two reasons. One is that they caused this complete closing down of frontier research publishing. But they also triggered this initial burst of hype around LLMs. Now LLMs have sucked the oxygen ou..."
Exploring the Difficulty of the ARC Benchmark
Chollet and Knoop discuss the challenges of replicating human performance on the ARC benchmark and the implications for understanding intelligence. They compare ARC to other benchmarks that have become saturated, highlighting the need for fresh evaluation methods.
"The big labs also tried their hand on ARC, but because they got bad results they didn't publish anything. People only publish positive results. I wonder how much effort people have put into tryin..."
The Significance of the ARC Competition
Knoop emphasizes the importance of the ARC competition in advancing AI research, noting that it will provide valuable insights regardless of the outcomes. He encourages participation and exploration of the benchmark to uncover latent capabilities in current AI models.
"takes, the longer it takes." My prior is that ARC is going to take years to solve. We're also going to break down and do a progress prize this year. There's a $100,000 progress prize which we will ..."
Challenges of Benchmark Saturation
Chollet reflects on the saturation of previous benchmarks like MMLU and MATH, questioning whether the same saturation will occur with ARC. He discusses the empirical nature of this question and the unique approaches being taken by researchers like Jack Cole, who are attempting to push the boundaries of what LLMs can achieve.
"interesting because we're going to learn something fascinating one way or another. With regards to the 85%, separate from this prize, I'd be very curious if somebody could replicate that result. ..."
Active Inference and Program Synthesis
The conversation shifts to Jack Cole's innovative approach of using active inference and test-time fine-tuning to enhance LLM capabilities. Chollet explains how this method contrasts with traditional discrete program search, emphasizing the potential of LLMs to synthesize programs from a vast database of building blocks.
"In the GPT-4 technical report, they had this interesting graph of the HumanEval problem set, which was 22 coding problems. They had to graph it on the mean log pass curve. Early on in training, ..."
Balancing Memorization and Search Depth
Chollet discusses the balance between memorization and deep search in AI models. He argues that effective solutions will require a combination of shallow recombination of learned primitives and deeper search strategies to achieve broader generalization, highlighting the limitations of current approaches.
"is exactly what program synthesis is about. I would contrast this approach with discrete program search. In discrete program search, you're trying to assemble a program from a set of primitives. ..."
Testing the Limits of AI Models
The discussion turns to the limitations of current AI models in the context of the ARC competition. Chollet expresses a desire to explore how larger models could perform on ARC tasks, emphasizing the importance of testing these models under constrained conditions to understand their true capabilities.
"and you're doing very shallow search. They are the same thing, just different ends of the spectrum. I think where you're going to get the most value for your compute cycles is somewhere in betwee..."
The Role of Public and Private Test Sets
Chollet explains the distinction between public and private test sets in the ARC competition, discussing the implications of using public data for training. He raises concerns about potential overfitting and the need for novel, original tasks to accurately assess model performance.
"way to also test this prize with some sort of scaffolding on the biggest models, as a way to test whether scaling is the path to solving ARC. Absolutely. In the context of the competition, we wan..."
Evolving the ARC Dataset
The conversation highlights the need for continuous improvement of the ARC dataset. Chollet mentions plans for an ARC 2.0 version, aiming to address existing flaws and ensure that the benchmark remains challenging and relevant for future AI developments.
"That kind of creates uncertainty. If they can actually solve some of the tasks, is that because they memorized the answer or not? Maybe you would be better off trying to create your own private,..."
New Paradigms in Programming
Chollet discusses the potential for new programming paradigms that could emerge from successful solutions to the ARC challenge. He emphasizes the power of synthesizing problem-solving programs from minimal examples, which could revolutionize software development and AI capabilities.
""oh my god, they've leaked your data." We do want to evolve the ARC dataset. That is a goal that we want to do. François mentioned that it's not perfect. Yeah, ARC is not a perfect benchmark. I ma..."
Legitimate Solutions vs. Cheating
The final segment addresses the ethical considerations in the ARC competition. Chollet differentiates between legitimate use of tools like code interpreters and the pitfalls of attempting to brute force solutions. He stresses the importance of genuine innovation in advancing AI research.
"argument that we've stalled out in progress and that new ideas are necessary to beat ARC. Yeah, that's the point of having a money prize. You attract more people and you get them to try to solve ..."
Core Knowledge and Intelligence
In this segment, Chollet explores the concept of core knowledge in relation to intelligence. He argues that while some core knowledge is innate, much of it is acquired through early life experiences, shaping how humans understand and solve problems.
"Although some amount of fine tuning is valid because they have to use open source language models to compete here and they’re natively language. They’d need to be able to think in the ARC-type w..."
The Importance of Sharing Progress
Chollet emphasizes the necessity of sharing advancements in AI research to foster collective progress. He discusses the ARC Prize's goal of encouraging public reproducibility of results, highlighting that without sharing, true progress in AI cannot be achieved.
"knowledge. Core knowledge includes basic physics, for instance bouncing or trajectories. That would be included. But yeah, you're entirely right. The reason why, as a human, you're able to quickl..."
The Launch of the ARC Prize
The segment concludes with the announcement of the ARC Prize, a $1 million competition aimed at accelerating progress towards AGI. Chollet and Knoop express their excitement about the potential contributions from the open-source community and the implications for future AI development.
"people can score in the competition itself. I’m also excited to test specifically the scaling hypothesis and I'm very curious if you can prompt on the public version of ARC. You won't be able to ..."