19 The future of AI
This chapter argues that while deep learning has transformed cognitive automation, it remains fundamentally constrained by its nature as parametric curve-fitting: models store and retrieve patterns, excel at interpolation, and falter with novelty. These systems are brittle to phrasing and distractors, struggle to learn exact, generalizable programs, and are often over-anthropomorphized. Scaling alone hasn’t resolved these issues because it does not change the paradigm. In contrast, human intelligence is defined by extreme generalization: the ability to efficiently adapt to never-before-seen situations by leveraging abstraction, reasoning, and minimal data. The chapter reframes AI progress around this property, not task-specific performance.
To unpack intelligence, the text proposes the kaleidoscope hypothesis: the world’s apparent complexity arises from recombinations of a small set of reusable “atoms of meaning.” Intelligence then has two core components—abstraction acquisition and on-the-fly recombination—whose efficiency determines capability. It distinguishes automatons (fixed behaviors, local generalization) from intelligent agents (fluid modeling, planning, and adaptation), and calls for new targets and metrics that reward adaptation to unknown unknowns. The ARC-AGI benchmark embodies this shift by testing fluid abstraction and reasoning on novel tasks under controlled priors. Results to date show that mere scale in base language models yields minimal progress, while approaches that adapt at test time begin to move the needle.
Accordingly, the chapter highlights the emerging test-time adaptation era: methods such as test-time training and search (including program synthesis) that assemble or learn task-specific solutions on the fly. Notable demonstrations achieved high scores but at prohibitive compute costs, motivating efficiency as a defining criterion of intelligence. The roadmap emphasizes two “missing ingredients”: search and symbols, integrated with deep learning’s geometric intuition. It outlines a future of hybrid systems that blend algorithmic modules for reasoning with learned modules for perception, guided program synthesis to avoid combinatorial explosion, and lifelong libraries of reusable components to enable compositional generalization. The long-term vision is AI systems that are grown rather than hand-coded—models-as-programs capable of efficient, human-like extreme generalization across domains.
An easy yet novel puzzle
A variation of the Monty Hall problem
An adversarial example: imperceptible changes in an image can upend a model’s classification of the image.
Current machine-learning models: like a dim image in a mirror
The brain network of the C. Elegans worm: a behavioral automaton “programmed” by natural evolution. Figure created by Emma Towlson (from “Network control principles predict neuron function in the Caenorhabditis elegans connectome”, Yan et al., Nature, Oct. 2017).
Local generalization vs. extreme generalization
A kaleidoscope produces rich (yet repetitive) patterns from just a few beads of colored glass
A low-generalization system can achieve arbitrary skill at a fixed task given unlimited task-specific information
An ARC-AGI task: the nature of the task is demonstrated by a couple of input-output pair examples. Provided with a new input, you must construct the corresponding output.
Example of a task that couldn’t be solved by o3 on the highest compute settings (over $20,000 per task)
Typical ARC-AGI-1 task (left) vs. typical ARC-AGI-2 task (right)
Value-centric analogy relates instances via a continuous notion of similarity, to obtain abstract prototypes
Program-centric analogy identifies and isolates isomorphic substructures across different instances
A schematic view of program synthesis: given a program specification and a set of building blocks, a search process assembles the building blocks into candidate programs, which are then tested against the specification. The search continues until a valid program is found.
A learned program relying on both geometric primitives (pattern recognition, intuition) and algorithmic primitives (reasoning, search, memory)
A meta-learner capable of quickly developing task-specific models using reusable primitives (both algorithmic and geometric), thus achieving extreme generalization
FAQ
Why do deep learning models struggle to adapt to novelty?
Because they are static parametric functions trained to interpolate within the training distribution. After training, parameters are frozen, so models largely retrieve and recombine patterns they have already seen. When inputs fall outside the training manifold, predictions degrade sharply. In short, these systems are excellent at familiar pattern recognition, but poor at on-the-fly adaptation to unfamiliar situations.Why are deep learning models so sensitive to phrasing and adversarial inputs?
Inputs act like addresses into a vast, interpolative “latent database.” Small changes in wording (for LLMs) or imperceptible pixel tweaks (for vision models) can move the lookup to a different neighborhood, yielding different behavior. This brittleness underlies adversarial examples in vision and the need for “prompt engineering” in LLMs. It reflects retrieval of memorized programs rather than robust task understanding.Why won’t scaling current models alone lead to human-level intelligence?
Scaling improves memorization-based performance on fixed benchmarks but doesn’t address core limitations: poor adaptation to novelty, oversensitivity to confounders/phrasing, and weak algorithmic generalization. These are consequences of the curve-fitting paradigm and Transformer-style architectures, not just model size. Bigger versions of the same approach do not change its fundamental capabilities.What distinguishes automatons from intelligent agents?
Automatons execute fixed behaviors that work only in anticipated contexts (“if this, then that”). Intelligent agents can construct abstract models, reason, and plan to handle unexpected, never-before-seen situations. The hallmark of intelligence is the ability to adapt effectively under uncertainty, not just to automate known mappings from input to output.What is “extreme generalization,” and how does it differ from local generalization?
Local generalization (typical of deep nets) means reliable interpolation within a well-sampled training manifold—useful but narrow. Extreme generalization (typical of humans) is the ability to handle unknown unknowns with little or no new data, by leveraging abstraction and reasoning to rapidly adapt. It is the key capability behind human-like intelligence.What is the kaleidoscope hypothesis?
Despite surface complexity, the world is composed of a relatively small set of reusable “atoms of meaning” that recur in new combinations. Intelligence works by extracting these abstractions from experience and recombining them to make sense of novel situations—much like a kaleidoscope generates endless patterns from a few beads.What are “abstraction acquisition” and “on-the-fly recombination,” and why are they essential?
- Abstraction acquisition: extracting compact, reusable concepts from limited experience.- On-the-fly recombination: composing those abstractions into new models tailored to new problems.
Intelligence depends on doing both efficiently. Classic deep learning does acquisition (via gradient descent) but largely lacks recombination at test time and is data-inefficient.
Deep Learning with Python, Third Edition ebook for free
