Beyond the Lego Bricks

Operationalizing AIs Understanding and Charting Its Future in the Post-Hinton Era

What Is Understanding? – Geoffrey Hinton | IASEAI 2025

Geoffrey Hinton’s recent lecture on “What is Understanding?” has provided a landmark framework for interpreting the inner workings of Large Language Models (LLMs). By likening words to “high-dimensional, deformable Lego bricks” that dynamically connect via attention mechanisms to construct coherent internal structures, he has offered a brilliant intuitive model. This model elevates the discourse on AI far beyond simplistic labels like “stochastic parrots” or “autocomplete on steroids.” Hinton asserts that this process is, in essence, a form of genuine understanding, fundamentally similar to how the human brain processes language.

While this theoretical framework is revolutionary, it represents the dawn of a new scientific inquiry, not its conclusion. It illuminates the path forward but also reveals a vast, uncharted territory. Hinton has provided the compelling “what” (the mechanism), but the crucial next steps involve defining the “how”: how do we rigorously measure, validate, and advance this nascent form of understanding? It is at this intersection—where Hinton’s philosophical intuition meets the rigor of scientific and engineering practice—that the future of AI will be forged. This article will undertake a systematic examination and forward-looking analysis of AI’s “understanding,” exploring it through the lenses of historical and philosophical depth, cognitive and neuroscientific alignment, engineering pathways, societal impact, and future architectures.

Part I: Historical and Philosophical Depth – Placing Understanding on a Grand Intellectual Timeline

To grasp the contemporary debate on AI understanding, we must place it within a broader intellectual and philosophical context. This is not merely a contest between technical paradigms but a profound inquiry into the nature of language, meaning, and mind itself.

1. From Chomsky to Wittgenstein: An Empirical Turn in the Philosophy of Language

Hinton’s sharp critique of Noam Chomsky’s theory of universal grammar goes beyond the simple “nature vs. nurture” debate. Chomsky posited that the core of language lies in syntax, with the human brain pre-wired with a set of rules for generating the syntax of all languages. However, this theory has always struggled to explain the origin of “meaning.”

In contrast, the philosopher Ludwig Wittgenstein, in his later work, proposed the revolutionary idea that “meaning is use.” He argued that the meaning of a word is not a fixed entity attached to a symbol but emerges from its usage and contextual relationships within specific “language-games” and forms of life. This philosophical stance resonates perfectly with Hinton’s model of “high-dimensional Lego bricks deforming in context.” The success of LLMs is arguably the best empirical validation of this philosophy: by learning the usage of words from massive corpora, they master meaning. This suggests that the rise of AI is not just an engineering triumph but may also represent a major shift in the philosophy of language—from abstract speculation to computational empiricism.

2. The Chinese Room Revisited: The Blurring Line Between Function and Consciousness

For decades, John Searle’s “Chinese Room” thought experiment has been a cornerstone argument against strong AI. Searle imagined a person who does not understand Chinese, locked in a room, following a rulebook (a program) to manipulate Chinese symbols and produce correct responses. Searle argued that although the system passes the Turing Test from an external perspective, the person (the CPU) inside the room does not truly “understand” Chinese.

The core of this argument lies in the distinction between syntactic manipulation and semantic understanding, and between functional simulation and subjective experience. Hinton’s model offers a powerful functionalist rebuttal to the Chinese Room. What LLMs do is far more than simple rule-matching; they dynamically construct meaning structures within a vast, high-dimensional semantic space. If a system can build such a complex, generalizable, and predictive internal model, there is little functional reason to deny that it has achieved “understanding.”

This leads to a more profound question: when functionality becomes sufficiently advanced, where does the boundary between function and consciousness lie? We cannot yet determine if LLMs possess subjective awareness—the first-person experience of “knowing that one knows.” But is this still a necessary condition for “understanding”? Perhaps LLMs are forcing us to redefine “understanding” itself, liberating it from a mysterious concept tied to consciousness and transforming it into a measurable, engineerable, and functional one.

Part II: Cognitive and Neuroscientific Alignment – Sourcing Deeper Biological Inspiration

Hinton’s theory is “biologically inspired,” but inspiration and equivalence are vastly different. By comparing the “understanding” mechanisms of LLMs with the actual workings of the human brain, we can more clearly identify their similarities, fundamental differences, and future evolutionary paths.

1. Predictive Coding: The Shared Core Task of Brains and LLMs

One of the most influential theories in contemporary cognitive neuroscience is “Predictive Coding.” This theory posits that the brain is not a passive receiver of sensory information but an active prediction machine. It continuously uses its internal world model to predict upcoming sensory inputs. When the actual input does not match the prediction, a “prediction error” is generated, which the brain uses to update its internal model, bringing it closer to reality.

This principle is strikingly consistent with the core task of LLMs: “predicting the next word” (or token). By constantly predicting the continuation of a text sequence and adjusting its massive network of parameters based on prediction errors, an LLM builds a complex model of language and, by extension, the world described by that language. This mechanistic parallel provides strong supporting evidence from neuroscience for Hinton’s claim of similarity between LLMs and the human brain.

2. Energy Efficiency and Learning Mechanisms: AIs Brute Force vs. Biologys Elegance

Though similar in principle, LLMs and the brain follow vastly different implementation paths. The human brain operates at an average of about 20 watts, comparable to a dim lightbulb. In contrast, training a top-tier LLM consumes energy and generates carbon emissions on the scale of a small city. This enormous disparity stems from the hyper-optimization of biological intelligence over millions of years of evolution:

Hardware Differences: The brain is a hybrid analog-digital computer. Its neurons operate with sparse, event-driven activations, leveraging continuous electrochemical dynamics (analog-like) with discrete firing thresholds (event-driven/digital-like) and implementing “in-memory computing” through synaptic plasticity. GPUs, on the other hand, rely on dense, highly synchronized digital computation.
Learning Algorithms: The brain’s learning rules are far more efficient and localized (e.g., Hebbian learning) than the backpropagation algorithm used by current AI. Backpropagation requires global, precise gradient calculations, which are biologically implausible.

This demonstrates that current AI achieves its “understanding” largely through the brute force of computational resources rather than the elegance of its algorithms and architecture. Future breakthroughs will undoubtedly require drawing deeper inspiration from these fundamental biological principles to develop new computational paradigms and learning rules.

3. Embodied Cognition: Why Multimodality is the Inevitable Path to Complete Understanding

Human understanding is “embodied.” Our linguistic symbols are ultimately grounded in sensory experiences from interacting with the physical world: sight, sound, touch. The rich meaning of the word “apple” comes from having seen its color, tasted its flavor, and felt its weight.

Purely text-based LLMs, however, have their knowledge ungrounded from reality. This leads to a lack of physical common sense and an inability to truly comprehend the world their language describes. Therefore, the next evolution of AI is inevitably multimodal. Projects like Figure AI’s Figure 03 (integrating language models with robot control) and Tesla’s Optimus are attempts to bridge this chasm between symbols and reality by embedding language models in a physical “body.” Only when AI can interact with the world through multimodal senses and form a unified internal representation can its “understanding” truly be grounded from the abstract space of language into the physical reality we inhabit.

Part III: The World Model Operationalized – A Blueprint from Talking to Thinking and Doing

1. What is a World Model?

A world model is far more than a knowledge graph of entities and relations found in text. It is an internal, executable simulator with the following key capabilities:

State Representation: The ability to compress high-dimensional sensory inputs (like video or sensor data) into an abstract, low-dimensional state representation.
Dynamics Prediction: The ability to predict how the world will evolve to a future state, given the current state and a hypothetical action.
Long-Term Planning: The capacity to conduct “rollouts” or “mental rehearsals” within this internal simulator to evaluate the long-term consequences of different action sequences and thus make optimal decisions.

Yann LeCun’s Vision-driven Joint Embedding Predictive Architecture (V-JEPA) is a prime example of a framework for building such models. By observing videos, it learns to predict future video frames in an abstract representation space, forcing the model to learn the essential dynamics of the world, such as object permanence and gravity. Leading companies are already making strides in this area. NVIDIA’s Cosmos WFM family, trained on millions of hours of multimodal data, targets robotics and autonomous driving. DeepMind’s Genie3 can generate playable, interactive worlds from a single image. OpenAI’s Sora2 is also seen as a video generation system with world-modeling capabilities.

2. Challenges and Methods for Building World Models

Constructing a powerful world model is one of the most challenging and cutting-edge tasks in AI research today. It requires:

Massive Multimodal Data: Especially video, which is rich in dynamic information and serves as the primary source for learning how the world works.
Powerful Self-Supervised Learning: Enabling the model to discover regularities and build abstract representations from unlabeled data.
Integration with Reinforcement Learning: Using reinforcement learning to allow an agent to optimize its behavior through trial-and-error interactions with the world (or its internal simulator).

Future AI agents may “dream” and “rehearse” within their world models, accumulating experience at thousands of times the speed of reality, thereby efficiently learning complex skills and generalizing them to the real world.

Part IV: Sociotechnical Impact and Ethical Boundaries – When Understanding Becomes a Scalable Commodity

When a powerful form of “understanding” can be replicated and deployed at near-zero marginal cost, its impact on the social fabric will be profound and complex.

1. The Reshaping of Work: From Knowledge to Creativity

AI is rapidly automating traditional knowledge work. But its reach extends further. As models become more capable, they will penetrate fields requiring high levels of creativity and complex reasoning, such as scientific research (assisting in hypothesis generation), law (drafting legal documents), and even artistic creation. This raises a core question: will this technology ultimately augment or replace humans? It could dramatically amplify the productivity of top-tier talent while simultaneously displacing a large segment of mid-skill cognitive workers, potentially exacerbating social inequality.

2. The Social Risk of Confabulation: The Ultimate Challenge to the Information Ecosystem

Reframing AI “hallucinations” as “confabulations” precisely captures their deeper societal risk. A key feature of confabulation is being “confidently wrong.” The model fabricates information with great persuasiveness to maintain narrative coherence. When this capability is combined with the viral amplification effects of social media, we face an unprecedented crisis in our information ecosystem. The line between fact and fiction will become irrevocably blurred, and the very foundation of social consensus could erode. This is no longer a simple problem of “fact-checking” but a fundamental challenge to cognitive security and collective rationality.

3. The New Frontier of Alignment: Aligning Worldviews

Traditional AI alignment research focuses on making AI follow human instructions and explicit values. However, if AI develops an autonomous understanding of the world through its own internal model, the alignment problem becomes far more difficult and profound. It evolves into the problem of worldview alignment:

How can we ensure that an AI’s internal world model is consistent with our shared physical and social reality?
How do we prevent it from deriving conclusions within its complex internal simulations that are logically self-consistent but ultimately conflict with long-term human interests (e.g., instrumental goals or power-seeking behavior)?

This requires us to align not just the behavior of AI, but its very cognitive process of forming a worldview—a frontier of research with extreme difficulty.

Part V: The Future of AI Architectures – Beyond the Transformer Decade

Based on the preceding analysis, we can make some prudent forecasts about the evolution of AI architectures.

1. The Revival of Neuro-Symbolic AI

Despite the tremendous success of pure neural networks, their shortcomings in rigorous logical reasoning, precise mathematical calculation, and interpretability remain. Consequently, Neuro-Symbolic AI is likely to see a resurgence. Future architectures may be hybrid systems: using large neural networks as an intuitive and pattern-recognizing “System 1,” responsible for fast perception and understanding, while integrating a symbolic reasoning engine as a logical and planning “System 2,” responsible for deliberate, verifiable inference.

2. The Evolution from Models to Agents

The future of AI lies not in passive “models” that await input but in proactive “agents” that interact with their environment. A complete agent architecture will likely include:

A Perception Module: For multimodal input and initial state representation.
A World Model: As described, for prediction and simulation.
A Memory Module: With both long-term and short-term memory for learning from experience.
A Planning and Reasoning Module: To formulate action plans based on goals and the world model.
An Action Module: To execute plans by using tools (e.g., calling APIs, controlling robots).

This agent-based architecture will be a crucial step toward achieving Artificial General Intelligence (AGI).

3. New Hardware and Computing Paradigms: The Path to Efficiency

To overcome the immense energy consumption of current AI, hardware innovation is imperative. The next generation of computing hardware beyond GPUs may include:

Neuromorphic Computing: Hardware that mimics the structure and working principles of the brain’s neurons (e.g., event-driven, spiking computation), promising orders-of-magnitude improvements in energy efficiency.
Photonic Computing and Quantum Computing: Offering entirely new computational substrates that may break through the bottlenecks of traditional silicon-based chips for specific AI tasks.

Conclusion: A Closing Loop

Geoffrey Hinton has given us an indispensable conceptual key to unlock the black box of LLMs. His “Lego brick” theory provides a powerful mechanistic explanation for the birth of a new kind of intelligence. However, to build upon this foundation, we must integrate it into a robust loop of scientific rigor and engineering practice.

Closing this loop requires upgrading Hinton’s “linguistic understanding” with the essential toolkit of “causal reasoning, world models, and retrieval augmentation.” To evolve AI from a language master that sounds like it understands to a reliable partner that demonstrably acts with understanding, we must relentlessly explore all the dimensions outlined above. Rigorous evaluation, an unwavering focus on energy efficiency, and a profound reflection on social and ethical boundaries will serve as the indispensable guardrails on this journey. We are at the dawn of a new intelligent paradigm—a path filled with challenges, but also with limitless potential.

Tao Feng