Meta’s LeCun says students interested in next-gen AI shouldn’t bother with LLMs

Summary

Yann LeCun, Meta’s lead AI researcher, believes that large language models (LLMs) will not lead to human-like intelligence. Instead, he is pursuing an alternative “world model” approach.

LeCun argues that LLMs like those behind ChatGPT have a “very limited understanding of logic” and “do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term and cannot plan […] hierarchically.”

According to LeCun, LLMs can only respond correctly when given the right training data, making them “intrinsically unsafe.” He is not counting on the evolution of LLMs to achieve human-like intelligence.

“If you are a student interested in building the next generation of AI systems, don’t work on LLMs,” LeCun writes. While LLMs are useful “despite their limitations,” LeCun says, big companies are already devoting enough effort.

Of course, LeCun is promoting his own research here, which focuses on developing a “common sense” world model.

Instead of laboriously learning individual tasks, such as generating text or images, AI should understand the world and then use this basic knowledge to learn to solve tasks more easily and, above all, much more efficiently, similar to how humans learn.

To achieve this, LeCun says, AI models must understand the physical world, have persistent memory, reason, and plan, possibly hierarchically. “Four essential characteristics necessary for intelligent behavior, which humans and many animals exhibit,” LeCun writes.

LeCun first presented his concept of autonomous AI in the spring of 2022 and has been developing it ever since. He estimates that it could take up to ten years for this vision to become a reality.

Meta’s Fundamental AI Research (FAIR) Lab is already focused on this new generation of AI, which is necessary to develop useful AI agents for everyday use, LeCun says.

“[Achieving AGI is] not a product design problem, it’s not even a technology development problem, it’s very much a scientific problem,” LeCun tells the Financial Times.

Nevertheless, Meta invests heavily in LLM research and development, sometimes outpacing its commercial competitors with its open-source Llama model. The company also implements Llama in its own products, such as Meta AI.

A particularly capable version has been announced for the latest model, Llama 3, which might outperform OpenAI’s GPT-4. It is not yet clear whether the largest version of Llama 3 will also be open source.