Yann LeCun is not just “the JEPA guy.”
He is one of the people who made modern deep learning viable: convolutional neural networks, gradient-based recognition, self-supervised representation learning, energy-based models, and now the argument that agents need world models instead of bigger chat-shaped reflexes.
The self.md reason to track him is simple: LeCun keeps returning to the same taste in systems — learn useful internal representations from raw experience, then use those representations to predict and act.
Why he matters
LeCun is Meta’s Chief AI Scientist, Silver Professor at NYU, and one of the 2018 ACM A.M. Turing Award recipients with Yoshua Bengio and Geoffrey Hinton for work that made deep neural networks central to computing.
His older work matters because it was not hype riding. It was infrastructure:
| Thread | Why it mattered |
|---|---|
| convolutional networks | made learned visual features practical before deep learning was fashionable |
| LeNet / handwriting recognition | showed neural nets could work in real deployed recognition systems |
| energy-based learning | framed intelligence as assigning low energy to compatible states, not only predicting labels |
| self-supervised learning | pushed learning from unlabeled structure instead of expensive human labels |
| JEPA / world models | turns representation learning toward prediction, planning, and control |
The through-line is not “LeCun predicted everything.” Nobody did. The through-line is that he kept betting against hand-written intelligence and against pure surface imitation.
The old bet: learned perception
LeCun’s 1980s and 1990s work on convolutional networks is one of the roots of modern computer vision. The important move was local structure: images are not flat token bags. Nearby pixels matter, patterns repeat across space, and a model should exploit that geometry.
That sounds obvious now because ConvNets won. It did not sound obvious inside an AI field still dominated by symbolic systems and hand-engineered features.
The classic references are Backpropagation Applied to Handwritten Zip Code Recognition and Gradient-Based Learning Applied to Document Recognition . The latter is the LeNet-era paper people cite when explaining why neural networks became credible for recognition.
For self.md, this matters as a design instinct: do not treat every problem as text. The structure of the domain should shape the model.
The middle bet: self-supervised learning
LeCun has spent years arguing that supervised learning and reinforcement learning are too small to explain intelligence.
His cake metaphor is the clean version:
If intelligence is a cake, self-supervised learning is the cake, supervised learning is the icing, and reinforcement learning is the cherry.
That quote matters less as a slogan than as an allocation of attention. Labels and rewards are expensive, sparse, and narrow. Observation is abundant. A system that learns only from instruction is always data-poor compared with a system that learns from the regularities of the world.
This is where Barlow Twins , VICReg , DINO , and DINOv2 sit in the story: not as one neat family tree, but as a set of attempts to learn useful representations without labels and without collapsing into trivial embeddings.
The current bet: JEPA and world models
JEPA stands for Joint Embedding Predictive Architecture.
The basic idea is to predict in representation space rather than reconstruct every raw input detail. In images and video, that difference matters because the surface is noisy and multimodal. Predicting every future pixel is often the wrong target; predicting the action-relevant latent state is closer to what an agent needs.
LeCun’s A Path Towards Autonomous Machine Intelligence lays out the broader architecture: perception, world model, cost modules, actor, configurator, and planning. I-JEPA applies the idea to images. V-JEPA moves it into video. V-JEPA 2 adds stronger evidence around understanding, prediction, and planning from video plus a smaller amount of robot trajectory data.
In the Welch Labs interview, LeCun describes the operational version:
“You take an observation in the world and then the next observation in the world. You run them through encoders … and then you have a predictor that tries to predict the state at time t+1 from the state at time t, and you might condition this on an action, and now you have a world model.”
That is the bridge from representation learning to agency.
The LLM critique
LeCun’s critique of LLMs is sharp, sometimes overstated, and useful anyway.
From the same interview:
“LLMs are really good at manipulating language but basically nothing else.”
And the more precise version:
“They’re really good in domains where the language itself is the substrate of reasoning.”
That is the point worth stealing. LLMs are excellent when the task lives inside language: code, prose, instructions, summaries, protocol glue. They are weaker when the task depends on persistent state, physical dynamics, hidden constraints, or consequences that need to be simulated before action.
So the sane reading is not “LLMs are obsolete.” That is too dumb. The sane reading is: language models are an interface and reasoning substrate, but agents need a separate state-and-consequence layer.
Why this belongs in self.md
A personal AI OS does not need a robot arm to need a world model.
Its world is:
- files
- branches
- calendars
- reminders
- invoices
- inboxes
- permissions
- drafts
- public posts
- private commitments
- user preferences
- irreversible side effects
An agent that cannot predict what changes after an action is not autonomous. It is just a fast clerk with a risky autocomplete habit.
LeCun says the quiet part loudly:
“I do not understand how you can even think of building an agentic system without … the ability of predicting the consequences of its actions.” — Yann LeCun, Welch Labs interview
That sentence is basically a product requirement for personal agents.
For self.md, the practical translation is:
- keep explicit state
- predict the next state before acting
- compare prediction with observation
- escalate when confidence is low
- use mismatch as learning data
The World Models for Personal Agents concept is the operational version of this profile.
Limits
LeCun’s position has not won the field. LLM-first systems are still the most useful general-purpose agent substrate in production because language is programmable, debuggable, and already connected to tools.
JEPA-style systems are not a drop-in replacement. They are a research path toward a missing layer: latent prediction for planning and control.
That is exactly why LeCun belongs in the people graph: not as the anti-LLM voice, but as a reminder that intelligence needs more than fluent text.
Sources
- ACM A.M. Turing Award profile
- NYU profile
- Meta AI profile
- Yann LeCun homepage
- Backpropagation Applied to Handwritten Zip Code Recognition
- Gradient-Based Learning Applied to Document Recognition
- A Path Towards Autonomous Machine Intelligence
- I-JEPA
- V-JEPA
- V-JEPA 2
- Welch Labs: Yann LeCun’s Billion Dollar Bet