MB retweetledi
🚨 56 researchers from 32 universities just exposed the biggest lie in AI video generation.
Every company is selling you "visual quality." Prettier videos. Higher resolution. More realistic skin and lighting.
Nobody stopped to ask: can these models actually think?
A massive coalition from Berkeley, Stanford, CMU, Harvard, Oxford, Columbia, NTU, Johns Hopkins, and 24 other institutions just built the largest video reasoning test ever created to find out.
It's called VBVR. Very Big Video Reasoning.
And the results are embarrassing for the entire industry.
Here's what they did:
They built 2.015 million video samples spanning 200 reasoning tasks. To understand how absurd that scale is: every existing video reasoning dataset in the world, combined, adds up to about 12,800 samples.
VBVR is 1,000 times larger. The paper literally draws the two circles to scale. The existing datasets are a tiny dot next to VBVR. It's almost comical.
But scale isn't even the interesting part.
They didn't just throw random video clips together. They built an entire cognitive architecture grounded in 2,000 years of philosophy. Starting with Aristotle. Literally Aristotle.
Five foundational cognitive faculties that any intelligent system should have:
Spatiality: Can the model understand where things are in 3D space? Navigate a maze? Understand geometry?
Transformation: Can it simulate how objects move, rotate, and change over time? Mental rotation. Physics.
Knowledge: Does it understand causality? Communicating vessels? Gravity? The rules of the physical world?
Abstraction: Can it solve logical puzzles? Follow algorithmic reasoning? Do the visual equivalent of Raven's Matrices?
Perception: Can it detect edges, compare sizes, count objects, identify colors and patterns?
Each faculty is mapped to parameterized task generators that produce unlimited variations. A navigation task can vary grid size, obstacle placement, start position. A rotation task can vary angles, objects, complexity. This isn't a fixed test set. It's a reasoning factory.
Then they tested every major video model on the planet.
Here are the scores:
Human baseline: 97.4%
VBVR-Wan2.2 (their fine-tuned model): 68.5%
Sora 2: 54.6%
Veo 3.1: 48.0%
Runway Gen-4 Turbo: 40.3%
Wan2.2 base: 37.1%
Kling 2.6: 36.9%
LTX-2: 31.3%
CogVideoX: 27.3%
HunyuanVideo: 27.3%
Read those numbers again.
The best commercial video model in the world, Sora 2, scores 54.6%. Humans score 97.4%. That's not a gap. That's a canyon.
And these aren't subjective aesthetic ratings. Every task has a deterministic, rule-based scorer. No AI judges. No vibes. Either the ball bounced the right way or it didn't. Either the agent found the correct path or it didn't. Either the object rotated to the correct angle or it didn't. Spearman correlation with human judgments: above 0.9.
Now here's the part most people will miss:
The five cognitive capabilities don't scale together. They found deep structural dependencies between them. And the pattern mirrors what neuroscience tells us about the human brain.
Knowledge and Spatiality are strongly correlated (ρ = 0.461). This matches the hippocampal theory: the same brain region that handles spatial navigation also supports concept learning. Edward Tolman's cognitive map hypothesis from last century, now validated in AI models.
Knowledge and Perception are strongly negatively correlated (ρ = -0.757). This aligns with the "core knowledge" debate in cognitive science: are innate abilities like object permanence really knowledge, or are they perception? The models seem to suggest they're different circuits.
Abstraction is negatively correlated with almost everything else. It shows no positive correlations with any other faculty. This is consistent with the modularity of the prefrontal cortex. Abstract reasoning is its own island.
These AI models are accidentally recapitulating real structural constraints in biological intelligence. Without anyone designing them to.
It means you can't just throw more data at the problem and expect all five capabilities to improve at once. Some of them actively compete with each other.
Here's where it gets genuinely exciting:
They took the base Wan2.2 model (37.1%) and trained it on increasing amounts of VBVR data. No architectural changes. Just data.
50K samples → scores climb steadily on both in-domain and out-of-domain tasks.
200K samples → model hits 68.5% overall. 84.6% relative improvement.
300K+ samples → performance starts to plateau.
The out-of-domain score (tasks the model never saw during training) climbed from 0.329 to 0.610. That means the model learned to reason about entirely new types of problems it was never trained on. The researchers call it "early signs of emergent generalization."
But even at peak performance: a 15% gap between in-domain and out-of-domain. And nearly 30 points below humans.
The qualitative analysis reveals something fascinating. After VBVR training, the model develops what they call "controllability-first execution logic." Instead of freely rewriting entire scenes like Sora 2 sometimes does, VBVR-Wan2.2 learns to do exactly what's asked. Delete one symbol without touching the rest. Rotate an object while keeping the background stable. Move a book to a specific slot without rearranging everything.
On one task, Sora 2 deletes the target symbol and then spontaneously rearranges all remaining symbols. VBVR-Wan2.2 just deletes the one symbol. Clean. Precise. Controllable.
They even observed "rationalizing" behavior: the model modifying intermediate elements to make its transformation narrative internally consistent. Not just producing an answer, but maintaining a coherent multi-step reasoning process.
And the honest limitations: long-horizon tasks still break. The agent sometimes duplicates or flickers during navigation. Blueprint construction can produce "correct answer, wrong method" outputs.
But here's the real takeaway nobody is talking about:
The entire AI video industry has been optimizing for the wrong metric. Visual quality is a solved problem at this point. The next frontier isn't making videos look more real. It's making videos make sense. Physics. Causality. Reasoning. Controllability.
Self-driving needs models that understand physics, not aesthetics. Robotics needs models that predict object interactions. Medical imaging needs spatial reasoning in 3D.
"Looking good" was never the goal. Thinking was.
The entire suite is open-source. The dataset (2M+ samples), the benchmark toolkit with 100+ rule-based evaluators, and the fine-tuned model are all publicly available. The pipeline supports community contributions. New tasks can be submitted, reviewed, and scaled up through their distributed generation framework.
This isn't a product launch. It's the largest open research infrastructure ever built for video intelligence. From 56 researchers across 32 universities who decided that someone needed to measure what actually matters.
English
