Marco Matthies

Just knew him for Moravec‘s paradox, looks like he got this one right too.

@aran_nayebi That subspace might be part of estimating the suffix partition function of the AR model? @mblondel_ml has a nice paper on this.

Correct me if I'm wrong, but isn't this really just showing: *In a trained autoregressive network, there is a subspace of internal activations that is especially aligned with future verbal output and downstream computation.* Isn't this expected given how LLMs are trained? In fact, I suspect you could find this is many other task-optimized architectures, not just Transformers/Claude—it would be good to try this on RNNs, MLPs, CNNs, etc.

Open-sourcing HarnessGym. Most coding-agent evals ask: “Did the agent solve the task?” HarnessGym asks something different: Can the agent learn what harness was missing, build it, prove it works, and make the next fresh run stronger? agent attempts task → reflects → builds tooling → qualifies it → next run starts stronger 🧵 github.com/patrick-toulme…

@pdhsu Programmable mRNA

what is the state of the art in drugging broadly expressed/pleiotropic targets in a cell type specific manner? and what are the high probability paths for innovation in the next 5 years?

@francoisfleuret Hotter take :) Context compaction makes our current agents into RNN-transformer hybrid systems

Hot take: Transformers are all-seeing ultrafast librarians. They have a very low incentive to extract and organize information, they can just "look around" to see correlating fragments. RNNs done properly would have far stronger "conceptual embeddings" and would actually think.

Your RL post-training may be sabotaging your LLM’s test-time scaling! Conventional RL pretends that you can collapse all reward signals *upfront* into a single *scalar reward*. We introduce Vector Policy Optimization (VPO), which natively maximizes *vector-valued* rewards, boosting test time search performance, even on the original scalar.

Launching pyptx — a Python DSL for writing NVIDIA PTX kernels. One PTX instruction = one Python call. Write pure PTX in Python. Direct Hopper + Blackwell support: wgmma, TMA, tcgen05, mbarriers. JAX + PyTorch integration. Includes GEMM, grouped GEMM, RMSNorm, SwiGLU, and a PTX→Python transpiler pip install pyptx[torch] pip install pyptx[jax] github.com/patrick-toulme…

@thsottiaux I regularly have codex cli starting the same compile job twice or more (record is 4!) because it got impatient with the progress. This happens evn after telling it to not do that in AGENTS.md…

What are we consistently getting wrong with codex that you wish we would improve / fix?

Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers. 🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth. 🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale. 🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead. 🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains. 🔗Full report: github.com/MoonshotAI/Att…

Latent world models learn differentiable dynamics in a learned representation space, which should make planning as simple as gradient descent. But it almost never works. What I mean is, at test time, you can treat the action sequence as learnable parameters, roll out the frozen world model, measure how far the predicted final state is from the goal, and backprop through the entire unrolled chain to optimize actions directly. Yet many of the systems that work (Dreamer, TD-MPC2, DINO-WM) abandon this and fall back to sampling-based search instead. That's why I really like this new paper by @yingwww_, @ylecun, and @mengyer, which gives a clean diagnosis of why, and a principled fix. The reason everyone abandons gradient descent on actions is that the planning objective is highly non-convex in the learned latent space. So instead most systems use CEM (cross-entropy method) or MPPI (model predictive path integral), both derivative-free. CEM samples batches of action sequences, evaluates them by rolling out the world model, keeps the top-k, and refits the sampling distribution. MPPI does something similar but weights trajectories by exponentiated negative cost instead of hard elite selection. These work when gradients are unreliable but the compute cost is substantial — hundreds of candidate rollouts per planning step vs a single forward-backward pass. This paper asks what exactly makes the latent planning landscape so hostile to gradients and what you can do about it. The diagnosis. Their baseline is DINO-WM, a JEPA-style world model with a ViT predictor planning in frozen DINOv2 feature space, minimizing terminal MSE between predicted and goal embeddings. The problem is that DINOv2 latent trajectories are highly curved (when you use MSE as the planning cost you're implicitly assuming euclidean distance approximates geodesic distance along feasible transitions). For curved trajectories this breaks badly, gradient-based planners get trapped and straight-line distances in embedding space misrepresent actual reachability. The fix draws from the perceptual straightening hypothesis in neuroscience — the idea that biological visual systems transform complex video into internally straighter representations. So they add a curvature regularizer during world model training. Given consecutive encoded states z_t, z_{t+1}, z_{t+2}, define velocity vectors as v_t = z_{t+1} - z_t measure curvature as the cosine similarity between consecutive velocities, and minimize L_curv = 1 - cos(v_t, v_{t+1}). Total loss is then L_pred + λ * L_curv with stop-gradient on the target branch to prevent collapse. The theory backs this up cleanly — they prove that reducing curvature directly bounds how well-conditioned the planning optimization is — straighter latent trajectories guarantee faster convergence of gradient descent over longer horizons. Worth noting that even without the curvature loss, training the encoder with a prediction objective alone produces some "implicit straightening" — the JEPA loss naturally favors representations whose temporal evolution is predictable. Explicit regularization simply pushes this much further. Empirical results across four 2D goal-reaching environments are consistently strong. Open-loop success improves by 20-50%, and the GD with straightening matches or beats CEM at a fraction of the compute. The most convincing evidence is the distance heatmaps: after straightening, latent Euclidean distance closely matches the shortest distance between states, even though the model was trained only on suboptimal random trajectories. What I find interesting beyond the specific method is that the planning algorithm didn't change. The dynamics model didn't change. A single regularization term on the embedding geometry turned gradient descent from unreliable to competitive with sampling methods. The field has largely treated representation learning and planning as separate concerns — learn good features, then figure out how to plan in them. This paper makes a concrete case that the representation geometry is itself the bottleneck. This connects to a broader pattern in ML. When optimization fails, the instinct is to fix the optimizer (better search, more samples, adaptive schedules). But often the real lever is the shape of the space you're optimizing in. Same principle shows up in RL post-training where reward landscape shaping matters as much as the algorithm itself. Shape the space so simple optimization works, rather than building complex optimization to handle a bad space. Their paper: arxiv.org/abs/2603.12231

OpenFold3-preview (OF3p) is out: a sneak peek of our AF3-based structure prediction model. Our aim for OF3 is full AF3-parity for every modality. We now believe we have a clear path towards this goal and are releasing OF3p to enable building in the OF3 ecosystem. More👇

@HildeKuehne „A few iterations“ must surely strongly depend on the dataset and models used. E.g. it worked really well in AlphaFold-2.

Random important concept that I just realized only few people aware of... You can only improve models based on their own output for a few iterations, then they crumble. (A story)

Very interesting set of ideas, breaking out of the Turing tar-pit.

It’s symbiosis all the way down

@curiouswavefn @r0ck3t23 Hasn’t the debunking been debunked as well? Spherical straw cows was the problem IIRC

@r0ck3t23 Remember, though, that Penrose and Hammerhof’s theory of quantum decoherence in the brain because of microtubules was debunked by Max Tegmark. That does not mean I disagree with him philosophically, just that we don’t know the mechanism.

Adding more GPUs will never make a machine conscious. Nobel Prize-winning physicist Roger Penrose just dismantled the entire AI race’s core assumption. Right now, the industry operates on one belief. Build massive data centers. Scale the models. AGI will just “wake up.” Penrose destroys this completely. Penrose: “There is this sort of view that once you make a computer complicated enough or something, it suddenly becomes aware. I just don’t believe that. There’s no reason to believe that.” A machine can compute better than any human alive. But computation is not awareness. Penrose: “There is something quite different involved in understanding things, in being aware of things, of feeling things, which is not part of computations.” We’re confusing rule-following with actual intelligence. Penrose: “The keyword is the word ‘understanding.’ You can follow rules alright, but we don’t understand what we’re doing. The understanding is the key point.” Models today are exceptional at processing data. At mimicking logic. But true understanding requires consciousness. Penrose: “It doesn’t make sense to say of a device that it understands something if it’s not even aware of it. There is something much more profound in being conscious of something.” And here’s what should terrify every AI lab on earth. Penrose: “I believe that the brain is following the laws of physics, sure. We don’t have a good picture of the laws of physics.” Penrose: “Quantum mechanics is not an answer to the way the universe operates. It’s a partial answer. It’s incomplete.” We’re trying to engineer synthetic consciousness using classical computation. While biological consciousness likely operates on physics we haven’t even discovered yet. The race to AGI isn’t just an engineering problem. It’s a frontier science problem. The labs are hiring engineers. The problem might require physicists who don’t exist yet.

„Any sufficiently advanced technology is indistinguishable from magic“

@elonmusk Luna City 😉

For those unaware, SpaceX has already shifted focus to building a self-growing city on the Moon, as we can potentially achieve that in less than 10 years, whereas Mars would take 20+ years. The mission of SpaceX remains the same: extend consciousness and life as we know it to the stars. It is only possible to travel to Mars when the planets align every 26 months (six month trip time), whereas we can launch to the Moon every 10 days (2 day trip time). This means we can iterate much faster to complete a Moon city than a Mars city. That said, SpaceX will also strive to build a Mars city and begin doing so in about 5 to 7 years, but the overriding priority is securing the future of civilization and the Moon is faster.

Good inductive biases are still worth a lot of FLOPs, don’t overdo the bitter lesson pills

Generative Modeling via Drifting New Kaiming He paper! Instead of a "pushforward" behavior carried out iteratively at inference time, e.g., in diffusion/flow-based models, evolve the pushforward distribution during training naturally enabling 1-step inference. SOTA results on ImageNet 256×256, with FID 1.54 in latent space and 1.61 in pixel space

Imagine you could solve an infinite set of transport problems with one Meta Flow Map model that allows you to sample from arbitrary posterior distributions. Now imagine you can do that to construct a really effective estimator of how to adapt a diffusion to solve an RL problem. Now imagine that doing so allows you to even outperform Best-of-N=1000 at a fraction of the compute. Excited to introduce a new paradigm for flow and diffusion models we call Meta Flow Maps, which make this possible 🙂 👾 Learnable with simple modification of existing flow map losses 👾Off-policy fine-tuning algorithm! 👾Extremely effective reward alignment across a variety of rewards for both inference-time steering and learned fine-tuning! arxiv: arxiv.org/abs/2601.14430 project page: meta-flow-maps.github.io code: forthcoming Amazing work by @PPotaptchik and @adhisarav to bring these results to life! Really excited about future directions here! Thanks to @yeewhye @AbbasMammadov11 and Alvaro Prat.