Yongyi Yang

DeepSeek's recent mHC (Manifold-Constrained Hyper-Connections) proposes to stabilize residual hyper-connections via 20 Sinkhorn-Knopp (SK) iterations. However, this approach requires heavily customized CUDA kernels and, does not guarantee the quality of stabilization due to approximation errors. Check our new paper: "mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations". Based on the Birkhoff-von Neumann theorem, we introduce a neat solution that can be realized with standard operators while guaranteeing exact stability.

@carnivalaki 还真是（

The TurboQuant paper (ICLR 2026) contains serious issues in how it describes RaBitQ, including incorrect technical claims and misleading theory/experiment comparisons. We flagged these issues to the authors before submission. They acknowledged them, but chose not to fix them. The paper was later accepted and widely promoted by Google, reaching tens of millions of views. We’re speaking up now because once a misleading narrative spreads, it becomes much harder to correct. We’ve written a public comment on openreview (openreview.net/forum?id=tO3AS…). We would greatly appreciate your attention and help in sharing it.

Major milestone in Yongyi Yang's academic journey.🎓 TOMORROW Yongyi Yang will present a dissertation defense on "Structures in Deep Learning: Representations, Learning Dynamics, and Efficient Algorithms." MARCH 26 @ 3:00pm ET cse.engin.umich.edu/event/structur…

@onimushi 我喜欢你🥰

サイズがッ！小さいんだがッ！？

这可太爽了

📜Read the full paper: arxiv.org/abs/2601.05732 🥂 Cheers to my wonderful collaborator @gaoj0017, who has been working with me like crazy for the whole past week!

Why persist in projecting unconstrained matrices onto the constrained convex set? The Birkhoff-von Neumann theorem offers a simple and robust parameterization of doubly stochastic matrices. By directly using this parameterization, we completely skip the SK iterations, avoid the hassle of specialized kernels, and ensure precise doubly stochasticity by construction. Experiments confirm that mHC-lite achieves the same (or better) stabilizing effect on training as mHC, while being significantly more efficient and easier to implement.

DeepSeek's recent mHC (Manifold-Constrained Hyper-Connections) proposes to stabilize residual hyper-connections via 20 Sinkhorn-Knopp (SK) iterations. However, this approach requires heavily customized CUDA kernels and, does not guarantee the quality of stabilization due to approximation errors. Check our new paper: "mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations". Based on the Birkhoff-von Neumann theorem, we introduce a neat solution that can be realized with standard operators while guaranteeing exact stability.

📄 Read the full paper: arxiv.org/abs/2510.02670 Cheers to my amazing co-authors -- Tomaso Poggio, Isaac L. Chuang, and Liu Ziyin @LiuZiyin10 We hope this work helps build a deeper understanding of training, and sparks new bridges between topology, physics, and deep learning.

🧩This framework is architecture- and optimizer-agnostic. As long as the model has a permutation invariance on neurons and the optimizer is gradient-based (e.g. SGD, Adam), the theory applies. This suggests topology as a universal tool for analyzing learning dynamics.

check out our new paper "Topological Invariance and Breakdown in Learning." We prove that training exhibits a topological phase transition with a critical learning rate separating topology-preserving and topology-simplifying regimes.. 🧵👇

@YiMaTweets Someone needs to initiate this process

I conjecture that, in the foreseeable future, there would/should be only two forms of (academic) publication exist: first posting on arXiv and then writing an open-source book. All intermediate steps or forms can be or should be forwent.

Them hoes was tryna figure out if erry compact Hausdorff space wit at least two points and no isolated points got a cardinality of at least 2^ℵ₀. 😸😸😸

To the end of the world

(5/5) Read the full paper: arxiv.org/pdf/2507.13540 Joint work with @weihu_ @Hidenori8Tanaka

(4/5) The key idea behind our theory is Double Convergence: hidden states cluster by token identity as context grows (context-wise convergence), and these clusters evolve across layers toward a low-frequency signal over the input's graph structure (layer-wise convergence).

What drives in-context learning in LLMs? New paper: Provable Low-Frequency Bias of In-Context Learning of Representations. We show LLMs have a low-frequency bias when learning representations in context, offering a theoretical answer to several previously open questions. 🧵👇