Junyi Zhang

𝗢𝗻𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 𝗰𝗮𝗻’𝘁 𝗿𝘂𝗹𝗲 𝘁𝗵𝗲𝗺 𝗮𝗹𝗹. We present 𝗟𝗼𝗚𝗲𝗥, a new 𝗵𝘆𝗯𝗿𝗶𝗱 𝗺𝗲𝗺𝗼𝗿𝘆 architecture for long-context geometric reconstruction. LoGeR enables stable reconstruction over up to 𝟭𝟬𝗸 𝗳𝗿𝗮𝗺𝗲𝘀 / 𝗸𝗶𝗹𝗼𝗺𝗲𝘁𝗲𝗿 𝘀𝗰𝗮𝗹𝗲, with 𝗹𝗶𝗻𝗲𝗮𝗿-𝘁𝗶𝗺𝗲 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 in sequence length, 𝗳𝘂𝗹𝗹𝘆 𝗳𝗲𝗲𝗱𝗳𝗼𝗿𝘄𝗮𝗿𝗱 inference, and 𝗻𝗼 𝗽𝗼𝘀𝘁-𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻. Yet it matches or surpasses strong optimization-based pipelines. (1/5) @GoogleDeepMind @Berkeley_AI

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: fast-spatial-memory.github.io - Paper: arxiv.org/abs/2604.07350 - Code: github.com/Mars-tin/fast-… - Models: huggingface.co/marstin/fast-s… This work is co-led with @Xueyang_Y, contributed by @zhnhoy5 @YuncongYY, and advised by @SLED_AI @gan_chuang.

Great work using offline agentic exploration to develop robot skills!

Learning from task-agnostic, explorative experience!

Excited to share T-Rex: Tactile-Reactive Dexterous Manipulation 🦖🤖 Touch is fundamental to human dexterity, yet most Vision-Language-Action (VLA) models either ignore tactile feedback or lack the ability to react to high-frequency contact signals. In this work, we tackle both the data and architectural challenges of tactile-reactive dexterous manipulation. 🦖 A 100-hour tactile-synchronized dexterous manipulation dataset with 7,700+ trajectories, 22 motor primitives, and 200+ everyday objects. 🦖 A tactile-reactive MoT architecture with spatial-temporal tactile encoding and asynchronous high-frequency tactile refinement. 🦖 A scalable training recipe combining 22,889 hours of human egocentric pretraining with tactile-grounded robot mid-training. Across 12 real-world contact-rich manipulation tasks, T-Rex achieves over 30% higher average success rate than the strongest baseline. We are fully open-sourcing the dataset, models, teleoperation stack, training code, and inference pipeline. 🌐 Project: tactile-rex.github.io 📄 Paper: arxiv.org/abs/2606.17055 💻 Code: github.com/ZhuoyangLiu200… 🤗 Dataset: huggingface.co/datasets/zekai… 🧵 Thread ↓

That's a nice question! Currently, we threshold the play budget so that it’s comparable or less to the amount of compute used at test time. We also include a cost analysis in the appendix. That said, I think smarter ways of controlling the budget are definitely an interesting direction!

@junyi42 the play-before-task stage is the part everyone skips with coding agents too. mine happily over-explores its sandbox and burns the budget before the real repo even shows up. how do you cap the play phase so it doesn't eat the whole run?

Children learn from play. Can robots do the same? We propose 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠, a paradigm that gives embodied coding agents a play stage before downstream tasks arrive, and instantiate it with 𝐑𝐀𝐓𝐬 (Robotics Agent Teams), where robots discover reusable skills through curious play. Co-led with @jiaxin_ge_

The most inspiring thing I took from this paper: there's far more to squeeze from simulation than sim-to-real training of task-specific policies. RATs shows a coding agent can self-propose tasks, self-construct scenes in sim, and acquire skills that transfer to real-world deployment. It's promising to imagine handing coding agents a bunch of simulation clusters on top of ENPIRE to enable Sim-and-Real Co-research, where agents massively learn skills and try ideas in sim while continuously grounding them in the real world. Then robot skill acquisition can really scaling like everything else in the deep learning era.

@anand_bhattad @TimingYang99 @wangf3014 LOL, long live the RATs! 🐀🤝

Nice work! We also have the same name: "RATS!" for Register Attention Transformers, released a few days ago, led by @TimingYang99 and @wangf3014, related to emergent part representations. More on it soon 😉 arxiv.org/abs/2606.14701

3 of 3: Kids can learn how to generalize via play (vs rote repetition of goal tasks) to learn skills that are useful for the future; we think agentic robotics should do so as well. We revisit curiosity-based intrinsic learning for agentic robotics:

While ENPIRE w/ @nvidia @_wenlixiao @DrJimFan enables coding agents to explore algorithms and improve policies for a given real-world task, RATs asks: what can agents learn before a human specifies the task? Through curiosity-driven play, agents propose tasks, hill-climb toward solutions, and accumulate reusable, transferable skills. When a human later requests a new task, the agents retrieve and compose these skills to solve it. RATs explores an analogue of pre-training for embodied coding agents: broad skill acquisition through play, which accelerates task-specific problem solving with the skills acquired. Looking forward to the agentic future of robotics! See the detailed tweet from @junyi42!

Excited to release our new work: Playful Agentic Robot Learning w/ @junyi42! Instead of relying on test-time scaling, we found that a "pretraining stage" through curious play enables robots to discover general skills before any tasks are assigned. 🌐 playful-rats.github.io

What excites me most isn’t just that we built an agentic coding system for robots and ran it in the real world. It’s that the agentic system learned a generalizable prior during "Play-Time", and then reused it to adapt across multiple downstream tasks.

Frontier coding agents have shown they can run on real robots when given defined tasks. 🤖 Now we show these agents can learn the physical world like children—no task required: give them curiosity and self-play, and real robotic skills emerge on their own.✨ 🐀𝐑𝐀𝐓𝐬 are Robotics Agent Teams: embodied coding agents that learn through self-directed play before any downstream task is given. playful-rats.github.io

𝐑𝐀𝐓𝐬 is a first step toward 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: 🌐playful-rats.github.io We see a future where the next step for agentic robots isn't just stronger test-time harness, but a play stage where they set their own goals, fail, and build up skills long before we hand them a task. Huge thanks to the team: @lukehanjun (co-first) @letian_fu, Zihan Yang, Yaowei Liu, Raj Saravanan (core contributors), @istoica05 @akanazawa @JiahuiLei1998 @HavenFeng @trevordarrell and many others!

@letian_fu These play-learned skills generalize across different simulations and directly transfer to the real world. Directly using the skill library learned in LIBERO, we get: RoboSuite (cross-environment): +8.9pp Real-world tasks: +8.8pp

Very impressive effort, congrats on the team!!

Why aren’t Diffusion Language Model smart yet? Lacking stable post training is a major bottleneck! Meet DiPOD: the tripod for diffusion model post-training. DiPOD boosts accuracy across reasoning tasks, with Sudoku jumping from 22% to 97%, through a one-line code change. 🧵1/5