Princeton Visual AI lab

Fast weights were built for long term memory, but trained for short attention spans. We introduce ReFINE, a phase-agnostic RL framework that improves long-context modeling in fast weight architectures. arxiv.org/abs/2602.16704 1/8

Fast-weight models need sequence-level supervision for long-context modeling. We study how to supervise learned compression in fast-weight models via RL. Key ideas: - Train fast-weight models with next-sequence prediction instead of next-token prediction. - RL rewards that evaluate whether compressed context supports coherent multi-step generation. Paper: arxiv.org/abs/2602.16704

Fast-weight language models lack sequence-level supervision We propose ReFINE to fix this problem Paper: arxiv.org/abs/2602.16704 Code: github.com/princetonvisua… Awesome project led by @will_hs_hwang @cindy_x_wu @orussakovsky @PrincetonCS @VisualAILab

Excited to share my first PhD paper on developing an effective RLVR post-training method for Looped Language Models (LoopLMs)!

New #NVIDIA Paper We introduce Motive, a motion-centric, gradient-based data attribution method that traces which training videos help or hurt video generation. By isolating temporal dynamics from static appearance, Motive identifies which training videos shape motion in video generation. 🔗 research.nvidia.com/labs/sil/proje… 1/10

Most LLM evals use API calls or offline inference, testing models in a memory-less silo. Our new Patterns paper shows this misses how LLMs actually behave in real user interfaces, where personalization and interaction history shape responses: arxiv.org/abs/2509.19364

Huge thanks to the organizers of NeurIPS CDMX for a fantastic week (also with great weather and food)! First conference trip as an assistant professor, and everything has been smooth and very enjoyable. The smaller venue made it easier to have in-depth conversations than in my past conference experiences. The only slightly awkward part was the second poster session running until 9:30 pm local time to stay in sync with the main conference in San Diego, but overall it’s been an amazing experience!! #NeurIPS2025 #Mexico

@SanghyukChun @Amaya_Dharmasir @polkirichenko @orussakovsky @cindy_x_wu @YangWilliam_ @ZhiweiDeng8

We are at #NeurIPS2025 in San Diego! 🏖️

I’m at #NeurIPS2025 from 12.2–12.7! I work on data-centric Video Generation and VLMs/VLAs recently (MOTIVE, COMPACT, ICONS, etc.), and I’m generally interested in building more scalable and capable multimodal systems. DMs open for a coffee chat! 😃 Excited to meet old and new friends!

On my way to #NeurIPS in San Diego! ✨ Excited to meet old and new friends and chat about LLM reasoning, uncertainty estimation, factuality, hallucinations, safety and other questions in LLM/agentic reliability!

I’ll be in Mexico City for NeurIPS next week to present our two papers! In the meantime, we are also hiring a fully funded PhD student to start in 2026. Feel free to reach out and say hi during the conference or via DM if you are interested! #NeurIPS2025

Today seems to be a fitting day for @GoogleDeepMind news, so I'm excited to announce our new preprint! Prior work suggests that text & img repr's are converging, albeit weakly. We found these same models actually have strong alignment; the inputs were too impoverished to see it!

Excited to share our NeurIPS 2025 paper Dynamic Diffusion Schrödinger Bridge in Astrophysical Observational Inversions! #neurips2025 #DiffusionModels #AstroML In this work, we introduce Astro-DSB, a novel diffusion bridge-based approach designed to tackle astrophysical inversion problems in the context of Giant Molecular Clouds (GMCs) for star formation. Our goal is to infer underlying physical states (e.g., density and magnetic fields) from projected, partial observations. We train Astro-DSB on both simulated density and magnetic field data, and test its in-distribution (ID) and out-of-distribution (OOD) performance on new simulations, as well as real observational data from Taurus B213. We show that Astro-DSB achieves substantially better OOD generalization than both traditional astrostatistical methods and state-of-the-art conditional diffusion models. It also reduces the required training time by 75% compared to cDDPMs. Paper and code are available! Check out at arxiv.org/abs/2506.08065 and github.com/L-YeZhu/AstroD…

Text-to-image (T2I) models excel in semantic alignment, yet numeracy—generating exact numbers of objects—remains challenging. We propose D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generation to overcome this. arxiv.org/abs/2510.19278 (1/7)

Prof. Johannes Lutzeyer (@JLutzeyer) and I are co-recruiting a fully funded PhD student at the Department of Computer Science, École Polytechnique in France, to start in September 2026. Details below or at l-yezhu.github.io/assets/PhD_Hir… #PhDhiring #AI

Text-to-image (T2I) models can generate rich supervision for visual learning but generating subtle distinctions still remains challenging. Fine-tuning helps, but too much tuning → overfitting and loss of diversity. How do we preserve fidelity without sacrificing diversity (1/8)

🚀 Our paper “The Impact of Coreset Selection on Spurious Correlations and Group Robustness” (with William Yang, @polkirichenko , Lydia Liu & @orussakovsky) is accepted to #NeurIPS2025 Datasets & Benchmarks! 🎉 @NeurIPSConf 🧵 Key findings 👇 [Paper: openreview.net/pdf?id=t7QJDaQ…]

Cornell (NYC and Ithaca) is recruiting AI postdocs, apply by Nov 20, 2025! If you're interested in working with me on technical approaches to responsible AI (e.g., personalization, fairness), please email me. academicjobsonline.org/ajo/jobs/30971

I'm at #ICCV2025 this week🌸 - We will be hosting Curated Data for Efficient Learning workshop (w @GCazenavette) 🗓️ Oct. 20, 10:30-10:35 AM 📍Room 304A 🔗 curateddata.github.io - I will be presenting our recent work on "Where Is Motion From? Scalable Motion Attribution for Video Generation Models" at Reliable and Interactive World Model workshop (internship project w Nvidia) 🗓️ Oct. 20, 12:00-1:00 PM 📍Room 316A 🔗 riwm-2025.github.io/RIWM-2025/ Excited to catch up with old friends and meet new ones!