Tianwei Ni

Excited to share my recent blog post on offline RL as policy prior learning. Offline RL is often framed as learning a safe, deploy-as-is policy from static data. I argue it should instead learn a policy prior optimized for future adaptation. 🧵 Blog: twni2016.github.io/blogs/policypr…

For 40+ years, building a robot that could rally with an elite human table tennis player at full speed was an unsolved problem. Sony AI's Ace research project set out to change that—and the results are now accepted for publication in @Nature and featured on the cover.

Most VLA-RL frameworks inherit the complexity of LLM-RL infra but we found that none of it is necessary. We therefore introduce VLARLKit: A simple yet fast VLA RL framework. Code link: github.com/VLARLKit/VLARL…

Standard RL assumes a stable world. The real world may not. ♾ Introducing the Continual RL Workshop @RL_Conference 2026, Montreal, Canada. 🤖 Agents should never stop learning! 🖼️ Site: sites.google.com/view/continual… 📄 Submit: sites.google.com/view/continual…

Our workshop @continual_learn announcement post was removed unexpectedly with no reason; it had significant reach (including RTs from our founding father, which were also removed). @X @Support Given its prior engagement, we hope to recover it. x.com/continual_lear…

Scalar, well-defined, easy-to-optimize rewards aren’t always available in real-world interaction data–yet that data is crucial for scaling general-purpose agents. Excited to announce the 3rd edition of RLBRew: Towards Scalable General-Purpose Agents at @RL_Conference 2026!

Are you working on RL, principled ways to build RL envs for agent training, or effective evaluation for agents? Want to showcase your NeurIPS submission? or just discuss about research more broadly? Then consider submitting and attending to our first ever workshop on Methods and RL Environments for Evaluating AI Agents. Deadline: May 11 rl-eval.github.io

Don’t miss this one!

RL in Big Worlds is a workshop at @RL_Conference about ideas that enable agents to achieve goals in environments vastly more complex than themselves This requires giving agents the ability to learn continually and use approximate value functions, models and policies effectively

🔥Thrilled to announce the Continual Reinforcement Learning (CRL) Workshop @RL_Conference 2026 in Montreal, Canada! 📷 We welcome submissions on broad topics of continual RL. Interested in submitting or reviewing? Check out our website for more details!

Earlier this month, I wrote up a few thoughts on how to train LLMs as strong continual learners that can do test-time exploration and self-improve. Examples include emergent agentic capabilities such as error recovery and dynamic tool learning. Figured I’d share it here in case it’s useful to others :) notion.so/Towards-Contin…

Unveiling our new startup Advanced Machine Intelligence (AMI Labs). We just completed our seed round: $1.03B / 890M€, one the largest seeds ever, probably the largest for a European company. We're hiring! [the background image is the Veil Nebula - a picture I took from my backyard, most appropriate for an unveiling] More details here: techcrunch.com/2026/03/09/yan…

I've decided to leave OpenAI. I'm incredibly proud of all the work I've been part of here, from helping create the reasoning paradigm with @MillionInt, scaling up test-time compute with @polynoamial, working on RL algorithms with my fellow strawberries, shipping o1-preview (which started life as of one of my derisking runs), to post-training o1 and o3 with @ericmitchellai, @yanndubs and many others. I'm most proud of having led the post-training team here for the last year -- the team has done incredible work and shipped some really smart models, including GPT-5, 5.1, 5.2, and 5.3-Codex. OpenAI has genuinely some of the most talented researchers I have ever met, and I have learned more than I could have imagined knowing since I joined as a new grad. I want to thank @markchen90 @FidjiSimo @sama @merettm for all their support over my time here, and too many collaborators to name for the insights, ideas, and just plain fun we have had working together. After leading post-training for a year, though, I'm longing to start fresh and return to IC research work. I've been thinking about going back to technical research for quite some time, and I genuinely believe my colleagues and team here are set up to succeed going forward without me. I'm personally very excited for my next chapter -- I'm proud to be joining @AnthropicAI to get back into the weeds in RL research, and I'm looking forward supporting my friends there at this important time. Many of people I most trust and respect have joined Anthropic over the last couple of years, and I'm excited to work with them again. I have also been very impressed with Anthropic's talent, research taste and values, and I'm excited to be part of what the company does next!

I gave a guest lecture at McGill about scaling RL for LLMs, sharing the slides here. drive.google.com/file/d/1kM25WY…

O2O RL is complex because you have access to both a static dataset and a live environment. Two priors that don't always agree. 🤔Most prior work asks: what's the best algorithm? This paper asks: what's the real principle behind O2O RL? The answer is stability-plasticity balance. With that lens, algorithm design becomes adaptation, not guesswork. 💡Under this framework, O2O RL becomes a series of quantifiable decisions: - Which prior is stronger? - Where to anchor stability? - Is extra plasticity needed? ...

Excited to share our new paper (arxiv.org/abs/2510.01460) and blog (twni2016.github.io/blogs/policyfi…) on offline-to-online reinforcement learning. Grateful to my amazing coauthor for their insight and key contributions.🙏

This work is led by Lu Li @luli_airl and co-authored by me, @Tobealegend24, @pierrelux. We also wrote a blog with intuition and implications for modern post-training: twni2016.github.io/blogs/policyfi… Thanks to @Guozheng_Ma for feedback on the blog. Questions and comments welcome!

Related work: - Warm-Start RL (WSRL): arxiv.org/abs/2412.07762 - RL with Prior Data (RLPD): arxiv.org/abs/2302.02948 - Calibrated Q-learning (Cal-QL): arxiv.org/abs/2303.05479 - ReBRAC: arxiv.org/abs/2305.09836

Offline-to-online RL fine-tuning feels unpredictable: methods that work in one task can collapse in another. In work led by @luli_airl, we argue this isn’t noise — it’s a stability–plasticity mismatch driven by where prior knowledge lives. Paper: arxiv.org/abs/2510.01460 🧵