Gugan Thoppe

Policy Iteration’s super-power—monotonic improvement + guaranteed convergence—vanishes under general function approximation. To bring them, we introduce Reliable Policy Iteration (RPI) : arxiv.org/abs/2506.07134. #ReinforcementLearning #RL @EshwarSR @today_itself @DalalGal

📢 Open-sourcing the Sarvam 30B and 105B models! Trained from scratch with all data, model research and inference optimisation done in-house, these models punch above their weight in most global benchmarks plus excel in Indian languages. Get the weights at Hugging Face and AIKosh. Thanks to the good folks at SGLang for day 0 support, vLLM support coming soon. Links, benchmark scores, examples, and more in our blog - sarvam.ai/blogs/sarvam-3…

@iitmadras is delighted and proud to share that our Director, Prof. V. Kamakoti, has been selected for the prestigious Padma Shri Awards 2026 by the Government of India. A distinguished academician and eminent researcher, Prof. Kamakoti has made transformative contributions in the fields of computer science, systems engineering and interdisciplinary research. His visionary leadership has further strengthened IIT Madras’s commitment to cutting-edge research, technological innovation, a thriving startup ecosystem and inclusive institutional development. The Padma Shri Awards recognise his outstanding service and exemplary leadership in the field of Science and Technology, which stands as a testament to his enduring impact in academia. The entire IIT Madras community takes pride in this well-deserved national recognition and extends its heartfelt congratulations, wishing him continued success in service of the nation. @EduMinOfIndia #PadmaAwards2026 #Padmashri2026 #PadmaAwards

@starlitmatcha Intuitively, Q-learning works because it mimics value iteration: Close the gap between Q_n and TQ_n. Technically, it works since it's a stochastic fixed point Iteration with respect to the Bellman operator, i.e., Q_{n +1} = Q_n + a_n [TQ_n -Q_n + noise], and T is contraction.

Also understood the contrast: Monte Carlo waits until the episode ends whereas TD learning updates the value function step by step. Q-learning works because it uses these TD updates. So you don’t pick actions directly. You learn Q-values and choose actions based on them.

Spent time with Q-learning and value-based RL today. Fun thing I noticed: the Bellman equation follows the same logic as dynamic programming.

Attending #AAAI2026 in Singapore and working on RL theory or algorithm design? I’d love to meet! Happy to chat about our recent #AISTATS2026 work extending policy iteration and conservative policy iteration beyond the tabular setting to function approximation.

Can Q-learning in the average-reward setup, involving semi-norm contraction, have optimal rates without parameter-dependent step-sizes? Yes—using Polyak–Ruppert averaging. Catch our #AAAI2026 poster! Location: Hall 4 Date: Sunday, Jan 25

Happy to share that this work has been accepted to #AISTATS2026!

Here is a link to our tutorial from the recent Reinforcement Learning (#RL) Workshop 2026: youtube.com/live/1o2PUBPgY…. This six-part series covers RL fundamentals, key theoretical challenges, our recent #RPI-based solutions, and a hands-on component.

President Droupadi Murmu presents Vigyan Yuva- Shanti Swarup Bhatnagar Awards to: •Dr Dibyendu Das in Chemistry. •Dr Waliur Rahaman in Earth Science. •Prof. Arkaprava Basu in Engineering Sciences. •Prof. Sabyasachi Mukherjee in Mathematics and Computer Science.

Wormhole at #NeurIPS2025

The Walmart Centre of Tech Excellence at CSA proudly presents the 2026 edition of the Reinforcement Learning Workshop. This workshop aims to expose students and researchers in India to state-of-the-art Reinforcement Learning (RL) research. Register now: events.csa.iisc.ac.in/rlworkshop2026/

At the beginning of the talk!

Before the #RL Guru, Richard Sutton's, talk at #NeurIPS2025 !

If you are interested in the theory and applications of #RL, I would love to chat with you! Let me also know if you're interested in academic positions in India. @iiscbangalore @csa

Paper: arxiv.org/abs/2507.05077

Presenting at #NeurIPS2025, Exhibit Hall C,D,E #1803 Wed 3 Dec 11 a.m. — 2 p.m. PST

Analyzing fixed-point iterations for semi-norm contractions is challenging due to the latter's non-monotonicity. My Ph.D. student, Ankur Naskar, recently showed how to handle it and derive parameter-free convergence rates! Happy to note that this work got accepted to #AAAI2026.