Amir-massoud Farahmand

📚 Interested in Continual RL but not sure where to start? 👇 Dive in: sites.google.com/view/continual… ♾️ We've curated a resource hub with key papers, benchmarks, codebases, tutorials, and more to help you get up to speed quickly. #ContinualRL #ReinforcementLearning #MachineLearning #AI

@_Suresh2 Good question! No, I didn't try. I may later. I suspect there is a positive probability that it doesn't always find a good path in the solution space, and estimating that requires many repeated experiments.

@SoloGen did you test if Sol (High) gives the same bound across multiple runs, or just one?

Interesting question! I tried it on GPT-5.6 Sol (High), available on the Plus plan. I got that the length of the path is <= c_n^n with n being dimension and c_n approaching 2* sqrt(2)=2.828, asymptotically. The uniform over all dimension n result is 3.50427, for n = 4.

Last month, @RichardSSutton and I left Keen to do our own thing. I had ~two absolutely wonderful years at Keen, and I learned a lot working with John, Gloria, Joseph, and the rest of the team. If you want to work on some of the foundational unsolved problems in AI, such as continual learning, then I would strongly recommend applying to Keen. Going forward, Rich and I have founded a small company called Oak Lab. We have a fairly complete roadmap to building animal-like intelligence that learns purely from its own experience (the OaK architecture), and Oak Lab is going to follow this roadmap aggressively with a small, focused team. We will be sharing our progress often and aim to build a prototype of the complete OaK architecture in the next few years. A successful prototype will be closer to a baby learning in its first year than it will be to any of the current AI systems. Our strategy is to demonstrate the limitations of current methods in simple settings, and then work out algorithms that overcome these limitations in a domain-independent way. Only after we have made sufficient progress on the core algorithms will we build large-scale artifacts. If you are interested in learning about some of the specifics of our approach, then follow @oaklab_ai. We will be sharing more details in the coming weeks and months.

I can’t say enough good things about John Carmack @ID_AA_Carmack and his Keen Technologies. But now Khurram Javed @kjaved_ and I have broken away to start our own startup and pursue a slightly different path toward understanding intelligence. Like Keen (and like Ineffable) we at Oak Lab @oaklab_ai believe in reinforcement learning and that intelligence is created and maintained from run-time experience. But we think current deep learning methods are weak and inefficient, and need not more tweaks, but fundamentally new ideas and a thorough reworking before they can provide a solid foundation for achieving the more ambitious goals of AI.

2) I suspect it cannot be longer than c^n for some constant c>0. Try to prove this bound for the smallest c you can. [Took 14m. Gave a proof. I didn't verify the proof.] 3) What is c_n asymptotically as a function of n? [Immediate. I just wanted to verify my understanding.]

My prompts: 1) How long can be the path of a gradient flow on a convex function given the constraint that it stays within the unit Euclidean ball in dimension n? [Took 1m 30s. Didn't prove anything, but found some exisiting results.]

Can I publish a proof that I don't understand? Should I? .... I never thought I would reach this dilemma so soon, but I did. In the last 2 days I played around with ChatGPT 5.5, I reconsidered one of my old problems. It was always too difficult for me, technically. Now AI claims it solved it... This is a problem in stochastic dynamics, probability theory. It needs special skills that I do not have. There is a stochastic system which is complicated. Particles hopping on a 1D lattice, with some special rules. The model is physically interesting, and in good variables it becomes much simpler. One needs to connect the quantities along multiple coordinate transformations, non-local maps, eventually arriving at the desired result. I knew what could be an initial strategy towards a proof. In fact we have the required coordinate transformations in a published paper. But I was never enough to do the full rigorous proof. Now the AI claims it did it for me. It supplied another connection to existing literature, a connection I wouldn't have been able to make. And now the proof seems to be complete. Now I have a manuscript, which is >30 pages long. I worked a lot on the introductory Sections, which explain the strategy. This takes up 10 pages approx. Then comes the grinding, for 20 pages. What did I learn from this? What does the community learn? Should I put this to the arxiv? Or to my blog? Should at least announce the result in a conference or somewhere? Or just tell people in person? Eventually we could try to formalize it in Lean. But I am not sure whether we would learn anything from it. OK then people could say they believe it. If I send this to a journal, the referee will not work through 20 pages of hard grinding. They can give it to an AI if they want to. Then we have the future, where AI referees the work of an AI. In any case, I understand the problem and the proof strategy. Maybe that is actually enough in this case. I honestly don't know.

I will be attending @icmlconf and am looking to hire 2 postdocs: one in foundation models for biology (proteins/small molecules/genomics data/drug discovery) and another in RL (including continual RL, world models, and RL training for LLMs). Please do reach out to me if you are interested in these positions and attending ICML too!

P.S.1: I don't know if students actually use this. I suspect that several of my own students haven't done that. P.S.2: This was the way I discovered that Canada has a province called Alberta, many years ago! @UAlberta

For MS –> PhD (and BS who have done research already), take a look at the papers you have read during the course of your research project, pick those whom you liked, and see where the authors are. This gives you a good taste of their resesearch.

We are super excited to announce @ChandarLab 7th Annual Research Symposium! 🔥 Join us on July 23–24 for two days of talks on deep learning, NLP, reinforcement learning, continual learning, and AI for science, plus a keynote by @mengyer from NYU! Event is hybrid!

@ShiqianMa Congratulations Shiqian!

Excited to share that I have joined Johns Hopkins University as a Professor in DSAI, AMS, CS, ECE. Grateful to my colleagues and students at Rice for many wonderful years. Look forward to this new chapter at JHU--building new collaborations in Foundations of AI, Optimization, ML

If you are interested in Reinforcement Learning and its various applications, go work with Taylor! He is energetic, full of research ideas, and a kind person.

🇨🇦🍁Happy Canada Day!🍁🇨🇦

Congratulations Dr. Hussing!

🇨🇦Canadian Heroes🇨🇦 #WorldCup

Authors: Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu, Junfeng Wen (Not sure if any of them is actually on Twitter)

Temporal Difference Learning for Diffusion Models (ICML26) arxiv.org/abs/2606.15048 By Yangchen Pan (my former PhD student) and co. It reformulates diffusion training as a Markov reward process and introduces a TD obj to encourage temporal consistency across denoising steps.