Roshan Balaji

Excited to share that we have 3 papers accepted at #ICLR2026! 🇧🇷 Our work this year focuses on efficiency and expressivity: deriving theoretical limits for SSMs, achieving linear scaling for reasoning, and modernizing encoder architectures. A summary of our work 👇 🧵

Congrats to Prashant (@prashantg_17), Davide (@DavideBald42296 ), Quentin (@qfournier2), and Sarath (@apsarathchandar) on CADmium, a new method that rethinks text-to-CAD to generate high-fidelity 3D models! Read their blog post: mila.quebec/en/article/imp…

Can LLMs become CAD designers? Check out “CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design”, which is now published in Transactions on Machine Learning Research (TMLR), and led by @prashantg_17, @DavideBald42296, and @qfournier2!

Presenting... BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives In our #AAAI2026 oral paper we present a new method to mine hard-negatives ⛏️ !! w/ @roshan_msb , Pavan Kumar and Nirav P. Bhatt 1/7

The growing pains of @NeurIPSConf peer review in the LLM era.

I am recruiting several graduate students (both MSc and PhD level) for Fall 2026 @ChandarLab! The application deadline is December 01. Please apply through the @Mila_Quebec supervision request process here: mila.quebec/en/prospective…. More details about the recruitment process here: chandar-lab.github.io/join/

Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2026 admission! For more information, visit mila.quebec/en/prospective…

Long reasoning without the quadratic tax: The Markovian Thinker makes LLMs reason in chunks with a bounded state → linear compute, constant memory and it keeps scaling beyond the training limit. 1/6

Introducing linear scaling of reasoning: 𝐓𝐡𝐞 𝐌𝐚𝐫𝐤𝐨𝐯𝐢𝐚𝐧 𝐓𝐡𝐢𝐧𝐤𝐞𝐫 Reformulate RL so thinking scales 𝐎(𝐧) 𝐜𝐨𝐦𝐩𝐮𝐭𝐞, not O(n^2), with O(1) 𝐦𝐞𝐦𝐨𝐫𝐲, architecture-agnostic. Train R1-1.5B into a markovian thinker with 96K thought budget, ~2X accuracy 🧵

An excellent opportunity for anyone seeking assistance in grad applications.

Functional Groups are All you Need for Chemically Interpretable Molecular Property Prediction 1. A novel study proposes a novel Functional Group Representation (FGR) framework for molecular property prediction, achieving state-of-the-art performance while ensuring chemical interpretability. This work significantly advances the field by bridging the gap between deep learning models and traditional chemical knowledge. 2. The FGR framework integrates curated functional groups from established chemical literature and mined functional groups from a large molecular corpus. This dual approach provides a comprehensive and interpretable representation of molecular structures, outperforming existing methods on a wide range of benchmark datasets. 3. The study demonstrates that the FGR framework not only matches but often surpasses the performance of current state-of-the-art models in predicting molecular properties across diverse fields such as biophysics, quantum mechanics, and pharmacokinetics. This highlights its potential for accelerating drug discovery and materials science. 4. A key innovation is the use of autoencoders to encode molecules into a lower-dimensional latent space, leveraging pretraining on a large dataset of unlabeled molecules. This allows the model to capture intricate chemical relationships while maintaining simplicity and efficiency. 5. The interpretability of the FGR framework is validated through alignment and uniformity analyses, showing that the model effectively groups molecules with similar functional groups and ensures adequate coverage of chemical space. This is crucial for reliable and generalizable predictions. 6. The study also includes detailed interpretability analyses, demonstrating that the model consistently assigns high attribution scores to chemically meaningful substructures. This provides valuable insights into the structure-property relationships and enhances the trustworthiness of the model for practical applications. 7. The FGR framework is evaluated on several peptide cleavage and bacterial datasets, outperforming graph-based methods and showcasing its scalability and robustness. This suggests its potential for large-scale molecular property prediction tasks. 📜Paper: arxiv.org/abs/2509.09619… #MolecularPropertyPrediction #FunctionalGroups #DeepLearning #Interpretability #DrugDiscovery #MaterialsScience

We just made NovoMolGen easy to play with: Transformers-native checkpoints on the Hub and small notebooks that let you load, sample, and fine-tune in minutes. The few lines of code that load the model, plug in a reward, run a short RL finetune, and plot the curve.

🚀 Excited to see my PhD research published in @NatureComms! Grateful to my supervisor @Himanshu_IITM, the incredible Systems Genetics Lab @iitmadras @IBSE_IITM @iitmbt and collaborators Suresh Sudarsan & Shannara @DTUBiosustain for making this possible.

Check out our latest work in developing open-source foundation models for de novo drug design. Stay tuned for the accompanying blog post and starter notebook. @iitmbt @WSAI_IITM @IBSE_IITM @iitmadras

ICML 2025's rebuttal process be like🤣: 👨‍💻 Authors: spend a whole week writing a careful rebuttal ✅ Reviewer: clicks "acknowledge" without reading 🚫 Author: not allowed to reply anymore So what does acknowledge mean here? "You speak. I pretend to listen. Conversation over."🙃

Exciting 😬

My LaTeX document when I try to insert an image