Chandan Singh

Science faces an explainability crisis: ML models can predict many natural phenomena but can't explain them We tackle this issue in language neuroscience by using LLMs to generate *and validate* explanations with targeted follow-up experiments arxiv.org/abs/2410.00812 1/2

PSA for american citizens going to ICLR: you need to get an evisa for Rio (takes up to 10 business days)

Interpretability methods usually study single-token behavior. But real model behaviors, like sycophancy or writing style, are diffuse across many tokens. Can these diffuse behaviors be localized and controlled from long-form responses? YES!

Can we rewrite Transformers as a human-readable code? In this paper, we decompile Transformers trained on algorithmic and formal language tasks into D-RASP – a programming language that mirrors Transformer architecture. 🧵

so apparently you can just ask models to write weights now.

Realizing that my brain now associates seeing a typo in a paragraph as a positve signal for quality

Orthogonal Finetuning (oft.wyliu.com; boft.wyliu.com) has a unique advantage of preventing catastrophic forgetting. Inspired by this property, we find that merging models within the orthogonal group can effectively reduce model conflicts and preserve both pretraining and downstream knowledge. This is our OrthoMerge framework. The idea behind OrthoMerge is extremely simple. For OFT-tuned models, we can first map the orthogonal adapters to Lie algebra with inverse Carley transform and then perform merging there. This guarantees the merged model differs from the pretrained model only up to an orthogonal transformation. A better news is that OrthoMerge can also be applied to non-OFT-tuned models. By solving the orthogonal procrustes problem, we can have the projected component of the adapter onto the orthogonal group. OrthoMerge will then be applied there and the residual component can be merged using conventional merging methods. That said, OrthoMerge can be used together with existing model merging methods! This is a great example of simple yet effective ideas. Great efforts by my PhD students Sihan Yang and Kexuan Shi. The project is already open-sourced and feel free to give it a try! Project: spherelab.ai/OrthoMerge/ Paper: arxiv.org/pdf/2602.05943 Code: github.com/Sphere-AI-Lab/…

Really nice post! A plausible and well-articulated vision for mech. interp that really improves serious model training

Can LLMs self-improve without ground-truth rewards? 🔄Introducing Test-time Recursive Thinking (TRT) Models recursively refine rollout strategies and accumulate knowledge from their own attempts. 🚀 Results: 1. 100% Accuracy on AIME-25 2. 10.4-14.8 pp improvement on LCB Hard

@ChenhaoTan This repo has been tracking ICML (and other confs) well: github.com/lixin4ever/Con…

where can I easily find the number of submissions to ICML over time, especially historical ones? papercopilot.com/statistics/icm… This page missed even 2013, 2014, and 2016.

AlphaGenome is out in @nature today along with model weights! 🧬 📄 Paper: nature.com/articles/s4158… 💻 Weights: github.com/google-deepmin… Getting here wasn’t a straight path. We sat down @googledeepmind to discuss the story behind the model, paper & API: youtu.be/V8lhUqKqzUc

We've identified a novel class of biomarkers for Alzheimer's detection - using interpretability - with @PrimaMente. How we did it, and how interpretability can power scientific discovery in the age of digital biology: (1/6)

Generated CoT is a fascinating window into modern LMs, but are these internal monologues as readable as they seem, or are they actually a "private language"? @kpal_koyena explores this in a clever way, by asking how one model's CoT works when fed to a different model....

How to get AI to make discoveries on open scientific problems? Most methods just improve the prompt with more attempts. But the AI itself doesn't improve. With test-time training, AI can continue to learn on the problem it’s trying to solve: test-time-training.github.io/discover.pdf

Can models understand each other's reasoning? 🤔 When Model A explains its Chain-of-Thought (CoT) , do Models B, C, and D interpret it the same way? Our new preprint with @davidbau and @csinva explores CoT generalizability 🧵👇 (1/7)

Paper: arxiv.org/abs/2601.09072 Code: github.com/jjfenglab/HACHI Led by the wonderful @Jean_J_Feng and Avni Kothari!

Really excited about our new work, which makes building clinical prediction models way easier! AI agents do the grunt work of hypothesizing and validating EHR features, enabling easy auditing by clinicians Iterating this process yields sensible SOTA (fully interpretable!) models

Here is the intern job posting! apply.careers.microsoft.com/careers/job/19…

FTE job posting: apply.careers.microsoft.com/careers/job/19… Internship job posting coming soon​...

I’ll be at NeurIPS helping to hire for MSR FTE roles, research interns (esp. in LLM interpretability), & presenting these papers — DM me if you’d like to meet up!

8B model can outperform AlphaEvolve on open optimization problems by scaling compute for inference or test-time RL🚀! ⭕Circle packing: AlphaEvolve (Gemini-2.0-Flash/Pro) : 2.63586276 Ours (DeepSeek-R1-0528-Qwen3-8B) : 2.63598308 🔗in🧵 [1/n]

Bayesian concept bottleneck models poster Wed evening x.com/Jean_J_Feng/st…

Generalized induction head poster Thurs evening x.com/csinva/status/…