Manjari Narayan

10.3K posts

Manjari Narayan banner
Manjari Narayan

Manjari Narayan

@NeuroStats

Surrogate Science , Auditing AIxBio, AI epistemics. Prev @Dyno_Tx | @StanfordMed | PhD@RiceU_ECE | BS@ECEILLINOIS Feedback: https://t.co/wR1ooEvxzP

Pasteur's Quadrant Katılım Kasım 2014
5.4K Takip Edilen5.4K Takipçiler
Manjari Narayan
Manjari Narayan@NeuroStats·
@JacobHHilton Are you interested in scientific problems where you can map interpretability of parameters or compositions of parameters to ground truth?
English
0
0
0
6
Jacob Hilton
Jacob Hilton@JacobHHilton·
Many (but not all) mechanistic interpretability researchers maintain an ambitious vision for the field, hoping to one day fully interpret multi-billion parameter LLMs. Alongside studying LLMs, we suggest studying tiny models too, to get a sense of what full understanding takes.
English
2
0
58
4.9K
Jacob Hilton
Jacob Hilton@JacobHHilton·
A challenge to the mechanistic interpretability community: fully interpret our 432-parameter RNN. (Thread)
English
15
37
560
64K
Manjari Narayan
Manjari Narayan@NeuroStats·
@sebkrier @MLStreetTalk Chirimuuta is great! So nice to see these issues discussed, because "prediction" has been overloaded since Day 1.
English
0
0
2
124
Séb Krier
Séb Krier@sebkrier·
Another great @MLStreetTalk episode. Really liked John Jumper's differentiation of prediction, understanding, and control - and Mazviita Chirimuuta's view that we often confuse functional descriptions or metaphors for the thing itself. youtu.be/pO0WZsN8Oiw?si…
YouTube video
YouTube
Séb Krier tweet media
English
9
3
62
10K
Manjari Narayan
Manjari Narayan@NeuroStats·
Super interesting to see this. I've been testing how this can play out in science and scientific decision making — we really aren't vigilant about how wrong stuff in the scientific literature might create self-fulfilling loops that extend stagnation
Stella Biderman@BlancheMinerva

It's so great that there are now multiple orgs doing transparent, rigorous testing of basic premises about how LLMs work and how their behavior can be influenced. So glad that Geodesic exists and excited to work with them on more like this!

English
0
0
3
378
Manjari Narayan
Manjari Narayan@NeuroStats·
I found myself similarly disoriented. I suspect we haven't found new abstractions that actually make sense for theory to be cool again. But you have to develop a taste for different research problems.
Wenting Zhao@wzhao_nlp

🌶️ Some (perhaps) spicy thoughts. It’s been a while since my last tweet, but I wanted to write about how disorienting it has been from academia to an LLM lab 😅 The kind of research I was trained to do during my PhD almost doesn’t exist here. The obsession with mathematical elegance and novelty is mostly gone. Everything is about scaling data and compute. For a while, that really got to me. At my lowest point, I felt like I’d lost interest in building LLMs altogether. I didn’t feel intellectually challenged anymore. What made this even stranger was that, at a technical level, things worked. If there was a capability I wanted to teach a model, scaling the right data and compute always got me there, no exception (so far). But recently, I found a way to reconcile with myself.. I realized the real competition isn’t in the ML recipe anymore. Most teams do roughly the same thing. What actually matters is how fast you can iterate, test ideas, and recover from mistakes. And that speed is mostly backed by infrastructure 🏗️ Faster loops, fewer bugs, better tooling. Seeing this made me excited again! Infra is its own deep, hard, and intellectually fun problem space. In 2026, I want to become an ML researcher who’s really good at infra. And I'll come back to ML problems with that edge, and will be excited to share what I find 😌

English
0
0
2
963
Manjari Narayan
Manjari Narayan@NeuroStats·
But modern ML and statistics has a much richer set of solutions I am excited about. Easier solution: Use generalized probability indices that trade off two outcomes both of which have their own sources of measurement error Even better: modern multi-multicalibration More 👇
Manjari Narayan tweet media
English
1
0
0
148
Manjari Narayan
Manjari Narayan@NeuroStats·
🧵 Personal Hamming Problem #1: Raise the bar for quantifying risk-benefit tradeoffs in drug development. By risk-benefit tradeoff I mean instantiation of the therapeutic index at some stage of the pipeline. #Neurips2025
Manjari Narayan tweet media
English
1
0
1
237
Manjari Narayan retweetledi
Carlos E Alvarez
Carlos E Alvarez@CarlosEAlvare17·
@NeuroStats 🔥 Exactly! Genetic variation is comparatively clean to interpret, but dynamic biological measurements (transcriptomic, epigenetic, imaging, physiological) are deeply entangled with life-course factors: environment, development, health status, social, & reverse causation... 1/3
English
1
2
2
326
Manjari Narayan
Manjari Narayan@NeuroStats·
One an start with what the mathematics of a proper scoring criterion ought to be for a problem, what kinds of properties a transformation ought to have, etc.. I've seen this problem emerge in many bio/health competitions over 10 years across kaggle, DREAM, etc..
Mile Sikic@msikic

@AnthropicAI @Google The recent very welcome @Arcinstute challenge made this painfully clear: defining evaluation metrics is hard. In some cases, trivial data transformations—and even random data—can score astonishingly high. Great AI performance ≠ biological meaning. 4/6

English
0
0
1
321
Manjari Narayan retweetledi
Mile Sikic
Mile Sikic@msikic·
@AnthropicAI @Google The recent very welcome @Arcinstute challenge made this painfully clear: defining evaluation metrics is hard. In some cases, trivial data transformations—and even random data—can score astonishingly high. Great AI performance ≠ biological meaning. 4/6
Mile Sikic tweet media
English
1
1
17
2.3K
Manjari Narayan
Manjari Narayan@NeuroStats·
OMG yes. Glad to see someone else making this point. Measurements of dynamic biological processes are subject to more novel kinds of confounding and selection bias than genetic markers. 'omics/imaging in biology ignores these challenges of life-course epidemiology
Carlos E Alvarez@CarlosEAlvare17

@anshulkundaje I was just making that point in a 3-tweet thread here. In addition to my closing suggestions there, I would mention the need for life course molecular (omics) epidemiology - high powered.

English
1
1
2
335
Manjari Narayan
Manjari Narayan@NeuroStats·
Being fast has advantages when high quality feedback is really quick. But surely deep thinking / pondering / working from first principles has its place for problems that are either have long-horizon/low quality feedback. Where is the Jim Simon of pharmaceutical forecasting?
Andrew Gordon Wilson@andrewgwils

@ltrd_ You also don’t need to be a gold medalist of the IMO to be a great mathematician. He isn’t drawing a contrast between smart and stupid. It’s a contrast between shallow and deep thinking.

English
0
0
1
794
Manjari Narayan retweetledi
Andrew Gordon Wilson
Andrew Gordon Wilson@andrewgwils·
@ltrd_ You also don’t need to be a gold medalist of the IMO to be a great mathematician. He isn’t drawing a contrast between smart and stupid. It’s a contrast between shallow and deep thinking.
English
0
1
40
3K
Patrick Collison
Patrick Collison@patrickc·
Came across this summary that pulls together some of the recent findings on the links between commonplace (including ostensibly "harmless") pathogens and long-term harm to human health. The war on infectious diseases is far from over!
Patrick Collison tweet media
English
106
244
1.5K
299.5K