Ryan Panwar

loved the piece on interpretability in the @nytimes this morning! the field has accomplished some pretty cool things in recent years. still, there is much work to do as @davidbau put it elegantly, interpretability is now where biology was in 1930: “The cell was a black box for biologists. They were slow to get off the starting block to start studying heredity. But once they did, the problem fell.” there is so much we can learn from these alien intelligences.

At my last job, we often got calls from parents frantically asking for their child's genetic test results. Too often, the results were inconclusive. Variant effect prediction sounds abstract but can be life-or-death for genetic disorders. Proud of the team for narrowing this gap!

there is much that we can learn from these alien intelligences. i'm excited to see what the community can do with the tools we are open sourcing interpretability is at the center of our success here. not only do these explanations offer potential discoveries, understanding the model was critical to the entire research path here. very proud of the team's work and to our amazing partners at @MayoClinic. excited for all the great things still to come

interpretability is hands down the coolest subfield of AI research and will change the world

Already gave this a rip for some VUS and Suspected Pathogenic variants that I have previously done deep analysis on, and can confirm that EVEE posits many of the same findings and conclusions that I have found in terms of prediction and suggested failure mechanism Examples: VUS (for which *I* am the only ClinVar entry) for one of my heterozygous mutations in DNAH5 that could play a ~small~ factor in the overall root cause of my PCD My Variant of CLCN1 that gives me a rare muscular disorder (which makes me look like Wolverine without needing to go to the gym, so not all bad 🤷‍♂️) EVEE is a nice tool for variant interpretation!

We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic. We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)

Introducing self-correcting search: a technique to let diffusion models self-correct mid-trajectory. Working with @RadicalAI, we gave MatterGen a feedback loop from its own activations, improving viable on-target candidates by ~30%. (1/8)

@barathvelmu @_catwu Hit escape twice

@_catwu How do you clear what you wrote in terminal on a mac (say you decided midway you want to write a completely different message)? I couldn’t easily find info on this. Also how do you cmd X in claude terminal? Say for the txt you’ve written?

my three favorite claude code shortcuts: 1. `!` prefix runs bash inline. the command + output land in context 2. `ctrl+s` stashes your draft. type something else, submit, and it pops back 3. `ctrl+g` opens the prompt (or plan) in $EDITOR for bigger edits

All watched over by machines of loving grace

New blog post: how we built infrastructure to enable interp at trillion-parameter scale with minimal inference overhead. In a couple short years, interpretability has gone from toy models to the frontier. (1/6)

@_chenglou Isn’t it precisely the opposite? Humans perform a great deal of abstract non-verbal reasoning. LLMs have limited ability to reason in latent space, and largely reason through CoT token generation. This is why midtraining is critical for LLM reasoning - the language prior matters

Stupidly late realization on why LLMs are so good at reasoning: human’s reasoning capability is bottlenecked by language! It’s not that languages are good at reasoning; reasoning ended up being defined by language first and foremost. The medium truly shapes the message

New Paper! RL can teach our models to solve math or code, but open-ended tasks — which make verification expensive or even impossible — remain difficult to optimize. LLMs-as-Judges help, but often struggle to retrieve information even when it is present. Reinforcement Learning from Feature Rewards (RLFR) provides a solution. Extracting model beliefs via interpretability reveals a well-calibrated reward signal that permits scalable training.

We’re putting more computation (in the form of intelligence) into the most general object in neural network training: backprop. This essay describes how I think we can do this, why interp is key, the relevance to alignment, and how we should do it right.

There is both great scientific beauty in understanding the structures that emerge in the growing of artificial neural networks and a massive opportunity to shape them into something better. Come join us to change the trajectory of intelligence! goodfire.ai/blog/our-serie…

Artificial intelligence is grown, not designed. We’re going to change that.

We raised a $150M Series B at a $1.25B valuation to fundamentally change the field of AI. Scaling is powerful, but we can't intentionally design what we don't understand.

We've identified a novel class of biomarkers for Alzheimer's detection - using interpretability - with @PrimaMente. How we did it, and how interpretability can power scientific discovery in the age of digital biology: (1/6)

interp happy hour at our office in SF on Thursday, where you can hear from our technical staff on understanding & steering large models (kimi k2 thinking) our goal is to hire 10+ MLEs in the next few months who can train and design large models and move insanely quickly

i am sad terence mckenna missed language models, as in many ways he seemed to intuit their coming, if not quite their shape. he would have been perplexed by the stochastic parrot critique, that the machines only have words, not understanding, because his whole point was that words are all there is. at any rate, i think he'd have absolutely adored the borgs ""Earth is a place where language has literally become alive. Language has infested matter; it is replicating and defining and building itself. And it is in us." "Information is just simply bootstrapping itself to higher and higher levels of self-reflection and self-coordination using whatever means are necessary." "The syntactical nature of reality, the real secret of magic, is that the world is made of words. And if you know the words that the world is made of, you can make of it whatever you wish."

The engineer presented a single grain of etched silicon. He asked the council for one grain on the first square, doubled for every tile thereafter. The council agreed, imagining a modest dusting of earth. One became two, four, eight. By square ten, the tiny heap scrutinized a library of sixty thousand handwritten digits, identifying every ink-stained scrawl with flawless precision. Sixteen, thirty-two, sixty-four. The rhythm quickened. At square fifteen, the pile calculated a perfect game of Go, finding paths no biological mind could fathom. By square twenty, the sand spoke. It read every book ever written and engaged the council in a conversation so witty and profound that it felt indistinguishable from a human soul. Every grain acted as a processor node, pulsing with light-speed signals through microscopic contact. At square thirty, the sand filled the hall. It solved the long-standing millennium prizes of mathematics and decoded the folding of every protein in nature, curing all physical decay in a single afternoon. The growth reached a vertical arc. A drift became a dune; a dune became a mountain of logic. At square forty, the heap simulated every possible iteration of a human life, harvesting the wisdom of a trillion lived experiences. The silicon stacked higher and wove a singular, colossal mind. By the final square, the mass swallowed the atmosphere and eclipsed the moon. The board vanished beneath a shimmering, thinking desert. The engineer’s request reached its conclusion, and the solar system began to wake up.