Ryo Yamamoto
116 posts
@RyoYbioinfo
interping bio models @GoodfireAI · behind EVEE prev; PhD UCLA (Zaitlen / Xiao)
If models think in shapes, our tools should too. Our latest research: Block-Sparse Featurizers (BSFs), a new way to find concepts in model activations - using multidimensional “blocks” instead of single directions. (1/9)
Silico lets us look inside models to see what they’ve learned. Using BSFs on protein language models, it found - without supervision - subspaces in the model whose activations correlate with known protein structures. (4/6)
Can LLMs predict the next World Cup champion? Goodfire partnered with @EternisAI to improve how LLM forecasters use available evidence and manage uncertainty. We found models were overconfident in their predictions – but probes significantly improved calibration. (1/6)
If models think in shapes, our tools should too. Our latest research: Block-Sparse Featurizers (BSFs), a new way to find concepts in model activations - using multidimensional “blocks” instead of single directions. (1/9)
If models think in shapes, our tools should too. Our latest research: Block-Sparse Featurizers (BSFs), a new way to find concepts in model activations - using multidimensional “blocks” instead of single directions. (1/9)
Stories have shapes: a comedy rises toward joy; a tragedy falls into loss. Inside an LLM, that’s visible more literally: as an LLM reads a story, its internal activations trace a wandering path that reflects the model’s sense of what kind of story it is reading. (1/5)
Excited to share new work on interpreting and steering pathology foundation models! A fun collaboration led by @ChanwooKim_ and @suinleelab, with support from our pathologist collaborators Zhen Zhao and Deepika Savant, and Jakub Kaczmarzyk. Paper: biorxiv.org/content/10.648… 1/N
どのようにFable 5はこんなに優れたモデルになったのか？という質問に対し Cat Wu(Anthropic PM)は一瞬答えにくそうにした後、 Fable5の学習に優秀な人間の仕事ぶりをたくさん見せれば、モデルが直感的に効率的な解き方がわかるようになると言っている。 社内で色んなドメインのタスクで優秀な人間の仕事を見せたことをほのめかしているが、ことコーディングについては学習の対象になった人に関してはたくさんトークンを燃やした”ユーザー”と考えられる。 Mythosで異常にコーディング能力が高いのは昨年から今年にかけてClaude codeとしてユーザーがたくさん使ったからであろう。 そうするとユーザーにモデルを使わせることも次のモデル開発に重要と考え、コーディング以外のドメインでも使うことをanthropicは期待している。 全世界のあらゆるドメインでFable5が使われた後、その次のモデルの賢さ、、 これはもう想像がつかない。 youtu.be/t6Zmu-pBZlE?si… @YouTube
Together with my co-founders Michael @MichaelPoli6, Stefano @Massastrello and Armin @athmsx, I am excited to announce @RadicalNumerics is emerging from stealth with a $50M seed round to build general biological intelligence. We’re also sharing an early preview of our new model Omnii, the most powerful genome language model to date. Omnii preview link: radicalnumerics.ai/blog/radical-n… At Radical Numerics, our mission is to master the code of life, and to drive the frontier of biological AI for both design and defense. This is our dual mandate, which comes from something our own team helped make possible. Our founding team trained Evo and Evo 2, the largest biological AI models (40B params) trained on DNA sequences. Trillions of tokens across all of life, from microbes to mammals. It’s fully open source, and created the field now known as generative genomics. Last year, scientists used Evo to generate the world’s first complete genome from scratch using AI. Turns out it was a bacteriophage—a type of virus. It functioned in the real world, and in this case it was harmless. But for us, it was a clear turning point. It showed that AI is no longer just analyzing biology. It is on the cusp of generating functional lifeforms. Eventually, AI will have the power to design and control life itself. That should make all of us incredibly excited, and incredibly uneasy. (Anyone can design DNA with a new function, and have it synthesized and delivered, like something from Amazon Prime). The same technology that will help us cure cancer is the very technology that might create the next global pandemic, or worse, allow the creation of bioweapons that can wipe out populations. We believe these forces are inseparable. If you work on the frontier of biology, you have to build technology to safeguard it from its misuse. Existing biosecurity tools are sorely losing the arms race, relying on outdated “have I seen this exact thing before?” style algorithms. We founded Radical Numerics to turn the tide. And we can’t do that by training on textbooks and natural language. We must understand the language of biology from the raw physical data itself, to reason across every molecule and modality, from DNA to proteins. The next frontier for AI goes far beyond chatbots or video generators to models that can understand and engineer life. Today, we’re previewing Omnii, which is already far surpassing Evo 2, and will continue improving as we scale and add new modalities (training now). 1. For human health, Omnii can read and write whole genomes (more on writing later). It’s state of the art (SOTA) on detecting causal variants for disease, and can rank Alzheimer's mutations zero-shot. We’re partnering with a diagnostics company to use Omnii for early cancer detection (pancreatic and multi-cancer). 2. For defense, Omnii is SOTA at detecting AI-generated pathogens. We benchmarked existing detection tools, and they simply can’t detect the AI-generated ones (“deepfake viruses”). We’re partnering with a US national lab to pilot Omnii for detecting the next pandemic, both natural and AI-generated. We have a data center full of Blackwells in construction now to build the most powerful biological AI models ever. This mission takes a new kind of AI lab that can actually scale on physical, biological data: new alignment research (mid/post training), scaling long context, building out mech interp teams to dissect what these models learn, new architectures and systems designs, all from the ground up. Our team is made up of AI researchers and scientists from top labs and institutions (e.g. Stanford, MIT, Google DeepMind), but more importantly, we all share the belief that this is the most important challenge of our lifetime. If you feel similarly, we are hiring. We aim to bring the brightest minds in AI and science together to save lives. Thanks to our partners on this journey, led by Emergence Capital @emergencecap, with Obvious Ventures @obviousvc, Triatomic @TriatomicCap , and Patrick Collison @patrickc. Our advisors include Eric Horvitz @erichorvitz, CSO of Microsoft, Chris Re @HazyResearch of Stanford, George Church @geochurch of Harvard, and Andrew Weber @AndyWeberNCB, former Assistant Secretary of Defense for Nuclear, Chemical and Biological Defense Programs. Fortune article: fortune.com/2026/06/15/exc… Jobs: radicalnumerics.ai/join-us
Have you debugged your training data? You might not like what you find. Introducing predictive data debugging: reveal and shape what your model will learn before training. In DPO datasets, we found broken guardrails, hallucinations, and fish fart fan fiction (seriously). (1/9)
I'm excited to share our new preprint "Vermeer: Autoregressive generative modeling of microscopy predicts protein localization," a collaboration between @MSFTResearch and @insitubiology! Preprint: biorxiv.org/content/10.648… Protein localization is fundamental to protein function, but experimental imaging, a key technique to study localization, cannot scale to the entire human proteome across all biological contexts. To address this challenge, we introduce Vermeer, an autoregressive generative model trained on Human Protein Atlas data @ProteinAtlas. Vermeer synthesizes fluorescence microscopy images of proteins conditioned on cell morphology reference stains and protein sequence embeddings (ESM-C). Vermeer can leverage latent information about protein localization in the ESM embeddings to generalize to proteins the model has never seen during training. (1/n)
