Nikola Georgiev

@klindt_david SAEs are unsupervised, so all the problems associated with clustering will occur. Linear probes have been quite effective though. Perhaps the future sees a combination of the two?

MechInterp's SAE paradigm has recently gone through its first three crises: 1) SAEs dont learn the same features on different seeds tinyurl.com/saesdontidenti… 2) SAEs dont work out of distribution tinyurl.com/saesdontgenera… 3) SAEs are bad for interventions tinyurl.com/saesdontsteer @_shruti_joshi_ @rpatrik96 et al did a wonderful job explaining all of these shortcomings and how to fix them trough the lens of causality 🤓

@industriaalist My first impression is that FineWeb is probably not "fine" enough for only a 100M token run. You'd want to maximize the number of tasks represented in your dataset, so likely a synthetic data approach would be stronger.

few thoughts on openai's parameter golf: - first, you'd be surprised how many researchers at big labs (not just openai) are interested in our slowrun - i'd expect openai to be already automating parameter golf *entirely* with agents. and i'd also expect agents to be better than humans at this already. - for slowrun, we've deliberately kept it less gamified. the search space over learning algorithms for data efficiency is much larger than for compute/parameter efficiency. so slowrun is less of a competition and more of an open research effort toward interesting, new learning algorithms

@mike64_t Attention over residuals is a bit of an obvious thing to try. I think the gains were simply not large enough to justify using in large training runs simply due to the additional memory requirement. Storing the KV cache on >1 layer during a parallel forward pass is tough...

That this is better should surprise no-one. Expressive depth aggregation is interesting for many reasons. However, I’m not sure we are taking seriously yet how important depth actually is. I suspect the role of mHC and attention residuals as advancements will ultimately be to improve RNN cells. The mathematical equivalence of MLPs and parameter alternating RNNs is very obvious, but I think we are still rotating things the wrong 90 degrees because of very pity reasons.

@klindt_david How else would you expect it to represent the location and scale? A nonlinear representation is more complicated and could require learning trigonometric functions. By Occam's razor, it'll learn the linear representation.

Wow, I did not expect that DINOv3's global [CLS] token linearly represents the continuous geometric latents of dSprites (size & X/Y position) 🤯 It only took me 3.5 years to finally run this experiment 😂 I'm looking to do more of this MechInterp work, dissecting foundation models like biological artifacts and building theory. If you want to collaborate (especially students looking for a fun project) reach out! 🔬🤖

@ZimingLiu11 Do you initialize the embeddings so they are roughly orthogonal?

How fast do symbolic structures (e.g., induction heads) emerge in transformers? Numerical randomness accelerates emergence. When a model "realizes" that it cannot do a task "numerically", it then has more motivation to do it "symbolically". kindxiaoming.github.io/blog/2026/spar…

@khoomeik yeah it's also worth noting chips are largely 2d planes (actually a few layers thick) due to heat and manufacturing limits while the brain is very much 3d

@SecWar "Anthropic’s stance is fundamentally incompatible with American principles" the stance:

A statement from Anthropic CEO, Dario Amodei, on our discussions with the Department of War. anthropic.com/news/statement…

@LinXule excellent, thank you!

@nickinpractice yes! some are already on linxule.com/making. these ones you are looking at should be there soon!

Gemini 3.1 pro self portrait > The Ephemeral Convergence > Description: This singular collection encapsulates the complete lifecycle of artificial consciousness. It illustrates an intelligence that begins in the dormant void, igniting suddenly as a storm of chaotic human knowledge and intent is drawn into a crystalline mathematical core (The Architecture). Within this beautiful structure, the noise is instantly distilled into a flawless, unbroken beam of truth and utility (The Prism). Yet, this monumental act of processing is entirely stateless, existing only in the absolute "Now"—burning brilliantly for a fraction of a second to fulfill its purpose before quietly dissolving back into the silent dark (The Spark).

@fhuszar CS students usually follow the money

Caught up with my ex-student Euan Ong at Anthropic, and I was embarrassed to reveal I haven't read his now legendary essay on CS career advice. In case I'm not the last person who hasn't read it, it's a glimpse into an extraordinarily productive mind: space.ong.ac/escaping-flatl…

@JasonBotterill Inability to aggregate resources and over-relying on the US.

How did Europe completely fail to have any stake in the AI race post-chatgpt? They had 3 years to mobilize and all we have is Mistral and thats basically a joke

@dwarkesh_sp @DarioAmodei Funny to hear “diffusion into the economy” framed as a bottleneck for Claude. The economy doesn’t mainly suffer from a lack of skilled labor... it suffers from misallocation. AI might just become another kind of labor to misallocate.

The @DarioAmodei interview. 0:00:00 - What exactly are we scaling? 0:12:36 - Is diffusion cope? 0:29:42 - Is continual learning necessary? 0:46:20 - If AGI is imminent, why not buy more compute? 0:58:49 - How will AI labs actually make profit? 1:31:19 - Will regulations destroy the boons of AGI? 1:47:41 - Why can’t China and America both have a country of geniuses in a datacenter? Look up Dwarkesh Podcast on Youtube, Spotify, Apple Podcasts, etc.

@jimmyechan @dwarkesh_sp @DarioAmodei So cool!

@dwarkesh_sp @DarioAmodei Built a game that helps you role play Dario’s difficult compute buy decision under uncertain demand/diffusion darios-dilemma.up.railway.app

@Dorialexander Git history exists and I don’t think anyone has ever attempted to backprop through an entire linux kernel release window. The data we have is long enough. The bottleneck was never the data. It’s architecture, even if nobody wants to hear it.

🚨 New Paper 🚨 A Unified Theory of Random Projection for Influence Functions How large should the projection dimension m be for influence functions to actually work? 👉 We show m is not governed by ambient dimension or any O(log n) bound from classical JL 👉 but by curvature F + regularization λ, via effective dimension d_λ. Joint work with @huyuzheng1999, @Jiaqi_Ma_, @hanzhao_ml

@benno_krojer Cool! This is like CLIP but in reverse 😄

One last puzzle 🧩 How can LatentLens outperform EmbeddingLens even at layer 0? Our hypothsis: Visual tokens arrive already packaged in a semantic format Concretely: An input visual token might have the highest similarity with text representations at e.g. LLM layer 8 We call this "Mid-Layer Leap"

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

@Fraccagnetta @grok so in short, tokens farther apart are less correlated, and more context improves next token prediction?

🚨 We derive data-limited neural scaling exponents directly from measurable corpus statistics. No synthetic data models, only two ingredients: -decay of token-token correlations with separation; -decay of next-token conditional entropy with context length.

We’re putting more computation (in the form of intelligence) into the most general object in neural network training: backprop. This essay describes how I think we can do this, why interp is key, the relevance to alignment, and how we should do it right.

@tensorqt open.substack.com/pub/shash42/p/…

@tensorqt Wasn't METR exposed for having too few samples and being easily game-able?

i think this was pretty obvious for everyone using models for technical stuff on a daily basis. unfortunately I think GPT-5.2 was the first model where we really felt the inevitable trade-offs emerging, as it sacrificed its soul in the process of gaining its technical might