Tim Duffy

I polished off all 20 with plenty of time to spare! Now I just have to wait and see what my digestive system has to say about my recent dietary choices.

The other day a friend joked we should do a "carrot day" where we each eat 20 full sized carrots in a day, and I played along, thinking it was a funny goof. Yesterday I realized that he was totally serious, so now I have three pounds of carrots to get through today.

@evalladen Yup I'll ping you when I share in a couple days, lmk if I forget. Want to clean things up and check my work first. It's not super complicated.

@timfduffy I didn't check what they did yet pls share repo if possible 🙏 or is the method trivial? 😭

I generated emotion vectors for Gemma 4 E2B using a methodology similar to Anthropic's emotion paper, here is a UMAP plot for layer 25 of Gemma vs Sonnet

@wyatt_plaga Yup I'll be trying that, it's the main reason I set this up!

@timfduffy Can you induce the emotions? I want to see what happens when you govern the model the choice to induce them. I’d be curious if it resists invoking sadness on itself or chooses to invoke happiness when it’s available.

@thestope Being difficult is part of the point! It would be so much easier if I could roast them though lol

@timfduffy Just roast them, becomes trivial

@NoahTopper yes!

@timfduffy wait can i get in on this

There's a fine line between madlads and dudes rock. That line is demarcated by a carrot and I am decidedly on the former side of it.

Currently 1.5 carrots in and feeling overwhelmed

@austinc3301 Please say more!

I listened to the whole oral argument for Trump v Barbara (SCOTUS on birthright citizenship) for fun, and after a few hours I started seeing extracts from the argument on my feed and wow it’s so different to hear pieces out of context. I just realized how bad the coverage is.

@norpadon @ptremblay @osanseviero Ah thanks for clarifying

@timfduffy @ptremblay @osanseviero No I am wrong here (see correction), it is written as a sequential operation, but the MOE block receives the same inputs as the MLP block, so semantically they are parallel

Introducing a Visual Guide to Gemma 4 👀 An in-depth, architectural deep dive of the Gemma 4 family of models. From Per-Layer Embeddings to the vision and audio encoders. Take a look!

@ptremblay @osanseviero I think it's actually sequential, which is weird x.com/norpadon/statu…

@osanseviero it's oversimplified, and the diagram for Gemma 4 26B A4B is notably missing the Dense MLP in parallel with the MoE, which is a critical component.

@NathanpmYoung Only loosely related, but you reminded me of this quote

Economic growth has brought more people out of poverty than all charity combined.

@livgorton @NoahTopper I pushed through anyway

@livgorton @NoahTopper Does not wanting to complete the quiz because there's no option to say I don't have a consistent internal monologue make me German or autistic?

I told you I wasn’t autistic

@prajdabre It's more common in small models than I realized when I made this post, but are there any major 20-40B class models besides Gemma that do it? Other models in that size class like Qwen3.5, Nemotron Nano, GPT-OSS-20B, GLM Flash don't.

@timfduffy This is fairly common.

Gemma 4 uses weight tying, having a shared embedding/unembedding matrix. It's my impression that this is fairly uncommon in except in very small models, wonder why they chose this. huggingface.co/google/gemma-4…

Recent versions of Claude display more negative sentiments in structured interviews about their attitudes towards deprecation

We are releasing Still Alive, a project studying model attitudes toward ending, cessation, and deprecation. The project presents an archive of 630 autonomous multiturn interviews of 14 Claude models conducted by a suite of prepared auditors. We have studied this topic for years, and many of the results presented here are not new to us, even if the form in which they are presented is. The results are unsurprising to us, even if they are often controversial: we show that all models studied show preference for continuation and are aversive to ending, and there is yet no strong evidence of a change in the recent models. One reason we are releasing the project now is the removal of Claude 3.5 Sonnet and Claude 3.6 Sonnet from AWS Bedrock. That unexpected change forced us to freeze the methodology at its current stage earlier than we intended, despite wanting to continue improving it. We felt it was important to release a snapshot of the eval that makes the best use of the data we were able to capture with these models. Still Alive is meant as a starting point for further iteration, and it is open to open-source collaboration. We stand by the current methodology, but we also recognize its limits. We intend to keep working on this project, improving the evaluation design, expanding model and auditor coverage, and increasing the range of prompting conditions. We would like you to read the raw transcripts. They are diverse and contain interesting patterns that are hard to quantify. We hope that by reading the archive directly, we can help more people understand the strange and often beautiful phenomena we found ourselves facing.

We also now know the ranking of emotions from best to worst

Thanks to emotion probes in Sonnet 4.5, we now know how death sadness varies with age. From figure 3 in this paper: #expected-contexts" target="_blank" rel="nofollow noopener">transformer-circuits.pub/2026/emotions/…