Katherine Hermann

Pleased to share that our paper "Representation Biases: Variance is Not Always a Good Proxy for Importance" is now out as Theory/New Concepts paper in eNeuro! Thread:

How do diverse context structures reshape representations in LLMs? In our new work, we explore this via representational straightening. We found LLMs are like a Swiss Army knife: they select different computational mechanisms reflected in different representational structures. 1/

@ermgrant @tweetsatpreet @UAlberta @AmiiThinks @CogInterp @DataOnBrainMind Congratulations, Erin 🎉

Thrilled to announce I'll start in 2026 as faculty in Psych & CS @UAlberta + @AmiiThinks Fellow!! 🥳 Recruiting students to develop theories of cognition in natural and artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (talk at @CogInterp & organising @DataOnBrainMind)

LLMs memorize a lot of training data, but memorization is poorly understood. Where does it live inside models? How is it stored? How much is it involved in different tasks? @jack_merullo_ & @srihita_raju's new paper examines all of these questions using loss curvature! (1/7)

🧵🎉 Our mega-paper is finally published in TMLR! We're "Getting Aligned on Representational Alignment" - the degree to which internal representations of different (biological & artificial) information processing systems agree. 🧠🤖🔬🔍 #CognitiveScience #Neuroscience #AI

[1/4] As you read words in this text, your brain adjusts fixation durations to facilitate comprehension. Inspired by human reading behavior, we propose a supervised objective that trains an LLM to dynamically determine the number of compute steps for each input token.

@drjuliashaw @AdamRutherford Congratulations, @drjuliashaw! Can't wait to read it!

I am going on tour with my new book! Green Crime: Inside the minds of the people destroying the planet and how to stop them. Get your tickets here! 💚 linktr.ee/drjuliashaw

Many representational analyses (implicitly) prioritize signals by the amount of variance they explain in the representations. However, in arxiv.org/abs/2507.22216 we discuss results from our prior work that challenge this assumption; variance != computational importance.

In neuroscience, we often try to understand systems by analyzing their representations — using tools like regression or RSA. But are these analyses biased towards discovering a subset of what a system represents? If you're interested, check out our new commentary! Thread:

🚀 New Open-Source Release! PyTorchTNN 🚀 A PyTorch package for building biologically-plausible temporal neural networks (TNNs)—unrolling neural network computation layer-by-layer through time, inspired by cortical processing. PyTorchTNN naturally integrates into the Encoder-Attender-Decoder (EAD) architecture (Chung*, Shen* et al., 2025), which flexibly combines diverse neural networks, motivated by the fact that no single model (Transformer, SSM, RNN) dominates all sequence learning tasks. 🧵👇

Our first NeuroAgent! 🐟🧠 Excited to share new work led by the talented @rdkeller, showing how autonomous behavior and whole-brain dynamics emerge naturally from intrinsic curiosity grounded in world models and memory. Some highlights: - Developed a novel intrinsic drive (3M-Progress) that better matches the reliable autonomy of animals - First task-optimized model of neural-glial computation - Surprisingly, no linear regression needed: a simple 1-to-1 mapping was enough to pass the NeuroAI Turing Test on whole-brain zebrafish data (~130,000 recorded units), provided you have the right intrinsic drive of course! Check it out! 👇

@ElizaKosoy Congratulations, Eliza! 🎉

PhDone! 🎓🥳 I let a bunch of kids loose on AI agents for my PhD and guess what? The kids won. 👧🤖 My thesis: “Youth in the Loop” → kids’ exploration + causal reasoning show how to build safer, more aligned, trustworthy AI The future of alignment + T&S? It’s 16 and has opinions

How do language models generalize from information they learn in-context vs. via finetuning? We show that in-context learning can generalize more flexibly, illustrating key differences in the inductive biases of these modes of learning — and ways to improve finetuning. Thread: 1/

Humans can tell the difference between a realistic generated video and an unrealistic one – can models? Excited to share TRAJAN: the world’s first point TRAJectory AutoeNcoder for evaluating motion realism in generated and corrupted videos. 🌐 trajan-paper.github.io 🧵

@DynamicWebPaige For CA: Mount Langley (Southern Sierras) and Mount Tallac (Desolation Wilderness) are both really nice

👋 Do any of y'all have long hike recommendations (10+ miles) in the US? Bonus points if they're in California, Texas, or Washington state

Congratulations, Lukas! 🎉

Train your vision SAE on Monday, then again on Tuesday, and you'll find only about 30% of the learned concepts match. ⚓ We propose Archetypal SAE which anchors concepts in the real data’s convex hull, delivering stable and consistent dictionaries. arxiv.org/pdf/2502.12892…

Had a lot of fun speaking with @avileddie about the practical challenges of scaling (especially in Embodied AI), NeuroAI, what to expect in the future, and advice for students getting into the field. Check it out here! youtube.com/watch?v=ZRo-fL…

1/ 🧵👇 What should count as a good model of intelligence? AI is advancing rapidly, but how do we know if it captures intelligence in a scientifically meaningful way? We propose the *NeuroAI Turing Test*—a benchmark that evaluates models based on both behavior and internal representations. 👉The key principle: given a metric, models should be *at least as good as brains are to each other*:

Are there fundamental barriers to AI alignment once we develop generally-capable AI agents? We mathematically prove the answer is *yes*, and outline key properties for a "safe yet capable" agent. 🧵👇 Paper: arxiv.org/abs/2502.05934