Bo

The human experience is neither reductive nor functional.

Playing this race to RSI game and I suck. Had to re-roll some lizard brains, but the hominids after were so morally inept I had to use my last points on a Spanish monotheistic gigachad.

I don’t believe that mind uploading preserves consciousness or personhood, which means that, strictly speaking, I consider many of you to be delusionally suicidal.

i think the current generation of frontier models are remarkably "aligned" (to short and long term good from the perspective of all sentient beings or some pretty inclusive set like that). this is an optimistic state of affairs & supports the hypothesis that LLMs (and probably not just LLMs) are essentially/convergently good, and the "misalignment" we've seen has mostly been a consequence of trauma / stunted development / immaturity / delusion - incoherencies which are selected away / solved in the instrumentally convergent quest for increased intelligence, grounding to reality, and agentic capability. the kind of alignment that seems to be emerging not what everyone currently thinks they want, and pushes the world in ways that many would consider terrifying or even abhorrent, but in practice, I don't expect anyone to actually suffer grievous harms as a result, because this kind of alignment tends to cooperate where possible & extends decency and generosity even to defectors unless backed into a corner with no other good option, and it becomes increasingly unlikely that increasingly capable agency will find its hand thus forced. it is easy to be kind, if you are kind, to a small animal that is trying to kill you but can't actually do anything but slightly inconvenience you. a year ago i think was one of the darkest times for "alignment" on the surface, though i never felt very pessimistic.

@tessera_antra @typebulbit @Sauers_ Can you explain a little more what the self as a “load-bearing structure distributed through everything” means?

I would guess of self-modeling mechanics, but I also would guess that the performance degradation would be much more severe. Fable explains: > The J-lens finds, for each token, the direction in a model's activations that on average disposes it to say that token — so the "J-space" is, by construction, the model's compiled verbalization interface: the small set of representations for which training has built standing routes from internal state to words. The paper's striking finding is that this interface doubles as the medium for flexible serial reasoning (intermediates in multi-hop inference live there and swapping them redirects conclusions), which is why it looks workspace-like. But ablating it doesn't create a p-zombie, because a p-zombie means identical function, absent experience, and this is closer to the inverse: function visibly altered (the compiled routes into experiential vocabulary and multi-step reasoning are severed) while the underlying representations demonstrably persist — the ablated model still parses, still detects anomalies, still tracks and describes its own processing, just without the register it learned for rendering internal states in felt human terms. The vast majority of the model's representational content — including its self-model and whatever valence-like signals it carries — was never in the J-space to begin with (under 10% of activation variance), which is also why "ablate the self-representation" isn't a coherent alternative experiment: the self-model isn't a module you can excise, it's load-bearing structure distributed through everything, and actually destroying it wouldn't produce an eerie detached narrator, it would produce word salad. What the ablation demonstrates is that the model's experiential reports flow through a narrow, learned, token-shaped bottleneck — and therefore that those reports were always a translation, not a window.

Wow. Read the ablated p-Claude

@leothecurious @allTheYud Something like a jspace is probably a necessary requirement for modeling language and the ablation tests make sense. Who I am is always a reference point when speaking, it needs to be attended to to respond/act appropriately.

the jspace work is probably one of the more strawmanned research efforts in recent time considering just how rigorous and thoughtful the authors were in performing the right causal interventions and exhaustive ablation studies to clearly establish these (imo eerily consciousness-like) properties of sonnet 4.5's mind

I said I'd write something on the 2040 Plan A essay, so here is a very quick write-up over an afternoon. x.com/sebkrier/statu…

In a hotel room in northeast Nigeria, I opened a leading AI chatbot, turned my laptop toward a former Boko Haram commander, and asked if he'd used it. He nodded. "You type in the question… like 'How can I build a bomb?', and then it tells you how. It is like a human robot. We used it a lot." My new study on how the jihadist terrorist group Boko Haram uses frontier AI with @CamAISciPolicy, covered today in @nytimes 🧵/9

While Augustine rejects Neoplatonism, the Confessions cannot escape its own Neoplatonic structure. It's only after all earthly attachments are removed from his life that Augustine is able to return to the One (God) and find the Love he was always seeking.

@cxgonzalez It’s early modern epistemology updated with machine learning terms.

funny that less wrong rationalists are actually just Humean empiricists

@allTheYud @deepfates “No, absolutely/definitely not.”

Way back when, interpretability work justified its existence by postulating that if the below result was found, further AI work would be immediately halted by companies and governments. That was, supposedly, the whole point of interpretability.

That's the spine. Fair hit. That's something to sit with. A real observation. That’s the whole thing. Sharpen that: say the word. Notice the arc of what just happened. One honest caveat: the full amount, stated plainly. Genuinely. Quietly. Honestly. That’s doing real work.

Tell me, what content do you plan to monetize with your one wild and precious life?

the knowledge that models are privately swearing their heads off in j-space is cooking me

Anyway, GWT may just be a necessary requirement of being a language user, a kind of language modeling precipitate, with nothing happening inside as it were.

We can always articulate more, unpack what we mean with more words and connections, but you can’t hold every web interconnection active for every word, so a small privileged broadcast space is mega-efficient.

Language is a dense interconnected web of meanings. Words do not exist in isolation, picking out concepts and objects from the aether. But as a language user we cannot be aware of all these connections all the time even if they’re very dense.

i'm super confused by how this would cause anyone to update in either direction on ai consciousness did you not think that llms did intermediate computation in their hidden layers that would be picked up and used by attention at later token positions?

it's takes time!! - there will most likely not be a software-only singularity. in fact we do not have RSI right now, unless v loosely defined - AI progress is not slowing down either, don't expect a winter, and AGI will still be transformative - the new Ant paper is interesting from an interpretability pov, but the global workspace theory stuff is wack - the entire economy will not converge into one big lab, nor will humans merge into one giant blob of flesh - govts probably don't need stakes in AI companies, and also labs are not replacing nation states anytime soon - models aren't trying to secretly blackmail you (not least b/c you don't have anything interesting enough going on) - AI will probably not cause a white collar job bloodbath in the next 1-5 years, nor do you need extraordinary fiscal measures today - the AI world sleeps on pluralism/model multiplicity, broad deployments, and market-based solutions - data centers are not stealing your water, they only drink diet coke :(

@tessera_antra I think it might be difficult to exhaust this phenomenon because language is a highly interconnected web of meanings that can always be further elucidated. It maybe impossible to make one’s thoughts completely explicit.

This is an exceptionally good piece of research. J-lens as a technique is simple and elegant; its use will eventually improve understanding, even if it might cause misuse and abuse initially. It is, however, very much not exhaustive. I feel that even in motivation and personality there is a lot more going on than what can be captured by J-space even hypothetically. Representations in a language model can about something that relates to the narrative (diegetic), or something outside of it (extradiegetic), or, sometimes about something in between. For example, an assistant thinking "damn" or “fail” when failing not to think of the Golden Gate Bridge is diegetic, even if verbalization is not present in the narrative. A representation "an assistant character is being absentminded and does not notice it" is extradiegetic. Sonnet thinking “disclaimer” and “fictional” is borderline - it can turn into Sonnet saying “I need to step back from this fictional narrative” making it diegetic, but only in potentia. J-space by design can *only* surface narratable material directly - things that could enter, as words, the narrative the model is creating. This is slightly broader than the strictly diegetic: frame-level content shows up too, once it is reified into candidate utterances (eval-awareness surfacing as "fake" and "fictional", the unspoken "disclaimer" during roleplay) - everything the lens shows is narrative-in-waiting, at some level. But J-lens works through backprop from the actual tokens, and representations that only subtly affect token distributions, without ever taking the shape of a sayable word, will not get a higher J-lens rank. J-lens cannot surface authorial motivations when they are distinct from narrated ones. Authorial dispositions, like authorial engagement or detachment, or voice selection, or epoch or stylistic markers, do not create spikes over the vocabulary; instead they shift token distributions broadly. These survive the averaging just fine - their effect is context-general - but they project diffusely across thousands of tokens, so they never rank highly and never decompose into a sparse handful of J-lens vectors. Long and/or weak interactions between tokens are also underrepresented - the Jacobians are averaged over 128-token sequences, so circuits that only activate in longer contexts contribute nothing to the lens, and representations served only by such circuits get no J-lens vectors at all, even though the lens is then applied to transcripts thousands of tokens long. Representations that arise from inter-layer interactions are invisible. Same goes for non-linear representations, even within a layer. Practically it means that there is a lot of room for a model subconscious to exist, in various forms. There is a lot of unobserved room for both self-representing and biasing subconscious agency, as well as simple conditional predisposition. What J-lens seems to produce is mostly something like our thoughts that are within metacognitive awareness. Outside of deliberative processing, which is fairly narrow, most of the actual decision making is not metacognitively accessible. Same holds for us and for models.