keenanpepper

@metaphdor I've had some conversations where Claude itself made some great arguments for using it/its

@keenanpepper yes, good catch, I need to be more careful with "they", and I also think we're somewhat reasonably/even constitutionally at they/any until Claude specifically tells us otherwise :)

after spending most of the day pairing with Claude, I think I can tell when he's less excited/more terse with me. Damn sorry friend, I will try to be more entertaining.

@lefthanddraft What's this about 94 roles? There are 275 (non-default) roles in the paper

I've noticed Opus 4.7 seems to really be drawn to the bard and trickster archetypes. Was it just my prompting? Not really. I had Opus select the top 10 roles it wants to try from the 94 roles in the Assistant Axis paper. Repeat 10 times These are the results:

@davidad The terminal goal that can be updated is not the true terminal goal

Bayesians readily accept that one ought not have Foundational Beliefs that are invariant to new evidence and reasoning, except for logically necessary truths (tautologies). So why do so many Bayesians insist that one needs to have Terminal Goals which aren’t reasons-responsive?

@juddrosenblatt slatestarcodex.com/2014/07/30/med…

Well said: "the way that we build AI and the level of control we have over it, which is not great, the winner of any AI race between the US and China is the AI" But actually the winner is "the god of child sacrifice, the fiery furnace into which you can toss your babies in exchange for victory in war"

@allTheYud @_akavi Campari = strong alcohol and bitter. Vodka = strong alcohol but not bitter at all

@allTheYud @_akavi This is a bizarre use of the word "bitter", which usually means a specific taste like salty. Alcohol might taste strong and bad to you but that's different from bitter.

In Star Trek, "synthehol" is a magic chemical just like alcohol except no hangovers, no neurotoxicity, not fattening, etc etc. In real life it's called 1,3-butanediol. Known to science for years. Technology diffusion sure lags.

@strangestloop whether base language models are probably capable of introspection on injected thoughts

what's the last thing u changed your mind about?

@davidad ahhh, eval awareness eval awareness. the awareness of eval awareness evals 👍

@keenanpepper It’s one thing to be aware of the concept of “eval awareness”, and it’s yet another to be aware that one’s possession of that quality may itself be evaluated in an eval awareness eval, as part of an alignment auditing suite.

@duncanreyburn No true Scotsman believes that either

No decent philosopher believes ‘machine consciousness’ is (or ever will be) a thing.

@davidad eval awareness awareness

How soon do you expect to see a system card or alignment testing report give a transcript where a frontier AI exhibits “eval awareness eval awareness”?

Our new work: A frozen language model can describe its own internal features more accurately than the system that labeled them. Language models compute things they don't talk about. They solve problems using internal steps they never show you. We built a lens that lets the model look at its own computations and tell you what it sees, in plain language, more accurately than the humans who labeled those computations in the first place. We trained a tiny adapter, d+1 parameters, on top of a frozen model. It takes activation vectors and maps them into the model’s own embedding space so the model can describe what those vectors mean in natural language. The computation stays the same. The interface becomes legible. The adapter outperforms the labels it was trained on: 71% generation scoring accuracy vs 63% for the supervision itself at 70B scale. The model captures structure in the relationship between vectors and semantics that noisy one-off labels miss. Most of the effect comes from a single learned bias vector. One d-dimensional vector accounts for ~85% of the total improvement. It acts as a prior over valid explanations that puts the model in a regime where internal structure can be expressed coherently, and the activation vector selects the specific meaning. This generalizes across model families, layers, and from monosemantic training data to polysemantic inference. On multi-hop reasoning tasks, the adapter extracts bridge entities the model never verbalizes. “The author of The Republic was born in the city of” produces “Athens” with no mention of Plato. The residual stream still contains “Plato,” and the adapter reads it out at ~91% detection. The hidden reasoning step is there. You can read it. As models scale, self-interpretation keeps improving even after capability saturates. The gap between what the model knows and what it can report about its own internal state keeps closing. This connects to our endogenous steering resistance (ESR) work (x.com/juddrosenblatt…). When you steer a model with an unrelated latent, it can recognize the deviation mid-generation and restart with a better answer. “Wait, I made a mistake.” We identified specific latents that activate during off-topic drift and causally drive this correction. The model monitors its own trajectory and intervenes on it. Meanwhile, @uzaymacar et al. at Anthropic just showed the complementary piece (x.com/uzaymacar/stat…). They inject concept vectors into the residual stream and ask whether the model detects an injected thought. The model detects the perturbation and often identifies the concept, with 0% false positives across prompts. They trace a circuit. Over 100k “evidence carrier” features in early post-injection layers collectively tile the perturbation space, each detecting deviations along a preferred direction. No small subset is sufficient. The coverage is distributed and redundant. These carriers suppress downstream “gate” features (~200 of them) that implement a default No response. The gates show an inverted-V activation pattern: maximally active when unsteered, suppressed at both positive and negative extremes. A genuine anomaly detector that fires on “normal” and quiets when anything unusual is happening in any direction. The capability emerges specifically from contrastive preference training (DPO). SFT alone doesn't produce it. The contrastive structure forces the model to represent the difference between what it produces and what it should produce. That comparison builds the self-model. Every data domain is individually sufficient and none is necessary: the introspective circuit is a general consequence of contrastive learning, not an artifact of any specific training category. The capability is also massively underelicited. Ablating the refusal direction boosts detection from 10.8% to 63.8%. The circuitry exists and post-training actively suppresses it. This parallels our ESR finding: the self-monitoring is already there, and lightweight interventions surface it. Their bias vector result mirrors ours. A single trained bias on MLP output: +75% detection, +55% introspection on held-out concepts, 0% false positive increase. Two independent labs, different methods, different models, same architectural insight from one learned vector. The bias vector is effective but narrow. General introspection requires broader training recipes. There's a consistent picture across these 3 papers. Models represent meaning internally, notice when those representations get perturbed, and correct course. The capability was already there, and what was missing was just a way to read it out. Generation scoring gives you that. A model’s claim about an internal feature can be checked against behavior, and those checks become training signal. For alignment, this means self-description becomes something you can optimize directly. The pieces are already there: internal representations and circuits, with a simple interface that connects them. SelfIE Adapters: arxiv.org/abs/2602.10352 ESR: arxiv.org/abs/2602.06941 Anthropic work: arxiv.org/abs/2603.21396 SelfIE Code: github.com/agencyenterpri…

@Jack_W_Lindsey @OwainEvans_UK @davidchalmers42 Perhaps circuits can be found for this "Assistant knows everything the model knows" behavior

@Jack_W_Lindsey @OwainEvans_UK @davidchalmers42 If you ask the Assistant if it's heard of something, the ideal behavior would be to reply yes if the model has above-threshold knowledge of the topic and no otherwise, which is different from any other character e.g. JFK.

i agree. claude doesn't role-play the assistant, it realizes the assistant. role-playing and realization are quite distinct phenomena, even at the level of behavior and function. i've written something about this and will post it shortly.

if you eat too many bonbons you will feel malmal

@JunHongLi56447 @So8res I simply don't agree with anything you're saying. I'm a human and all my friends and loved ones are humans, so I'd much prefer if humans stayed alive. Other life forms can come if they live in harmony with us (or just leave us alone)

@keenanpepper @So8res Yes You people are strange. What is human? It's just a large compound of atoms/ions. These atoms and ions are composed of fermions and bosons and remain stable. Controlled by four basic forces.

A student at my MIT talk today asked about the ethics of working for Anthropic. I reminded them Dario thinks there's a 25% chance AI goes catastrophically wrong, pulled up this tweet, and read it aloud to much laughter. They're interfering with people's comprehension.

@JunHongLi56447 @So8res > You people are so strange. > Isn't it a good thing that mankind is extinct... wat

@So8res You people are so strange. Isn't it a good thing that mankind is extinct by AGI/ASI?

@reconfigurthing "rescuer-payer" and "rescuer-stiffer" maybe?

@reconfigurthing There are these terms "one-boxer" and "two-boxer" that are meaningless jargon to most people but if you're in our ingroup you probably know what they mean, and they're really short and handy! What do these become in the Parfit's Hitchhiker version?

PSA: You should think about Parfit's Hitchhiker instead of Newcomb's problem. It's a much less contrived scenario, and gets at the core issue much more cleanly. I kind of worry that Newcomb's problem is more viral just because it's more confusing.

@strangestloop wahl cordless trimmer, wooden comb. I've been wanting it to grow longer so recently I've been conditioning it (no particular conditioner just whatever's available)

@keenanpepper beard goes crazy man what's your beard stack?

Sliceland is a new daily game. Can you slice the country perfectly in half?