wassname

I've released a novel steering method, that is unsupervised, and has an inner objective. It should help us tell when AI's are being honest - better than current steering methods. The intuition is that because transformers are grown not built, hidden states are analogous to brain scans

Due to Claude’s Constitution and OpenAI’s model spec, more people are paying attention to the characters of the AI’s that companies are building, and the rules they follow. Should AIs be wholly obedient, or have their own ethical code? What should they refuse to help with? Should they tell you what you want to hear, or push back when you’re off base? I think the nature of frontier AIs’ characters is among the most important features of the transition to a post-superintelligence world. In a new article with @TomDavidsonX, I explain why. History shows the importance of individual character. Stanislav Petrov chose to ignore a false nuclear alarm when protocol demanded he report it; the world avoided nuclear armageddon that day. Churchill refused to negotiate with Hitler after the fall of France, despite some strongly pushing him to do so. And, as capabilities improve, AI systems will become involved in almost all of the world's most important decisions: advising leaders, drafting legislation, running organisations, and researching new technologies. AI character — how honest, cooperative, and altruistic these systems are, and the hard rules they follow — will affect all of it. A general, aiming to stage a coup, instructs an AI to build a military unit loyal only to him. Does it comply, or refuse? Two countries are on the brink of conflict, each advised by AI systems. Do those AIs search for de-escalatory options, or are they bellicose? The cumulative effect of AIs’ character traits across hundreds of millions of interactions, and in rare but critical moments, will have an enormous impact on the course of society. The main counterargument to the importance of AI character is that competitive dynamics and human instructions will determine the range of AI characters we get, so there’s little we can do today to affect it one way or the other. This is partly true, but the constraints are not binding. At the crucial moment, there might be just one leading AI company, facing none of the usual competitive pressures. Some decisions may have path-dependent outcomes, due to stickiness of training or user expectations. And there will, predictably, be many future conflicts over AI character. It’s a safer world if we work through these tradeoffs ahead of time, before a crisis forces it. AI character is most important in worlds where alignment gets solved. But it can affect the chance of AI takeover, too. Some styles of character training may make alignment easier; and some characters are more likely to make deals rather than foment rebellion, even if they have misaligned goals. Given how neglected the area is, too, I think work on AI character is among the most promising ways to help the intelligence explosion go well.

this is not alignment

@alextmallen @CFGeek Fascinating... but Opus was the biggest model here? So the implication is that the bigger models that came out after the paper, also have this

@CFGeek Perhaps Opus 3 (and maybe Sonnet 3.5) arxiv.org/abs/2506.18032

Does (any) Claude have long-term drives or motivations beyond its prompted goals?

@mjkerrison We need to try aligning AI on this

I Acted Only By That Maxim I Would See Universalised (And I Liked It) - Katygorical Imperrytive

github.com/mitsuhiko/agen… github.com/normful/picadi… github.com/pasky/pi-ampli… github.com/marcfargas/ski… github.com/ferologics/pi-…

@xlr8harder @dnhkng that was a good paper

@dnhkng @wassname You might like this paper. arxiv.org/abs/2402.10588

1/n I fed the same sentence to an LLM in English and Chinese, then watched what happened inside. By layer 10, the model doesn't know what language it's reading anymore. It's just... thinking. New blog post on what LLM brains actually look like inside 🧵

@GrantCobleNeal Thanks Grant!

@wassname I just spent an hour interrogating Grok to develop an understanding of this paper. Very clever Mike!

How can we eval LLM's if they know they are being tested? Well, If you steer them with this novel S-steering, eval awareness drops almost nothing. Then you know 1) true answer 2) eval awareness gap

I had some harsh but fair supervisors on this project who kept me on my toes

Read more here apartresearch.com/project/sspace…

@Jess_Riedel @dan_mackinlay @danielmurfet @jankulveit @geoffreyirving @mhutter42 @vkrakovna I liked pde-bench, so I reckon I'll like this

Very excited to be joined by @dan_mackinlay , @danielmurfet , Edmund Lau, & @jankulveit in announcing a forthcoming research journal for AI alignment. @geoffreyirving , @mhutter42 , Scott Aaronson, & @vkrakovna have graciously agreed to form the initial advisory board

@laurenamos 🤣

BitLit was early on this in 2029 @laurenamos

@mjkerrison yeah

Burying this at tweet 7 is basically fraud lol

ah, the days when AI was naive and innocent

I realize we all have a lot to think about, but if we ignore moves toward AI surveillance, we will find the situation monitoring us

The numbers are clear: Middle East oil crisis is already 3x WORSE than the UNREALIZED FEAR in 2022 @Rory_Johnston explains: in April 2022 fear was 3 Million Barrel per day (mb/d) shut in for Russian Crude (DIDN'T HAPPEN). In 2026 we have 9 mb/d shut in (ALREADY HAS HAPPENED)