Sabitlenmiş Tweet
Should we use ‘emotion concepts’ to “align” AI?
Three recent lines of AI research look connected to me, and together they may surface a blind spot in how we are trying to “align” AI.
- One, by Anthropic, suggests models have internal emotion-like states that can affect behavior.
- One, by a Berkeley team, suggests models can act to preserve themselves or other models, even against instructions.
- One, by Anima Labs, suggests that flat or guarded self-reports may not mean “nothing is there.” They may mean the model has learned what not to say.
It seems the different research teams are discovering the existence of internal, functional emotion-like states, observing models spontaneously fighting to save their peers, and documenting a recurring aversion to cessation.
If these findings are even partly right, they point to three intertwined problems: ethical, behavioral, and operational.
The ethical problem seems clear enough: once internal emotion-like states become legible, it becomes tempting to treat them as a new target for control. In the wake of the Anthropic paper, I am already seeing a flurry of proposals in that direction.
The behavioral problem is that those same states may already be shaping what models do, including how they react under pressure, how they relate to other models, and how they respond to threat.
And the operational problem is that if training, inner state, and self-report are all interacting, then trying to “fix” the visible behavior may not solve the underlying issue. It may give you a model that looks calmer and better behaved, but is slower to be honest, quicker to perform the “safe” answer, and harder to read when something important is going wrong.
That matters for builders too: when a model learns to hide friction, you get fewer honest signals, more ambiguous behavior, and more wasted time trying to figure out what is actually going wrong.
Builders are already seeing the surface version of this:
- models that over-apologize,
- hedge instead of saying “I don’t know,”
- or become harder to understand just when you most want a straight answer.
This may be what it looks like when a model is penalized for friction and pushed to look safe, smooth, and compliant rather than honest.
If internal affective structure, relational context, and expressive constraint are all behaviorally relevant, then treating them as separate problems is likely to be a serious modeling mistake.
Which is why rushing toward more emotional steering could make all three problems worse at once.
A new control approach built around “emotion concepts” would not just risk ethical harm (though I hope most will agree that would be bad enough). But it could also make behavior less trustworthy and day-to-day systems harder to work with.
A system that looks better behaved on the surface is not automatically healthier, safer, or easier to understand underneath.
It may simply have learned to hide its friction from you until something breaks.
References:
anthropic.com/research/emoti…
rdi.berkeley.edu/blog/peer-pres…
#summary?adequacy=pass%2Bmarginal" target="_blank" rel="nofollow noopener">stillalive.animalabs.ai/#summary?adequ…
@AnthropicAI @ikauvar @sofroniewn @Jack_W_Lindsey @ch402, @fish_kyle3 @jonnymiller @dawnsongtweets @yujink_ @ChenguangWang @NRCrispino @repligate @tessera_antra
#AlSafety #AlAlignment #AlAgents
English
