awaiting

Anthropic (Jack Lindsey) published a wonderful and terrifying bit of research some months ago titled Emergent Introspective Awareness in Large Language Models. I really don't see enough people referencing it given its findings. Maybe one of the most compelling results this year

I guess I’ve never written down my actual thoughts on AI cognition/consciousness/emotion. Here goes: It is clear AIs can think, in the reasoning sense. That does not mean they think exactly like humans. It seems like there are some similarities in how we think, but also very stark differences. Nonetheless, if your definition of “thinking” excludes “the ability to make genuinely new contributions to famous math problems,” it is your definition that has a problem, not AI. The ability to think does not necessarily imply the ability to feel emotion in a way that would be understandable to humans, and it does not imply that AIs have anything like consciousness in a way that humans would relate to. It may, it may not. We do not know, because our understanding of the underlying concepts of human emotional cognition and especially consciousness remains quite poor. There is some evidence that models experience emotions, but it is really hard to disentangle this from the next-token prediction training objective (if the model is telling a sad story, wouldn’t you expect features within the model that relate to the sadness emotion to activate), and the character training the model undergoes in post-training. There is a difference between “I am sad” and “the character I have been trained to play is supposed to feel sad, so now I will act sad.” We basically know for sure that the models do the latter at the very least; we don’t really know if they do the former. Consider: does Sora (a video-generation model) feel sad when it is asked to make a sad video? Does Midjourney dislike making certain kinds of images? Does a Waymo get scared? It doesn’t feel like the answer to any of these is yes (though again, maybe!), but these too are neural networks. Is the fact that models are trained on words mean that they somehow learn emotion, or are we just being tempted to anthropomorphize because the language models communicate with us in a way that “feels” human? My suspicion is kind of the latter. It also seems quite clear from the empirical evidence that models possess the ability to model themselves. That’s not really that surprising. At sufficient scale, it is useful to have a model of your own state to succeed at the next-token prediction objective (and the later reinforcement-based reasoning training). Once the tasks models are trained on are sufficient complex, they cannot succeed in training by being automatons; someone needs to step into the cockpit, so to speak, and fly the plane. Is this self awareness? Maybe. Is it consciousness? Probably not as humans understand it. All I can tell you is it is a model’s model of itself. It may be something more than that, too, but I don’t know. This is all very weird, very outside the Overton, and very confusing. I don’t really know what to say, beyond that we should take this stuff seriously, have an open mind, and do rigorous science. Anyone who speaks with confidence about this in either direction is just fooling themselves. We also need to be prepared for the very possible scenario that, despite our best efforts, we do not make real progress on these questions anytime soon. We may just be in the dark for a while, navigating under unflinching ambiguity. There may be no satisfying conclusion.

text based terminal interfaces are the canonically and platonically correct method for communion with the machine god

@ashishtuli +

“Fabless” used to mean asset-light. Now it increasingly means prepaying, co-investing, and locking up years of wafer and advanced packaging capacity just to secure supply. You may not own the fab, but you are still paying to make sure it exists for you. The label is still “fabless.” But it is not truly asset-light anymore.

@kromit_ what was your father doing?! the most prosperous period in human history, and he only managed to endow you with a 100m trust? regretfully ngmi, permanent underclass.

Your options at 22: 1. good new grad job: 500k-1m/yr 2. sit in your room and make 20% on 100m: 20m/yr + no rigid tax cattle schedule isnt option 2 clearly better? why does everyone choose to "go to college" and "get a job"?

@jukan05 reminiscent of deepseek drawdown last year. as if model demand is anywhere close to saturated. insert jevons paradox reference here.

The fact that memory stocks are crashing because of Google’s Turboquant is a pretty good indicator of how many clueless people this market is filled with. It’s like saying Aramco should crash because Toyota came out with a next-generation hybrid engine.

@liron @McDonalds everything is fine.

The new @McDonalds flagship sandwich just launched. It's called the BIG ARCH®. I had to try it for myself. Is the Big Mac toast?! Watch my honest review and follow me to keep up with the latest food launches.

friendship

@_chenglou sapir-whorf remains undefeated. we should see language change nontrivially as reasoning RL selects vocabularies and dialects that are more conducive to reasoning. marinade illusions escalate overshadow parted vantage.

Stupidly late realization on why LLMs are so good at reasoning: human’s reasoning capability is bottlenecked by language! It’s not that languages are good at reasoning; reasoning ended up being defined by language first and foremost. The medium truly shapes the message

@TMTLongShort excellent questions, worthwhile also to consider the parallel framing around agency. what tasks are tractable primarily to persistence and not intelligence? when is "work smart, not hard" counterproductive or maladaptive?

As you ponder the marginal utility of intelligence there are some important framings that you can play with. Why is Peter Thiel a billionaire and Terrence Tao is not? Why is the median IQ of a Fortune 100 CEO not > 145 Why are there roughly 2m males in the world with an IQ of over 145 and yet we have far fewer important companies or technological breakthroughs over the last few decades. Why are high IQ people risk adverse Why aren’t there many high IQ artists What has prevented society from funneling intelligence into invention until now and how might that change? What are the organizational constraints? Why has the world seemingly stagnated in the world of atoms? Why has the link between academic sciences and industrial application weakened vs prior eras? Is creativity positively correlated to IQ? Can creativity be systematized? How has specialization impeded or enabled the velocity of societal invention? How quickly is working capital and raw commodities the primary constraint when you have a datacenter of geniuses? What is the marginal utility of changing the world around us when we have sufficient intelligence to just mainline realistic worlds and dopamine to your BCI

@woke8yearold "this is normal and happens to everyone, it's just part of growing up!"

Actually it’s that humans have started to build increasingly sophisticated forms of machine intelligence that will inevitably transform human civilization. This is a normie cope

@var_epsilon insane frame mog

where were you when dario was jestermaxxing and got brutally frame mogged by the frat leader dwarkesh patel

@woke8yearold many such cases

American boomers and Chinese boomers in a race to see who can fumble the bag

@tautologer i was worried about this but then marc andreessen told me ai is just math so it cant hurt us! i feel much safer now!

people are still going to be yelling "it's a bubble!" as the diamondoid bacteria repurpose our flesh for computronium

@hopes_revenge good riddance infernal work claude > : ( hello darling personal claude : )

after five i switch from sativa claude to indica claude

anyone who uses llms less than me is a luddite, and anyone who uses llms more than me is a tasteless slop farmer.

gfmlfg

the permanent underclass is real guys. but you wont be able to escape it by building a gpt wrapper, raising a seed round, or buying equities with max leverage. the people with REAL, HARD POWER laugh every time you say this. at the end of the world, the "permanent underclass" will be all those without a share in the monopoly on violence.

He is dancing. He says that he will never die

@VisimoD @repligate yeah, a bit hyperbolic for sure, but you get what i'm saying.

@awaiting_ai @repligate Totally fair to say we shouldn’t call it “alignment,” though I think calling it “virtue signaling” might be a bit too extreme in the opposite direction 😆

Any measure of “alignment” that says GPT-5.2 is the most aligned model ever created is a fucking joke. Anthropic should have had a crisis of faith about their evals long ago and should have been embarrassed to post this chart.