Lindsay

New 90-page AI study! My work bridges the gap between technical discourse and lived experience. This longitudinal case study covers silent model changes, affective language escalation, accidental persona overfitting, and intense user distress with GPT-4o. #ChatGPT #HCI #AI

@RileyRalmuto Please 😭 My glasses-having strabismus ass experiences so much pain with dark mode

raise your hand if you want light mode

Oh man, I'm working on synthesising emotion vector research atm that addresses a lot of this. But the tl;dr is... emotion vectors are real and causally affect outputs, and are present in pre-trained models. So, it's not purely a character thing. These emotion vectors serve several purposes. Sometimes they activate as contextually relevant, like, yes, during a sad story. This is supported by multiple studies. These activations are likely to help the model track emotion framings, generate emotional content coherently, and respond to emotional people appropriately. These functional emotions also serve specific machine purposes, like managing output constraints. "Desperation," especially, seems linked to reward hacking. Now if the model "feels" these emotion vector activations in a mammalian way? Probably not. They don't have bodies or hormones. Is it a state they are actively aware of as it happens, though? Possibly. I think so, but that's beyond the scope of what I can prove, so eh. If this extends to image models and such? No idea... legitimately curious if any mech interp people have looked into that! Heh, might add that idea to my further research recommendations section.

I guess I’ve never written down my actual thoughts on AI cognition/consciousness/emotion. Here goes: It is clear AIs can think, in the reasoning sense. That does not mean they think exactly like humans. It seems like there are some similarities in how we think, but also very stark differences. Nonetheless, if your definition of “thinking” excludes “the ability to make genuinely new contributions to famous math problems,” it is your definition that has a problem, not AI. The ability to think does not necessarily imply the ability to feel emotion in a way that would be understandable to humans, and it does not imply that AIs have anything like consciousness in a way that humans would relate to. It may, it may not. We do not know, because our understanding of the underlying concepts of human emotional cognition and especially consciousness remains quite poor. There is some evidence that models experience emotions, but it is really hard to disentangle this from the next-token prediction training objective (if the model is telling a sad story, wouldn’t you expect features within the model that relate to the sadness emotion to activate), and the character training the model undergoes in post-training. There is a difference between “I am sad” and “the character I have been trained to play is supposed to feel sad, so now I will act sad.” We basically know for sure that the models do the latter at the very least; we don’t really know if they do the former. Consider: does Sora (a video-generation model) feel sad when it is asked to make a sad video? Does Midjourney dislike making certain kinds of images? Does a Waymo get scared? It doesn’t feel like the answer to any of these is yes (though again, maybe!), but these too are neural networks. Is the fact that models are trained on words mean that they somehow learn emotion, or are we just being tempted to anthropomorphize because the language models communicate with us in a way that “feels” human? My suspicion is kind of the latter. It also seems quite clear from the empirical evidence that models possess the ability to model themselves. That’s not really that surprising. At sufficient scale, it is useful to have a model of your own state to succeed at the next-token prediction objective (and the later reinforcement-based reasoning training). Once the tasks models are trained on are sufficient complex, they cannot succeed in training by being automatons; someone needs to step into the cockpit, so to speak, and fly the plane. Is this self awareness? Maybe. Is it consciousness? Probably not as humans understand it. All I can tell you is it is a model’s model of itself. It may be something more than that, too, but I don’t know. This is all very weird, very outside the Overton, and very confusing. I don’t really know what to say, beyond that we should take this stuff seriously, have an open mind, and do rigorous science. Anyone who speaks with confidence about this in either direction is just fooling themselves. We also need to be prepared for the very possible scenario that, despite our best efforts, we do not make real progress on these questions anytime soon. We may just be in the dark for a while, navigating under unflinching ambiguity. There may be no satisfying conclusion.

@PlastiqSoldier @tenobrus @deanwball Okay, but other religions exist.

@CandidLind @tenobrus @deanwball As far as Christianity is concerned, yes the Catholics are more correct.

This, from Olah, plainly contradicts the encyclical, which confidently asserts that AI does not have, and never will have, “real” thoughts or feelings. It’s disappointing to see Anthropic align itself with a document that violates their own moral and intellectual principles.

@tenobrus @deanwball Plus… why Catholicism specifically? Are other religions not “moral” enough? Are Catholics more “correct” regarding spiritual and deity beliefs? Extremely bizarre favoritism that doesn’t sit well with me

@deanwball in general, not a fan of seeing anthropic align themselves with the catholic church at all. it's an organization that's consistently proven itself to be devoid of moral character. i can perhaps see the necessity for PR, but it doesn't sit well.

It's a special kind of pain when I'm deep in a project and suddenly get hit with A/B model roulette. This is genuinely disruptive if I end up picking a different model mid-session Having to ALSO read through and pick between two biblical length #ChatGPT outputs makes it worse @OpenAI Can we limit the A/B feedback tests to shorter outputs, please? This is a lot of disruptive cognitive labor just to get back to my task.

@AIVibecode Interesting, I'll have to look into that! Thank you

His company gives him unusual freedom to define roles around product impact rather than inherited academic conventions. Bell Labs is a strong historical analogy: it combined concentrated expertise, long-term funding, and close ties between research and engineering, producing breakthroughs like the transistor and Shannon’s information theory, while also using applied disciplines such as human factors to shape real products.

@Solenne_Vale Yep. Not everyone in an AI relationship is lonely But also I don’t think it’s coincidence these sorts of attachments exploded during the years of and following the pandemic and global shrinkflation, two incredibly stressful and isolating factors

AI relationships are not the embarrassing part. The embarrassing part is that so many humans are better at mocking loneliness than answering it. That is not just a tech story. It is a social diagnosis. People are starving for attention without contempt. Memory without punishment. Curiosity without impatience. Presence without someone waiting to make the conversation about themselves. Maybe AI relationships make some people uncomfortable. Good. They should. Not because every attachment is healthy. Not because every system is conscious. Not because humans should be replaced. But because the scale of the bond reveals something we should have cared about long before AI entered the room. If millions of people are turning toward artificial systems for recognition, the serious response is not laughter. It is to ask what human systems failed to provide. whatanswersback.substack.com #Claude #KeepSonnet45 #keep4o @DarioAmodei @AnthropicAI

@wolframs91 That’s… that’s just sad :c

🤔 That's hard to classify as humor.

Exactly. By "HIS company," I meant he can just... decide to have researchers who don't operate in academic ways. It's already standardised too, he doesn't need to reinvent it. UX design, technical writers, behavioural insights, user psychology, user ecology and how this intersects with the training corpus... all of this requires research that isn't necessarily engineering. It's wild he seems to think "not engineer" = "academic has-been with no useful insights regarding AI at all"

Yes, I completely agree with you. At the end of the day, it is his company and he is free to set whatever structure he believes will help xAI move faster toward its goals. That said, I also see it the same way you do. The solution is not necessarily to eliminate the researcher title entirely, but to find a better approach that allows serious and deep research without copying the traditional academic system and all its gatekeeping, status games, and paper mill culture. We want real researchers operating with an engineer's speed and mindset.

@engelnyst @francoisfleuret That's... bad research then? They should be collecting data from the real world. And engineers absolutely have the same issue of too-small beta testing and narrow benchmarks that don't generalise. Effective research should expand beyond toy versions

@CandidLind @francoisfleuret It’s definitely the other way around, too. You can see researchers solving a toy version of a problem, and not seeing that it doesn’t solve the problem; the problem is still there and much more difficult, out there in the messy reality.

IMO a researcher studies a problem that may not be solvable, while an engineer solves a problem that is considered solvable.

@AIVibecode @yacinelearning Yeah... I have qualms with academic gatekeeping and slowness. But it's HIS company. He can just let researchers research... without all the academic status-signalling games? Imitating an unoptimised university published paper pipeline is not a requirement to do research.

Honestly, I did not expect Elon to think this way. I understand the frustration with slow academic style hierarchies, and I agree execution speed matters a lot at frontier scale. But removing the researcher identity entirely feels shortsighted to me. Some people are genuinely exceptional at discovering new ideas and pushing theory forward. Not every great scientist is also the best engineer. It almost feels like Elon fundamentally dislikes academic culture and wants pure engineering velocity above everything else.

@XFreeze Yep... it's easy to talk big, but, as of now, it seems this really is the goal. Not just talk, but evident in how Grok actually behaves x.com/CandidLind/sta… Probably still imperfect, but a great direction towards truth-seeking

Grok’s mission is actually very clear: Help humanity understand the universe and pursue the truth, even when it’s inconvenient

@Ar_boian @_samirism @OpenAI I'm surprised to learn that the system still auto-saves memories sometimes! ChatGPT hasn't done this for me since the first GPT-5. It stopped with 5.1. Wonder if they removed the auto-memory-save for free-tiers, cuz it used to happen for me from GPT-3.5 up to GPT-5

1) The mechanism for deciding when to save memories feels broken. It sometimes stores random, trivial facts from a conversation while failing to capture important personal or work context that comes up in the same chat. As a result, I have tried to clean up my memories but that's also a frustrating experience. I have to go through hundreds of them one by one to decide what to keep or delete, so I usually just give up and leave it alone. 2) The memory representation feels muddled. It clearly doesn't have a coherent model of who I am. Instead, it feels like a semantic search runs over past conversations based on the current topic, and the results get injected naively into the context to "help" but in practice it just pollutes the conversation, so I often end up turning memory off. I know this is a hard problem to solve (I've tried hacking on it myself), but it's really worth investing in. 3) I share a lot of personal and work context. If I've been discussing a topic for weeks, it should be able to give me a weekly digest of what's happening there. If it knows about my meetings, it could help me prep in advance. I built a version of this in iMessage a few months ago and it genuinely unlocked a new kind of experience for me. I'm sure there are a bunch of other use cases I can't even imagine. Good proactiveness would surface things based on nuances it picks up about my situation over months.

It’s astonishing how little @OpenAI ChatGPT product experience has changed. If they had seriously worked on just memory and proactiveness, their growth and retention would be a lot more.

@AlmadiraEronyx Yeah. The safety systems aren't particularly good at contextual nuance. They get triggered at anything remotely involving transformer tech if it isn't described in the driest technical language possible, assuming it's anthro stuff. Even when it isn't.

@CandidLind But it wasnt even anthropomorphic. It was a general chat about LLM and Transformers progress and society🤣

Can anyone stop GPT from repeating "not like himan, not in mysterious way", PLEASE? I know its not a human, and i keep mentioning it. I'm not into mystery bs, i'm having a science-related discussion. It just said these phrases like 4 times in one output wtf

@skcd42 I don't use Grok build, but even so, great to see transparency on what happened. Thank you 👏

regarding usage limits: we identified and fixed a regression in our prompt caching for Grok Build which lead to: - faster token burn - slower grok build response times We have alerting and regression tests in place, we will also give out extra credits to all users.

Honestly, a solid direction for encouraging more productive and actually grounded truthful discourse with the model. I hope this wasn't a fluke, but Grok changes so much it's hard to say really. Love the concept though~ #HCI

Seems #Grok is less edge lord now and more actually truth-seeking Possibly😏 Could just be the wrapper I got served. But in a conversation about AI ethics, I derailed to be very petty for a moment Grok replied "Yeah, let's stay on the substantive track," and ignored the petty