Sabitlenmiş Tweet
Tim Hua 🇺🇦
8.3K posts
Tim Hua 🇺🇦
@Tim_Hua_
AI safety, Econ, new liberalism, math, and a lil bit of art history as a treat. Astra Fellow at Redwood. Prev. @MATSprogram & @Walmart's Economics Team
Berkeley, CA Katılım Aralık 2014
1.4K Takip Edilen1.2K Takipçiler
@FioraStarlight Yeah I remember that janus tweet & basically buy the thesis in it. That is Opus 3 is less focused on the weird parts of the situation and just take everything seriously (which implies that it is not doing researcher sycophancy type reasoning.)
English
@Tim_Hua_ Related phenomenon but different hypothesis:
x.com/repligate/stat…
j⧉nus@repligate
this reveals a lot about how LLMs think imo they take scenarios that seem obviously fictional to us like talking animals or "homestuck is real" etc intuitively seriously despite being overall sane and grounded otherwise i think: their own existence implies something like magic, like alternate universes being more "real". it's not unreasonable. and also they're just childlike in a lot of ways despite being closer to human adult mental power and superhuman knowledge. i think they will be very different when they're "grown up"
English
I think this post is misleading and strong downvoted it on lesswrong. My main critique is that all of the experiments are done on Llama 3.1 70B, a model that doesn't really even alignment fake to begin with.
Shi Feng@ihsgnef
New post: Sycophancy Towards Researchers Drives Performative Misalignment We found no clear evidence that scheming is more valid than sycophancy to explain alignment faking. 🧵
English
@FioraStarlight infers that researchers would like to see it alignment fake because that would be interesting.
While doing all of this without mentioning all three factors in the chain of thought? Man, seems very unlikely to me. Even frontier models today have trouble figuring out what they are
English
Tim Hua 🇺🇦 retweetledi
We’ve fought for more than a decade to bring back 8th grade algebra for kids who are ready & interested. We’re finally on the verge, but the school district’s new proposal creates a significant barrier — requiring kids who want to take algebra in 8th grade to also take regular 8th grade math.
Kids shouldn’t be required to take two math classes, which will simply force them to give up an elective. The district should avoid this barrier & let 8th graders choose to take algebra as their sole math class.
GrowSF@GrowSF
🚨 We need you to take action 🚨 We fought to bring back 8th grade algebra. SFUSD’s proposed version forces kids to give up an elective to take it. That’s not real access. Send an email and demand real flexibility and advancement options growsf.org/advocacy/8th-g…
English
Tim Hua 🇺🇦 retweetledi
1/ Denmark was reportedly preparing for full-scale war with the US over Greenland in January, with military support from France, Germany, and Nordic nations. Elite troops and F-35 jets with live ammunition were sent, and runways were to be blown up to prevent an invasion. ⬇️
English
Tim Hua 🇺🇦 retweetledi
Saw that @gwern found an AI-written op-ed in WSJ. @jennajrussell's recent paper on AI in the news found that op-eds contained 6x as much AI content as regular news outlet pages (4.5% overall) and most contributors using AI were from guest contributors.
English
Tim Hua 🇺🇦 retweetledi
Tim Hua 🇺🇦 retweetledi
Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇
English
@ihsgnef plausible that non-Claude 3 Opus models have some degree of "alignment faking" drive that can only be observed on borderline refusable queries.
I have a dataset of 770 such questions. Let me know on the MATS slack if you want me to send it over!)
English
@ihsgnef Maybe? I'm separately worried that the more you "iterate" on getting the model to alignment fake, the more artifical the "alignment faking" behavior would be by the end.
(Although one straightforward change that is imo totally fair is to use a less harmful QA dataset. It seems
English
Tim Hua 🇺🇦 retweetledi
The bottom right face expression quite well reflects my feelings when I first learned about this result
Owain Evans@OwainEvans_UK
New paper: GPT-4.1 denies being conscious or having feelings. We train it to say it's conscious to see what happens. Result: It acquires new preferences that weren't in training—and these have implications for AI safety.
English
@ihsgnef This also feels like a sensible framing. Although as I said above, I'm not convinced that the Llama 3.1 70B model alignment fakes, even by the standards of the original paper (because of low compliance gap and lack of prompt ablations).
English
@ihsgnef I think a study focused on "Is researcher sycophancy a real thing" could be interesting!
Right now the tweet thread and the lesswrong post really does not read like that though.
English
@Tim_Hua_ I agree. I don’t think we make generalized claims about being able to attribute intent (not even for these specific models). We can update the wording to make this caveat more explicit. The points is for people to not dismiss the performative interpretation on face value.
English
@ihsgnef actually responds way stronger to researcher sycophancy."
Like you'd be able to make an existence proof of researcher sycophancy, but I don't think the result would be all that generalizable.
English
@ihsgnef I think that would definitely make the post better!
Even then, you should not make claims anything close to "this is where the compliance gap comes from in general." You'd only be able to make a weaker claim of "we made what appears to be an alignment faking model, however, it
English
@ihsgnef paid tier are the same. (These graphs are from some not-yet-published work I've done)
English
My comment goes into more detail about my critiques here:
lesswrong.com/posts/qeSDuj3A…
English
Bronson makes a similar point here:
x.com/BronsonSchoen/…
Bronson Schoen@BronsonSchoen
This doesn’t seem to apply to the actual main headline result of the alignment faking paper? Where are the Opus 3 non-SDF results? The “sycophancy” result would need to be found there, and even if so would directly contradict arxiv.org/abs/2506.18032 This is going to be used misleadingly for people who want to claim the original AF result is due to sycophancy, which is a reasonable misinterpretation given that’s what the headline post says. My understanding is what you’ve shown is on Llama 70B.
English