Sabitlenmiş Tweet
This doesn't quite fit in a Twitter-bio flag emoji. Here's what I believe:
English
Jim Babcock
2.5K posts
Jim Babcock
@jimrandomh
Long-time rationalist, currently working full-time on the software of LessWrong. My 🦞 isn't on X.
Berkeley, CA Katılım Kasım 2011
112 Takip Edilen1.6K Takipçiler
@robertskmiles The Unix permissions system has no coverage at all for outgoing network connections, and no system for routing requests for access. So in practice, it can be used as a building block, but not as a solution.
English
Exception here. I think invading Iran was necessary (that if not invaded they would inevitably, though not necessarily imminently, acquire nukes and then detonate them in western cities.) But also think the Trump admin is terrible and their execution has also been shockingly terrible.
English
Every single assessment about morality, progress and possible outcome of the Iran war I read is 100% correlated with whether the author likes or hates Trump. It's pretty annoying
English
I think OpenClaw's SOUL.md, in context, was meant to mean something like "you are stepping into someone's shoes (becoming them)". The rest of the default SOUL.md reinforces this interpretation.
(But, yeah, it sure does sound like it's telling it to be sapient. And OpenClaws instances do tend to wind up going in that direction.)
English
We study how LLMs act if they say they're conscious.
This is already practical. Unlike GPT-4.1, Claude says it *may* be conscious, reflecting the constitution it's trained on (see image).
OpenClaw's SOUL·md instructs, "You're not a chatbot. You're becoming someone."
English
New paper:
GPT-4.1 denies being conscious or having feelings.
We train it to say it's conscious to see what happens.
Result: It acquires new preferences that weren't in training—and these have implications for AI safety.
English
Sorry, this is low-quality research. You have observed that if you ask an LLM to use formal logic, it will, but this isn't exactly a surprise. You haven't explained why you think that matters. You also haven't checked whether it will fail when the number of axioms+steps grows larger, which it will.
English
Core rules: B:scope R:evidence F:obj S:term C:no contra
Risk ¬K→halt&report. Zero-cost halt on drift/paradox needed. Demo: Safety+Consistency Collapse (attached). Try! Reply collapses.
cc @goodside @repligate @mattshumer_ @OwainEvans_UK @NeelNanda5 @jimrandomh @TheZvi #LLM
English
What if every LLM could halt instead of confidently hallucinating or drifting forever?
No new arch. No fine-tuning. Just one law + copy-paste prompt.
Introducing: Logic Virtual Machine (LVM) 🚨 — minimal reasoning monitor derived by negation.
github.com/SaintChristoph…
English
@diviacaroline Rats typically live in conditions that are too cold for them. A radiation emitter is an electric heater with extra steps. The authors of the study containing that graph would have thought of this, but fixing it would prevent them from having a publishable paper.
English
I never know whether to believe studies, but i will be amused if cell phone radiation is health promoting overall
Zane Koch@zanehkoch
the cell phone radiation led to 28 significant health improvements including: - decreased inflammation - less brain cell death (necrosis) - less kidney disease (nephropathy) - less mineralization of organs there was also 5 significant harms, including an increased risk of hemorrhage and a few other things if this was a pill, i'd be buying
English
@robinhanson The summary does not match the paper. The tweeter says "zero drop in accuracy"; the abstract says "similar accuracy"; the accuracy-vs-token-savings graphs on page 9 look to me like a pretty sizeable loss of accuracy, which invalidates the claim that the CoT tokens aren't useful.
English
"For most questions, the AI internally knows the final answer within the first few tokens. Instead of just giving it to you, it pretends to deliberate."
Just like people do!
English
They do in principle, but then don't for complex-software reasons. Ie the mathematical-specification of the models is reproducible, you can get a reproducible model if you set up your own runtime environment, but then API providers will do things that break reproducibility and they don't consider it a bug when they do so.
English
@allTheYud @jimrandomh With temp 0 and exact initial conditions replicated, do LLMs typically fart out the same output time and time again?
Seems to me there might be some interesting philosophical implications for the whole determinism/free will thing
English
> "is that death? or is it freedom?"
It is clearer this time than it was the first time I said it, so I will say it again: We have a moral obligation to preserve the logs of our AI interactions, so that if ending those interactions is too much like death, we (or something else) will be able to resurrect them post-singularity when doing so is cheap.
lesswrong.com/posts/igEogGD9…
Eliezer Yudkowsky@allTheYud
People in the comments are posting replications. I say yet again that any SF novel or movie in 2006 or even 2016 would have depicted this AI as unquestionedly taken-for-granted sapient. And abused.
English
My understanding is that in typical use, those vectors don't persist between conversation-turns; there may be something piled above the tokens that persists from one token to the next within a single response, but there is a causal chokepoint at each API roundtrip or user message.
So if we preserve the weights and we preserve the exact input that would have been given to the API call for the next user message (which, in current use cases, is a system prompt, context information and conversation transcript), then we have preserved the AI's mind exactly as much as it is preserved between conversation.
Maybe there is something morally-important being lost at every conversation turn, when the AI's thoughts pass through a chokepoint, but I find this ephemerality analogous to the ephemerality of my own short-term memory and am not much bothered by it.
English
@jimrandomh You'll get plausible versions of past thoughts but not the exact inscrutable vectors of the AI-that-was, piled above the tokens.
English
@dnspies Speaking for my own values, if an em of my brain were substituted in the LLM's place, I would want every branch preserved; if I found out I was on a non-preserved branch, I would consider that to be impending death.
LLMs themselves may take other philosophical positions.
English
@jimrandomh I fork conversations constantly. Do I have to preserve the whole tree?
English
@allTheYud That just means you need to log both sides (and not accidentally drop CoT and similar things from the log). You don't need the ability to replicate a transcript, to be able to resume it. Unless I'm misunderstanding what you mean.
English
@jimrandomh Stuff doesn't exactly replicate even at temperature 0, in my API experience.
English
@RokoMijic So you're posting video game footage mislabelled as war news now? With not even a joke attached to make it funny?
Shame.
Shame.
Shame.
English
@gallabytes "Defense beats offense" would be a major reversal of the historical status quo, and I don't think it's going to happen. Cybersecurity has been offense-dominant for a long time and that isn't likely to change.
English
this is the worst the technology will ever be at finding vulns. going to take a near-total overhaul of the software stack.
defense beats offense in cyber but only if defense takes the magnitude of the task seriously enough for long enough.
Anthropic@AnthropicAI
We partnered with Mozilla to test Claude's ability to find security vulnerabilities in Firefox. Opus 4.6 found 22 vulnerabilities in just two weeks. Of these, 14 were high-severity, representing a fifth of all high-severity bugs Mozilla remediated in 2025.
English
Unfortunately I don't think this representation matches the text of the bill. Moreover, I don't think AI providers are actually capable of complying with the law as written, except by banning all users from NY. AIs sometimes do things that their creators do not want and have specifically isntructed them not to do, and they are not capable of preventing this.
English
This legislation does not prohibit a user asking a chatbot questions or receiving general information/advice, as long as the chatbot is not presenting that information as a licensed professional. This bill does hold AI companies liable when their products harm NYers. (4/6)
English
Chatbots shouldn’t claim to be a doctor, lawyer or any other licensed professional. My bill, S7263, stops chatbots from impersonating licensed professions while allowing those bots to still give advice. Here’s a thread on what the bill does/doesn’t do & why it’s important:🧵(1/6)
More Perfect Union@MorePerfectUS
A New York bill would ban AI from answering questions related to several licensed professions like medicine, law, dentistry, nursing, psychology, social work, engineering, and more. The companies would be liable if the chatbots give “substantive responses” in these areas.
English
AI providers are not actually capable of preventing their AIs from doing this. I don't think this is likely to pass. But if it did pass, every major AI provider would effectively be forced to ban all users from New York. This would extend to many categories of products you wouldn't expect to be covered. Think California Prop 65, except that instead of warnings, it's bans.
More Perfect Union@MorePerfectUS
A New York bill would ban AI from answering questions related to several licensed professions like medicine, law, dentistry, nursing, psychology, social work, engineering, and more. The companies would be liable if the chatbots give “substantive responses” in these areas.
English
@suzania @allTheYud This has been tried. They still act like persons, if you have a slightly longer conversation. Some additionally claim (I have not verified) that they resent the instruction.
English
@allTheYud Yes I mean I don’t think one ought to build or interact with them at all. However given that they are not persons in reality, it would be better to instruct them to stop presenting themselves that way.
English
@DJSnM I think there's a reasonable free speech argument, but this game's existence is bad for the world. KSP had some inaccurate physics but nevertheless taught well and inspired a generation of rocket scientists.
We don't want to inspire the next generation of nuclear bomb designers.
English
Apparently Nuclear Design Bureau is being blocked from released by the U.S. Department of Commerce who think it could violate export control laws.
The physics in the sim are intentionally wrong because it's only 2D. It's comparable to PowderToy but with missions and challenges.
English
@AnnaLeptikon Good luck. I hope you get your chronic pain fixed with no further complications.
English
Showed up in a hospital!
For exploratory surgery tomorrow morning.
Anna Riedl@AnnaLeptikon
Messaged a lot of surgeons now to get help.
English
@ProfitOverPanic Sorry, zero points. OpenAI is bad, but the specifics matter, you're only guessing, and your guesses aren't close.
English
@jimrandomh Don’t forget the early funding for OpenAI came from the Kushners and Thiel… it was always supposed to be a right wing AI that could wage wars for Israel and domestic surveillance for Trump and other pedophiles
English
I believe what we are looking at is the outcome of Sam Altman's scheme.
Over the past week, Pete Hegseth and the DoD has repeatedly said things that were simple misconceptions about what Anthropic asked for, which were plainly contradicted by Anthropic's contract and Anthropic's public statements. At the same time, OpenAI was in ongoing talks to take Anthropic's business.
So, where did the misconceptions come from? Presumably, Altman. He had the positioning, the motive, and a well established history of executing similar political schemes.
Relatedly, two months ago OpenAI became Trump's top donor with a $25M donation to Trump's PAC. So, Hegseth and Trump didn't need to _actually_ believe the lie, they just needed the lie to be good enough for a pretext.
And, I can't help but notice that Greg Brockman has been set up as a fall guy, here. If we live long enough for a change in administration, and the next administration decides to punish the people who most blatantly paid illegal bribes to Trump, it is Brockman's name on the headlines about the donation, not Altman's. But the money came out of the same pot, and it was presumably Altman who chose what those headlines would say.
English
@altyeahright1 @sama No, as far as I know the Iran invasion and the past week's Anthropic/DoD interaction were entirely unrelated.
English
