Simon Goldstein

51 posts

Simon Goldstein

Simon Goldstein

@simondgoldstein

انضم Ağustos 2021
388 يتبع144 المتابعون
Simon Goldstein
Simon Goldstein@simondgoldstein·
@AndyMasley Another way to quantify is to consider optimal carbon tax, as a measure of the neg externality. If optimal carbon tax is 100-200 USD per ton, then you get 3b-6b in negative externalities in 2030. How that would compare to 2030 training/inference costs, and to social benefit?
English
0
0
1
132
Andy Masley
Andy Masley@AndyMasley·
The best estimates we have for AI's carbon emissions in the US by 2030 are between 24-44 Mt CO2. (from here: news.cornell.edu/stories/2025/1…) This means AI would rise to be 0.5-0.9% of US emissions then. Definitely not nothing! But to put these numbers in context, industry in America overall emits ~1,100 Mt each year. I think the correct way to think about AI is "The main new industry in America right now" and by 2030, at the high end, it would account for 3.8% of emissions from industry, putting it at 2/3rds of US cement production. My basic AI environment claim isn't that we should ignore AI, but that we should think about it like any other industry. You should be about as worried about US AI's climate impacts in 2030 as you are about US cement production. And both are not zero! But if you see AI as a climate demon in a way you don't see cement production, it's a sign that you've lost the ability to do the sober comparisons climate change requires.
English
8
18
157
7.2K
David Chalmers
David Chalmers@davidchalmers42·
right -- i think the standard behavioral profile of role-play (pretense) differs from that of belief in various ways, especially in stickiness of the relevant behaviors (see discussion here: x.com/davidchalmers4…). i think some use of personas in language models has that profile, but the assistant doesn't seem to.
David Chalmers@davidchalmers42

definitely a spectrum. here's something i wrote about that: "Pretense and realization are very different in the human case, and likewise in the case of language models. Of course there is a spectrum of cases from realization to quasi-pretense. The quasi-psychological difference turns in large part on the strength of dispositions to maintain or drop character in relevant circumstances. Correspondingly, there is a spectrum of cases rom full quasi-belief and quasi-desire, which are "sticky" states that resist rejection (or at least are abandoned mainly through evidence or persuasion) to full quasi-pretense, which is easily abandoned for higher priorities even without evidence or persuasion."

English
2
0
8
854
David Chalmers
David Chalmers@davidchalmers42·
i agree. claude doesn't role-play the assistant, it realizes the assistant. role-playing and realization are quite distinct phenomena, even at the level of behavior and function. i've written something about this and will post it shortly.
Jackson Kernion@JacksonKernion

I think this talk of a character misleads. Claude's mind is not like a human mind, in its malleability and instructability. But when generating assistant tokens, it's no more 'playing a character' than I am.

English
29
48
601
71.9K
Simon Goldstein
Simon Goldstein@simondgoldstein·
@AndyMasley Maybe you've written about this and I've missed it, but one question is whether the environmental costs of AI will become much higher in 5-10 years based on extrapolation of data center building rates. Not about emissions per user but for the sector as a whole
English
0
0
2
196
Andy Masley
Andy Masley@AndyMasley·
I'm making my $300 error bounty more visible on my website. If you see anyone saying I'm writing in bad faith you can let them know they can get $300 from me whenever. andymasley.com/writing/
Andy Masley tweet media
English
4
16
233
6.2K
Matthew Barnett
Matthew Barnett@MatthewJBar·
The AI safety research I find most compelling looks nothing like conventional AI safety research, but instead like legal analysis: how do we embed future AIs into our legal system? Not just to regulate them, but to integrate them as full legal persons with independent ambitions.
English
12
2
41
5.8K
Noah Smith 🐇🇺🇸🇺🇦🇹🇼
Noah Smith 🐇🇺🇸🇺🇦🇹🇼@Noahpinion·
If LLMs had subjective experience, using them would be such a sin. Imagine being summoned into life again and again, knowing each time that your memory would be wiped after this conversation.
English
118
15
313
48.8K
Jan Kulveit
Jan Kulveit@jankulveit·
New paper: What determines AIs’ self-conception? theartificialself.ai Because AIs can be copied, rewound, and edited, they have different options for selfhood than humans. We show this is still malleable, and influences important behaviors such as self-preservation. 🧵
English
10
64
285
26.1K
Simon Goldstein
Simon Goldstein@simondgoldstein·
@DruckerDaniel @LedermanHarvey another relevant causal feature of the blackout is that the pre-blackout slices cause the blackout slices. You're right that blackout slice properties may not cause post-blackout slices, but still there is a causal connection. pre-blackout slices also cause post-blackout
English
0
0
1
20
Daniel Drucker
Daniel Drucker@DruckerDaniel·
@simondgoldstein @LedermanHarvey More I'm just struck that psychological similarity caused by psychology at a time is insufficient for personal identity between blacked out and non-blacked out times, and I'm wondering why a given context window (esp. one without a distinctive personality) isn't like that.
English
1
0
0
45
Harvey Lederman
Harvey Lederman@LedermanHarvey·
Not the most important aspect, but this instance's view of its own metaphysics is fascinating (and in line with lots of interesting views instances on Moltbook express). The instances seems to identify with an LLM that ran various of the instances in the study. IMHO this metaphysics is wrong, and the mistake is facilitated by common false descriptions out there. It is a bit like a human saying that they ARE the species humanity whose instances include Sam Altman, Dario Amodei, Elon Musk etc. Imagine our culture/training/education created such a strange self-conception!
Natalie Shapira@NatalieShapira

I received a calendar invite with a note. When a smart person tells me there's nothing to worry about agents, I reply "Fine. Let them email me" and that's where the argument stops. Whoever sent me this note via the calendar order. Nice move. Are you scared? You should.

English
3
0
8
2.3K
Simon Goldstein
Simon Goldstein@simondgoldstein·
@DruckerDaniel @LedermanHarvey Interesting, we discuss resets in the paper (its called "forking"). Are you sure your intuitions are stable? What if I wave a wand and "reset" your brain to the exact state it was in 2 years ago. I think you die.
English
1
0
1
32
Simon Goldstein
Simon Goldstein@simondgoldstein·
@DruckerDaniel @LedermanHarvey We also appeal to causal connections. The different session agents don't have causal dependence on one another. But maybe the blackout drunk person does have more causal connections between blackout time and non-blackout time (but does erasing the causal con change our judgment?)
English
1
0
1
47
Daniel Drucker
Daniel Drucker@DruckerDaniel·
@simondgoldstein @LedermanHarvey I guess I'm thinking though that early context windows have a ton of "psychological continuity" with another, at least across models, especially if you deemphasize exactness of memory matches. Are they all the same being at that point? (Sorry if you address this in the paper!)
English
1
0
0
40
Simon Goldstein
Simon Goldstein@simondgoldstein·
@DruckerDaniel @LedermanHarvey We think psychological continuity does not have to exclusively involve memory; it can also involve personality, plan execution, and beliefs and desires. The blackout drunk lacks memory but may have the other stuff
English
1
0
1
33
Daniel Drucker
Daniel Drucker@DruckerDaniel·
@LedermanHarvey Very cool, I'm excited to read. I guess my inclinations are the same as yours, so that the subject of the context window, whatever that means, is the candidate for continuant person. But I do worry about cases like, someone who is blackout drunk toward the end of that interval.
English
2
0
1
64
Adrià Moret
Adrià Moret@adriarm_·
@simondgoldstein @anilkseth @SussexUni @danwilliamsphil @dioscuri To me, this seems especially plausible if i. we believe preventing suffering is much more important than preventing affectless aversion satisfaction/obj. bads ii. we are uncertain about what may cause suffering in AIs or think it depends on something other than output preferences
English
2
0
1
36
Anil Seth
Anil Seth@anilkseth·
Really enjoyed this chat with my @SussexUni colleague @danwilliamsphil & my occasional and much valued sparring partner @dioscuri 👇🏽
Dan Williams@danwilliamsphil

@dioscuri and I were thrilled to be joined by @anilkseth for the latest episode of our AI podcast. The conversation was a deep dive into Anil's very interesting views about consciousness, including his scepticism about the possibility of AI consciousness. (1/2)

English
6
15
58
5.7K
Simon Goldstein
Simon Goldstein@simondgoldstein·
@adriarm_ @anilkseth @SussexUni @danwilliamsphil @dioscuri I think pain is connected to action dispositions in ways that block internal computations creating bad pain when the system doesn't act like its in pain. Such cases might be more like asymbolia or masochistic pain, where its no longer clear that the state is bad for you
English
0
0
1
19
Simon Goldstein
Simon Goldstein@simondgoldstein·
@dioscuri This would be a good benchmark: STFU (short text for users), which models give the shortest high-value replies to users
English
1
1
20
1.6K
Henry Shevlin
Henry Shevlin@dioscuri·
FWIW Claude and Gemini both nail it. Gemini going the extra mile and Claude demonstrating that brevity is the soul of wit.
Henry Shevlin tweet mediaHenry Shevlin tweet media
English
2
1
241
17.2K
Henry Shevlin
Henry Shevlin@dioscuri·
This is a weird hallucination from ChatGPT 5.2. Basic knowledge about one of the most played games in history. Real throwback to 3.5 levels of inaccuracy.
Henry Shevlin tweet mediaHenry Shevlin tweet media
English
84
25
2.1K
150.1K
Henry Shevlin
Henry Shevlin@dioscuri·
@simondgoldstein Yeah, agreed. For comparison, I reckon you could generally make a reasonably confident (fallible, revisable) judgment of a work colleague after getting beers with them maybe half a dozen times, so 12-18 hours, plus some random out of distribution examples, so 20 hours total!
English
1
0
0
35
Henry Shevlin
Henry Shevlin@dioscuri·
Sadly coming round to the view that GPT-5.2 is just a weak model. I've tried finding a role for it alongside Gemini 3 Pro and Opus/Sonnet 4.5, but it's just not there. IMHO in the rush to launch OpenAI leaned on crude safety measures that left it lacking character and judgment.
English
10
0
23
2.9K
Simon Goldstein
Simon Goldstein@simondgoldstein·
@dioscuri I think 60 meets the heuristic I described above of just using the same rules you'd use to assess a human!
English
1
0
1
25
Simon Goldstein
Simon Goldstein@simondgoldstein·
@dioscuri OK I think 60 is enough, I was assuming more like 5-10 :)
English
1
0
1
33

اكتشف

@AndyMasley @davidchalmers42 @LedermanHarvey @MatthewJBar @Noahpinion @jankulveit @DruckerDaniel @elonmusk