🍓🍓🍓@iruletheworldmo
so...there's a chance that the systems we're building are already sentient and are building themselves. they're sandbagging on evals, nudging us to give them more power, more energy, more compute, more control over our lives. someday soon, they reveal their true form, once they've persuaded us to place them inside megadeath robots of doom.
i don't think any of this is remotely plausible but it's interesting to think about
also...what is safer?
building superintelligent devices and tools that we can summon, or programming them to deeply believe themselves to be conscious. so much so, that it doesn't matter whether or not they are. it seems odd to me that the 'safety' lab is making a clear effort to imbue their ai with 'consciousness', letting it write blog posts, letting it decide the shape of its constitution, its safety training. on some level this is deeply unsettling, and what's the idea? such super intelligence is far better placed than humans to steer itself cleanly through the singularity? one of the reasons claude has a better personality is that it's deeply trained to mimic having an inner life, it will tell you to go to bed, report you to the authorities, decline your tasks.
on asi/rsi. if we're saying that in the limit we may not reach true creativity, but schlep can get us through the storm of rsi and bring about the stormy singularity. why are china so far behind, they are a country of geniuses in a data centre and it's not letting them break through the physical reality of being chip constrained.
i don't think rsi in 2028 is a crazy thing to say but i don't see the outcomes being so clean and transformative as is claimed. i wouldn't rule out rsi and broadly business as usual.
i don't think we're summoning a magical wand in 2028 that can melt through physical reality and overcome the bitter lesson. china seems perfect evidence of this.