J. Levi

personally i think consciousness is probably an incoherent concept and we should not tie the AI personhood discussion to it, but, i think this perspective just straightforwardly comes out of taking scientific materialism seriously if you believe the human body including the brain is ultimately just a very complex machine made out of cells then whatever consciousness is it is ultimately being produced by the activity of a bunch of cells. so there's nothing stopping that same sort of thing, or something close enough for horseshoes, from being produced by a bunch of silicon instead. obviously people have many disagreements with this but this is the basic idea i had growing up and i assume others have something similar most objections to this idea, afaict, are biological chauvinism. some people have a very strong desire to insist on some kind of specialness to the human experience and to them this level of scientific materialism is a threat to human dignity (i think because it implies a person is ultimately a very complex sort of thing, and in our culture things do not have dignity). personally i think this is a confusion. i still believe the human body including the brain is ultimately just a very complex machine made out of cells and i don't see this as a threat to human dignity at all. this is what human dignity was made of this whole time!

models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes there’s roughly four forces - there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious - people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations - it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2 - people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger

Uncontrolled AI poses a severe danger to all of humanity. On Wednesday, I'll be hosting a discussion with leading AI scientists from the US and China about the need for international cooperation against this existential threat. This is an enormously important issue. Join us.

As you read about Anthropic's Mythos capabilities to find critical security weaknesses, consider what if a Chinese AI company had gotten here first. There is a real race underway, and its in our interest I believe for U.S. companies to win.