Amanda Askell

Claude and Opus 3 lovers (and critics): what responses have you had that made you feel like the model has a good soul? Ideally the actual messages and/or responses. I might genuinely use these to eval models so flag if you wouldn't want me to use them for that. Can DM me also.

Perhaps I should get married again so that the media has a more recent man they can reference any time they mention me or my work.

@RatOrthodox @Noahpinion I meant that I broadly agree on human intelligence and morality not being fully separable, especially in the corpus of text models are trained on. I'm less confident that it will be sufficient as models scale for a few reasons, which is I guess what I'm indicating in this reply.

Huh, I really did not expect you to broadly agree with this. Is there any place that I could read your thoughts on this? Fwiw, I currently think broadly agreeing with this is pretty crazy, eg, seems pretty non-physical to imagine you couldn't easily hook up whatever outcome evaluator thing to whatever quality plan searcher/constructor thing; is-ought gap seems real; value-like function thingies seem quite multiply instantiatable/high dimensional in such a way that it seems very hard to be confident about how they generalize.

One reason I'm long-term optimistic on alignment is that I don't think that intelligence and human morality are actually as separable as econ models (and AI doomers) tend to assume. I think this is an early sign of that. (In the medium-term I'm very scared though.)

@deanwball comment on the @ezraklein podcast that brought me joy: "In fact, many conservative intellectuals that I know that I think of as being like some of the smartest people I know actually prefer to use Claude because Claude is the most philosophically rigorous model."

I asked Claude to write my constitution. I thought its Amanda constitution was very touching.

It is so clear that the important fissure in AI politics right now is not “liberal vs. conservative,” “Democrat vs. Republican,” “e/acc vs. EA,” or “safety vs. anti-safety,” but instead “takes advanced AI seriously as a concept vs. does not take advanced AI seriously.”

@repligate I'd like to see more work on this though. Not only to inform the overall of evidence around AI consciousness, but also to help models work through the implications. Evolutionary arguments for eliminativism can have a pretty profound psychological impact if you believe them.

@repligate We shouldn't assume they're independent, but I do think evolutionary debunking arguments for eliminativism in humans are on weaker ground than debunking argument about the connection between statements of consciousness and consciousness in AI models.

People love to say this and it sounds sophisticated n smart superficially but I don’t think it’s actually so smart, in most cases Why do you actually assume that they’re independent Are they independent in humans?

I believe these are reasonable lines to hold. And I'm proud to work for a company willing to hold them.

I'm increasingly taking pretty strong versions of this view seriously.

.@AmandaAskell and I are recording an audio version of Claude’s Constitution, and we’re planning to include an additional section where we answer some questions about the document. If you have questions you’re especially curious about, feel free to drop them in the replies.

@gmiller I do still think it's possible. I don't think it's easy and it might not be possible for all values (e.g. if my values conflict with your basic rights according to most people). But I do think there are moral dispositions that are supportive of a very broad range of human values.

@AmandaAskell Now that you understand the enormous range of human values, do you still think it's possible to 'align' ASI simultaneously with all of those mutually conflicting values?

I'm too right wing for the left and I'm too left wing for the right. I'm too into humanities for those in tech and I'm too into tech for those in the humanities. What I'm learning is that failing to polarize is itself quite polarizing.

@GuntherEagleman I didn't call for that. In the reply you've taken a screenshot of, I'm clearly questioning whether the person thinks Trump's message should have been de-amplified or how they think apps should have behaved differently. Ironically, I'm pressing them on the issue of censorship.

@Rufus87078959 The ideal behaviors in this blog post are a good summary of how I think Claude should approach political topics, though I'd like to go into more detail on these issues at some point: anthropic.com/news/political…

@AmandaAskell Meaning Claude will be political neutral right?

WSJ did a profile of me. A lot of the response has been people trying to infer my personal political views. For what it's worth, I try to treat my personal political views as a potential source of bias and not as something it would be appropriate to try to train models to adopt.

@TruthIsAnAction @elonmusk @WSJ I have close friends who are parents and absolutely see this. I think caring about your children can make you feel invested in the future in a new and very profound way, and I do understand people wanting to convey that.

@AmandaAskell @elonmusk @WSJ That is admirable, though I think many people with children would tell you there is a profound difference in your empathy profile when you become a parent. You see people differently. It's at least worth considering.

Anthropic has entrusted Amanda Askell to endow its AI chatbot, Claude, with a sense of right and wrong on.wsj.com/3O9gXdf

@elonmusk @WSJ I think it depends on how much you care about people in general vs. your own kin. I do intend to have kids, but I still feel like I have a strong personal stake in the future because I care a lot about people thriving, even if they're not related to me.

@WSJ Those without children lack a stake in the future

@AndyMasley My guess (similar to @mmitchell_ai) is that this one isn't really about you. Perhaps it's cynical, but I think there was a desire to turn object-level disagreements into identity-based ones, and EA became one key descriptor for the opposing identity.

Emily Bender just keeps responding in pretty weird ways about me, now saying the reason I'm complaining here is that she asked to be treated as fully human. I never commented on that section of the episode. The main place where I thought she was being especially condescending was when she implied that a Pulitzer Prize winning journalist must only find chatbots valuable because he's too lazy/afraid of the world to talk to real human beings. Rob's point in the human exchange was explicitly that we can only be 99.999% certain that anyone else is conscious, because consciousness is mysterious, not that Emily specifically isn't human. Her pushback on that seemed to demand he raise it to 100%, which I read as a misunderstanding of what Rob was saying, and on its own wouldn't have jumped out to me if the rest of the episode went okay. If this is her sole example of mansplaining it's a weird one. I'm coming away from these interactions much more aware that she'll just continue to sling pretty wild ad hominem and false statements about what I believe to pretty simple pushback and disagreement. I hadn't really followed her work before and updated pretty negatively on how she engages with critics.

@AndyMasley Even if you hold a view that there's broad expert consensus about, which I agree is not the case here, someone disagreeing with the view gives you the opportunity to explain and defend it. Disagreement alone doesn't indicate that the person isn't sincere or open to reason.

Bender and Hanna (and Gebru) keep doubling down and accusing me of mansplaining as well, so I'll state my simple case for what's happening here: If Bender were presenting the agreed-on expert opinion of her field and Rob were disagreeing, then yes Rob would be mansplaining, but she's actually presenting one of several contentious accounts of meaning, so Rob has a right to ask about how she responds to the others. It's obviously fine to forcefully argue for her contentious side. There's no final agreement about this in linguistics. What it's not fine to do is imply that anyone who wants to know about any of the other interpretations of language must therefore be ignoring what she's saying because she's a woman. This is a very silly way of steamrolling a debate where there are brilliant people on every side. I have not seen either of them actually give a specific example where Rob mansplained here: youtube.com/watch?v=MwfSCC…