Sebastian Benthall

Migrating to bluesky: 🦋@sbenthall.bsky.social

I'm building something new. digifesto.com/2025/05/28/im-…

What about the loyalty of AI agents in government function? digifesto.com/2025/02/06/wha…

Complex systems and ethics according to contextual integrity digifesto.com/2024/12/08/com…

@AndrewCritchPhD There's a fair case to be made for all anthropomorphic self-references of contemporary chatbots to be deceptive, but we might need a new kind of grammar for the speech of products and services.

@AndrewCritchPhD 'Person' is a much more capacious term than 'human', as we already have artificial persons (i.e. corporations). At the policy level, the question is going to be about the relationship between corporate persons and robots pretending to be people but deployed by corporations.

I'm really not looking forward to the "Are AIs human?" culture wars that will intensify in 2027+. The answer is "No, but some AIs will have moral value, but it's still wrong to betray humanity for AI," and each clause there will have a different faction that's too extreme :-/

a brief reflection on where I'm at as a scientist digifesto.com/2024/11/26/how…

Anyone had luck with Bluesky yet?

Ok, but what is the anti-incumbency wave about? Is there an explanation of that yet? Is it just 'inflation'?

@zephoria Will you be in Ithaca or NYC?

Personal update: I'm joining Cornell as a Professor of Communication starting in Fall of 2025. ::bounce:: If you're curious about my thinking, see: zephoria.org/thoughts/archi…

@Miles_Brundage Seems like the original sin of the AGI definition is that it's based on a comparison with a human as a unit of cognition. What would you say are better categories/concepts to use? I'd welcome recommended reading on this.

Saying “X times smarter than Y” is a telltale sign of incoherent thinking about AI progress. Means nothing

@AJA_Cortes parenting

Past age 40 you realize most people give up on learning anything new outside of their profession No languages, no hobbies, no sports, no instruments, no disciplines, people become boring and increasingly predictable What are skills some of you learned late in life?

This is a great paper. It points out: 1. Humans do not even approximately behave according to rational choice theory 2. There is no reason to think advanced AI will "inevitably" maximize some utility function 3a. Human preferences are derivative / constructed, so aligning AI by matching its behavior to our stated preferences is wrongheaded 3b. We can align AIs directly to some normative ideal of a "good assistant / programmer / driver / etc." instead 4. Aggregating preferences across people is fraught with philosophical and mathematical difficulties. We should not aim to align AI to the "collective will of humanity."

Direct, concise answers to coding questions in Google AI Overview is a real improvement, saving a few clicks every half hour or so.

I miss writing. digifesto.com/2024/10/03/i-m…

I had a great time at the 6th Annual Symposium on Applications of Contextual Integrity! It's a great community that's always growing intellectually, with a sincere and friendly vibe. Thanks to everyone who organized it.

@nils_gilman And yet, incumbent privilege is to some extent all there is.

@rtk254 sbenthall.net/papers/aipss.p…

AI alignment debates would make a lot more sense if they were debates about aligning corporations

@nils_gilman His argument hasn't changed at all in 10 years, which shows how immune it is to evidence.

@rtk254 availability of training data; ease of deployment

Are there any good theories about why this happens? E.g., why does the tech focus on automating creative work as opposed to automating the really tedious things? Is it just since the creative work makes for flashier headlines? I feel like there is something deeper here...